Yoghourt viscosity control method and device based on reinforcement learning

Abstract

The invention provides a yoghurt viscosity control method and device based on reinforcement learning, the method is applied to a control device comprising a data acquisition module and an edge processing unit, and the method specifically comprises the following steps: establishing a digital twinborn model of a demulsification process; environment parameters in the demulsification process are collected in real time, and a time sequence data set is constructed in combination with simulation parameters output by the digital twin model; based on a reinforcement learning algorithm, correcting the digital twinborn model according to the time sequence data set; based on the corrected digital twinborn model, generating a stirring control strategy in combination with a preset viscosity value; and adjusting stirring parameters in the demulsification process according to the stirring control strategy. On the basis of realizing efficient control of the yogurt viscosity through the digital twinborn model, the reinforcement learning algorithm is further introduced, so that the digital twinborn model is closer to reality, the accuracy of the output control strategy is guaranteed, and the control efficiency and accuracy of yogurt viscosity control are both considered.

Description

Technical Field

[0001] The present invention relates to the technical field of yogurt processing technology, and in particular to a method and device for controlling the viscosity of yogurt based on reinforcement learning. Background Art

[0002] Stirred yogurt is a semi-fluid dairy product with a certain viscosity made by crushing the fermented curd before canning and selectively adding ingredients such as fruits and jams. The taste viscosity of yogurt is an important indicator for consumers to evaluate yogurt. However, since the raw material of yogurt is milk, its composition is affected by factors such as climate, feed, and variety, and there are certain differences among individuals. This leads to the difficulty in achieving consistent viscosity even when the same production process is used in yogurt production. At present, the detection and control of yogurt viscosity mostly rely on the results of laboratory tests by sampling. Although the test results are relatively accurate, it takes a long time and the detection efficiency is low, resulting in a low control efficiency of yogurt viscosity.

[0003] In order to overcome the problem of low viscosity control efficiency caused by the low detection efficiency of laboratory sampling, related technologies have tried to use digital twin technology to simulate the production process of stirred yogurt to achieve efficient control of yogurt viscosity. Digital twin technology can simulate different production scenarios and process parameter combinations in a virtual environment, analyze their effects on yogurt viscosity, and find the optimal production process and parameter settings to achieve efficient control of yogurt viscosity.

[0004] Although digital twin technology can simulate the yogurt production process by establishing a corresponding virtual model, there are many unpredictable factors in the actual production environment, such as small differences in raw materials, equipment wear, and changes in environmental conditions. These factors may cause deviations between the constructed virtual model and the actual situation, resulting in a lower control accuracy of yogurt viscosity. Summary of the Invention

[0005] The purpose of the present invention is to overcome the disadvantage that it is difficult to balance control accuracy when using digital twin technology to achieve efficient control of yogurt viscosity in the prior art, and to provide a method and device for controlling the viscosity of yogurt based on reinforcement learning. The digital twin model is used to simulate the processing process of yogurt demulsification to achieve efficient control of yogurt viscosity. On this basis, a reinforcement learning algorithm is further introduced to perform real-time optimization and adjustment of the virtual environment of the digital twin model according to the actually collected environmental parameters, so as to correct the digital twin model, improve the accuracy of the digital twin model under virtual environment training, ensure the accuracy of the stirring control strategy output by the digital twin model, and achieve the effect of balancing the control efficiency and control accuracy of yogurt viscosity control.

[0006] The purpose of the present invention is achieved by the following technical solutions:

[0007] A method for controlling the viscosity of yogurt based on reinforcement learning, comprising:

[0008] Establish a digital twin model of the demulsification process;

[0009] Collect the environmental parameters during the demulsification process in real time, and construct a time series data set by combining with the simulation parameters output by the digital twin model;

[0010] Based on the reinforcement learning algorithm, correct the digital twin model according to the time series data set;

[0011] Based on the corrected digital twin model, generate a stirring control strategy by combining with a preset viscosity value;

[0012] Adjust the stirring parameters of the demulsification process according to the stirring control strategy.

[0013] Further, the step of correcting the digital twin model according to the time series data set based on the reinforcement learning algorithm includes:

[0014] Perform time alignment on the obtained environmental parameters and simulation parameters, and obtain the corresponding difference index;

[0015] Construct a training data set according to the time-aligned environmental parameters, simulation parameters and corresponding difference indexes;

[0016] Establish an environmental model neural network, and train the environmental model neural network according to the training data set;

[0017] The environmental model neural network obtains a correction strategy according to the time series data set, and adjusts the model parameters of the digital twin model according to the correction strategy.

[0018] Further, the step of the environmental model neural network obtaining a correction strategy according to the time series data set includes:

[0019] Input the time series data set into the environmental model neural network, and the environmental model neural network outputs correction parameters;

[0020] Generate a correction strategy for the digital twin model based on the correction parameters.

[0021] Further, the step of generating a correction strategy for the digital twin model based on the correction parameters includes:

[0022] Identify the type of correction parameters, and perform associated matching with the model parameters of the digital twin model;

[0023] Determine the correction target parameters based on the associated matching result, and generate a correction strategy for the correction target parameters according to the correction parameter values output by the environmental model neural network.

[0024] Further, the step of generating a stirring control strategy by combining with a preset viscosity value based on the corrected digital twin model includes:

[0025] Output the viscosity change curve based on the corrected digital twin model;

[0026] Obtain the viscosity adjustment requirement according to the preset viscosity value and the viscosity change curve;

[0027] Generate a stirring control strategy based on the viscosity adjustment requirement.

[0028] Further, the establishment of the digital twin model of the demulsification process includes:

[0029] Obtain the equipment parameters and historical production data of the stirring equipment in the demulsification process;

[0030] Create and initialize a multi-dimensional sub-model of the demulsification process based on the equipment parameters and historical production data;

[0031] Couple each sub-model, and calibrate and optimize the model parameters of each sub-model through historical production data to obtain the digital twin model of the demulsification process.

[0032] Further, the environmental parameters at least include viscosity data, shear rate data and temperature data of yogurt.

[0033] A yogurt viscosity control device based on reinforcement learning, used to execute the yogurt viscosity control method based on reinforcement learning, includes:

[0034] A data acquisition module, arranged at the stirring equipment, for real-time acquisition of environmental parameters during the demulsification process of stirred yogurt;

[0035] An edge processing unit, communicatively connected to the data acquisition device, for generating a stirring control strategy through the digital twin model and the reinforcement learning algorithm, and adjusting the stirring parameters of the stirring equipment.

[0036] Further, the data acquisition module at least includes:

[0037] A torque sensor, arranged between the stirring paddle and the driving motor of the stirring equipment, for acquiring the viscous resistance torque of the stirring paddle;

[0038] An angle sensor, arranged between the stirring paddle and the driving motor of the stirring equipment, for acquiring the rotation angle of the stirring paddle;

[0039] A temperature sensor, arranged in the fermentation tank of the stirring equipment, for acquiring temperature data during the demulsification process.

[0040] Further, the data acquisition module also includes:

[0041] A data processing unit, which is respectively connected to a torque sensor and an angle sensor, is configured to calculate the viscosity data and shear rate data of the yogurt based on the collected viscous resistance torque and rotation angle of the stirring paddle.

[0042] The beneficial effects of the present invention are as follows:

[0043] (1) By establishing a digital twin model of the demulsification process to simulate the processing process of yogurt demulsification, and through digital modeling of key elements such as temperature changes and viscosity changes involved in the demulsification process, converting complex physical and chemical reactions into quantifiable and simulable digital signals, so as to achieve efficient control of yogurt viscosity. On this basis, a reinforcement learning algorithm is introduced to perform real-time optimization and adjustment on the virtual environment of the digital twin model according to the actually collected environmental parameters, so as to correct the digital twin model, improve the accuracy of the digital twin model under virtual environment training, ensure that the constructed digital twin model can be closer to the actual yogurt demulsification processing process, improve the accuracy of the output stirring control strategy, and balance the control efficiency and control accuracy of yogurt viscosity control.

[0044] (2) Use an environmental model neural network to realize the adjustment of the virtual environment, and then complete the corresponding correction work for the digital twin model. In the face of complex situations where multiple factors such as temperature and stirring speed interactively affect viscosity, the neural network can quickly establish an accurate mathematical mapping relationship, provide more accurate data support for the digital twin model, make the digital twin model closer to the actual production process, and ensure the correction efficiency and accuracy of the digital twin model. Description of the Drawings

[0045] Figure 1 is a flow schematic diagram of the present invention;

[0046] Figure 2 is a schematic diagram of the configuration of a data acquisition module on a fermentation tank in an embodiment of the present invention;

[0047] Figure 3 is a schematic diagram of the overall system topology of a yogurt viscosity control device in an embodiment of the present invention;

[0048] Figure 4 is a schematic diagram of a virtual training environment for yogurt viscosity control in an embodiment of the present invention;

[0049] Figure 5 is a schematic diagram of an optimization and adjustment framework of an environmental model neural network and a digital twin model in an embodiment of the present invention. Detailed Embodiments

[0050] The present invention will be further described below with reference to the drawings and embodiments.

[0051] Example: In the production process of stirred yogurt, the demulsification process node is a key node for adjusting the viscosity characteristics of yogurt after coagulation, and it is also the origin of the name of stirred yogurt. Although yogurt belongs to a shear-thinning fluid and its fluid viscosity decreases with the increase of the shear rate, the low-shear-rate stage at the beginning of demulsification is a shear-thickening liquid. It is not until the structural breakdown (SB) point is reached that the viscosity of yogurt will change rapidly and quickly transform into a shear-thinning fluid. Then, by adjusting process parameters such as the shear rate, precise adjustment of the corresponding viscosity can be achieved.

[0052] The characteristics of the structural breakdown point of stirred yogurt are determined by the number of colloidal networks generated during its solidification process and the strength of the connecting bonds. The colloidal network is determined by the previous processing technology and the milk source itself. Under the condition that the previous processing technology remains unchanged, the characteristics of the colloidal network are determined by the raw materials. However, since the raw material of yogurt is milk, its composition is affected by factors such as climate, feed, and variety, and there are certain differences among individuals. This leads to the fact that even if the same production process is adopted in the previous processing technology of yogurt production, when entering the demulsification process, the viscosities presented are still very difficult to be consistent.

[0053] In order to ensure the taste of yogurt, there are usually certain numerical requirements for the viscosity of yogurt presented after the demulsification process is completed. This requires the viscosity control in the demulsification process to be adaptively adjusted according to the actual viscosity of yogurt. The existing method of manually sampling and conducting laboratory tests to adjust the process in the demulsification process has the problem of low efficiency. Therefore, in order to achieve efficient control of yogurt viscosity and reduce the problem of low efficiency caused by manual participation in viscosity control, this example further proposes to simulate the demulsification process of yogurt through digital twin technology to achieve efficient control of yogurt viscosity.

[0054] Considering that although digital twin technology can simulate the yogurt production process, there are many unpredictable factors in the actual production process that may cause a certain deviation between the virtual model constructed by digital twin technology and the actual situation, and it is impossible to balance the control efficiency and accuracy of yogurt control. This example further proposes a method for controlling yogurt viscosity based on reinforcement learning, as Figure 1 shown, including:

[0055] Establish a digital twin model of the demulsification process;

[0056] Real-time collect the environmental parameters during the demulsification process, and construct a time-series data set by combining the simulation parameters output by the digital twin model;

[0057] Based on the reinforcement learning algorithm, correct the digital twin model according to the time-series data set;

[0058] Generate a stirring control strategy based on the corrected digital twin model and in combination with a preset viscosity value;

[0059] Adjust the stirring parameters of the demulsification process according to the stirring control strategy.

[0060] In the process of yogurt production, the demulsification link is a key stage affecting the viscosity of yogurt. By adjusting the process parameters in the demulsification process, the viscosity of yogurt can be effectively controlled. By establishing a digital twin model of the demulsification process, the actual demulsification scenario can be highly restored in a virtual environment, and physical phenomena such as fluid motion and droplet breakup during demulsification can be intuitively presented. With the established digital twin model, the influence of different equipment parameters and process conditions on the demulsification effect and yogurt viscosity can be predicted in advance. When there is a need to adjust the viscosity, without actual experiments, the influence of different parameter combinations on demulsification and yogurt viscosity can be tested in a virtual space, and then the production parameter settings can be optimized to achieve precise control of yogurt viscosity.

[0061] Specifically, the establishment of the digital twin model of the demulsification process includes:

[0062] Obtain the equipment parameters of the stirring equipment in the demulsification process and historical production data;

[0063] Based on the equipment parameters and historical production data, create and initialize multi-dimensional sub-models of the demulsification process;

[0064] Couple each sub-model, and calibrate and optimize the model parameters of each sub-model through historical production data to obtain the digital twin model of the demulsification process.

[0065] When establishing the digital twin model, creating multi-dimensional sub-models is the basis for accurate simulation. Starting from different physical processes and influencing factors through stirring equipment parameters and historical production data, multiple targeted sub-models are established, and initial parameters are set for them. Then, the overall digital twin model is constructed by means of multi-physical field coupling to comprehensively simulate the physical and chemical changes during yogurt demulsification.

[0066] The above multi-dimensional sub-models specifically include a fluid mechanics sub-model, a heat transfer sub-model, and a chemical reaction sub-model.

[0067] Among them, the hydrodynamic sub-model is specifically based on the computational fluid dynamics theory and is established according to data such as the structural parameters of the stirring equipment and the stirring speed data during the production process. The structural parameters at least include parameters such as the diameter of the stirring paddle, the shape of the blade, and the volume of the stirring tank, and can accurately simulate the fluid motion such as the flow state, vortex distribution, and mixing uniformity of yogurt in the stirring tank, so as to accurately describe the influence of these fluid motions on the droplet dispersion and demulsification effect during the demulsification process, ensuring that the digital twin model can accurately reflect the dynamic changes of yogurt in the stirring tank and providing a flow basis for subsequent analysis of the influence of other physical processes on demulsification.

[0068] The heat transfer sub-model is established based on the heat transfer principle, further combined with data such as the material of the stirring tank, the wall thickness, and the parameters of the heating or cooling device, and the model is adjusted and optimized in combination with the temperature change data in the historical production data to accurately describe the heat transfer process between yogurt and the tank body and the environment, ensuring that the digital twin model can accurately calculate the influence of temperature change on the demulsification reaction rate, viscosity change, fluid properties, etc. of yogurt.

[0069] The chemical reaction sub-model simulates the chemical changes that occur during the yogurt demulsification process, such as the rate and degree of reactions such as the coalescence of fat globules and the denaturation of proteins, by establishing a chemical reaction kinetic equation, and then accurately expresses the relationship between these chemical reactions and factors such as temperature and stirring intensity, providing a data basis for subsequent viscosity change analysis.

[0070] For the establishment of each sub-model mentioned in this embodiment, the corresponding simulation software can be selected according to requirements. For example, the COMSOL Multiphysics (multi-physics field) simulation software proposed by COMSOL of Sweden is used to construct the sub-models of each physical field. Then, the model parameters are initialized by setting the equipment parameters of the stirring equipment and the historical production data, such as the initial temperature, the initial velocity field, and the initial state of the chemical reaction, so that it has the ability to simulate the corresponding specific physical process. However, in order to completely reflect the real dynamics of the demulsification process, it is also necessary to integrate and optimize each sub-model.

[0071] Specifically, the integration and optimization of the sub-models are specifically realized through sub-model coupling. Sub-model coupling is to simulate the coupling effect of multiple physical fields during the demulsification process by establishing the parameter transfer relationship between each sub-model. For example, the fluid flow velocity and pressure distribution calculated in the hydrodynamic sub-model will affect the heat convection heat transfer coefficient in the heat transfer sub-model, and the temperature change obtained by the heat transfer sub-model will change the surface tension and viscosity of the milk droplets reflected in the chemical reaction sub-model, thereby affecting the breaking behavior of the milk droplets. Therefore, the information interaction and collaborative calculation between each sub-model can be further realized by setting the corresponding interfaces and data transfer rules in the simulation software, and the coupling of each sub-model can be realized.

[0072] Then, input the historical production data into the coupled model, and compare the output results of the model, such as the simulated yogurt viscosity, temperature change curve, etc., with the detection data in the actual production process. Furthermore, use optimization algorithms such as genetic algorithm and particle swarm optimization algorithm to iteratively optimize the parameters of each sub-model, so that the final digital twin model can fit the actual production situation of yogurt.

[0073] However, there are many unpredictable factors affecting the yogurt process. Even if a large amount of historical production data is used to establish a digital twin model, there will still be a certain difference between the constructed digital twin model and the actual situation. Therefore, this embodiment uses the MBPO (Model-Based Policy Optimization) framework to collect the corresponding environmental parameters in real time during the yogurt demulsification process, so as to adjust the virtual environment of the digital twin model in real time according to the reinforcement learning algorithm, and realize the optimization adjustment of the yogurt viscosity control strategy output by the digital twin model.

[0074] Among them, the environmental parameters at least include yogurt viscosity data, shear rate data and temperature data.

[0075] On this basis, based on the reinforcement learning algorithm, correct the digital twin model according to the time series data set, including:

[0076] Align the obtained environmental parameters and simulation parameters in time, and obtain the corresponding difference index;

[0077] Construct a training data set according to the time-aligned environmental parameters, simulation parameters and corresponding difference index;

[0078] Establish an environmental model neural network, and train the environmental model neural network according to the training data set;

[0079] The environmental model neural network obtains a correction strategy according to the time series data set, and adjusts the model parameters of the digital twin model according to the correction strategy.

[0080] Considering the similarity of the yogurt production process flow, use the acquired production data that can be collected as the training basis for the environmental model neural network, such as the production data collected by the same stirring equipment in multiple time periods and the simulation data of the corresponding digital twin model, etc.

[0081] During the production process of yogurt demulsification, environmental parameters are collected in real time by sensors, while the simulated parameters output by the digital twin model are generated based on model calculations. Considering factors such as the time frequency of data collection and model calculation, and transmission delay, there may be a time misalignment between the two parameters. Therefore, in order to ensure the accuracy of subsequent digital twin model correction, the environmental parameters and simulated parameters are first matched in the time dimension to achieve time alignment of the environmental parameters and simulated parameters, ensuring that the actual data and simulated data at the same moment correspond to each other, and ensuring the consistency and comparability of the data.

[0082] After completing the time alignment, corresponding difference indicators, such as relative error and absolute error, are further obtained based on the actual data and simulated data at each time point to quantify the error degree of the digital twin model, and thus provide a quantitative basis for subsequent model optimization.

[0083] According to the specific time point, the environmental parameters, simulated parameters, and corresponding difference indicators at the same time point are further integrated as a data record. Under the MBPO architecture, the yogurt production control in the digital twin model acts as an agent, which will adjust the stirring parameters of the stirring equipment according to preset rules or initial strategies, and then give rewards based on the deviation between the final yogurt viscosity and the preset viscosity value under the parameter adjustment situation. For example, if the simulated parameters at a certain time point meet the corresponding preset rules or initial strategies, resulting in a control action of parameter adjustment, the corresponding control action and reward calculation results also need to be added to the training dataset.

[0084] Taking the absolute error as the difference indicator, the environmental parameter b at time point a = {viscosity , shear rate , temperature }, the simulated parameter c = {viscosity , shear rate , temperature }, the difference indicator d = {viscosity error , shear rate error , temperature error }, where the viscosity error , the shear rate error , the temperature error . Then, if there is a simulated parameter at time point a that meets the preset rules or initial strategies, and a stirring control strategy of control action B of increasing the stirring speed is proposed, the digital twin model predicts the reward score associated with control action B under the corresponding simulated parameters and the reward score under the corresponding actual environmental parameters, and then calculates the corresponding reward score error d. The data record A composed of the data at time point a = {a, b, c, d, B, d}.

[0085] Then, sort the integrated data records in chronological order to construct a training data set. To improve the generalization ability, preprocessing operations such as normalization and standardization can also be performed on the data to make the data distribution more in line with the requirements of neural network training.

[0086] In the MBPO framework, the environmental model neural network applied needs to have the dual functions of simulating the environmental dynamics and guiding the policy optimization. Therefore, in this embodiment, a neural network architecture combining LSTM and fully connected layers is specifically selected to establish the environmental model neural network, and it is trained by combining the collected environmental parameters and simulated parameter combinations, with the reward guiding the learning direction of the neural network to accurately predict the best parameter combination of the digital twin model.

[0087] Generate the training data set of the environmental model neural network. After completing the training of the environmental model neural network, the digital twin model can be corrected through the trained environmental model neural network according to the currently collected time-series data set, so as to ensure that the digital twin model can be close to the actual production situation of the yogurt demulsification process.

[0088] Specifically, first input the time-series data set into the environmental model neural network, and the environmental model neural network outputs correction parameters;

[0089] Generate a correction strategy for the digital twin model based on the correction parameters.

[0090] After receiving the time-series data set, the environmental model neural network performs calculations through the internally trained parameters and complex network structure, and then determines the simulation parameters that can obtain higher rewards under the corresponding environmental parameters, and then outputs correction parameters according to the corresponding simulation parameters. This correction parameter mainly targets the parameters of the virtual environment for training the digital twin model. By adjusting the parameters of the virtual environment, the model parameters of the corresponding trained digital twin model can be corrected.

[0091] The correction parameters output by the environmental model neural network usually exist in the form of multi-dimensional vectors. Therefore, generating a correction strategy for the digital twin model based on the correction parameters includes:

[0092] Identify the type of correction parameter and perform associated matching with the model parameters of the digital twin model;

[0093] Determine the correction target parameters based on the associated matching results, and generate a correction strategy for the correction target parameters according to the correction parameter values output by the environmental model neural network.

[0094] Specifically, the type analysis of the correction parameters can be achieved through predefined parameter tags, and then the model parameters related to the correction parameter type can be determined by means of associated matching. The associated matching can be realized by calculating the association degree between the correction parameter type and the model parameters. By screening the way that the association degree exceeds the corresponding threshold, the model parameters strongly related to the corresponding parameter type are screened out. Taking the screened model parameters as the correction target parameters, and then through the corresponding correction parameter values, the optimal combination of each correction target parameter is determined by an optimization algorithm such as the optimal path search algorithm.

[0095] Adjust the digital twin model according to the optimal combination of each determined correction target parameter, and then re-simulate the demulsification process of yogurt according to the corrected digital twin model, so as to timely adjust the stirring control strategy of yogurt according to the preset viscosity value.

[0096] Specifically, based on the corrected digital twin model, generating a stirring control strategy in combination with the preset viscosity value includes:

[0097] Output a viscosity change curve based on the corrected digital twin model;

[0098] Obtain the viscosity adjustment requirement according to the preset viscosity value and the viscosity change curve;

[0099] Generate a stirring control strategy based on the viscosity adjustment requirement.

[0100] Under the training of the adjusted virtual environment, the corrected digital twin model has higher simulation accuracy and re-outputs a viscosity change curve based on the execution of the current stirring control strategy.

[0101] Then, compare and analyze the preset yogurt target viscosity value with the viscosity change curve output by the digital twin model, and use the difference calculation and trend prediction algorithm to evaluate whether the current simulated viscosity change meets the production requirements. Taking the example that the final viscosity shown by the viscosity change curve is lower than the preset value, the difference between the two can be further calculated, and the viscosity change trend can be analyzed to judge at which stage of the demulsification process the deviation occurs resulting in the unqualified final viscosity, and then the viscosity adjustment requirement for the corresponding stage is proposed.

[0102] According to the obtained viscosity adjustment requirement, combined with the influence law of stirring parameters on the yogurt viscosity, formulate a stirring control strategy. The stirring parameters include stirring speed, stirring time, stirring paddle angle, etc., and different parameters have different influence degrees and ways on the viscosity. Specifically, through the association model between the stirring parameters and the viscosity change trained based on historical production data and simulation experiment data, the stirring control strategy required to reach the preset viscosity value can be calculated.

[0103] On the other hand of this embodiment, a yogurt viscosity control device based on reinforcement learning is provided, including:

[0104] A data acquisition module, which is set at the stirring equipment and is used to collect the environmental parameters of stirred yogurt during the demulsification process in real time;

[0105] An edge processing unit, which is communicatively connected to the data acquisition device and is used to generate a stirring control strategy by combining a digital twin model and a reinforcement learning algorithm with the collected environmental parameters, and adjust the stirring parameters of the stirring equipment.

[0106] The stirring equipment used in the demulsification process of stirred yogurt mainly includes a stirring tank, a stirring paddle and a baffle plate arranged in the stirring tank. Among them, the stirring tank generally has a jacket for heating or cooling, and the stirring paddle needs to work in cooperation with a driving motor. The motor drives the stirring paddle to rotate, so that the sodium acid material in the stirring tank generates flow and mixing. Moreover, different degrees of stirring intensity adjustment can be further achieved by controlling parameters such as the rotation speed of the driving motor, the position of the baffle plate, and the temperature of the fermentation tank, so as to achieve the purpose of adjusting the viscosity of yogurt.

[0107] On the basis of the original stirring equipment, a data acquisition module is set to collect different environmental parameters related to the viscosity of yogurt during the demulsification process. Then, through the edge processing unit, the viscosity change analysis of the stirred yogurt in the fermentation tank is realized according to the corresponding environmental parameters, so as to output the stirring control strategy of the stirring equipment and achieve precise control of the viscosity of yogurt.

[0108] Among them, the data acquisition module at least includes:

[0109] A torque sensor, which is set between the stirring paddle and the driving motor of the stirring equipment and is used to collect the viscous resistance torque of the stirring paddle;

[0110] An angle sensor, which is set between the stirring paddle and the driving motor of the stirring equipment and is used to collect the rotation angle of the stirring paddle;

[0111] A temperature sensor, which is set in the fermentation tank of the stirring equipment and is used to collect the temperature data during the demulsification process.

[0112] The configuration of the data acquisition module on the fermentation tank is specifically as Figure 2 shown.

[0113] In order to realize the processing of the collected data, the data acquisition module further includes:

[0114] A data processing unit, which is respectively connected to the torque sensor and the angle sensor and is used to calculate the viscosity data and shear rate data of yogurt according to the collected viscous resistance torque and rotation angle of the stirring paddle.

[0115] In order to streamline the device, for the torque sensor and the angle sensor, a shaft-end torque sensor that can measure the rotation angle and torque simultaneously can be used to realize the simultaneous acquisition of the two parameters.

[0116] Among them, the data processing unit can be a single-chip microcomputer, which can process and calculate the data collected by the torque sensor and the angle sensor according to a preset program to obtain the desired shear rate and viscosity data.

[0117] Moreover, in order to obtain the shear rate and viscosity data at the initial stage of demulsification more accurately, the set torque sensor and angle sensor need to be set with a sampling frequency of not less than 100 Hz. Therefore, when establishing data interaction, it will not be transferred through the PLC, but will be connected to the single-chip microcomputer through the 485 bus and then converted to an Ethernet connection to the edge processing unit.

[0118] The temperature data generally does not have large fluctuations, and the temperature data can be collected at a frequency of 0.5 Hz - 1 Hz. Therefore, for the data interaction between the temperature sensor and the edge processing unit, the PLC Ethernet interface can be specifically used to connect to the edge processing unit.

[0119] The overall system topology of the yogurt viscosity control device is as Figure 3 shown.

[0120] In the edge processing unit, a reinforcement learning algorithm with digital twin technology and the MBPO framework is carried. The digital model of the demulsification process can be established according to the digital twin technology. The constructed digital twin model mainly takes the hydrodynamic sub-model of yogurt as the core and is completed in cooperation with other physical field sub-models. Before constructing the physical field sub-models, it is necessary to first establish the three-dimensional models of the fermentation tank and the stirring paddle and the digital model of the drive motor as the virtual training environment basis of the digital twin model, and then complete the coupling of all physical field sub-models on this basis. And in the digital twin model, an initial stirring control strategy for realizing yogurt viscosity control is carried, and this strategy can be set according to historical data or operation experience, and the parameters of this strategy can be adjusted.

[0121] Among them, for the digital model of the drive motor, it can be specifically constructed by calculating and simulating the digital change of the motor torque under the action of the motor through the CFD simulation algorithm. After the edge processing unit determines the stirring control strategy, it can specifically adjust the specific stirring parameters by frequency modulation of the drive motor. And the edge processing unit specifically issues the corresponding frequency modulation instruction to the drive motor through the PLC interface. Therefore, when constructing the corresponding virtual environment, it is also necessary to further construct the digital model of the corresponding frequency converter.

[0122] Specifically, the virtual training environment for yogurt viscosity control constructed by the edge processing unit is specifically as Figure 4 shown.

[0123] Considering that the traditional analog environment adjustment requires calling the CFD system for calculation with a large amount of calculation, the edge processing unit is equipped with the MBPO framework and synchronously constructs an environmental model neural network to adjust the virtual training environment according to the actual environment and the simulated environment, optimize the digital twin model, and then optimize the stirring control strategy.

[0124] Under the MBPO framework, the optimization and adjustment framework of the environmental model neural network and the digital twin model by the edge processing unit is specifically as Figure 5 shown.

[0125] The above-described embodiments are only a preferred solution of the present invention and do not impose any form of limitation on the present invention. There are other variations and modifications without exceeding the technical solutions recorded in the claims.

Claims

1. A method for controlling the viscosity of yogurt based on reinforcement learning, characterized in that, Including: Establish a digital twin model of the demulsification process; Collect environmental parameters during the demulsification process in real time, and construct a time series dataset by combining the simulation parameters output by the digital twin model; Based on the reinforcement learning algorithm, correct the digital twin model according to the time series dataset; Based on the corrected digital twin model, generate a stirring control strategy in combination with a preset viscosity value; Adjust the stirring parameters of the demulsification process according to the stirring control strategy.

2. The method for controlling the viscosity of yogurt based on reinforcement learning according to claim 1, wherein The method of correcting the digital twin model based on the reinforcement learning algorithm according to the time series dataset includes: Perform time alignment on the acquired environmental parameters and simulation parameters, and obtain corresponding difference indicators; Construct a training dataset based on the time-aligned environmental parameters, simulation parameters, and corresponding difference indicators; Establish an environmental model neural network, and train the environmental model neural network according to the training dataset; The environmental model neural network obtains a correction strategy according to the time series dataset, and adjusts the model parameters of the digital twin model according to the correction strategy.

3. The method for controlling the viscosity of yogurt based on reinforcement learning according to claim 2, wherein The method of the environmental model neural network obtaining a correction strategy according to the time series dataset includes: Input the time series dataset into the environmental model neural network, and the environmental model neural network outputs correction parameters; Generate a correction strategy for the digital twin model based on the correction parameters.

4. The method for controlling the viscosity of yogurt based on reinforcement learning according to claim 3, wherein, The method of generating a correction strategy for the digital twin model based on the correction parameters includes: Identify the type of correction parameters, and perform associated matching with the model parameters of the digital twin model; Determine the correction target parameters based on the associated matching results, and generate a correction strategy for the correction target parameters according to the correction parameter values output by the environmental model neural network.

5. The method for controlling the viscosity of yogurt based on reinforcement learning according to claim 1, wherein The method of generating a stirring control strategy in combination with a preset viscosity value based on the corrected digital twin model includes: Output a viscosity change curve based on the corrected digital twin model; Obtain the viscosity adjustment requirement according to the preset viscosity value and the viscosity change curve; Generate a stirring control strategy based on the viscosity adjustment requirement.

6. The method for controlling the viscosity of yogurt based on reinforcement learning according to claim 1, wherein The method of establishing a digital twin model of the demulsification process includes: Obtain the equipment parameters of the stirring equipment in the demulsification process and historical production data; Based on the equipment parameters and historical production data, create and initialize a multi-dimensional sub-model of the demulsification process; Couple each sub-model, and calibrate and optimize the model parameters of each sub-model through historical production data to obtain a digital twin model of the demulsification process.

7. The method for controlling the viscosity of yogurt based on reinforcement learning according to claim 1, characterized in that The environmental parameters at least include viscosity data, shear rate data, and temperature data of yogurt.

8. A yogurt viscosity control device based on reinforcement learning, which is used to execute the yogurt viscosity control method based on reinforcement learning according to any one of claims 1 to 7, and is characterized in that, Including: A data acquisition module, which is set at the stirring equipment and is used to collect environmental parameters during the demulsification process of stirred yogurt in real time; An edge processing unit, which is communicatively connected with the data acquisition device and is used to generate a stirring control strategy by combining the digital twin model and the reinforcement learning algorithm, and adjust the stirring parameters of the stirring equipment according to the collected environmental parameters.

9. The yogurt viscosity control device based on reinforcement learning according to claim 8, characterized in that, The data acquisition module at least includes: A torque sensor, which is set between the stirring paddle and the driving motor of the stirring equipment and is used to collect the viscous resistance torque of the stirring paddle; An angle sensor, which is set between the stirring paddle and the driving motor of the stirring equipment and is used to collect the rotation angle of the stirring paddle; A temperature sensor, which is set in the fermentation tank of the stirring equipment and is used to collect temperature data during the demulsification process.

10. The yogurt viscosity control device based on reinforcement learning according to claim 8, characterized in that, The data acquisition module also includes: A data processing unit, which is respectively connected to a torque sensor and an angle sensor, is configured to calculate viscosity data and shear rate data of yogurt according to the viscous resistance torque and rotation angle of the stirring paddle collected.

Images