Methods and systems for repairing filling valves and pressure disturbances in food packaging systems.
By combining reinforcement learning and deep reinforcement learning models, and utilizing neural networks to optimize the control strategy of the filling valve, the shortcomings of filling valve control are solved, achieving more efficient and flexible filling control, and reducing packaging waste and setup time.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TETRA LAVAL HOLDINGS & FINANCE SA
- Filing Date
- 2021-12-17
- Publication Date
- 2026-05-26
AI Technical Summary
Existing technologies are inadequate in controlling filling valves and repairing pressure disturbances, resulting in overfilling or underfilling. Furthermore, PID controllers require manual adjustment, have poor adaptability, and are difficult to cope with complex changes in the factory environment.
By combining reinforcement learning and deep reinforcement learning models with a local control model, the control parameters of the filling valve are adjusted by receiving local and remote variable values. The control strategy is optimized using neural networks to achieve precise control of the filling valve.
It improves filling accuracy, reduces packaging waste, shortens setup time, and enhances the system's adaptability and operational efficiency.
Smart Images
Figure CN116568601B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to food packaging systems, and more particularly to repairing filling valves and pressure disturbances that may occur in food packaging systems. Background Technology
[0002] Today, automated control systems are widely used in manufacturing and processing environments, and their complexity is constantly increasing. A common approach to managing this complexity is to divide the system into subsystems and develop appropriate control mechanisms for each subsystem. However, this approach does not always provide the optimal solution for the entire system.
[0003] As systems become increasingly complex and the number of influencing factors grows, capturing these factors from different sources becomes increasingly difficult. This complexity increases further when the relationships between influencing factors, control variables, and the system itself are non-linear and / or difficult to model.
[0004] Regarding the levels of abstraction in industrial control, there are two main perspectives: low-level control and high-level control. Low-level control refers to the management of individual automated components (such as actuators, servo motors, heaters, and many other devices). High-level control can abstract from the subsystem level to the system level, and further to the orchestration of the entire plant with multiple systems and subsystems that need to operate in coordination.
[0005] For example, food processing and packaging equipment typically comprises several subsystems, such as filling systems, sterilization systems, and packaging folding systems. Each subsystem contains many different components (e.g., pneumatic actuators, servo motors, DC motors, AC motors, sensors, and other actuators). These individual components are typically controlled by a low-level local control system that utilizes conventional control techniques (e.g., proportional-integral-derivative (PID) controllers) to control the target variable. Feedback loops are used to keep the controller's error relative to the target operating point of the component, system, or subsystem low.
[0006] However, PID controllers require tuning for their specific applications and are typically optimized for a particular operating range and dynamics. They are also less suited to unforeseen situations or operating conditions outside the normal operating range. When these conditions change (e.g., different working environments, changes in automation components, changes in manufacturing processes, etc.), the parameters of the PID controller usually need to be adjusted and recalibrated. This can be a time-consuming and complex process, requiring a significant amount of manual input from experienced personnel, especially when a large number of components and / or subsystems are involved, as is often the case in food processing and packaging equipment.
[0007] A filling machine is an example of a complex system that packages liquid, semi-liquid, or pourable foods, such as juice, UHT (ultra-high temperature) milk, wine, and ketchup, into multi-layered composite packaging materials for distribution and sale. A typical example is the Tetra Brik Aseptic. TM The pourable food packaging is a parallelepiped shape made by sealing and folding laminated strip packaging material. The packaging material has a multi-layered structure, including cardboard and / or paper base layers, with heat-sealable plastic material (e.g., polyethylene) layers on both sides. In the case of aseptic packaging for long-term storage products, the packaging material also includes an oxygen barrier layer, such as aluminum foil, which is stacked on top of the heat-sealable plastic material layers and then covered by another heat-sealable plastic material layer, forming the inner surface of the package that ultimately comes into contact with the food.
[0008] The filling machine begins with a roll of multi-layer composite packaging material (wound from a reel). The roll is fed through the filling machine, where it forms a tube by creating a longitudinal seal. A specified amount of food is fed into the tube through the tube; then the lower end of the tube is fed into a folding device, where a transverse seal is created. The tube is folded along a folding line (also called a weakening line) and then cut, thus forming a composite package filled with liquid food.
[0009] The amount of food delivered to the tubes that are subsequently cut into individual packages is regulated by a filling valve. When the filling valve is controlled using conventional control techniques such as a PID controller, it can be susceptible to changes in events and operating conditions. Furthermore, the pressure of the food in the line leading to the filling valve (also referred to herein as “product pressure”) is an unmeasurable noise factor that can negatively impact the control performance of the filling valve. PID controllers used for filling valves may react slowly to pressure changes, potentially leading to filling problems such as overfilling. Additionally, the PID gain typically needs to be manually adjusted by a technician for each volume of the intended package and each type of food to be filled into the package.
[0010] Therefore, there is a need for improved techniques for controlling filling valves and repairing pressure disturbances, taking into account a range of events that occur within the packaging machine or sometimes even outside of it. Summary of the Invention
[0011] One object of the present invention is to overcome, at least in part, one or more limitations of the prior art. Specifically, one object is to provide methods and systems that improve the control of the filling valve of a food packaging machine in response to various events occurring inside or outside the filling machine by considering not only the measured parameter values of the local filling valve subsystem, but also the measured parameter values of other remote subsystems within or even outside the food packaging machine. Therefore, correct packaging filling (i.e., the exact amount, neither overfilling nor underfilling) can be achieved, making the setup process during initial installation of the food packaging machine faster and handling of unexpected events better, ultimately resulting in fewer packages needing to be discarded.
[0012] In one aspect of the invention, this is achieved by a method for filling packages with food in a food packaging machine, wherein the food packaging machine includes multiple subsystems, and the method includes:
[0013] • Receive one or more local variable values, which represent measurements of one or more physical parameters of the local filling subsystem of the food packaging machine;
[0014] • Receive one or more remote variable values, which represent measurements of one or more physical parameters of one or more physical parameters of one or more remote subsystems of the food packaging machine (100);
[0015] • By using a reinforcement learning model and a local control model to process the remote variable values and the local variable values, one or more control parameter values for the local filling subsystem of the food packaging machine are determined;
[0016] • Adjust one or more control parameters of the local filling subsystem according to the determined control parameter values; and
[0017] • The food packaging machine (100) is controlled to fill the packaging with food according to the adjusted one or more control parameters.
[0018] Utilizing local variables and inputs from remote subsystems allows for more precise control of the filling valve in the event of unexpected pressure changes in the line containing the food to be filled into the packaging, resulting in more flexible operation. This leads to less packaging (and food) waste, making the food packaging machine more efficient and environmentally friendly to operate. Given the improved control over the packaging process, time to market for new products and / or configurations may also be shortened due to the need for less manual testing. This further enhances the ability of control strategies to learn in a simulation environment, eliminating the need for manual configuration of the food packaging machine "from scratch."
[0019] In one implementation, the reinforcement learning model is a deep reinforcement learning model that includes a neural network. Deep reinforcement learning is particularly useful when developing control strategies for subsystems that must consider a large number of variables (the internal relationships between these variables and their effects on the subsystem may be unknown), and provides a more sophisticated method for determining one or more control parameter values for the local filling subsystem of a food packaging machine than methods that traditional reinforcement learning without neural networks might offer.
[0020] In one implementation, the method includes receiving one or more remote variable values representing measurements of one or more physical parameters from one or more systems outside the packaging machine. This enables the packaging machine to account for events that might occur without the packaging machine itself when determining and adjusting control parameters to fill the package.
[0021] In one implementation, a local filling subsystem is connected via a line to a central storage tank containing food and is configured to dispense a specific amount of food into each package, with a remote variable value representing the pressure of the food in the line. This allows multiple packaging machines to be connected to the same central storage tank containing food and enables each machine's filling subsystem to respond to pressure changes that may occur in its own line due to events occurring in or related to other packaging machines.
[0022] In one embodiment, adjusting one or more control parameters of the filling subsystem includes adjusting one or more of the following: the time the filling valve is opened, the time the food passes through the filling valve when it is added to the package; and the degree to which the filling valve is opened when the food is dispensed into the package. That is, by more precisely controlling how long the filling valve is open and the degree to which it is open, based on information received from various subsystems and external systems, the amount of food filled in each package can be controlled.
[0023] In one implementation, the neural network is a convolutional neural network, a recurrent neural network, a long short-term memory neural network, or a fully connected neural network. These are different types of networks well known to those skilled in the art, and therefore easier to incorporate into existing food packaging machine setups.
[0024] In one embodiment, the one or more local variable values include: filling valve dynamics reflecting transients when the filling valve opens and closes, and a filling valve control signal reflecting the degree of opening of the filling valve; and the one or more remote variables include: product type, the number of lines connected to a central product storage tank, the operating status of the lines connected to the central product storage tank, and pressure changes of the food at the input of the filling subsystem. These are common parameters measured in most conventional packaging and production systems. Using these parameters to better control the local filling subsystem (as achieved through the data-driven approach of the various embodiments described herein) significantly enhances the operation of the filling subsystem and thus the overall operation of the packaging machine.
[0025] Other aspects of the invention include systems and computer programs for filling food packages in a food packaging machine. The features and advantages of these aspects of the invention are substantially the same as those discussed above with respect to the described method.
[0026] Other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description and the accompanying drawings. Attached Figure Description
[0027] Embodiments of the invention will now be described by way of example with reference to the accompanying schematic diagrams.
[0028] Figure 1 This is a schematic diagram of a part of a food packaging machine according to one implementation plan.
[0029] Figure 2 This is a schematic diagram of a controller in a food packaging machine according to one implementation scheme.
[0030] Figure 3 A schematic diagram of a filling subsystem according to one embodiment of the present invention is shown. Detailed Implementation
[0031] As described above, the objective of various embodiments of the present invention is to provide improved control technology for equipment and systems related to food processing and packaging (particularly concerning food filling packages). Filling packages with the correct quantity of food is important not only from a "customer needs" perspective but also from a functional perspective, as overfilling or underfilling can lead to prolonged machine downtime when correcting problems and wasteful packaging, which is undesirable from both a food waste and environmental perspective. By applying the general concept of reinforcement learning and / or deep reinforcement learning techniques to control the filling system of a food packaging machine, a wider range of factors can be considered, and the product can be adjusted very precisely compared to existing systems and food filling methods. This avoids overfilling or underfilling, and allows for more efficient use of the food packaging machine with less food packaging waste.
[0032] Reinforcement learning and deep reinforcement learning are both examples of machine learning techniques. Generally, reinforcement learning (RL) can be characterized as dynamic learning using positive or negative rewards. System performance is evaluated based on a desired objective. A positive reward is given if the objective is achieved or not, and a negative reward is given if the objective is not achieved. As positive and negative rewards accumulate over time, the RL model evolves a control policy for the system, aiming to maximize the outcome. Deep reinforcement learning (DRL) can be characterized as an enhancement of RL, where RL is used in conjunction with a neural network in evolving the system's control policy.
[0033] In the context of food processing and packaging, RL (i.e., agent-environment interaction) can be used to develop control strategies for food processing and / or packaging machines. DRL (i.e., RL and neural networks) is particularly useful when developing control strategies for subsystems (e.g., filling subsystems) that must consider a large number of variables whose internal relationships and effects on the system may be unknown. Furthermore, it should be noted that RL and DRL techniques can also be used to improve existing local control techniques, essentially “filling” the gaps in traditional control techniques by using this data-driven approach. Therefore, DRL algorithms can then directly (or indirectly through other control layers, e.g., by adjusting the gain of a traditional PID controller to allow the PID controller to operate more efficiently compared to traditional control techniques) control actuators (e.g., servo motors, pneumatic actuators, or other actuators), thereby mitigating filling valve and pressure disturbances that may occur in food packaging systems.
[0034] To further illustrate these principles, various embodiments of the invention will now be described more fully with reference to examples of controlling a filling subsystem in a food packaging machine, and with reference to the accompanying drawings, which show some, but not all, embodiments of the invention. The invention can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. For example, while various embodiments of the invention will be described with reference to roll-to-roll carton packaging machines, other embodiments of the invention can be applied to situations where discrete packaging has been formed and can have any shape or form or be made of any material (to name just a few examples, such as PET bottles, glass bottles, or cans made of metal). Although such containers may be formed using different processes than those described below, or by other machines, or even in other production facilities, the same general principles apply to filling food into these containers, and therefore the same control methods used for filling subsystems of machines designed for filling these different types of containers can also be applied to these settings.
[0035] As mentioned above, the filling subsystem is an important component of a food packaging machine and its operation requires careful control to ensure that the correct amount of food is filled into the packaging and that overfilling or underfilling does not occur.
[0036] Figure 1 A food packaging machine 100 is shown in general. In the example shown, the food packaging machine 100 is a roll-to-roll carton packaging machine. The general principle 20 of such a machine is to form a roll 102 from a roll of packaging material. The food packaging machine 100 may include a roll receiver (not shown) for receiving the roll of packaging material. Although not shown, if food safety regulations are required, the roll 102 may be sterilized using a hydrogen peroxide bath, a low-pressure electron beam (LVEB) device, or any other device capable of reducing the number of unwanted microorganisms.
[0037] After sterilization, the roll 102 can be formed into a tube 104 using a tube former. According to a non-limiting example, the tube former can be a longitudinally sealing device. Once the tube has been formed, food products such as milk can be supplied from a product filling device into the tube 104 through a product conduit 106 placed at least partially within the tube 104. In this context, food refers to anything ingested, eaten, and / or drunk by people or animals, or absorbed by plants, including but not limited to liquid, semi-liquid, viscous, dry, powdered, and solid foods, beverage products, and water. For the avoidance of doubt, food also includes ingredients used in the preparation of food. Some examples of food include milk, water, and fruit juice.
[0038] To form package 112 from tube 104 filled with product, a lateral seal can be achieved at the lower end of the tube using a sealing subsystem 110 (often also referred to as a "jaw system"). Typically, the sealing subsystem 110 has two main functions: providing a lateral seal by welding the two opposite sides of tube 104 together, separating the product placed below tube 104 from the product placed above tube 104, and cutting off the lower portion of tube 104 to form package 112. Alternatively, instead of providing the lateral seal and cutting the lower portion in the same apparatus as shown, the step of cutting the lower portion can be performed in subsequent steps using different equipment, or by the consumer if the packages are intended to be sold in a multi-pack.
[0039] In addition, the controller receives input from one or more remote subsystems of the food packaging machine 100, as well as from one or more remote systems outside the food packaging machine 100, all of which may experience events that similarly affect the operation of the filling subsystem. For example, a production plant may include several packaging machines 100, each of which can be connected via wiring to a central food storage tank 301, such as a tank containing liquid that will be filled into individual packages 112 by the packaging machine 100. If one of the packaging machines 100 encounters a problem, this may cause changes in the pressure in the wiring leading to the other packaging machines 100. The filling subsystems of these other packaging machines 100 need to react to this change to avoid overfilling or underfilling the corresponding package 112. Similarly, the density and / or viscosity of the product may affect the behavior of the filling valve of the packaging machine. For example, when filling a package 112 with a viscous or semi-solid liquid (e.g., beans or crushed tomatoes) compared to a smooth liquid (e.g., water or apple juice), the filling valve may need to open more or remain open for a longer period of time. In yet another example, the filling of package 112 may be affected by the ambient temperature in the production plant (e.g., some liquids flow more smoothly at higher temperatures) or by the physical configuration of the production plant (e.g., whether the can containing the food is located at a height lower or higher than the packaging machine, making gravity a factor to consider). As those skilled in the art will recognize, numerous local and remote factors can influence the filling of food into package 112 and these factors need to be considered to achieve improved control of the filling subsystem. Several food storage tanks 301 may also be connected to the packaging machine 100.
[0040] These events and external factors can be represented by a set of variables, the values of which represent various states at different subsystems of the food packaging machine 100, or the states of various systems outside the food packaging machine 100. This is in Figure 2The diagram is schematically shown, illustrating how input variables 116 from local sensors of the filling subsystem are fed into controller 114 along with input values 204 from other subsystems of the food packaging machine.
[0041] Figure 3 A schematic diagram of a filling subsystem 300 according to one embodiment of the present invention is shown. Figure 3 As shown, line 302 connects to food storage tank 301, through which food is transferred from the food storage tank to containers (which can be tubes formed of rolled material or discrete containers). As mentioned, filling valve 304 can be opened to different degrees and maintained for a specific period of time to dispense a certain amount of food into the containers via line 306. Filling valve 304 is controlled by controller 114. Figure 3 In the embodiment of the filling subsystem 300 shown, there are two sensors 116a and 116b. The first sensor 116a is positioned in line 302 before valve 304 and measures the pressure in line 302. The second sensor 116b measures the amount of food being delivered to food packaging 112, for example, by measuring the liquid level of the food. It should be noted that these are merely two examples of process variables that can be measured by sensors 116a and 116b, and in other embodiments, other parameters may be measured based on the specific configuration at hand. Generally, the values measured by sensors 116a and 116b will be referred to hereinafter as local filling subsystem input variable 116.
[0042] In one implementation, some examples of variables representing physical parameters from the local filling subsystem include:
[0043] • Filler valve dynamics, which reflects the transients during the opening and closing of the filler valve (i.e., the brief pressure drop or rise that occurs when the valve is opened or closed, and which quickly stabilizes again), and
[0044] • The filling valve control signal reflects the degree to which the filling valve is open.
[0045] In one implementation, some examples of variables from other subsystems of the packaging machine or from external systems outside the packaging machine include:
[0046] • Food type (and / or product viscosity),
[0047] • The number of lines connected to the central food storage unit 301 (e.g., more lines generally result in more events and may affect how the filling valve is controlled, compared to having fewer lines and thus fewer events).
[0048] • The operational status of the lines connected to the central food storage 301 (e.g., the lines of each filling machine can operate in specific states, such as preparation, production, cleaning, stopping, or allowing food to pass through the lines at different speeds, depending on what the food packaging machine 100 is doing at any given time), and
[0049] • Pressure changes at the food input point of the filling subsystem.
[0050] It should be recognized that these are merely a few examples of possible influencing factors from other subsystems or external systems and should not be considered an exhaustive list. However, they do represent influencing factors that traditional filling valve control systems cannot consider, as it is difficult or impossible to determine how the various possible combinations of these factors should affect the operation of the filling valve subsystem.
[0051] According to the various embodiments described herein, controller 114 uses local control model 210 to process the local filling subsystem input variable 116, and combines it with reinforcement learning model 206 to process input values from other subsystems, as well as any input variables from external systems, to determine how all measured variables as a whole collectively affect the operation of the filling subsystem. Local control model 210 may be an algorithm executed by a PID controller. Reinforcement learning model 206 may be a deep reinforcement learning model comprising one or more neural networks, as described above. In some embodiments, local subsystem input variable 116 may be processed by reinforcement learning model 206. In some embodiments, reinforcement learning model 206 may be used to compute how different combinations of local and remote variables should affect the filling subsystem and use this insight to improve local control model 210. Based on the results of this processing and determination, controller 114 generates a set of output control signals 208 for the local filling subsystem, which controls the filling valve to fill the correct quantity of food into package 112. Typically, for filling valves, the controlled parameters include a parameter specifying how far the filling valve should open (e.g., 0% represents fully closed, and 100% represents fully open) and a parameter specifying how long the filling valve should remain open at the desired level. However, this will of course vary depending on the specific type of filling valve used and is a design choice by the systems engineer.
[0052] Examples of neural networks that can be used in implementations employing deep reinforcement learning models include, for example, convolutional neural networks (CNNs) already trained using reinforcement learning and deep reinforcement learning, recurrent neural networks (RNNs), such as long short-term memory (LSTM) neural networks frequently used in the field of deep learning, or fully connected neural networks. LSTM networks can be particularly useful because, unlike standard feedforward neural networks, LSTMs have feedback connections. This allows LSTMs to process not only single data points but also entire sequences of data, which is particularly useful in the design of food packaging machines used to generate large quantities of packages 112.
[0053] Traditional control techniques typically require manual calibration for each different operating setting (e.g., packaging size, food type, etc.), which is usually a very time-consuming process. In contrast, this embodiment of the invention provides a training environment where different parameter variations can be simulated, allowing the controller 114 to learn the optimal control strategy given the objective of the filling subsystem. This can save significant time in setting up the packaging machine, thereby shortening the time to market for new packaging and products. In some embodiments, the output of the reinforcement learning model can be used to adjust the gain of a conventional PID controller, enabling the PID controller to operate more efficiently than traditional control techniques that rely solely on local variable values.
[0054] It should be noted that although the subsystem is referred to above as filling system, sterilization system, packaging folding system, etc., it can also refer to a part of the aforementioned subsystem or a separate component.
[0055] It should be noted that in some implementations, the control model of controller 140 may reside within controller 140 itself, such as... Figure 2 As shown. In other embodiments, they may reside in external hardware / software (e.g., an external computer or similar processing device) and be operated by that external hardware / software to further accelerate the required computation, and the controller 140 in the food packaging machine may be a simpler controller that only performs the functions determined by the external hardware / software.
[0056] The systems and methods disclosed herein can be implemented as software, firmware, hardware, or a combination thereof. In a hardware implementation, the task division among functional units or components referred to in the above description does not necessarily correspond to the division of physical units; on the contrary, a physical component can perform multiple functions, and a task may be completed collaboratively by multiple physical components.
[0057] Some or all of the components may be implemented as software executed by a digital signal processor or microprocessor, or as hardware or application-specific integrated circuits. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other storage technologies, optical or magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer.
[0058] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or part of an instruction, comprising one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a system based on dedicated hardware that performs a specific function or action, or by a combination of dedicated hardware and computer instructions.
[0059] As can be seen from the above description, although various embodiments of the present invention have been described and shown, the present invention is not limited thereto, but may be embodied in other ways within the scope of the subject matter defined by the appended claims.
Claims
1. A method for filling food into a package (112) in a food packaging machine (100), wherein, The packaging is formed from a tube, which is formed from a roll of packaging material. The food packaging machine (100) includes multiple subsystems, and the method includes: Receive one or more local variable values (116) representing measurements of one or more physical parameters of the local filling subsystem (300) of the food packaging machine (100); wherein the one or more local variable values include: filling valve dynamics reflecting transients when the filling valve (304) is opened and closed, and filling valve control signals reflecting the degree to which the filling valve (304) is opened; Receive one or more remote variable values (204) representing measurements of one or more physical parameters of one or more remote subsystems of the food packaging machine (100); One or more control parameter values for the local filling subsystem (300) of the food packaging machine (100) are determined by processing the remote variable values (204) and the local variable values (116) using a reinforcement learning model (206) and a local control model (210); Adjust one or more control parameters of the local filling subsystem (300) according to the determined control parameter values; wherein adjusting one or more control parameters of the local filling subsystem (300) includes adjusting one or more of the following: the time when the filling valve (304) is opened, the food passing through the filling valve (304) when added to the package (112); and the degree to which the filling valve (304) is opened when the food is dispensed into the package (112); According to the adjusted one or more control parameters, the food packaging machine (100) is controlled to fill food into the package (112); Receive one or more remote variable values representing measurements of one or more physical parameters of one or more systems outside the food packaging machine (100), wherein the local filling subsystem (300) is connected via line (302) to a central storage tank (301) containing the food and is configured to dispense a specific amount of the food into each package (112), and one of the remote variable values represents the pressure of the food in the line (302).
2. The method according to claim 1, wherein the reinforcement learning model (206) is a deep reinforcement learning model including a neural network.
3. The method according to claim 2, wherein, The neural network is one of the following: convolutional neural network, recurrent neural network, long short-term memory neural network, and fully connected neural network.
4. The method according to claim 1 or 2, wherein: The one or more remote variable values include: food type, number of lines connected to the central food storage facility (301), operating status of the line (302) connected to the central food storage facility (301), and pressure variation of the food at the input of the local filling subsystem (300).
5. A system (300) for filling food into a package (112) in a food packaging machine (100), wherein, The packaging is formed from a tube, which is formed from a roll of packaging material. The food packaging machine (100) includes multiple subsystems, the systems including: Memory; and processor, The memory contains instructions that, when executed by the processor, cause the processor to execute a method, the method comprising: Receive one or more local variable values (116) representing measurements of one or more physical parameters of the local filling subsystem (300) of the food packaging machine (100); wherein the one or more local variable values include: filling valve dynamics reflecting transients when the filling valve (304) is opened and closed, and filling valve control signals reflecting the degree to which the filling valve (304) is opened; Receive one or more remote variable values (204) representing measurements of one or more physical parameters of one or more remote subsystems of the food packaging machine (100); One or more control parameter values for the local filling subsystem (300) of the food packaging machine (100) are determined by processing the remote variable values (204) and the local variable values (116) using a reinforcement learning model (206) and a local control model (210); Adjust one or more control parameters of the local filling subsystem (300) according to the determined control parameter values; wherein adjusting one or more control parameters of the local filling subsystem (300) includes adjusting one or more of the following: the time when the filling valve (304) is opened, the food passing through the filling valve (304) when added to the package (112); and the degree to which the filling valve (304) is opened when the food is dispensed into the package (112); According to the adjusted one or more control parameters, the food packaging machine (100) is controlled to fill food into the package (112); Receive one or more remote variable values representing measurements of one or more physical parameters of one or more systems outside the food packaging machine (100), wherein the local filling subsystem (300) is connected via line (302) to a central storage tank (301) containing the food and is configured to dispense a specific amount of the food into each package (112), and one of the remote variable values represents the pressure of the food in the line (302).
6. The system according to claim 5, wherein the reinforcement learning model (206) is a deep reinforcement learning model including a neural network.
7. The system according to claim 6, wherein, The neural network is one of the following: convolutional neural network, recurrent neural network, long short-term memory neural network, and fully connected neural network.
8. The system according to claim 5 or 6, wherein: The one or more remote variable values include: food type, number of lines connected to the central food storage facility (301), operating status of the line (302) connected to the central food storage facility (301), and pressure variation of the food at the input of the local filling subsystem (300).
9. A computer program product comprising a computer-readable storage medium having instructions adapted to perform the method according to any one of claims 1-4 when executed by a processor.