Methods and systems for controlling the orientation of pipes in food packaging systems
By combining reinforcement learning and deep reinforcement learning models with local control models, the problem of tube twisting in food packaging machines was solved, resulting in more efficient production and less packaging waste, and simplifying the system adjustment process.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- TETRA LAVAL HOLDINGS & FINANCE SA
- Filing Date
- 2021-12-17
- Publication Date
- 2026-05-26
Smart Images
Figure CN116568598B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to food packaging systems, and more particularly to the orientation of pipes within food packaging systems. Background Technology
[0002] Today, automated control systems are widely used in manufacturing and processing environments, and their complexity is constantly increasing. A common approach to managing this complexity is to divide the system into subsystems and develop appropriate control mechanisms for each subsystem. However, this approach does not always provide the optimal solution for the entire system.
[0003] As systems become increasingly complex and the number of influencing factors grows, capturing these factors from different sources becomes increasingly difficult. This complexity increases further when the relationships between influencing factors, control variables, and the system itself are non-linear and / or difficult to model.
[0004] Regarding the levels of abstraction in industrial control, there are two main perspectives: low-level control and high-level control. Low-level control refers to the management of individual automated components (such as actuators, servo motors, heaters, and many other devices). High-level control can abstract from the subsystem level to the system level, and further to the orchestration of the entire plant with multiple systems and subsystems that need to operate in coordination.
[0005] For example, food processing and packaging equipment typically comprises several subsystems, such as filling systems, sterilization systems, and packaging folding systems. Each subsystem contains many different components (e.g., pneumatic actuators, servo motors, DC motors, AC motors, sensors, and other actuators). These individual components are typically controlled by a low-level local control system that utilizes conventional control techniques (e.g., proportional-integral-derivative (PID) controllers) to control the target variable. Feedback loops are used to keep the controller's error relative to the target operating point of the component, system, or subsystem low.
[0006] However, PID controllers require tuning for their specific applications and are typically optimized for a particular operating range and dynamics. They are also less suited to unforeseen situations or operating conditions outside the normal operating range. When these conditions change (e.g., different working environments, changes in automation components, changes in manufacturing processes, etc.), the parameters of the PID controller usually need to be adjusted and recalibrated. This can be a time-consuming and complex process, requiring a significant amount of manual input from experienced personnel, especially when a large number of components and / or subsystems are involved, as is often the case in food processing and packaging equipment.
[0007] A filling machine is an example of a complex system that packages liquid, semi-liquid, or pourable foods, such as juice, UHT (ultra-high temperature) milk, wine, and ketchup, into multi-layered composite packaging materials for distribution and sale. A typical example is the Tetra Brik Aseptic. TM The pourable food packaging is a parallelepiped shape made by sealing and folding laminated strip packaging material. The packaging material has a multi-layered structure, including cardboard and / or paper base layers, with heat-sealable plastic material (e.g., polyethylene) layers on both sides. In the case of aseptic packaging for long-term storage products, the packaging material also includes an oxygen barrier layer, such as aluminum foil, which is stacked on top of the heat-sealable plastic material layers and then covered by another heat-sealable plastic material layer, forming the inner surface of the package that ultimately comes into contact with the food.
[0008] The filling machine begins with a roll of multi-layer composite packaging material (wound from a reel). The roll is fed through the filling machine, where it is formed into a tube by creating a longitudinal seal. A specified amount of food is fed into the tube through the tube; then the lower end of the tube is fed into a folding device (often also called a "gripper system"), where a transverse seal is created. The tube is folded along a fold line (also called a weakening line) and then cut, thus forming a composite package filled with liquid food.
[0009] When tubes are formed, a problem known as "tube twisting" can occur. This is an unstable behavior pattern of packaging material tubes, where the tube formed from the packaging roll rotates clockwise or counterclockwise around its central axis due to lateral displacement of the roll along one or more rollers in a food packaging machine. Some common causes of this displacement include poor roll alignment, splicing events, "long side" defects (elongated cuts) in the packaging material roll, or incorrect / uneven interaction between the grippers / wings and the tube. Tube twisting negatively impacts the throughput and packaging quality of the packaging system. For example, the tube may fail to align properly with the gripper system, which can lead to design problems (both aesthetically and in terms of packaging integrity) because misalignment can result in improper sealing of the packaging (e.g., areas on the packaging roll that should be sealed are no longer within the area where the equipment can achieve a seal) and being cut by the gripper system. This is a particularly significant issue when the packaged contents need to remain sterile.
[0010] Because many of these events can occur in other subsystems of the food packaging machine, the local PID controller of the pipe orientation subsystem cannot account for them and therefore cannot proactively consider these event factors. Consequently, the local PID controller may overcompensate for any problems it detects, leading to poor correction of pipe orientation issues. Furthermore, whenever the type of packaging produced by the food packaging machine changes, the PID gain typically requires manual adjustment by technicians, often resulting in costly downtime. Therefore, improved techniques are needed to control pipe orientation overall and specifically pipe torsion, taking into account the range of events occurring within the packaging machine. Summary of the Invention
[0011] One object of the present invention is to overcome, at least in part, one or more limitations of the prior art. In particular, one object is to provide methods and systems that enable improvements in tube orientation within the tube orientation subsystem of a food packaging machine by responding to various events occurring in the machine, taking into account not only measured parameters of the local tube orientation subsystem but also measured parameters of other remote subsystems within the food packaging machine, to avoid tube twisting and other tube orientation problems. Therefore, stability of the tube forming subsystem against adverse events can be achieved, requiring less packaging to be discarded, ultimately resulting in a beneficial impact on the environment and the amount of food wasted.
[0012] In one aspect of the invention, this is achieved by a method for managing the orientation of tubes in a food packaging machine, wherein the food packaging machine includes multiple subsystems. The method includes:
[0013] • Receive one or more variable values, which represent measured values of one or more physical parameters in one or more subsystems of the food packaging machine, the one or more physical parameters affecting the tube orientation;
[0014] • By processing the received variable values using a reinforcement learning model and a local control model, one or more control parameter values are determined for the one or more subsystems; and
[0015] • Adjust one or more control parameters of the one or more subsystems according to the determined control parameter values.
[0016] Utilizing local variables and inputs from remote subsystems allows for more precise control of tube orientation and more flexible operation, reducing the likelihood of tube twisting or other problems when unexpected malfunctions occur in the tube forming subsystem or other remote subsystems of the food packaging machine. As mentioned above, this results in less packaging (and food) waste, making the food packaging machine more efficient and environmentally friendly to operate. Given the improved control over tube orientation, time to market for new products and / or configurations may also be shortened due to the need for less manual testing. This further enhances the ability of control strategies to learn in a simulation environment, eliminating the need for manual configuration of the food packaging machine "from scratch."
[0017] In one implementation, the reinforcement learning model is a deep reinforcement learning model that includes a neural network. Deep reinforcement learning is particularly useful when developing control strategies for subsystems that must consider a large number of variables (the internal relationships between these variables and their effects on the subsystem may be unknown), and provides a more sophisticated method for determining one or more control parameter values for the local tube forming subsystem of a food packaging machine than methods that traditional reinforcement learning without neural networks might offer.
[0018] In one implementation, adjusting one or more control parameters includes adjusting the tilt of one or more rollers in the food packaging machine to move the roll of material laterally along the length of the roller. For example, imagine a horizontal roller on which the roll of material runs. By tilting the roller, i.e., moving the right or left edge of the roller up and down in the vertical direction, the roll of material entering the roller will move left or right (i.e., laterally toward either end of the roller) along the length of the roller as it aligns its direction of travel perpendicular to the roller axis. This lateral movement of the roll of material causes the tube to twist in a clockwise or counterclockwise direction as the tube is being formed, and can therefore be used as a means of mitigating tube twisting problems.
[0019] In one embodiment, adjusting the tilt of one or more rollers further causes the tube formed from the roll to twist around the central axis of the tube in a clockwise or counterclockwise direction, as described above.
[0020] In one implementation, the neural network is a convolutional neural network, a recurrent neural network, a long short-term memory neural network, or a fully connected neural network. These are all different types of neural networks well known to those skilled in the art, and therefore easier to incorporate into existing food packaging machine setups.
[0021] In one implementation, one or more variable values include measurements related to one or more of the following: packaging roll motion and control variables, roll tension variables, packaging material properties, and food type. These are common parameters measured in most conventional packaging and production systems. Using these parameters to better control the subsystems of a food packaging machine (as achieved through the data-driven approach of the various implementations described herein) significantly enhances the operation of the tube forming subsystem and thus the overall operation of the food packaging machine.
[0022] Other aspects of the invention include systems and computer programs for tube orientation in food packaging machines. The features and advantages of these aspects of the invention are substantially the same as those discussed above with respect to the described method.
[0023] Other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description and the accompanying drawings. Attached Figure Description
[0024] Embodiments of the invention will now be described by way of example with reference to the accompanying schematic diagrams.
[0025] Figure 1 This is a schematic diagram of a part of a food packaging machine according to one implementation plan.
[0026] Figure 2 This is a schematic diagram of a controller in a food packaging machine according to one implementation scheme.
[0027] Figure 3A This illustrates an implementation scheme. Figure 1 This is a schematic diagram of a roll of material moving laterally on a roller in a food packaging machine, with the roller in a horizontal position.
[0028] Figure 3B This illustrates an implementation scheme. Figure 1 This is a schematic diagram of a roll of material moving laterally on a roller in a food packaging machine, with the roller rotating slightly counterclockwise.
[0029] Figure 3C This illustrates an implementation scheme. Figure 1 This is a schematic diagram of a roll of material moving laterally on a roller in a food packaging machine, with the roller rotating slightly clockwise. Detailed Implementation
[0030] As described above, the objective of various embodiments of the present invention is to provide improved control techniques for equipment and systems related to food processing and packaging (particularly concerning tube orientation in food packaging machines). Also as mentioned above, tube twisting is a problem that can lead to packaging waste, which is undesirable from both a food waste and environmental perspective. Furthermore, currently, setting up the tube forming subsystem to minimize the risk of tube twisting when a food packaging machine is first put into operation involves a significant amount of time and labor. By applying the general concept of reinforcement learning and / or deep reinforcement learning techniques to control the tube forming subsystem of a food packaging machine, a wider range of factors can be considered compared to existing systems and possible factors in the tube. Furthermore, tube orientation can be precisely adjusted by modifying local parameters of the tube forming subsystem and / or parameters in other subsystems of the food packaging machine, making tube twisting less likely and allowing for more efficient use of the food packaging machine and reduced food packaging waste.
[0031] Reinforcement learning and deep reinforcement learning are both examples of machine learning techniques. Generally, reinforcement learning (RL) can be characterized as dynamic learning using positive or negative rewards. System performance is evaluated based on a desired objective. A positive reward is given if the objective is achieved or not, and a negative reward is given if the objective is not achieved. As positive and negative rewards accumulate over time, the RL model evolves a control policy for the system, aiming to maximize the outcome. Deep reinforcement learning (DRL) can be characterized as an enhancement of RL, where RL is used in conjunction with a neural network in evolving the system's control policy.
[0032] In the context of food processing and packaging, RL (i.e., agent-environment interaction) can be used to develop control strategies for food processing and / or packaging machines. DRL (i.e., RL and neural networks) is particularly useful when developing control strategies for subsystems (e.g., filling subsystems) that must consider a large number of variables whose internal relationships and effects on the subsystem may be unknown. Furthermore, it should be noted that RL and DRL techniques can also be used to improve existing local control techniques, essentially “filling” the gaps in traditional control techniques by using this data-driven approach. Thus, the DRL algorithm can then directly (or indirectly through other control layers, e.g., by adjusting the gain of a traditional PID controller to allow the PID controller to operate more efficiently compared to traditional control techniques) control actuators (e.g., servo motors, pneumatic actuators, or other actuators) to prevent pipe twisting and maintain pipe orientation stability.
[0033] To further illustrate these principles, various embodiments of the invention will now be described more fully with reference to examples of controlling the tube-forming subsystem in a food packaging machine, and with reference to the accompanying drawings, which show some, but not all, embodiments of the invention. The invention can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. As mentioned above, the tube-forming subsystem is an important component of the food packaging machine, and its operation requires careful control to ensure that tube twisting does not occur in the event of malfunctions in other subsystems of the food packaging machine.
[0034] Figure 1 A food packaging machine 100 is shown in general. In the example shown, the food packaging machine 100 is a roll-to-roll carton packaging machine. The general principle 20 of this food packaging machine is to form a roll 102 from a roll of packaging material. The food packaging machine 100 may include a roll receiver (not shown) for receiving the roll of packaging material. Although not shown, if food safety regulations need to be met, the roll 102 may be sterilized using a hydrogen peroxide bath, a low-pressure electron beam (LVEB) device, or any other device capable of reducing the number of unwanted microorganisms.
[0035] After sterilization, the roll 102 can be formed into a tube 104 in the tube forming subsystem using a tube forming device. According to a non-limiting example, the tube forming device may include a longitudinal sealing device that seals the long sides of the roll together to form a tube. Once the tube has been formed, food, such as milk, can be supplied from the food filling device into the tube 104 through a conduit 106 at least partially placed within it. In this context, food refers to anything ingested, eaten, and / or drunk by people or animals, or absorbed by plants, including but not limited to liquid, semi-liquid, viscous, dry, powdered, and solid foods, beverage products, and water. For the avoidance of doubt, food also includes ingredients used in the preparation of food. Some examples of food include milk, water, and fruit juice.
[0036] To form package 112 from tube 104 filled with product, a lateral seal can be achieved at the lower end of the tube using a sealing subsystem 110 (often also referred to as a "clamp system"). Typically, the sealing subsystem 110 has two main functions: providing a lateral seal by welding the two opposite sides of tube 104 together, separating the product placed below tube 104 from the product placed above tube 104, and cutting off the lower portion of tube 104 to form package 112. Alternatively, instead of providing the lateral seal and cutting the lower portion in the same apparatus as shown, the step of cutting the lower portion can be performed in subsequent steps using different equipment, or by the consumer if the packages are intended to be sold in a multi-pack.
[0037] The tube forming subsystem is controlled by controller 114, such as Figure 2 As schematically shown, it receives inputs from various subsystems of the food packaging machine 100, which may experience events that also affect the operation of the tube-forming subsystem. These events and external factors can be represented by a set of variables whose values represent various states at different subsystems of the food packaging machine 100. Figure 2 This shows how inputs from the local sensor 116 of the tube forming subsystem are fed into the controller 114 along with input values 204 from other remote subsystems of the food packaging machine.
[0038] In one implementation, some examples of variables that can affect the physical parameters of the tube forming subsystem include:
[0039] • Roll material motion and control variables (e.g., start-up, stop-up, acceleration, and deceleration of roll material 102),
[0040] • Roll tension variables (e.g., roll tension setpoint (i.e., the desired roll tension for a specific type of roll 102 in the food packaging machine 100), and / or the current roll tensioning system position (i.e., the current roll tension recorded by the roll tensioning subsystem of the food packaging machine 100),
[0041] • Packaging material characteristics (e.g., length and width of rolls, stiffness and thickness of packaging materials, etc.), and
[0042] Food type (e.g., density, volume, etc.).
[0043] Roll motion and roll tension variables can be considered "dynamic" variables, while packaging material properties and food type variables are static variables, meaning they relate to physical properties. It should be recognized that these are merely a few examples of possible influencing factors from other subsystems or external systems and should not be considered an exhaustive list. However, they do represent influencing factors that traditional tube forming systems cannot consider, as it is difficult or impossible to determine how the various possible combinations of these factors should affect the operation of the tube forming subsystem.
[0044] According to the various embodiments described herein, controller 114 uses local control model 210 to process local tube forming subsystem input variables 116 (e.g., signals from edge detectors or from position markers on the roll), combined with reinforcement learning model 206 to process input values from other subsystems of the filling machine, to determine how all measured variables as a whole collectively affect the operation of the tube forming subsystem. Local control model 210 may be an algorithm executed by a PID controller. Reinforcement learning model 206 may be a deep reinforcement learning model comprising one or more neural networks, as described above. In some embodiments, local subsystem input variables 116 may be processed by reinforcement learning model 206. In some embodiments, reinforcement learning model 206 may be used to compute how different combinations of local and remote variables should affect the tube forming subsystem and use this insight to improve local control model 210. Based on the results of this processing and determination, controller 114 generates a set of output control signals 208 for the local tube forming subsystem, which control actuators, such as servo motors, pneumatic actuators, or other actuators, to prevent tube twisting (e.g., by changing the inclination of one or more rollers, as described above).
[0045] According to one implementation plan, Figures 3A-3C This schematically illustrates how changing the tilt of the rollers affects the orientation of the tube. Figure 3A The intermediate position is schematically shown, in which the roll 102 runs from the bottom of the figure toward the top of the figure over the roller 300, wherein the tube 104 is formed by bonding the long sides of the roll 102 together. To control the orientation of the tube, the roller 300 can be tilted clockwise or counterclockwise by an actuator (not shown). Figure 3B The diagram schematically illustrates the state in which roller 300 has been rotated counterclockwise a few degrees. As a result, the roll 102 moves laterally across roller 300, toward the right side of the figure, attempting to align perpendicularly to roller 300. This lateral movement causes tube 104 to twist counterclockwise. Similarly, Figure 3C The diagram schematically illustrates the state of roller 300 rotating a few degrees clockwise. As a result, roll 102 moves laterally across roller 300, toward the left side of the figure, again attempting to align perpendicularly to the roller. This lateral movement causes tube 104 to twist clockwise. It can be understood that adjusting the tilt of roller 300 is a possible mechanism to prevent tube twisting in the food packaging machine 100.
[0046] In some implementations, controller 114 also generates a set of control signals 208 for other subsystems of the food packaging machine 100 to correct problems that occur in these subsystems and affect tube orientation, i.e., essentially resolving problems that "fundamentally cause" any tube orientation, such as tube twisting. Having methods for resolving tube orientation problems both locally (i.e., in the tube forming subsystem) and remotely (i.e., in other subsystems of the food packaging machine 100) can further improve the tube forming process.
[0047] Examples of neural networks that can be used in implementations employing deep reinforcement learning models include, for example, convolutional neural networks (CNNs) already trained using reinforcement learning and deep reinforcement learning, recurrent neural networks (RNNs), such as long short-term memory (LSTM) neural networks frequently used in the field of deep learning, or fully connected neural networks. LSTM networks can be particularly useful because, unlike standard feedforward neural networks, LSTMs have feedback connections. This allows LSTMs to process not only single data points but also entire sequences of data, which is particularly useful in the design of food packaging machines used to generate large quantities of packages 112.
[0048] Traditional control techniques typically require manual calibration for each different operating setting (e.g., packaging size, food type, etc.), which is usually a very time-consuming process. In contrast, this embodiment of the invention provides a training environment where different parameter variations can be simulated, allowing the controller 114 to learn the optimal control strategy given the objectives of the tube forming subsystem. This can save significant time in setting up the packaging machine, thereby shortening the time to market for new packaging and products. As mentioned above, in some embodiments, the output of the reinforcement learning model can be used to adjust the gain of a conventional PID controller, enabling the PID controller to operate more efficiently compared to traditional control techniques that rely solely on local variable values.
[0049] It should be noted that although the subsystem is referred to above as tube forming system, filling system, sterilization system, packaging folding system, etc., it can also refer to a part of the above subsystem or a separate element.
[0050] It should be noted that in some implementations, the control model of controller 140 may reside within controller 140 itself, such as... Figure 2 As shown. In other embodiments, they may reside in external hardware / software (e.g., an external computer or similar processing device) and be operated by that external hardware / software to further accelerate the required computation, and the controller 140 in the food packaging machine may be a simpler controller that only performs the functions determined by the external hardware / software.
[0051] The systems and methods disclosed herein can be implemented as software, firmware, hardware, or a combination thereof. In a hardware implementation, the task division among functional units or components referred to in the above description does not necessarily correspond to the division of physical units; on the contrary, a physical component can perform multiple functions, and a task may be completed collaboratively by multiple physical components.
[0052] Some or all of the components may be implemented as software executed by a digital signal processor or microprocessor, or as hardware or application-specific integrated circuits. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other storage technologies, optical or magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer.
[0053] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or part of an instruction, comprising one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a system based on dedicated hardware that performs a specific function or action, or by a combination of dedicated hardware and computer instructions.
[0054] As can be seen from the above description, although various embodiments of the present invention have been described and shown, the present invention is not limited thereto, but may be embodied in other ways within the scope of the subject matter defined by the appended claims.
Claims
1. A method for managing the orientation of pipes in a food packaging machine (100), wherein the food packaging machine (100) includes a plurality of subsystems, the method comprising: Receive variable values (116, 204), which represent the measured values of multiple physical parameters in one or more subsystems of the food packaging machine (100), the multiple physical parameters affecting the tube orientation, wherein the variable values include measured values related to packaging roll motion and control variables, roll tension variables, and packaging material properties, wherein the packaging roll motion and control variables include roll start, stop, acceleration, and deceleration, and the packaging material properties include roll length and width, packaging material stiffness and thickness; By processing the received variable values using a reinforcement learning model (206) and a local control model (210), one or more control parameter values are determined for one or more subsystems in the subsystem; and Adjust one or more control parameters of one or more subsystems in the subsystem according to the determined control parameter values; Adjusting one or more control parameters includes: Adjust the inclination of one or more rollers (300) in the food packaging machine (100) to move the roll laterally along the length of the rollers (300).
2. The method according to claim 1, wherein the reinforcement learning model (206) is a deep reinforcement learning model including a neural network.
3. The method of claim 1, wherein adjusting the inclination of the one or more rollers (300) further causes the tube formed by the roll to twist clockwise or counterclockwise about the central axis of the tube.
4. The method of claim 2, wherein, The neural network is one of the following: convolutional neural network, recurrent neural network, long short-term memory neural network, and fully connected neural network.
5. A system for managing the orientation of pipes in a food packaging machine (100) having multiple subsystems, the system comprising: Memory; as well as processor, The memory contains instructions that, when executed by the processor, cause the processor to perform a method comprising: Receive variable values (116, 204), which represent the measured values of multiple physical parameters in one or more subsystems of the food packaging machine (100), the multiple physical parameters affecting the tube orientation, wherein the variable values include measured values related to packaging roll motion and control variables, roll tension variables, and packaging material properties, wherein the packaging roll motion and control variables include roll start, stop, acceleration, and deceleration, and the packaging material properties include roll length and width, packaging material stiffness and thickness; By processing the received variable values using a reinforcement learning model (206) and a local control model (210), one or more control parameter values are determined for one or more subsystems in the subsystem; and Adjust one or more control parameters of one or more subsystems in the subsystem according to the determined control parameter values; Adjusting one or more control parameters includes: Adjust the inclination of one or more rollers (300) in the food packaging machine (100) to move the roll laterally along the length of the rollers (300).
6. The system according to claim 5, wherein the reinforcement learning model (206) is a deep reinforcement learning model including a neural network.
7. The system of claim 5, wherein adjusting the inclination of the one or more rollers (300) further causes the tube formed by the roll to twist clockwise or counterclockwise about the central axis of the tube.
8. The system according to claim 6, wherein, The neural network is one of the following: convolutional neural network, recurrent neural network, long short-term memory neural network, and fully connected neural network.
9. A computer program product comprising a computer-readable storage medium having instructions adapted to perform the method according to any one of claims 1-4 when executed by a processor.