Method and system for controlling tube orientation in a food packaging system

Abstract

Methods and apparatus, including computer program products, are described for managing tube orientation in a food packing machine, the food packing machine comprising a plurality of subsystems. One or more variable values ​​indicative of measurements by the food packing machine of one or more physical parameters of one or more subsystems, the one or more physical parameters affecting tube orientation, are received. One or more control parameter values ​​are determined for the one or more subsystems by processing the received variable values ​​using a reinforcement learning model and a local control model. One or more control parameters of the one or more subsystems are adjusted according to the determined control parameter values.

Description

[Technical Field]

[0001] FIELD OF THE INVENTION The present invention relates to food packaging systems, and more particularly to tube orientation in food packaging systems. [Background technology]

[0002] Automation control systems are currently used in a variety of manufacturing and processing sites and are continually becoming more complex. A common approach to managing this complexity is to divide the system into subsystems and develop appropriate control mechanisms for each subsystem. However, this approach does not always yield an optimal solution for the system as a whole.

[0003] As systems become more complex and the number of influencing factors increases, capturing these influencing factors from various sources becomes increasingly difficult. This complexity is further compounded when the relationships between influencing factors, control variables, and the system itself are nonlinear and difficult to model.

[0004] There are two main perspectives regarding abstraction levels in industrial control: low-level control and high-level control. Low-level control refers to the management of individual automation components (e.g., actuators, servo motors, heaters, and many other devices). High-level control can increase the level of abstraction from the subsystem level to the system level, and even to the orchestration of an entire plant with multiple systems and subsystems that need to work together.

[0005] As an example, food processing and packaging equipment typically includes several subsystems, such as a filling system, a sterilization system, a package folding system, etc. Each subsystem includes many different elements (e.g., pneumatic actuators, servo motors, DC motors, AC motors, sensors, other actuators, etc.). These individual elements are typically controlled by low-level local control systems that utilize traditional control techniques, such as proportional integral derivative (PID) controllers, to control target variables. Feedback loops are used to keep the controller error low relative to the target operating point of the element, system, or subsystem.

[0006] However, PID controllers must be tuned for their application and are typically optimized for a specific operating range and dynamics. They are also poorly suited to adapting to unexpected events or operating conditions outside of their traditional operating range. When these conditions change (e.g., a different operating environment, changes in automation elements, changes in the manufacturing process, etc.), the PID controller parameters must be adjusted and recalibrated. This can be a time-consuming and complex process, requiring significant manual input from experienced personnel, especially when numerous elements and / or subsystems are involved, as is typical with food processing and packaging equipment.

[0007] A filling machine is an example of a composite system that packages liquid, semi-liquid, or pourable foods, such as fruit juice, UHT (ultra-high temperature processed) milk, wine, and tomato sauce, into composite packages made of multilayer composite packaging materials for distribution and sale. A typical example is the parallelepiped pourable food package known as "Tetra Brik Aseptic®," which is made by sealing and folding laminated strips of packaging material. This packaging material has a multilayer structure, with a carton or paper substrate layer covered on both sides with a heat-sealable plastic material such as polyethylene. For long-term, aseptic packaging, the packaging material also includes a layer of oxygen barrier material, e.g., aluminum foil, which overlaps the heat-sealable plastic layer and is then covered with another layer of heat-sealable plastic to form the inner surface of the package that ultimately contacts the food.

[0008] The filler starts with a web of multi-layer composite packaging material (wound from a reel). The web is fed through the filler, which creates a longitudinal seal to form a tube. Liquid food is fed through a pipe into the tube, and the bottom end of the tube is fed into a folding device, which creates a transverse seal and folds the tube along fold lines, also known as lines of weakness, and cuts it to form a composite package filled with the liquid food.

[0009] During tube formation, a problem known as "tube twist" can occur. This is an unstable behavioral mode of the packaging material tube, where the lateral displacement of the web along one or more rollers in a food packaging machine causes the tube formed by the packaging material web to rotate clockwise or counterclockwise around its central axis. Common causes of this misalignment include web misalignment, splicing, "long edge" defects (slits) in the packaging material reel, and improper / uneven interaction between the jaws / volume flaps and the tube. Tube twist can negatively impact the production yield and package quality of a packaging system. For example, the tube may not be properly aligned with the jaw system. This misalignment can result in improperly sealed packages (e.g., the area on the packaging web that should be sealed is not within the range of the sealing device) and can be severed by the jaw system, potentially creating design issues (both aesthetic and package integrity). This is particularly important when the package contents must remain sterile.

[0010] Because many of these events can occur in other subsystems of the food packaging machine, there is no way for the local PID controller in the tube orientation subsystem to account for these events and, therefore, to proactively consider these factors. As a result, the local PID controller may overcompensate for detected problems and not properly correct the tube orientation problem. Furthermore, every time the type of package produced by the food packaging machine changes, a technician must manually adjust the PID gains, often resulting in costly downtime for the food packaging machine. Therefore, an improved technique is needed to control tube orientation in general, and tube twist in particular, that takes into account the various events that occur within the packaging machine. Summary of the Invention [Problem to be solved by the invention]

[0011] The present invention aims to at least partially overcome one or more limitations of the prior art. In particular, it is an object of the present invention to provide a method and system that can improve tube orientation in a tube orientation subsystem of a food packer and avoid tube kinking and other tube orientation problems in response to various events occurring within the food packer by taking into account parameter values ​​measured not only in the local tube orientation subsystem but also in other remote subsystems within the food packer. As a result, the tube forming subsystem is more resilient to adverse events, resulting in fewer discarded packages, ultimately having a beneficial effect on not only the amount of food waste but also the environment.

[0012] In one aspect of the present invention, this is accomplished by a method for managing tube orientation in a food packing machine, the food packing machine being comprised of multiple subsystems, the method including: · receiving one or more variable values ​​indicative of measurements by the food packaging machine of one or more physical parameters of one or more subsystems, the one or more physical parameters affecting tube orientation; · determining control parameter values ​​for one or more subsystems by processing received variable values ​​using a reinforcement learning model and a local control model; · Adjusting one or more control parameters of one or more subsystems according to the determined control parameter values.

[0013] By utilizing both local variables and inputs from remote subsystems, tube orientation is more precisely controlled, resulting in more resilient operation that is less susceptible to tube kinking or other issues due to unexpected adverse events in the tube forming subsystem or other remote subsystems of the food packaging machine. As noted above, this results in less wasted packaging (and food), making food packaging machine operations more efficient and environmentally friendly. Better control of tube orientation also reduces manual testing, potentially shortening time-to-market for new products and configurations. This benefit is further enhanced because control strategies can be learned in a simulation environment, eliminating the need to manually configure the food packaging machine from scratch.

[0014] In one embodiment, the reinforcement learning model is a deep reinforcement learning model that includes a neural network. Deep reinforcement learning is particularly useful when evolving control policies for subsystems that must consider a large number of variables whose internal relationships and effects on the subsystems may be unknown, and offers a more sophisticated approach to determining one or more control parameter values ​​for a local subsystem of a food packaging machine than is possible using traditional reinforcement learning that does not include a neural network.

[0015] In one embodiment, adjusting one or more control parameters includes adjusting the tilt of one or more rollers of the food packaging machine to move the web laterally along the length of the roller. For example, imagine a horizontal roller across which a web is moving. By tilting the roller, i.e., moving the right or left end of the roller vertically up or down, the web entering the roller will align its direction of travel perpendicular to the axis of the roller, causing the web to move laterally to the left or right along the length of the roller, i.e., toward either end of the roller. This lateral movement of the web causes the tube to twist in either a clockwise or counterclockwise direction as the tube is formed, and can be used as a means to mitigate tube twist problems.

[0016] In one embodiment, adjusting the tilt of one or more rollers further causes the tube formed by the web to twist clockwise or counterclockwise about the central axis of the tube, as described above.

[0017] In one embodiment, the neural network is a convolutional neural network, a recurrent neural network, a long short-term memory neural network, or a fully connected neural network, all of which are different types of conventional neural networks that are well known to those skilled in the art and therefore may be more easily integrated into existing food packing machine setups.

[0018] In one embodiment, the one or more variable values ​​include measurements of one or more of packaging web movement and control variables, web tension variables, packaging material properties, and food type, all of which are common parameters measured in conventional packaging and production systems. As achieved by the data-driven approach of various embodiments described herein, these parameters are used to better control the food packer's subsystems, significantly improving the operation of the tube forming subsystem and, in turn, the overall operation of the food packer.

[0019] Other aspects of the invention include systems and computer programs for tube orientation in food packing machines. The features and advantages of these aspects of the invention are substantially the same as those for the methods described above.

[0020] Further objects, features, aspects and advantages of the present invention will become apparent from the following detailed description and drawings. [Means for solving the problem]

[0021] Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings, in which: FIG. [Brief explanation of the drawings]

[0022] [Figure 1]1 is a schematic diagram showing a part of a food packaging machine according to an embodiment. [Figure 2] FIG. 2 is a schematic diagram of a controller in a food packaging machine according to one embodiment. [Figure 3A] 2 is a schematic diagram illustrating a web moving laterally across the rollers of the food packer of FIG. 1 when the rollers are in a horizontal position, according to one embodiment. [Figure 3B] 2 is a schematic diagram showing a web moving laterally across the rollers of the food packer of FIG. 1 when the rollers are rotated slightly counterclockwise, according to one embodiment. [Figure 3C] 2 is a schematic diagram showing a web moving laterally across the rollers of the food packer of FIG. 1 when the rollers are rotated slightly clockwise, according to one embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0023] As described above, a goal of various embodiments of the present invention is to improve control techniques for devices and systems related to food processing and packaging, particularly with regard to tube orientation in food packing machines. As described above, tube twist is a problem that can lead to package waste, which is undesirable from a food waste and environmental perspective. Additionally, currently, significant time and effort is required to set up the tube forming subsystem when the food packing machine is first operated to minimize the risk of tube twist. By applying the general concepts of reinforcement learning and / or deep reinforcement learning techniques to control the tube forming subsystem of a food packing machine, a wider range of factors can be considered compared to what is possible with existing systems, and by adjusting local parameters of the tube forming subsystem and / or parameters of other subsystems of the food packing machine, tube orientation can be adjusted with considerable precision, reducing the likelihood of tube twist, resulting in fewer wasted food packages, and allowing for more efficient use of the food packing machine.

[0024] Reinforcement learning and deep reinforcement learning are both examples of machine learning techniques. In general, reinforcement learning (RL) is characterized by dynamic learning using positive or negative rewards. A system's performance is evaluated against a desired goal. If the goal is reached, a positive reward is given, and if the goal is not reached, a negative reward is given. As positive and negative rewards accumulate over time, the RL model evolves the system's control policy with the goal of maximizing the outcome. Deep reinforcement learning (DRL) is an enhancement of RL, characterized by the combined use of RL and neural networks to evolve the system's control policy.

[0025] In the context of food processing and packaging, RL (i.e., agent-environment interaction) can be used to evolve control policies for food processing and / or packaging machines. DRL (i.e., RL combined with neural networks) is particularly effective when evolving control policies for subsystems, such as the filling subsystem, that must consider a large number of variables whose internal relationships and influences on the subsystem are unknown. Furthermore, it is noteworthy that RL and DRL techniques can also be used to improve existing local control techniques, allowing this data-driven approach to "fill in the gaps" of traditional control techniques. Thus, DRL algorithms can directly (or through other control layers, e.g., by adjusting gains so that traditional PID controllers can operate more efficiently compared to traditional control techniques) control actuators (e.g., servo motors, pneumatic actuators, or other actuators) that prevent tube twisting and keep tube orientation stable.

[0026] To further illustrate these principles, various embodiments of the present invention will now be described in more detail, using the control of a tube forming subsystem in a food packing machine as an example, with reference to the accompanying drawings, in which some, but not all, embodiments of the present invention are shown. The present invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. As noted above, the tube forming subsystem is a critical part of the food packing machine, and its operation must be carefully controlled to prevent tube kinking when adverse events occur in other subsystems of the food packing machine.

[0027] FIG. 1 generally illustrates a food packer 100. In the illustrated example, the food packer 100 is a roll-fed carton packer. The general principle of such food packers is that a web 102 is formed from a roll of packaging material. The food packer 100 may include a roll receiver (not shown) for receiving the roll of packaging material. Although not shown, if necessary to meet food safety regulations, the web 102 may be sterilized using a hydrogen peroxide bath, a low-voltage electron beam (LVEB) device, or any other device capable of reducing unwanted microorganisms.

[0028] After sterilization, the web 102 can be formed into a tube 104 in a tube forming subsystem using a tube former. By way of a non-limiting example, the tube former can be a longitudinal sealing device. Once the tube is formed, food, such as milk, can be delivered into the tube 104 from a product filling device through a product pipe 106 disposed at least partially within the tube 104. Food, in this context, refers to anything ingested, eaten, and / or drunk by humans or animals, or absorbed by plants, and includes, but is not limited to, liquid, semi-liquid, viscous, dry, powdered, and solid food and beverage products, and water. For the avoidance of doubt, food also includes ingredients for preparing food. Examples of food include milk, water, juice, etc.

[0029] To form a package 112 from the product-filled tube 104, a sealing subsystem 110, referred to as a "jaw system," can be used to form a transverse seal at the bottom end of the tube. Generally, the sealing subsystem 110 has two primary functions: to perform the transverse seal, i.e., to weld two opposing sides of the tube 104 together so that the product in the bottom of the tube 104 located below the sealing subsystem 110 is separated from the product in the tube 104 located above the sealing subsystem 110, and to trim the bottom of the tube 104 to form the package 112. Alternatively, instead of providing the transverse seal and the bottom trimming in the same apparatus as shown, the bottom trimming step may be performed in a subsequent step by a separate apparatus or by the consumer if the package is intended to be sold in a multipack.

[0030] The tube forming subsystem is controlled by a controller 114, shown schematically in Figure 2, which receives inputs from various subsystems of the food packer 100 that may experience events that affect the operation of the tube forming subsystem. These events and external factors may be represented by a set of variables, the values ​​of which indicate various conditions in the different subsystems of the food packer 100. Figure 2 shows how inputs from local sensors 116 of the tube forming subsystem are input to the controller 114, along with input variables 204 from other remote subsystems of the food packer.

[0031] In one embodiment, some examples of variables representing physical parameters that may affect the tube forming subsystem include: Web motion and control variables (e.g., starting, stopping, accelerating, and decelerating the web 102); web tension variables (e.g., a web tension set point (i.e., a desired web tension for a particular type of web 102 used in the food packaging machine 100) and / or a current web tension adjustment system position (i.e., a current web tension registered by a web tension adjustment subsystem of the food packaging machine 100); · The characteristics of the packaging material (e.g. length and width of the web, hardness and thickness of the packaging material, etc.) and type of food (density, volume, etc.).

[0032] The variables of web motion and web tension can be considered "dynamic" variables, while the variables of packaging material properties and food type are static variables, i.e., related to physical properties. As can be appreciated, these are only a few examples of factors that may be influenced by other subsystems or external systems and should not be considered an exhaustive list. However, these factors represent influencing factors that cannot be considered in conventional tube forming systems, and it is difficult or impossible to determine how various possible combinations of these factors should affect the operation of the tube forming subsystem.

[0033] According to various embodiments described herein, the controller 114 uses a local control model 210 to process local tube forming subsystem input variables 116 (e.g., signals from an edge detector or signals from position markings on the web) and, in combination with a reinforcement learning model 206, process input values ​​from other subsystems of the filling machine to determine how all measured variables collectively affect the operation of the tube forming subsystem. The local control model 210 is an algorithm implemented by a PID controller. The reinforcement learning model 206 may be a deep reinforcement learning model including one or more neural networks, as described above. In some embodiments, the local subsystem input variables 116 may be processed by the reinforcement learning model 206. In some embodiments, the reinforcement learning model 206 can figure out how different combinations of local and remote variables should affect the tube forming subsystem and use this insight to improve the local control model 210. Based on the results of this processing and decisions, the controller 114 generates a set of output control signals 208 for the local tube forming subsystem, which control actuators such as servo motors, pneumatic actuators, etc. to prevent kinking of the tube (e.g., by changing the tilt of one or more rollers, as described above).

[0034] 3A-3C schematically illustrate how varying the tilt of the rollers affects the orientation of the tube in one embodiment. FIG. 3A schematically illustrates a neutral position in which the web 102 travels across the roller 300 from bottom to top, forming the tube 104 by bonding the long edges of the web 102 together. To control the orientation of the tube, the roller 300 can be tilted clockwise or counterclockwise by an actuator (not shown). FIG. 3B schematically illustrates the roller 300 rotated counterclockwise several degrees. As a result, the web 102 moves laterally across the roller 300 toward the right side of the figure, attempting to align perpendicular to the roller 300. This lateral movement causes the tube 104 to twist counterclockwise. Similarly, FIG. 3C schematically illustrates the roller 300 rotated clockwise several degrees. As a result, the web 102 moves laterally across the roller 300 toward the left side of the figure, again attempting to align perpendicular to the roller. This lateral movement causes a clockwise twist in the tube 104. Adjusting the tilt of the rollers 300 is one possible mechanism for preventing tube twist in the food packer 100.

[0035] In some embodiments, the controller 114 generates sets of control signals 208 for other subsystems of the food packing machine 100 to correct problems that occur in those subsystems and that affect tube orientation, essentially addressing the "root cause" of any tube orientation problems, such as tube kinking. Having an approach in which tube orientation problems are addressed both locally (i.e., in the tube forming subsystem) and remotely (i.e., in other subsystems of the food packing machine 100) can further improve the tube forming process.

[0036] Examples of neural networks that can be used in embodiments using deep reinforcement learning models include, for example, convolutional neural networks (CNNs) trained using reinforcement learning and deep reinforcement learning, recurrent neural networks (RNNs) such as long short-term memory (LSTM) neural networks commonly used in the field of deep learning, or fully connected neural networks. LSTM networks are considered particularly useful because, unlike standard feedforward neural networks, LSTMs have feedback connections. This allows LSTMs to process entire sequences of data, not just single data points, which is particularly useful in the context of food packaging machines designed to produce large numbers of packages.

[0037] Traditional control techniques often require manual calibration for different operational settings, e.g., package size, food type, etc., which is a very time-consuming process. In contrast, this embodiment of the present invention allows for a training environment in which simulations can be run to see how different parameters vary, allowing the controller 114 to learn an optimal control policy given goals for the tube forming subsystem. This can save significant man-hours when setting up a packaging machine, thereby reducing the time to market for new packages and products. In some embodiments, outputs from a reinforcement learning model can be used to adjust the gains of a traditional PID controller, allowing the PID controller to operate more efficiently compared to traditional control techniques that rely only on local variable values.

[0038] It should be noted that while subsystems are referred to above as a tube forming system, a filling system, a sterilization system, a package folding system, etc., these may refer to parts of the above subsystems or individual elements.

[0039] Note that in some embodiments, the control models for the controller 140 may reside in the controller 140 itself, as shown in Figure 2. In other embodiments, they may reside in and operate from external hardware / software (e.g., an external computer or similar processing device) to further accelerate the necessary calculations, and the controller 140 within the food packing machine may be a simpler controller that simply performs functions determined by the external hardware / software.

[0040] The systems and methods disclosed herein can be implemented as software, firmware, hardware, or a combination thereof. In a hardware implementation, the division of tasks among functional units or components mentioned in the above description does not necessarily correspond to a division into physical units; conversely, one physical component may perform multiple functions, and one task may be performed jointly by multiple physical components.

[0041] Particular components, or all components, may be implemented as software executed by a digital signal processor or microprocessor, or as hardware or application-specific integrated circuits. Such software may be distributed on computer-readable media, including computer storage media (or non-transitory media) and communication media (or transitory media). As known to those skilled in the art, the term computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, optical or magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer.

[0042] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowcharts or block diagrams may represent a module, segment, or portion of instructions, which includes one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions shown in the blocks may be executed out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may be executed in the reverse order, depending on the functionality involved. Each block of the block diagrams and / or flowchart diagrams, and combinations of blocks in the block diagrams and / or flowchart diagrams, may be implemented by a special-purpose hardware-based system that performs the specified functions or operations or executes a combination of special-purpose hardware and computer instructions.

[0043] From the foregoing description, while various embodiments of the present invention have been described and illustrated, the present invention is not limited thereto and may be embodied in other ways within the scope of the subject matter defined in the following claims.

Claims

1. A method for managing tube orientation in a food packer (100), the food packer (100) comprising a plurality of subsystems, the method comprising: receiving variable values ​​(116, 204) indicative of measurements by the food packer (100) of one or more physical parameters for one or more subsystems, the one or more physical parameters affecting tube orientation, the variable values ​​including measurements related to one or more of packaging web movement and control variables, web tension variables, packaging material properties, and food type; determining one or more control parameter values ​​for the one or more subsystems by processing the received variable values ​​using a reinforcement learning model (206) and a local control model (210); adjusting one or more control parameters of the one or more subsystems in accordance with the determined control parameter values. How to prepare for this.

2. The reinforcement learning model (206) is a deep reinforcement learning model including a neural network. The method of claim 1.

3. adjusting the one or more control parameters adjusting the tilt of one or more rollers (300) of the food packer (100) to move the web (102) laterally along the length of said rollers (300); Including, 3. The method according to claim 1 or 2.

4. By adjusting the inclination of the one or more rollers (300), the tube (104) formed by the web (102) is twisted clockwise or counterclockwise around the central axis of the tube (104). The method of claim 3.

5. the neural network is one of a convolutional neural network, a recurrent neural network, a long short-term memory neural network, and a fully connected neural network; The method of claim 2.

6. The variable values ​​include measurements related to packaging web movement and control variables, web tension variables, packaging material properties, and food type. The method according to any one of claims 1 to 5.

7. 1. A system for managing tube orientation in a food packing machine (100) having multiple subsystems, the system comprising: Memory and a processor; Equipped with The memory includes instructions that, when executed by the processor, cause the processor to perform a method including: receiving variable values ​​(116, 204) indicative of measurements by the food packer (100) of one or more physical parameters for one or more subsystems, the one or more physical parameters affecting tube orientation, the variable values ​​including measurements related to one or more of packaging web movement and control variables, web tension variables, packaging material properties, and food type; determining one or more control parameter values ​​for the one or more subsystems by processing the received variable values ​​using a reinforcement learning model (206) and a local control model (210); adjusting one or more control parameters of the one or more subsystems in accordance with the determined control parameter values. A system that includes:

8. The reinforcement learning model (206) is a deep reinforcement learning model including a neural network. The system of claim 7.

9. adjusting the one or more control parameters adjusting the tilt of one or more rollers (300) of the food packer (100) to move the web (102) laterally along the length of said rollers (300); Including, 9. A system according to claim 7 or 8.

10. By adjusting the inclination of the one or more rollers (300), the tube (104) formed by the web (102) is twisted clockwise or counterclockwise around the central axis of the tube (104). The system of claim 9.

11. The neural network is one of a convolutional neural network, a recurrent neural network, a long short-term memory neural network, and a fully connected neural network. The system of claim 8.

12. The variable values ​​include measurements related to packaging web movement and control variables, web tension variables, packaging material properties, and food type. A system according to any one of claims 7 to 11.

13. A computer program product comprising a computer readable storage medium having instructions adapted to perform the method of any one of claims 1 to 6 when executed by a processor.

Images