Reinforcement learning based enhancement of package formation in food packaging systems

Abstract

Methods and apparatus, including computer program products, are described for forming individual packages in a food packaging machine (100) including a plurality of subsystems. One or more local variable values (142) are received representing measurements of one or more physical parameters of a local subsystem (120) of the food packaging machine (100). One or more remote variable values (204) are received representing measurements of one or more physical parameters of one or more remote subsystems of the food packaging machine (100). One or more control parameter values for the local subsystem (120) of the food packaging machine (100) are determined by processing the remote variable values (204) and the local variable values (142) using a reinforcement learning model (206) and a local control model (210). One or more control parameters of the local subsystem (120) are adjusted in accordance with the determined control parameter values. Forming of individual packages (122) by the food packaging machine (100) is controlled in accordance with the adjusted one or more control parameters.

Description

Technical Field

[0001] This invention relates to food packaging systems, and more particularly to controlling how individual packages are formed within a food packaging system. Background Technology

[0002] Today, automated control systems are widely used in manufacturing and processing environments, and their complexity is constantly increasing. A common approach to managing this complexity is to divide the system into subsystems and develop appropriate control mechanisms for each subsystem. However, this approach does not always provide the optimal solution for the entire system.

[0003] As systems become increasingly complex and the number of influencing factors grows, capturing these factors from different sources becomes increasingly difficult. This complexity increases further when the relationships between influencing factors, control variables, and the system itself are non-linear and / or difficult to model.

[0004] Regarding the levels of abstraction in industrial control, there are two main perspectives: low-level control and high-level control. Low-level control refers to the management of individual automated components (such as actuators, servo motors, heaters, and many other devices). High-level control can abstract from the subsystem level to the system level, and further to the orchestration of the entire plant with multiple systems and subsystems that need to operate in coordination.

[0005] For example, food processing and packaging equipment typically comprises several subsystems, such as filling systems, sterilization systems, and packaging folding systems. Each subsystem contains many different components (e.g., pneumatic actuators, servo motors, DC motors, AC motors, sensors, and other actuators). These individual components are typically controlled by a low-level local control system that utilizes conventional control techniques (e.g., proportional-integral-derivative (PID) controllers) to control the target variable. Feedback loops are used to keep the controller's error relative to the target operating point of the component, system, or subsystem low.

[0006] However, PID controllers require tuning for their specific applications and are typically optimized for a particular operating range and dynamics. They are also less suited to unforeseen situations or operating conditions outside the normal operating range. When these conditions change (e.g., different working environments, changes in automation components, changes in manufacturing processes, etc.), the parameters of the PID controller usually need to be adjusted and recalibrated. This can be a time-consuming and complex process, requiring a significant amount of manual input from experienced personnel, especially when a large number of components and / or subsystems are involved, as is often the case in food processing and packaging equipment.

[0007] A filling machine is an example of a complex system that packages liquid, semi-liquid, or pourable foods, such as juice, UHT (ultra-high temperature) milk, wine, and tomato sauce, into multi-layered composite packaging materials for distribution and sale. A typical example is the TetraBrik Aseptic. TM The pourable food packaging is a parallelepiped shape made by sealing and folding laminated strip packaging material. The packaging material has a multi-layered structure, including cardboard and / or paper base layers, with heat-sealable plastic material (e.g., polyethylene) layers on both sides. In the case of aseptic packaging for long-term storage products, the packaging material also includes an oxygen barrier layer, such as aluminum foil, which is stacked on top of the heat-sealable plastic material layers and then covered by another heat-sealable plastic material layer, forming the inner surface of the package that ultimately comes into contact with the food.

[0008] The filling machine begins with a roll of multi-layer composite packaging material (wound from a reel). The roll is fed through the filling machine, where it forms a tube by creating a longitudinal seal. Liquid food is fed into the tube through the tube; then the lower end of the tube is fed into a folding device, where a transverse seal is created. The tube is folded along a fold line (also called a weakening line) and then cut, thus forming a composite package filled with liquid food.

[0009] The machine module or subsystem responsible for packaging formation, lateral sealing, and cutting is called the "gripper system." It consists of pairs of grippers whose synchronized movement allows the tube of packaging material to be pulled down and the filled package to be completely closed. The gripper system is a crucial component of the filling machine because the coordinated movement of the two gripper pairs is responsible for the proper shaping of the package. Furthermore, the grippers must move up and down without interfering with each other and must remain closed for given time intervals so that the sealing system can perform its task. Simultaneously, the system should be designed and controlled to adjust its motion curves according to the volume and size of different packaging formats to increase the machine's flexibility.

[0010] If the movement of the gripper system is not precisely controlled, misalignment may occur between the design on the packaging material and the sealing and cutting processes within the gripper system. This can lead to unsightly appearances and issues with the folding and integrity of the packaging material. Furthermore, even if the gripper system itself can be well controlled as a subsystem, events can still occur during the packaging process (e.g., splicing events (i.e., when the end of a used packaging roll is joined to the front end of a new packaging roll at the beginning of the food packaging machine to form a continuous packaging roll, resulting in a section of roll with two layers instead of a single layer), acceleration, deceleration, stopping, changes in packaging format, changes in food type, etc.). These events can affect the robustness of low-level controls and cause misalignment between the design on the packaging material and the sealing and cutting processes of the gripper system. Therefore, enhanced control technology is needed that also considers events occurring outside the gripper system itself that may affect the formation of individual packages. Summary of the Invention

[0011] One object of the present invention is to overcome at least in part one or more limitations of the prior art. Specifically, one object is to provide methods and systems that enable control of a food packaging machine's local subsystem (e.g., a gripper system) by considering not only measured parameter values ​​of the local subsystem itself within the food packaging machine, but also measured parameter values ​​of other, remote, subsystems within the food packaging machine. As a result, improved individual package formation can be achieved.

[0012] In one aspect of the invention, this is achieved by a method for forming a single package in a food packaging machine, wherein the food packaging machine includes multiple subsystems, the method comprising:

[0013] • Receive one or more local variable values, which represent the food packaging machine's measurement of one or more local physical parameters of the local subsystem;

[0014] • Receive one or more remote variable values, which represent the food packaging machine's measurement of one or more physical parameters of a or a remote subsystem;

[0015] • By using a reinforcement learning model and a local control model to process the remote variable values ​​and the local variable values, one or more control parameter values ​​for the local subsystem of the food packaging machine are determined;

[0016] • Adjust one or more control parameters of the local subsystem according to the determined control parameter values; and

[0017] • Control the formation of individual packages through the food packaging machine according to the adjusted one or more control parameters.

[0018] Utilizing local variables and inputs from remote subsystems will result in more precise control of the packaging process and greater operational flexibility in the event of unforeseen events in the food packaging machine. This leads to less packaging (and food) waste, making the food packaging machine more efficient and environmentally friendly to operate. Given the improved control over the packaging process, time to market for new products and / or configurations may also be shortened due to the need for less manual testing. This further enhances the ability of control strategies to learn in a simulation environment, eliminating the need for manual configuration of the food packaging machine "from scratch."

[0019] In one implementation, the reinforcement learning model is a deep reinforcement learning model that includes neural networks. Deep reinforcement learning is particularly useful when developing control strategies for subsystems that must consider a large number of variables (the internal relationships between these variables and their effects on the subsystem may be unknown), and provides a more sophisticated method for determining one or more control parameter values ​​for the local subsystem of a food packaging machine than methods that traditional reinforcement learning without neural networks might offer.

[0020] In one embodiment, the local subsystem is a gripper system configured to form individual packages from a tube of packaging material filled with food. Gripper systems are common subsystems in many conventional food packaging machines. The ability to deploy various embodiments of the invention into existing food packaging machines and systems enhances the versatility of the invention.

[0021] In one implementation, adjusting one or more control parameters of the gripper system includes adjusting the timing of engagement between the sealing gripper and the packaging material tube to form a single package, and / or adjusting the position of engagement between the sealing gripper and the packaging material tube to form a single package. These are two critical operations, each extremely important, and require precise control to correctly form the single package. Therefore, as achieved by the data-driven approach of the various embodiments described herein, the control of these parameters is improved, significantly enhancing the operation of the gripper system and thus significantly improving the formation of the single package.

[0022] In one implementation, the neural network is a convolutional neural network, a recurrent neural network, a long short-term memory neural network, or a fully connected neural network. These are all different types of convolutional neural networks well known to those skilled in the art, and therefore easier to incorporate into existing food packaging machine setups.

[0023] In one implementation, one or more local variables include measurements related to synchronization marks printed on the packaging roll, gripper system motion curves, or the state of mechanical forming adjustment tools, and one or more remote variable values ​​include measurements related to packaging roll motion and control variables, packaging roll tension variables, packaging fill state variables, and packaging material. These are different categories of variables that are used in various combinations in different food packaging machines. By using neural networks, any single variable (or combination thereof) belonging to these categories can be accommodated, greatly increasing the flexibility of the system.

[0024] Other aspects of the invention include systems and computer programs for forming individual packages in a food packaging machine. The features and advantages of these aspects of the invention are substantially the same as those discussed above with respect to the described method.

[0025] Other objects, features, aspects and advantages of the present invention will become apparent from the following detailed description and the accompanying drawings. Attached Figure Description

[0026] Embodiments of the invention will now be described by way of example with reference to the accompanying schematic diagrams.

[0027] Figure 1 This is a schematic diagram of a part of a food packaging machine according to one implementation plan.

[0028] Figure 2 This is a schematic diagram of a controller in a food packaging machine according to one implementation scheme. Detailed Implementation

[0029] As described above, the objective of various embodiments of the present invention is to provide improved control techniques for equipment and systems related to food processing and packaging, particularly concerning the formation of individual packages by food packaging machines. Properly formed packages are important, not only from a design and aesthetic perspective but also from a functional perspective, because very small inaccuracies in the formation of individual packages can affect their functionality. For some packages, very high precision (typically at the sub-millimeter level) is required. By applying the general concept of reinforcement learning and / or deep reinforcement learning techniques to control the gripper system, misalignments (e.g., misalignments between the design on the packaging material and the sealing and cutting processes in the gripper system) can be corrected with very high level precision.

[0030] Reinforcement learning and deep reinforcement learning are both examples of machine learning techniques. Generally, reinforcement learning (RL) can be characterized as dynamic learning using positive or negative rewards. System performance is evaluated based on a desired objective. A positive reward is given if the objective is achieved or not, and a negative reward is given if the objective is not achieved. As positive and negative rewards accumulate over time, the RL model evolves a control policy for the system, aiming to maximize the outcome. Deep reinforcement learning (DRL) can be characterized as an enhancement of RL, where RL is used in conjunction with a neural network in evolving the system's control policy.

[0031] In the context of food processing and packaging, RL (i.e., agent-environment interaction) can be used to develop control strategies for food processing and / or packaging machines. DRL (i.e., RL and neural networks) is particularly useful when developing control strategies for subsystems (e.g., filling subsystems) that must consider a large number of variables whose internal relationships and effects on the subsystem may be unknown. Furthermore, it should be noted that RL and DRL techniques can also be used to improve existing local control techniques, essentially “filling” the gaps in traditional control techniques by using this data-driven approach. Therefore, DRL algorithms can then directly (or indirectly through other control layers, e.g., by adjusting the gain of a traditional PID controller to allow the PID controller to operate more efficiently compared to traditional control techniques) control actuators (e.g., servo motors, pneumatic actuators, or other actuators), thereby controlling how individual packages are formed in a food packaging system.

[0032] To further illustrate these principles, various embodiments of the invention will now be described more fully with reference to the accompanying drawings, by way of example of controlling a gripper system in a food packaging machine to perform alignment correction throughout the machine, wherein some, but not all, embodiments of the invention are shown. The invention may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein.

[0033] As mentioned above, the gripper system is a crucial subsystem of a food packaging machine, requiring precise control to conform to the packaging material design and correctly form individual packages. Misalignment can lead to issues with the folding and integrity of the packaging material.

[0034] Figure 1A schematic diagram of a food packaging machine 100 is shown, wherein a roll of packaging material 102, preferably including at least one sealable surface 104 thereon, is fed forward 106 via a roll feeder on guide rollers 108, 110 and formed into a tube 112. The longitudinally overlapping side edges 114, 116 of the roll 102 are sealed to close the tube along its longitudinal edges. The side edges may overlap opposite each other with the bottom edge, or overlap with the bottom edge in the same direction. Adhesive strips (not shown) may be provided along one or both of the longitudinal edges 114, 116 to aid in tube formation.

[0035] Food is supplied from the food filling equipment into the formed tube via a food tube 118, which is at least partially placed within the formed tube. Herein, food refers to anything ingested, eaten, and / or drunk by people or animals, or absorbed by plants, including but not limited to liquid, semi-liquid, viscous, dry, powdered, and solid foods, beverage products, and water. For the avoidance of doubt, food also includes ingredients used in the preparation of food. Some examples of food include milk, water, and fruit juice. The filled tube is then conveyed to a gripper system 120, where the transverse seals of the package 122 are preferably formed at equally spaced locations along the length of the tube, although unequal lengths may be formed if desired. Sealing can occur by heating or other known methods. After sealing, the tube is cut along its length and within the boundaries of the transversely sealed area to form individual packages filled with product. Typically, when producing packages of the same size, each package is filled with a consistent volume of product. Especially in food packaging machines, volume consistency is provided by ensuring that individual packages have the same volume upon sealing. Therefore, the individual transverse seals are preferably formed at equally spaced locations along the length of the roll.

[0036] exist Figure 1 In a preferred embodiment of the food packaging machine shown, the gripper system 120 includes first and second sealing gripper assemblies 124 and 126, respectively disposed on opposite sides of the tube. These subassemblies 124, 126 include at least one bracket 128, 130 and preferably include multiple brackets. The brackets 128, 130 are preferably mounted along a closed-loop path on their respective tracks 132, 134. Alternatively, the brackets may be mounted on an open-loop path. Preferably, instead of changing the speed of the roll 102, the positioning of the brackets 128, 130 and their associated scaling grippers 136, 138 is controlled by a controller 140 or other control mechanism to ensure that each pair of sealing grippers 136, 138 is aligned with the appropriate portion of the tube at a pre-selected position. This is used to ensure proper packaging 122 dimensions.

[0037] The controller 140 receives input from the alignment sensor 142, such as an optical sensor capable of optically detecting synchronization marks 144 spatially spaced on the packaging roll. The synchronization marks 144 are constructed to make them virtually impossible for the alignment sensor 142 to misread. For example, they may have high contrast with the background and / or have an easily identifiable shape. An example of a synchronization mark 144 is a UPC (Universal Product Code) barcode. In some embodiments, the alignment sensor 142 may be an infrared or fluorescent ink sensor or a proximity probe, or any other type of position sensing device, such as a sensor capable of detecting magnetic ink.

[0038] In addition, the controller also receives input from remote subsystems of the food packaging machine 100, which may experience events that could affect the operation of the local gripper subsystem. Some examples of such events may include shearing events; acceleration, deceleration, or stopping of the packaging roll; changes in package format; product changes, and so on.

[0039] These events can be represented by a set of remote variables, whose values ​​represent various states of different subsystems of the food packaging machine. This is in Figure 2 The diagram schematically illustrates how the input from the alignment sensor 142 of the local gripper subsystem, together with the input value 204 from the remote subsystem of the food packaging machine, is fed into the controller 140.

[0040] In one implementation, some examples of variables representing physical parameters from the local gripper system include:

[0041] • Synchronization markings printed on the packaging materials.

[0042] • Gripper system motion curve (i.e., stored motion data describing the motion of the gripper system over a period of time, for example, by recording the motion of the servo motor controlling the gripper system in a PLC (Programmable Logic Controller)).

[0043] • The physical location of the mechanical forming adjustment tool (this location may vary, for example, depending on the specific type of packaging produced by the food packaging machine).

[0044] In one implementation, some examples of variables representing physical parameters from a remote subsystem include:

[0045] • Roll material movement and control variables, which are represented, for example, splice detection or packaging dimensions.

[0046] • Roll tension variables, which represent, for example, the position and / or pressure on various rollers in a food packaging machine as the roll passes through it.

[0047] • Fill status, such as fill flow rate and product level.

[0048] • Packaging material characteristics, such as packaging material stiffness, presence of a seal, and packaging volume.

[0049] It should be recognized that these are just a few examples of possible influencing factors from remote subsystems and should not be considered an exhaustive list. However, they do represent influencing factors that conventional control systems used today cannot account for. Both local and remote variables affect pipe position in their own ways, and conventional control systems find it difficult or impossible to determine how the various possible combinations of these remote and local variables should affect the operation of the local gripper subsystem.

[0050] According to the various embodiments described herein, controller 140 uses local control model 210 to process local subsystem input variables 142 and combines it with reinforcement learning model 206 to process input values ​​from remote subsystems to determine how the measured variables collectively affect the operation of the local gripper subsystem. Local control model 210 may be an algorithm executed by a PID controller. The reinforcement learning model may be a deep reinforcement learning model comprising one or more neural networks, as described above. In some embodiments, local subsystem input variables 116 may be processed by reinforcement learning model 206. In some embodiments, reinforcement learning model 206 may be used to compute how different combinations of local and remote variables should affect the roll tension subsystem and use this insight to improve local control model 210. Based on the results of this processing and determination, controller 140 generates a set of output control signals 208 for local gripper system 120, which control the engagement of the sealing grippers of the two sub-components with the moving tube 112 and the timing of their movement to engage with the moving tube 112 to form a lateral seal.

[0051] Examples of neural networks that can be used in implementations employing deep reinforcement learning models include, for example, convolutional neural networks (CNNs) already trained using reinforcement learning and deep reinforcement learning, recurrent neural networks (RNNs), long short-term memory (LSTM) neural networks such as those frequently used in the field of deep learning, or fully connected neural networks. LSTM networks can be particularly useful because, unlike standard feedforward neural networks, LSTMs have feedback connections. This allows LSTMs to process not only single data points but also entire sequences of data, which is particularly useful in the design of food packaging machines used to produce large quantities of packages.

[0052] Therefore, if the speed of the moving tube 112 changes, for example due to variations in tension within the tube or due to inaccurate operation of one or more mechanical components of the filling machine, the data-driven approach allows the controller 140 to detect such changes in tube speed and adjust the position of the aligned sealing jaws to ensure that the sealing jaws engage the tube at the appropriate time, thereby avoiding misalignment relative to the design of individual packages. Thus, compared to existing solutions that may not account for these variables, the food packaging machine can operate more efficiently, requiring less packaging to be discarded, resulting in financial and environmental advantages.

[0053] Furthermore, in some embodiments, the output from the reinforcement learning model can be used to adjust the gain of a conventional PID controller, allowing the PID controller to operate more efficiently compared to conventional control techniques where the PID controller relies on local variable values. Therefore, embodiments of the present invention are advantageous even when the only device used to control the gripper system is a PID controller. Moreover, particularly due to the flexibility of the system in positioning the sealing grippers based on variables collected from different subsystems of the food packaging machine, the system can be used to produce any of a variety of packaging sizes without any mechanical modifications to the system.

[0054] It should be noted that although the subsystem is referred to above as a gripper system, filling system, sterilization system, packaging folding system, etc., it can also refer to a part of the above subsystem or a separate component.

[0055] It should be noted that in some implementations, the control model of controller 140 may reside within controller 140 itself, such as... Figure 1 As shown. In other embodiments, they may reside in external hardware / software (e.g., an external computer or similar processing device) and be operated by that external hardware / software to further accelerate the required computation, and the controller 140 in the food packaging machine may be a simpler controller that only performs the functions determined by the external hardware / software.

[0056] The systems and methods disclosed herein can be implemented as software, firmware, hardware, or a combination thereof. In a hardware implementation, the task division among functional units or components referred to in the above description does not necessarily correspond to the division of physical units; on the contrary, a physical component can perform multiple functions, and a task may be completed collaboratively by multiple physical components.

[0057] Some or all of the components may be implemented as software executed by a digital signal processor or microprocessor, or as hardware or application-specific integrated circuits. Such software may be distributed on a computer-readable medium, which may include computer storage media (or non-transitory media) and communication media (or transient media). As is known to those skilled in the art, the term computer storage media includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules, or other data. Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other storage technologies, optical or magnetic storage devices, or any other medium that can be used to store desired information and is accessible to a computer.

[0058] The flowcharts and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or part of an instruction, comprising one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the figures. For example, two blocks shown consecutively may actually be executed substantially simultaneously, or these blocks may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented by a system based on dedicated hardware that performs a specific function or action, or by a combination of dedicated hardware and computer instructions.

[0059] As can be seen from the above description, although various embodiments of the present invention have been described and shown, the present invention is not limited thereto, but may be embodied in other ways within the scope of the subject matter defined by the appended claims.

Claims

1. A method for forming a single package in a food packaging machine (100), wherein the food packaging machine (100) includes a plurality of subsystems, the method comprising: Receive one or more local variable values ​​(142) representing measurements of one or more local physical parameters of a local subsystem of the food packaging machine (100), the local subsystem including a gripper system configured to form a single package (122) from a tube (112) filled with packaging material (102) of food, the food including liquid or semi-liquid food, the gripper system including a machine module or subsystem responsible for package formation, lateral sealing and cutting; Receive one or more remote variable values ​​(204) representing measurements of one or more physical parameters of one or more physical parameters of one or more remote subsystems of the food packaging machine (100); One or more control parameter values ​​for the local subsystem of the food packaging machine (100) are determined by processing the remote variable values ​​(204) and the local variable values ​​(142) using a reinforcement learning model (206) and a local control model (210); Adjust one or more control parameters of the gripper system according to the determined control parameter values; as well as Based on the adjusted one or more control parameters, the food packaging machine (100) is controlled to form the individual package (122); The one or more local variable values ​​include measurements related to one or more of the following: synchronization marks printed on the packaging roll, the motion curve of the gripper system, and the physical position of the mechanical forming adjustment tool; The one or more remote variable values ​​include measurements related to one or more of the following: packaging roll movement and control variables, packaging roll tension variables, packaging fill state variables, and packaging material property variables; The adjustment of one or more control parameters of the gripper system includes adjusting one or more of the following: the time when the sealing gripper engages with the tube (112) of the packaging material (102) to form a single package (122), and the position at which the sealing gripper engages with the tube (112) of the packaging material (102) to form a single package (122).

2. The method according to claim 1, wherein the reinforcement learning model (206) is a deep reinforcement learning model including a neural network.

3. The method according to any one of claims 2, wherein, The neural network is one of the following: convolutional neural network, recurrent neural network, long short-term memory neural network, and fully connected neural network.

4. A system for forming a single package in a food packaging machine (100) having multiple subsystems, said system comprising: Memory; and processor, The memory contains instructions that, when executed by the processor, cause the processor to perform a method, the method comprising: Receive one or more remote variable values ​​(204) representing measurements of one or more physical parameters of one or more physical parameters of one or more remote subsystems of the food packaging machine (100); Receive one or more local variable values ​​(142) representing measurements of one or more physical parameters of a local subsystem of the food packaging machine (100), the local subsystem including a gripper system configured to form a single package (122) from a tube (112) filled with packaging material (102) of food, the food including liquid or semi-liquid food, the gripper system including a machine module or subsystem responsible for package formation, lateral sealing and cutting; One or more control parameter values ​​for the local subsystem of the food packaging machine (100) are determined by processing the remote variable values ​​(204) and the local variable values ​​(142) using a reinforcement learning model (206) and a local control model (210); Adjust one or more control parameters of the gripper system according to the determined control parameter values; and Based on the adjusted one or more control parameters, the food packaging machine (100) is controlled to form the individual package (122); The one or more local variable values ​​include measurements related to one or more of the following: synchronization marks printed on the packaging roll, the motion curve of the gripper system, and the physical position of the mechanical forming adjustment tool; The one or more remote variable values ​​include measurements related to one or more of the following: packaging roll movement and control variables, packaging roll tension variables, packaging fill state variables, and packaging material property variables; The adjustment of one or more control parameters of the gripper system includes adjusting one or more of the following: the time when the sealing gripper engages with the tube (112) of the packaging material (102) to form a single package (122), and the position at which the sealing gripper engages with the tube (112) of the packaging material (102) to form a single package (122).

5. The system according to claim 4, wherein the reinforcement learning model (206) is a deep reinforcement learning model including a neural network.

6. The system according to any one of claims 5, wherein, The neural network is one of the following: convolutional neural network, recurrent neural network, long short-term memory neural network, and fully connected neural network.

7. A computer program product comprising a computer-readable storage medium having instructions adapted to perform the method according to any one of claims 1-3 when executed by a processor.

Images