Method for monitoring and modifying a trained intelligent algorithm for setting operating parameters of a container handling machine and device for carrying out the method
Patent Information
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- KRONES AG
- Filing Date
- 2024-06-05
- Publication Date
- 2026-06-10
AI Technical Summary
Container treatment machines face challenges in self-healing and adapting to changing framework conditions, such as varying ambient temperatures and material properties, which affect the accuracy of operating parameter settings, leading to inefficiencies and potential errors in producing containers with desired properties.
A procedure for monitoring and modifying intelligent algorithms that compares current and expected rewards by analyzing differences in container property measurements and simulations, allowing for automatic adjustments of operating parameters, training data expansion, or complete redesign of experiment runs to optimize performance and address errors.
Enables the intelligent algorithm to self-heal and adapt by automatically modifying operating parameters, improving the accuracy and efficiency of container treatment machines in producing consistent container properties, even under changing conditions, thereby enhancing the overall performance and reducing errors.
Smart Images
Figure EP2024065403_06022025_PF_FP_ABST
Abstract
Description
[0001] Method for monitoring and modifying a trained intelligent algorithm for setting operating parameters of a container treatment machine and device for carrying out the method
[0002] The invention relates to a method for monitoring and modifying an intelligent algorithm for setting operating parameters of a container treatment machine and to a device for carrying out the method according to the independent claims.
[0003] State of the art
[0004] US 2022 / 0147876 A1 discloses a method for providing an explainable agent for estimating an explainable reward function. The method comprises detecting an observed space comprising one or more states and one or more actions, and modeling the observed space as a plurality of explainable state-action pairs. The method further comprises forming one or more explainable models comprising a simulated environment and a reward function, and returning at least one explanation of at least one model of the one or more explainable models corresponding to at least one state and at least one reward.
[0005] To adjust the parameters of a container handling machine, monitoring can be provided on the machine, for example, using a programmable logic controller (PLC), or on an edge device. The work and performance can be monitored by an artificial intelligence (AI). The monitoring can transmit messages or information about problems and weaknesses of the container handling machine. A human operator can perform an analysis when anomalies occur, make decisions in the event of malfunctions, and initiate appropriate measures.
[0006] Task
[0007] The object of the invention is to provide a method for monitoring and modifying an intelligent algorithm for setting operating parameters of a container treatment machine and a device for carrying out the method, which enable self-healing of the intelligent algorithm when the framework conditions change.
[0008] Solution
[0009] The object is achieved by the method for monitoring and modifying an intelligent algorithm for setting operating parameters of a container treatment machine and the device for carrying out the method according to the independent claims. Further embodiments are disclosed in the subclaims.The method according to the invention for monitoring and modifying an intelligent algorithm for setting operating parameters of a container treatment machine comprises determining a current reward from a comparison of a target value of a container property to a measured actual value of the container property, determining an expected reward from a comparison of the target value to an expected, simulated actual value of the container property, wherein the expected, simulated actual value is based on the intelligent algorithm, automatically analyzing the current reward and the expected reward, and modifying at least one manipulated variable of the intelligent algorithm based on a result of the automatic analysis for setting operating parameters of a container treatment machine.
[0010] A container treatment machine may include a blow molding machine for producing PET containers from preforms, a filling system for filling product into containers, a closing device for closing containers filled with product, a pasteurizing device for pasteurizing containers filled with product, a labeling machine for applying labels to containers, and / or a cleaning device for cleaning containers, or the like.
[0011] The operating parameters may include a heating setting for heating the preforms and / or a blowing pressure for forming a container from a preform in the blow mold (for example in a blow molding machine), filling speed and / or filling temperature of the product (for example in a filling system), application pressure of a closure and / or application speed of a closure (for example in a closing device), a pasteurization temperature and / or a residence time below the pasteurization temperature (for example in a pasteurization device), an operating speed (for example in a labeling machine), a heating temperature of a caustic bath and / or a residence time of a container in a caustic bath (for example in a cleaning device).
[0012] Adjusting operating parameters may include adjusting a single operating parameter or adjusting multiple operating parameters. The multiple operating parameters may be independent of one another, at least partially dependent on one another, or interdependent.
[0013] In addition to the operating parameters of the container treatment machine, other conditions can influence a container being treated in the container treatment machine. For example, an ambient temperature (i.e., a boundary condition) in which the container treatment machine is operated can have a first temperature during a first operating cycle and a second temperature during a second operating cycle; for example, a difference between the first and second temperatures can be 30% or more, or even less. Humidity can also influence the results of the container treatment machine.For example, a wall thickness (i.e. a boundary condition) of preforms can vary per batch, so that this can have an influence on the wall thickness to be achieved for a container and, for example, for one of the batches, a desired wall thickness (setpoint) of the container may not be achievable using the container treatment machine under the prevailing operating parameters.
[0014] The containers may include bottles, for example made of PET, glass, fiber material (such as wood fibers or other biodegradable fibers) or the like, or cans.
[0015] A target value of a container property may be a wall thickness or a wall thickness distribution of the container, a number of pasteurization units of the product accommodated in a container, the position of a label on the container and / or the position of a closure on the container and / or the like.
[0016] The measured actual value of the container property can be a wall thickness determined by measurement or a wall thickness distribution of the container, a number of pasteurization units of the product contained in a container determined by measurement, the position of a label on the container determined by measurement and / or the position of a closure on the container determined by measurement and / or the like.
[0017] The expected, simulated actual value can be obtained by simulating a process of the container treatment machine, whereby the process can include the corresponding container treatments (for example, blow molding a container from a preform in a blow mold that is heated to a temperature and shaped with a blowing pressure) and whereby the manipulated variables available to the intelligent algorithm can be used.
[0018] By comparing the current reward and the expected reward, conclusions can be drawn as to whether the intelligent algorithm, with the control variables provided to it, can control the container treatment machine by adjusting operating parameters in such a way that the measured actual value corresponds to the target value to a specified extent.
[0019] Such conclusions can be obtained, for example, by automatically analyzing the current reward and the expected reward. Based on the result of the automatic analysis for setting the operating parameters of a container handling machine, at least one control variable of the intelligent algorithm can then be modified if necessary.
[0020] The method may further comprise measuring an actual value of the container property to obtain the measured actual value. The method may further comprise simulating an actual value to obtain the expected, simulated actual value. The simulation may comprise a simulation of a process of the container treatment machine, wherein the process may include the corresponding container treatments and wherein the manipulated variables available to the intelligent algorithm may be used.
[0021] The automatic analysis may include determining that the difference between the current reward and the expected reward is greater than a threshold, and the modification may include automatically expanding a dataset of a design-of-experiment (DOE) run by automatically initiating a training run with additional setpoints and automatically deploying the intelligent algorithm. The threshold may, for example, be 50.
[0022] For example, during operation of the container handling machine, it may be detected that the current reward is decreasing or settling at a low level. For example, the current reward may decrease during operation to a value below 100 out of a maximum of 250 or settling at a low level of below 100 out of a maximum of 250. This may mean that the intelligent algorithm cannot deliver a better result based on the at least one manipulated variable available to it. The expected, simulated actual value may be in line with the target. The difference between the current and expected reward may be large. It may be useful to enlarge the data set of the DOE with which the intelligent algorithm works. The training data set of the intelligent algorithm can be increased by additional set points. The new training run can be initiated, and the intelligent algorithm can thus be deployed.
[0023] The automatic analysis may include determining that the difference between the current reward and the expected reward is greater than a threshold value and / or that the difference between the target value and the expected, simulated actual value is greater than a further threshold value. The modification may include a completely new design-of-experiment run by initiating a new training run with set points from the new design-of-experiment run and automatically deploying the intelligent algorithm. The threshold value may, for example, be 50. The further threshold value may be a light transmittance difference. The magnitude of the further threshold value may, for example, be given by maximum deviations from the target value during a DOE run.
[0024] For example, during operation of the container handling machine, it may be detected that the current reward is decreasing or settling at a low level. For example, the current reward may decrease to a value below 100 out of a maximum of 250 during operation, or it may settling at a low level of below 100 out of a maximum of 250. The difference between the current and expected reward may be large. The expected reward may be good, which may mean that the difference between the setpoint and the expected, simulated actual value may be small. The expected reward may be good for values greater than 180 out of 250. The difference between the setpoint and the expected, simulated actual value may be a difference in light transmittance. The magnitude can be given, for example, by small deviations from the setpoint during a DOE run, for example, within 10% of the maximum deviation.It may be useful to perform a complete, new DOE run with all setpoints, for example, 126. Retraining the intelligent algorithm may be necessary based on the current situation at the container handling machine. Thus, a new training dataset across all setpoints can be obtained by initiating a new training run and deploying the intelligent algorithm.
[0025] The automatic analysis may include determining that the difference between the current reward and the expected reward is less than a threshold value and that at least one of the manipulated variables is at the limit of a value range. The modification may include automatically increasing the value range for the at least one manipulated variable and automatically adjusting it via an extrapolation of the at least one manipulated variable. The threshold value may be 50, for example.
[0026] For example, during operation of the container handling machine, it may be detected that the current reward is decreasing or settling at a low level. For example, the current reward may decrease to a value below 100 out of a maximum of 250 during operation, or settling at a low level of below 100 out of a maximum of 250. This may mean that the intelligent algorithm cannot deliver a better result based on the at least one manipulated variable available to it. The expected, simulated actual value may be realistic. It may be useful to allow a larger value range for the at least one manipulated variable. The settings can be determined by extrapolating the manipulated variables.
[0027] The current reward and the expected reward can be determined during operation of the container handling machine. This allows the temporal behavior of the current and expected reward to be analyzed. Likewise, the temporal progression of the measured actual value and the expected, simulated actual value can be analyzed.
[0028] The automatic analysis can include determining that both the current reward and the expected reward are greater or less than a limit value and that a difference between the setpoint and the expected, simulated actual value is less or greater than a further limit value. The modification can include an automatic shift of basic setpoints defined via basic manipulated variables and an automatic assignment of at least one new manipulated variable with an offset to the intelligent algorithm. The further limit value can be a light transmittance difference. The magnitude can be given, for example, by small deviations from the setpoint during a DOE run, for example within 10% of the maximum deviation or, for example, 60% of the maximum deviation.
[0029] This can occur when the intelligent algorithm reaches the limits of a manipulated variable's value range. The current reward may be poor, and the measured actual value may deviate significantly from the setpoint. The expected reward may also be poor. The current reward may be less than 100 out of a maximum of 250. The measured actual value may be a light transmittance difference. The magnitude can be determined, for example, by small deviations from the setpoint during a DOE run, such as 60% of the maximum deviation. The expected reward may be less than 100 out of a maximum of 250.
[0030] The base setpoints can be part of or include the setpoints of a DOE run.
[0031] The method may further comprise automatically calculating the offset. Alternatively, the offset may be defined manually, for example, by a human operator of the container handling machine.
[0032] Automatically calculating the offset may involve using limits of a range of base setpoint values as new base setpoints.
[0033] Furthermore, error messages from the container handling machine can be taken into account during automatic analysis.
[0034] The error messages can include faults such as container bursting on the blow molding machine or container bursting on the filling system or whitening of containers in the container handling machine.
[0035] Automatic analysis may include determining that the number of error messages occurring is increasing and that the current reward is greater than a threshold. Modification may include using new operating parameters to obtain new control variables and automatically passing the new control variables to the intelligent algorithm. The threshold may, for example, be 180 out of 250.
[0036] For example, unfavorable material distributions may lead to a good measured actual value, but errors may still occur, for example due to too much material in the bottom and too little material in the neck of the container or vice versa.
[0037] Automatic analysis may include determining that the number of error messages occurring is increasing and that the current reward is greater than a threshold. Modification may include using new operating parameters and a completely new design-of-experiment run by initiating a new training run with setpoints from the new design-of-experiment run and automatically deploying the intelligent algorithm. The threshold may, for example, be 180 out of 250.
[0038] Furthermore, a device is provided which comprises a computer-readable storage device having instructions stored thereon which, when executed by a processor of the device, cause the processor to carry out the method as described above or further below.
[0039] Short character description
[0040] The attached figure illustrates aspects and / or embodiments of the invention by way of example for better understanding and illustration. It shows:
[0041] Figure 1 is a block diagram describing the method for monitoring and modifying an intelligent algorithm for setting operating parameters of a container handling machine.
[0042] Detailed character description
[0043] Figure 1 shows a block diagram describing a method for monitoring and modifying an intelligent algorithm 1 for setting 3 operating parameters 4 of a container treatment machine 5. The intelligent algorithm 1 can determine the setting 3 of one or more operating parameters 4 of the container treatment machine 5 using manipulated variables 2. A container 6 can be treated in the container treatment machine 5. A measured actual value 8 of a container property can be determined for this container 6 using a measurement 7. A target value 9 can be known for this container property.
[0044] By entering 11 the measured actual value 8 and entering 12 the setpoint value 9, a current reward 13 can be determined from a comparison of the setpoint value 9 to the measured actual value 8.
[0045] An expected, simulated actual value 10 can result from a simulation of a process of the container treatment machine 5, wherein the process can include the corresponding container treatments and wherein the manipulated variables 2 can be used that are available to the intelligent algorithm 1.
[0046] By entering 14 the target value 9 and by entering 15 the expected, simulated actual value 10, an expected reward 16 can be determined from a comparison of the target value 9 to the expected, simulated actual value 10.
[0047] An input 17 of the current reward 13 and an input 18 of the expected reward 16 can be entered into an automatic analysis 19. Furthermore, one or more error messages 20 can be entered into the automatic analysis 19. The current reward 13 and the expected reward 16, as well as any one or more error messages 20, can be analyzed in the automatic analysis 19.
[0048] Based on a result of the automatic analysis 19, a modification 21 of at least one of the manipulated variables 2 of the intelligent algorithm 1 for a setting 3 of operating parameters 4 of the container treatment machine 5 can take place.
[0049] For example, the container treatment machine 5 can be a blow molding machine in which preforms are formed into containers 6 by inflating them with a blowing pressure in a heated blow mold. The operating parameters 4 of the blow molding machine can be a heating setting for heating the preforms and the blowing pressure. Using these operating parameters 4, a container 5 can now be formed in the blow molding machine, whereby one container property to be measured can be a wall thickness. The measurement 7 of the wall thickness of the container 5 results in a measured actual value 8 of the wall thickness. For containers produced with the blow molding machine, a target value 9 of the wall thickness can be specified. In addition, an expected, simulated actual value 10 of the wall thickness can be determined.This value may correspond to a theoretical value of the wall thickness that a container should theoretically have if all operating parameters 4 of the blow molding machine and an assumed thickness of the preform as well as, for example, assumed environmental parameters of the blow molding machine are as they should be theoretically.
[0050] The current reward 13 can be determined by comparing the target value 9 of the wall thickness with the measured actual value 8 of the wall thickness. The expected reward 16 can be determined by comparing the target value 9 of the wall thickness with the expected, simulated actual value 10 of the wall thickness.
[0051] An automatic analysis 19 of the current reward 13 and the expected reward 14 can be performed in order to, based on a result of the automatic analysis 19, optionally carry out a modification 21 of at least one manipulated variable 2 of the intelligent algorithm 1 for a setting 3 of operating parameters 4, i.e., for a setting 3 of the heating temperature and / or the blowing pressure of the blow molding machine. A modification 21 of at least one manipulated variable 2 may be necessary if the assumed wall thickness of the preforms changes, for example, between different batches, and / or if the ambient parameters change significantly during operation of the blow molding machine.
[0052] The automatic analysis 19 can also incorporate one or more error messages 20 from the blow molding machine, for example, if the blow molding process is not carried out properly. Furthermore, an automatic analysis 19 can also initiate a modification 22 of the intelligent algorithm 1, whereby a new training and deployment replaces the intelligent algorithm 1 with an optimized algorithm.
Claims
Claims 1. A method for monitoring and modifying an intelligent algorithm (1) for setting (3) operating parameters (4) of a container treatment machine (5), the method comprising: - Determining a current reward (13) from a comparison of a target value (9) of a container property to a measured actual value (8) of the container property, - determining an expected reward (16) from a comparison of the target value (9) to an expected, simulated actual value (10) of the container property, wherein the expected, simulated actual value (10) is based on the intelligent algorithm (1), - automatic analysis (19) of the current reward (13) and the expected reward (14), - Modifying (21) at least one manipulated variable (2) of the intelligent algorithm (1) based on a result of the automatic analysis for a setting (3) of operating parameters (4) of a container treatment machine (5).
2. The method of claim 1, further comprising measuring (7) an actual value of the container property to obtain the measured actual value (8).
3. The method of claim 1 or 2, further comprising simulating an actual value to obtain the expected simulated actual value (10).
4. The method according to one of claims 1 to 3, wherein the automatic analysis (19) comprises determining that a difference between the current reward (13) and the expected reward (16) is smaller than a threshold value, and wherein the modification (21) comprises automatically expanding a data set of a design-of-experiment run by automatically initiating a training run with further set points and automatically deploying the intelligent algorithm (1).
5. The method according to one of claims 1 to 3, wherein the automatic analysis (19) comprises determining that a difference between the current reward (13) and the expected reward (16) is greater than a limit value and that a difference between the target value (9) and the expected, simulated actual value (10) is smaller than a further limit value, and wherein the modification (21) comprises a completely new design-of-experiment run by initiating a new training run with set points of the new design-of-experiment run and automatically deploying the intelligent algorithm (1).
6. The method according to one of claims 1 to 3, wherein the automatic analysis (19) comprises determining that a difference between the current reward (13) and the expected reward (16) is smaller than a limit value and that at least one of the manipulated variables (2) is at a limit a value range, and wherein the modifying (21) comprises an automatic enlargement of the value range for the at least one manipulated variable (2) and an automatic adjustment via an extrapolation of the at least one manipulated variable (2).
7. The method according to one of claims 1 to 3, wherein the automatic analysis (19) comprises determining that both the current reward (13) and the expected reward (16) are greater or less than a limit value and that a difference between the setpoint value (9) and the expected, simulated actual value (10) is less than or greater than a further limit value, and wherein the modification (21) comprises an automatic shift of basic set points defined via basic manipulated variables and an automatic assignment of at least one new manipulated variable (2) with offset to the intelligent algorithm (1).
8. The method of claim 7, further comprising automatically calculating the offset.
9. The method of claim 8, wherein automatically calculating the offset comprises using boundaries of a range of values of the base set points as new base set points.
10. The method according to one of claims 1 to 9, wherein further error messages (20) of the container treatment machine (5) are taken into account during the automatic analysis (19).
11. The method according to claim 10, wherein the automatic analysis (19) comprises determining that a number of occurring error messages (20) is increasing and that the current reward (13) is greater than a limit value, and wherein the modification (21) comprises using new operating parameters (4) and obtaining new manipulated variables (2) therefrom and automatically transferring the new manipulated variables (2) to the intelligent algorithm (1).
12. The method of claim 10, wherein the automatic analyzing (19) comprises determining that a number of occurring error messages (20) is increasing and that the current reward (13) is greater than a threshold, and wherein the modifying (21) comprises using new operating parameters (4) and a completely new design-of-experiment run by initiating a new training run with set points of the new design-of-experiment run and automatically deploying the intelligent algorithm (1).
13. An apparatus comprising a computer-readable storage device having instructions stored thereon which, when executed by a processor of the apparatus, cause the processor to carry out the method of any one of claims 1 to 12.