Control parameter adjustment method, program, and recording medium

Abstract

To provide a control parameter adjustment method, a program, and a recording medium for efficiently adjusting control parameters of a transport apparatus by using a simulator model.SOLUTION: A method comprises: a step S32 for measuring a tension fluctuation value if a transport apparatus is operated with an existing PI parameter; a step S33 for predicting the tension fluctuation value for the existing PI parameter by using a plurality of simulator models; a step S35 for selecting a target simulator model from the plurality of simulator models based on the measured tension fluctuation value and the predicted tension fluctuation value; a step S36 for predicting an optimal PI parameter by using the target simulator model; a step S37 for measuring the tension fluctuation value if the transport apparatus is operated with the predicted optimal PI parameter; and an additional step S39 for training the target simulator model by using the measured tension fluctuation value as training data if the measured tension fluctuation value is outside the allowable range.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The subject matter disclosed in this specification relates to a control parameter adjustment method, a program, and a recording medium. [Background technology]

[0002] In the manufacturing industry, there are many cases where desired objectives are achieved by adjusting control parameters for devices based on experience or know-how. For example, in a printing device that records on a continuous flexible medium (hereinafter referred to as a "web"), it is necessary to adjust, for example, the PID parameters for motor control in order to transport the web with high precision. Such adjustments have traditionally been performed by engineers through trial and error, which requires a lot of man-hours. Therefore, reducing the number of man-hours required for parameter adjustment has become a challenge.

[0003] In recent years, there has been a movement to automate such parameter adjustment by incorporating machine learning. For example, in Patent Document 1 (JP 2019-098439 A), acceleration / deceleration parameters are adjusted using machine learning so that vibrations generated during acceleration / deceleration operations of a controlled robot are kept below a predetermined threshold. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Publication No. 2019-098439 Summary of the Invention [Problem to be solved by the invention]

[0005] When using machine learning for parameter adjustment, it is necessary to "actually operate the device" or "create and operate a simulator model of the device" in order to obtain training data for the machine learning model. When "actually operating the device," data must be obtained in advance when performing supervised learning or unsupervised learning. However, it is difficult to know in advance how much data will be required to achieve the desired accuracy.

[0006] On the other hand, with methods such as reinforcement learning and Bayesian optimization, data is actively acquired, reducing the need to know the amount of data required in advance. However, the amount of data required to achieve the desired accuracy may be unrealistic for actually operating the device, making data acquisition costs an issue.

[0007] The problem of data acquisition costs can be reduced by creating and running a simulator model of the equipment model. However, it is difficult to formulate the controlled object as a physical model and build a simulator that accurately reproduces the equipment.

[0008] To address the above problem, a method has been proposed in which a simulator model that simulates device behavior using a simple machine learning model is first created, and then the model is trained using an actual device. However, the versatility and scope of application of simulator models based on machine learning are not necessarily clear. For example, in a printing device, if control parameters cannot be optimized due to changes in the web type, environmental changes, or other factors, a trained simulator model must be reconstructed, making it difficult to efficiently adjust control parameters using the simulator model.

[0009] An object of the present invention is to provide a technique that can efficiently adjust the control parameters of a transport device using a simulator model. [Means for solving the problem]

[0010] In order to solve the above problem, a first aspect is a control parameter adjustment method for adjusting control parameters for controlling a conveying device that conveys a web, the method including: a) a step of measuring conveying characteristics when the conveying device is operated with first control parameters; b) a step of predicting the conveying characteristics for the first control parameters using a plurality of simulator models that have control parameters as input and conveying characteristics as output; c) a step of selecting a target simulator model from the plurality of simulator models based on the conveying characteristics measured in step a) and the conveying characteristics predicted in step b); d) a step of predicting control parameters using the target simulator model that will bring the conveying characteristics into an acceptable range; e) a step of measuring the conveying characteristics when the conveying device is operated with the control parameters predicted in step d); and f) a step of training the target simulator model using the measured conveying characteristics as training data if the conveying characteristics measured in step e) are outside the acceptable range.

[0011] A second aspect is a control parameter adjustment method of the first aspect, wherein the target simulator model selected in step c) is the simulator model among the plurality of simulator models that outputs transport characteristics that are the smallest in difference from the transport characteristics measured in step a).

[0012] A third aspect is the control parameter adjusting method of the first or second aspect, wherein steps d) to f) are repeatedly executed until the transport characteristics of the transport device fall within an allowable range.

[0013] A fourth aspect is the control parameter adjustment method of the third aspect, wherein the step d) is performed based on a reinforcement learning or Bayesian optimization algorithm.

[0014] A fifth aspect is the control parameter adjusting method according to any one of the first to fourth aspects, wherein the plurality of simulator models receive control parameters as input and output tension fluctuation values ​​of the web, respectively.

[0015] A sixth aspect is a computer-executable program that causes the computer to execute the control parameter adjustment method according to any one of the first to fifth aspects.

[0016] A seventh aspect is a computer-readable recording medium on which the program of the sixth aspect is recorded. [Effects of the Invention]

[0017] According to the control parameter adjustment method of the first aspect, even if a change in the situation of the transport device occurs, a simulator model corresponding to the change in situation can be obtained by training a target simulator model selected from a plurality of simulator models, and therefore, the control parameters can be adjusted efficiently using the simulator model.

[0018] According to the second aspect of the control parameter adjustment method, a target simulator model that exhibits behavior similar to that of a transport device in which a situation change has occurred is selected, and the target simulator is trained as necessary, thereby enabling efficient construction of a simulator model that can respond to situation changes.

[0019] According to the control parameter adjustment method of the third aspect, the target simulator model is trained using newly acquired training data, and it becomes possible to predict optimal control parameters using the target simulator model.

[0020] According to the control parameter adjustment method of the fourth aspect, training data can be automatically acquired based on a reinforcement learning or Bayesian optimization algorithm, and the target simulator model can be trained using the training data. [Brief explanation of the drawings]

[0021] [Figure 1] FIG. 2 is a diagram illustrating a configuration of a transport device according to an embodiment. [Figure 2] FIG. 2 is a diagram illustrating a configuration of a control unit. [Figure 3]FIG. 10 is a diagram showing a flow of a simulator model creation process executed in the transport device. [Figure 4] FIG. 10 is a diagram showing sampling points sampled from the PI parameter range in which the transport device can operate. [Figure 5] FIG. 10 is a diagram showing output results of a simulator model. [Figure 6] FIG. 10 is a diagram showing a flow of a fine adjustment process of a simulator model executed in the transport device. [Figure 7] FIG. 10 is a diagram showing the flow of a PI parameter readjustment process executed in the transport device. DETAILED DESCRIPTION OF THE INVENTION

[0022] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. Note that the components described in the embodiment are merely examples and are not intended to limit the scope of the present invention. In the drawings, the dimensions and numbers of each part may be exaggerated or simplified as necessary to facilitate understanding.

[0023] <1. Embodiment> 1 is a diagram showing a conveying device 1 according to an embodiment. The conveying device 1 conveys a long strip-shaped web 9 along a predetermined conveying path along the longitudinal direction of the web 9. The web 9 is, for example, printing paper, a resin film, or a metal foil. The conveying device 1 includes an unwinding roller 11, multiple conveying rollers 121, two nip rollers 122 and 123, a tension measuring roller 124, a winding roller 13, multiple motors 14, a tension meter 15, and a control unit 2.

[0024] The unwinding roller 11 feeds out the wound web 9. The winding roller 13 winds up the web 9 fed from the unwinding roller 11. A plurality of conveying rollers 121, nip rollers 122 and 123, and a tension measuring roller 124 are arranged on the conveying path between the unwinding roller 11 and the winding roller 13, and each supports the web 9.

[0025] Each motor 14 is connected to the unwinding roller 11, the nip rollers 122 and 123, and the winding roller 13. Each roller connected to a motor 14 receives the driving force of the motor 14 and rotates about a rotation axis perpendicular to the conveying path, thereby conveying the web 9 downstream along the conveying path. The motors 14 are electrically connected to the control unit 2.

[0026] The nip rollers 122 and 123 contact both sides of the web 9 to grip the web 9, and are actively rotated by the driving force of the motor 14. The nip roller 123 is located downstream of the nip roller 122 on the conveyance path.

[0027] The tension measuring roller 124 is located between the nip roller 122 and the nip roller 123 on the transport path of the web 9. The tension meter 15 is a device that measures the tension applied to the web 9, and measures the force that the tension measuring roller 124 receives from the web 9. The tension meter 15 is connected to the control unit 2 so as to be able to communicate with it.

[0028] FIG. 2 is a diagram showing the configuration of the control unit 2. The control unit 2 controls the operation of each part of the conveyance device 1. The control unit 2 includes a processor 21, a memory unit 23, a display 25, and an input device 27. The processor 21 is configured, for example, by a CPU. The memory unit 23 is configured by a RAM or a hard disk drive. The display 25 displays various information. The input device 27 is configured by a keyboard, a mouse, a touch panel, or the like. The memory unit 23, the display 25, and the input device 27 are electrically connected to the processor 21 via a bus.

[0029] The storage unit 23 stores a program 231 and a simulator model 235. The program 231 is provided by a recording medium M. The recording medium M records the program 231 so as to be readable by the control unit 2, which is a computer. The recording medium M is, for example, a USB (Universal Serial Bus) memory, an optical disk such as a DVD (Digital Versatile Disc), or a magnetic disk.

[0030] The processor 21 of the control unit 2 reads out the program 231 from the storage unit 23 and operates in accordance with the program 231. This controls the operation of each unit of the conveyance device 1, and the conveyance device 1 executes each process.

[0031] As shown by dashed lines in FIG. 1 , the control unit 2 is electrically connected to each motor 14 and tension meter 15. The control unit 2 controls the operation of each motor 14 to control the rotation speed of each roller. The control unit 2 applies tension to the web 9, for example, by making the rotation speed of nip roller 123 greater than the rotation speed of nip roller 122. Applying an appropriate tension to the web 9 suppresses the occurrence of slack or wrinkles in the web 9 during transport. The control unit 2 also measures tension fluctuations based on the output of the tension meter 15.

[0032] For example, the conveying device 1 may be incorporated into a printing device. In this case, a head that ejects ink by an inkjet method may be disposed between the nip rollers 122 and 123.

[0033] As will be described later, the control unit 2 creates a simulator model 235 and adjusts control parameters using the simulator model 235. As will be described later, the simulator model 235 receives control parameters as input and outputs the transport characteristics of the transport device 1.

[0034] The simulator model 235 is, for example, a Gaussian process regression model. Note that the simulator model 235 is not limited to a Gaussian process regression model, and any machine learning model such as a neural network or a support vector machine (SVM) may be applied. Furthermore, when Bayesian optimization is used in additional training of the simulator model 235, which will be described later, a Bayesian model is adopted.

[0035] Specifically, the control parameters are PI parameters (P gain and I gain) used when the control unit 2 performs PI control on the motor 14 of the nip roller 123. Furthermore, specifically, the conveying characteristics are tension fluctuation values ​​detected by the tension meter 15.

[0036] <Simulator model creation process> Fig. 3 is a diagram showing the flow of the process of creating the simulator model 235 executed in the conveyance device 1. For example, when no simulator model 235 exists, such as when the conveyance device 1 is started up, the process of creating the simulator model shown in Fig. 3 is executed. The processor 21 of the control unit 2 operates in accordance with the program 231 from the storage unit 23 to execute each step shown in Fig. 3.

[0037] First, the control unit 2 samples a set of PI parameters to be measured as training data from the PI parameter range in which the transport device 1 can operate according to design (sampling step S11). Note that, for example, Monte Carlo sampling can be applied as a sampling algorithm.

[0038] Fig. 4 is a diagram showing sampling points sampled from the PI parameter range in which the conveying device 1 can operate. In the example shown in Fig. 4, the PI parameter range is set to a P gain of 0.2 or more and 0.8 or less, and an I gain of 0.0 or more and 0.4 or less, and tension fluctuation values ​​are measured for each PI parameter sampled within that range.

[0039] In the sampling step S11, the number of sampling points and the sampling method can be set arbitrarily. For example, if the output accuracy is low, the number of sampling points may be increased. Also, if the number of PI parameters to be sampled is large, Latin square sampling may be applied.

[0040] The control unit 2 operates the conveying device 1 based on each PI parameter sampled in the sampling step S11, and measures the tension fluctuation value of the web 9 (measurement step S12). As a result, the tension fluctuation value for each PI parameter is acquired.

[0041] The control unit 2 trains the untrained simulator model 235 using a plurality of training data sets, each of which has each PI parameter as an input and each tension fluctuation value obtained in the measurement step S12 as an output (training step S13), thereby creating a trained simulator model 235.

[0042] Fig. 5 is a diagram showing the output results of the simulator model 235. As shown in Fig. 5, the trained simulator model 235 is trained in the training step S13 to output a tension fluctuation value for each input (PI parameter).

[0043] <Fine-tuning of simulator model 235> Fig. 6 is a diagram showing the flow of the fine-tuning process of the simulator model 235 executed in the conveyance device 1. The fine-tuning process shown in Fig. 6 is performed to improve the accuracy of the simulator model 235 created in the creation process shown in Fig. 3. The processor 21 of the control unit 2 operates in accordance with the program 231 in the storage unit 23 to execute each step shown in Fig. 6.

[0044] First, the control unit 2 predicts the optimal PI parameters that will optimize the output tension fluctuation value (prediction step S21) using the simulator model 235. After the prediction step S21, the control unit 2 transports the web based on the predicted optimal PI parameters and measures the tension fluctuation value during web transport (measurement step S22).

[0045] After the measurement step S22, the control unit 2 determines whether the measured tension fluctuation value falls within the target allowable range (determination step S23). If the measured tension fluctuation value falls within the allowable range (Yes in determination step S23), the control unit 2 ends the fine adjustment of the simulator model 235. If the measured tension fluctuation value is out of the allowable range (No in determination step S23), the control unit 2 uses the predicted optimal PI parameters and the measured tension fluctuation value as training data to additionally train the simulator model 235 (additional training step S24). After the additional training step S24, the control unit 2 executes the prediction step S21 again. That is, until the measured tension fluctuation value falls within the allowable range, the control unit 2 repeatedly executes from the prediction step S21 to the additional training step S24.

[0046] In the prediction step S21, reinforcement learning may be used as an algorithm for predicting the optimal PI parameters. Also, as the reinforcement learning agent, for example, SAC (Soft-Actor-Critic) can be applied. Also, as the reward function, for example, the following normalized one can be applied. When the tension fluctuation value fails to reach the target: -1 When the tension fluctuation value reaches the target: 1

[0047] Note that Bayesian optimization may be used as an algorithm for predicting the optimal PI parameters. Also, the reward function is not limited to the above. For example, a feature amount may be calculated from the tension fluctuation value measured in the measurement step S22, and the calculated feature amount may be used as the reward.

[0048] The control unit 2 constructs a plurality of learned simulator models by executing the flows shown in FIGS. 3 and 6 multiple times. When constructing each simulator model of 235, the conveyance conditions of the conveyance device 1 may be changed. For example, by changing the type of the web 9, a simulator model 235 showing the behavior of the conveyance device 1 for each type of the web 9 is created.

[0049] <Readjustment of PI Parameters> 7 is a diagram showing the flow of the PI parameter readjustment process executed in the conveyance device 1. The processor 21 of the control unit 2 operates in accordance with the program 231 in the storage unit 23 to execute each step shown in FIG.

[0050] First, the control unit 2 determines whether a situation change has occurred (determination step S31). The situation change may be, for example, a change in the type of web 9 being conveyed by the conveying device 1, a change in the conveying conditions (conveying speed, etc.) of the web 9, or a change in the environment. The control unit 2 may also determine that a situation change has occurred when the tension fluctuation value measured by the tension meter 15 falls outside the allowable range. The control unit 2 may also determine that a situation change has occurred based on a predetermined instruction input by the user.

[0051] When a change in the situation occurs (Yes in the determination step S31), the control unit 2 conveys the web based on the existing PI parameters and measures the tension fluctuation value (measurement step S32).The control unit 2 also predicts the tension fluctuation value for the existing PI parameters using a simulator ensemble consisting of multiple simulator models 235 (prediction step S33).

[0052] The control unit 2 calculates the difference between the actual measured value of the tension fluctuation value obtained in the measurement step S32 and the output value of each simulator model 235 obtained in the prediction step S33 (difference calculation step S34). This difference is, for example, the Kullback-Leibler (KL) distance. However, the difference may also be another value (for example, the Wasserstein distance).

[0053] After the difference calculation step S34, the control unit 2 selects the simulator model 235 with the smallest difference from the multiple simulator models 235 (selection step S35). Hereinafter, the selected simulator model 235 will be referred to as the "target simulator model 235." The control unit 2 uses the target simulator model 235 to predict optimal PI parameters that will optimize the tension fluctuation value, which is the output value (prediction step S36). The algorithm applied in the prediction step S36 may be the same as the algorithm applied in the prediction step S21 (for example, reinforcement learning).

[0054] After the prediction step S36, the control unit 2 measures the tension fluctuation value while transporting the web based on the predicted optimal PI parameter (measurement step S37). After the measurement step S37, the control unit 2 determines whether the measured tension fluctuation value falls within the target allowable range (determination step S38). If the measured tension fluctuation value falls within the allowable range (Yes in the determination step S38), the control unit 2 ends the PI parameter readjustment process. In this case, the tension fluctuation value can be brought within the allowable range by transporting the web based on the optimal PI parameter predicted in the prediction step S36.

[0055] If the tension fluctuation value measured in the prediction step S36 falls outside the allowable range (No in the determination step S38), the control unit 2 uses a set of the predicted optimal PI parameter and the measured tension fluctuation value as new training data and performs additional training on the target simulator model 235 (additional training step S39). After the additional training step S39, the control unit 2 executes the prediction step S36 again. That is, the prediction step S36 to the additional training step S39 are repeatedly executed until the measured tension fluctuation value falls within the allowable range.

[0056] When the PI parameters are readjusted in response to a change in the type of web 9, if the characteristics of the web 9 do not change much before and after the change (specifically, if the difference in Young's modulus is small), the amount of change in the PI parameters is generally small. On the other hand, if the characteristics of the web 9 change significantly before and after the change, the amount of change in the PI parameters is generally large. Here, changes in the simulator model 235 are represented by changes in the internal parameters of the model (weighting coefficients and biases in the case of a linear model). Therefore, if the characteristics of the web 9 do not change much before and after the change, the amount of change in the internal parameters of the simulator model 235 required to accommodate the change will be relatively small.

[0057] In the PI parameter readjustment process shown in FIG. 7, retraining is performed using a target simulator model 235 that is closest to the state of the transport apparatus 1 after a situation change, from among multiple simulator models 235 constructed in advance. This allows the amount of training data to be reduced compared to training an unlearned simulator model from scratch. Since changes in the situation of the transport apparatus 1 are handled by adjusting the internal parameters of the simulator model, there is no need to redesign the simulator model. Therefore, a simulator model that represents the behavior of the transport apparatus 1 in response to situation changes can be efficiently constructed, allowing for efficient readjustment of the PI parameters using the simulator model.

[0058] Furthermore, in the prediction step S36 to the additional training step S39, a learning algorithm such as reinforcement learning or Bayesian optimization is applied, which allows the necessary training data to be acquired automatically.

[0059] <2. Variations> Although the embodiments have been described above, the present invention is not limited to the above and various modifications are possible.

[0060] In the above embodiment, the control parameter is a PI parameter, but it may be another parameter (such as a PID parameter). Also, in the above embodiment, the transport characteristic is a tension fluctuation value of the web 9, but it may be another physical quantity. For example, the transport characteristic may be the transport speed (movement speed of the web 9), the amount of meandering of the web 9, etc.

[0061] Although the present invention has been described in detail, the above description is merely illustrative in all respects and does not limit the present invention. It is understood that countless variations not illustrated can be envisioned without departing from the scope of the present invention. The configurations described in the above embodiments and variations can be combined or omitted as appropriate as long as they are not mutually inconsistent. [Explanation of symbols]

[0062] 1. Conveyor device 2. Control section 9. Web 231 Programs 235 Simulator Model M Recording medium

Claims

1. 1. A control parameter adjustment method for adjusting a control parameter for controlling a conveying device that conveys a web, comprising: a) measuring conveyance characteristics when the conveyance device is operated with existing control parameters; b) predicting the transport characteristics for the existing control parameters using a plurality of simulator models that have control parameters as inputs and transport characteristics as outputs; c) selecting a target simulator model from the plurality of simulator models based on the transport characteristics measured in the step a) and the transport characteristics predicted in the step b); d) predicting control parameters that result in an allowable range of transport characteristics using the target simulator model; e) measuring conveyance characteristics when the conveyance device is operated using the control parameters predicted in step d); f) if the transport characteristics measured in step e) are outside the allowable range, training the target simulator model using the measured transport characteristics as training data; A control parameter adjustment method comprising:

2. 2. The control parameter adjustment method according to claim 1, a control parameter adjusting method, wherein the target simulator model selected in step c) is a simulator model among the plurality of simulator models that outputs a transport characteristic that is the smallest difference from the transport characteristic measured in step a).

3. 3. The control parameter adjustment method according to claim 1 or 2, The control parameter adjustment method includes repeatedly executing steps d) to f) until the transport characteristics of the transport device fall within an allowable range.

4. 4. The control parameter adjustment method according to claim 3, The control parameter adjustment method, wherein step d) is performed based on a reinforcement learning or Bayesian optimization algorithm.

5. 5. The control parameter adjustment method according to claim 1, further comprising: A control parameter adjustment method, wherein the plurality of simulator models receive control parameters as input and output tension fluctuation values ​​of the web, respectively.

6. A computer-executable program, A program causing the computer to execute the control parameter adjustment method according to any one of claims 1 to 5.

7. A computer-readable recording medium on which the program according to claim 6 is recorded.

Images