Injection molding machine, method of controlling injection molding machine, and program of controlling injection molding machine

Abstract

To provide a technique capable of performing efficient temperature control of a to-be-temperature controlled part using machine learning.SOLUTION: An injection molding machine 1 comprises: a heater 5; a controller 7 for controlling the heater 5; and a to-be-temperature controlled part to be temperature-controlled by the heater 5. The controller 7 has: a function of calculating a control deviation on the basis of a measured temperature and a set target temperature of the to-be-temperature controlled part; a function capable of implementing a first control law for calculating a manipulated variable of the heater 5 as a result of machine learning based on the control deviation, and a second control law for calculating a manipulated variable of the heater 5 on the basis of the control deviation without using the machine learning; and a function of selecting for implementing either one of the first control law or the second control law.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to an injection molding machine, a control method for the injection molding machine, and a program therefor. [Background technology]

[0002] In recent years, attempts have been made to use machine learning to control the temperature of temperature control devices such as heaters in injection molding machines using AI (Artificial Intelligence). However, AI control using machine learning has the problem of learning disturbances as well. For example, strong disturbances such as temperature fluctuations occur during injection molding. Therefore, the effects of the disturbances are incorporated into the parameters that are the learning results, and as a result, the parameters change due to the disturbances. In this way, since AI learns regardless of the presence or absence of disturbances, the way the temperature changes changes when the parameters change, and as a result, the temperature control becomes unstable. When the temperature control is unstable, the quality of the molded product may vary. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2022-67240 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a technology capable of efficiently controlling the temperature of a temperature-adjustable part using machine learning. [Means for solving the problem]

[0005] One embodiment is an injection molding machine equipped with a control device having a function of calculating a control deviation based on a measured temperature and a set target temperature of a temperature-controlled part, a function of executing a first control law that calculates an operation amount of the temperature control device as a result of machine learning based on the control deviation, and a second control law that calculates the operation amount of the temperature control device based on the control deviation without using machine learning, and a function of selecting to execute either the first control law or the second control law. [Brief description of the drawings]

[0006] [Figure 1] 1 is a schematic vertical sectional view showing an injection molding machine according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram for explaining a control system relating to feedback control for a heater of the injection molding machine according to the first embodiment. [Diagram 3] FIG. 2 is a block diagram for explaining reinforcement learning by the control device according to the first embodiment. [Figure 4] FIG. 4 is a diagram showing a state value table according to the first embodiment. [Diagram 5] 2 is a block diagram showing a hardware configuration of a control device according to the first embodiment. FIG. [Figure 6] 2 is a block diagram showing a functional configuration of a control device according to the first embodiment. FIG. [Figure 7] 4 is a flowchart showing a learning feedback control process according to the first embodiment. [Figure 8] 5 is a flowchart showing an operation amount calculation process according to the first embodiment. [Figure 9] 4 is a flowchart showing a reinforcement learning process according to the first embodiment. [Figure 10] 4 is a flowchart showing a learning parameter update process according to the first embodiment. [Figure 11] 5 is a flowchart showing a learning stop operation amount calculation process according to the first embodiment. [Figure 12]FIG. 4 is a diagram for explaining a filter target temperature according to the first embodiment. [Figure 13] FIG. 4 is a diagram for explaining the effect of a filter target temperature according to the first embodiment. [Figure 14] 4 is a flowchart showing a filter target temperature calculation process according to the first embodiment. [Figure 15] 11A to 11C are diagrams for explaining the relationship between the magnitude of the parameter θ and the elapsed time and the relationship between the measured temperature and the elapsed time, which are obtained as a result of the first feedback control using reinforcement learning in the comparative example. [Figure 16] FIG. 13 is a diagram for explaining the relationship between the magnitude of the parameter θ and the elapsed time and the relationship between the measured temperature and the elapsed time, which are obtained as a result of the tenth feedback control using reinforcement learning according to the comparative example. [Figure 17] 11A to 11C are diagrams for explaining the relationship between the magnitude of the parameter θ and the elapsed time and the relationship between the measured temperature and the elapsed time, which are obtained as a result of the n-th feedback control using reinforcement learning according to the comparative example. [Figure 18] 4A to 4C are diagrams for explaining the relationship between the magnitude of a learning parameter θ and elapsed time, and the relationship between a measured temperature and elapsed time, which are obtained as a result of a first learning feedback control process according to the first embodiment. [Figure 19] 10A to 10C are diagrams for explaining the relationship between the magnitude of the learning parameter θ and the elapsed time, and the relationship between the measured temperature and the elapsed time, which are obtained as a result of the tenth learning feedback control process according to the first embodiment. [Figure 20] 10A to 10C are diagrams for explaining the relationship between the magnitude of a learning parameter θ and elapsed time, and the relationship between a measured temperature and elapsed time, which are obtained as a result of an n-th learning feedback control process according to the first embodiment. [Figure 21] 5 is a flowchart showing a user notification process according to the first embodiment. [Figure 22] FIG. 2 is a schematic diagram showing a screen displayed on a display during AI control according to the first embodiment. [Figure 23]4 is a schematic diagram showing a screen on which a countdown for control switching is displayed on a display according to the first embodiment; FIG. [Figure 24] FIG. 4 is a schematic diagram showing a screen displayed on a display during standard PID control according to the first embodiment. [Diagram 25] 10 is a schematic diagram showing a screen on which the control status of each heater is displayed on a display according to the application example of the first embodiment; FIG. [Figure 26] FIG. 1 is a diagram for explaining fluctuations in measured temperature that occur as disturbances during injection molding. [Figure 27] FIG. 13 is a diagram for explaining parameter fluctuations when a disturbance occurs during injection molding. [Figure 28] 5A to 5C are diagrams for explaining parameter fluctuations when a disturbance occurs during injection molding in the learning feedback control process according to the first embodiment. [Figure 29] FIG. 4 is a schematic vertical cross-sectional view showing a tie bar and a mold according to an application example of the first embodiment. [Diagram 30] 5A to 5C are diagrams for explaining a processing result obtained by a learning feedback control process targeted at a cooling device according to an application example of the first embodiment. [Diagram 31] FIG. 11 is a diagram for explaining bumpless switching according to an application example of the first embodiment. [Diagram 32] FIG. 11 is a block diagram showing a functional configuration of a control device according to a second embodiment. [Diagram 33] 10 is a flowchart showing an operation amount calculation process according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In this specification and the drawings, components having substantially the same functions are denoted by the same reference numerals.

[0008] <First embodiment> (Configuration of injection molding machine) The configuration of the injection molding machine according to this embodiment will be described below. Fig. 1 is a schematic vertical cross-sectional view showing the injection molding machine according to this embodiment.

[0009] As shown in FIG. 1, the injection molding machine 1 according to this embodiment includes a cylinder 2 extending in the conveying direction to melt the pellet-shaped resin material. The cylinder 2 is a temperature-adjusted part to be temperature-adjusted, and a hopper 3 into which the resin material is poured is provided at the upstream end (right side in the figure) of the resin material in the conveying direction. A single-shaft or double-shaft screw 4 is rotatably provided in the cylinder 2 to melt and knead the resin material and convey it to the tip of the cylinder 2 located downstream in the conveying direction. The tip of the cylinder 2 has a smaller diameter than the upstream side in the conveying direction, and a mold (not shown) is attached to it. The screw 4 advances at high speed in the conveying direction, so that the resin material melted and plasticized in the cylinder 2 is injected from the tip of the cylinder 2 into the mold. The injected resin material becomes a plastic part having a shape according to the mold.

[0010] The cylinder 2 is provided with a plurality of annular heaters (e.g., band heaters) 5 surrounding the outer periphery of the cylinder 2 along the conveying direction as temperature control devices. Each of the heaters 5 is supplied with power from a heater power source (not shown) via a corresponding SSR (Solid State Relay). The power supply to the heaters 5 is not limited to an SSR, and may be performed via a contact relay. The heaters 5 heat the contact portions of the cylinder 2 by being supplied with power. The cylinder 2 can be heated to different temperatures along the extension direction by the heaters 5. In general, each of the heaters 5 is controlled so that the temperature of the cylinder 2 increases from the upstream side to the downstream side in the conveying direction. A temperature sensor 6 is provided in the vicinity of each of the heaters 5.

[0011] Each of the multiple temperature sensors 6 measures the temperature of a portion of the cylinder 2 that is heated by the corresponding heater 5. Each of the multiple temperature sensors 6 is inserted, for example, into a through hole formed in the corresponding heater 5 and comes into contact with the cylinder 2. The temperature sensor 6 is configured, for example, as a thermocouple.

[0012] The heaters 5 and the temperature sensors 6 are connected so as to be controllable by a control device 7. The control device 7 performs feedback control of the heaters 5 based on the measured temperatures obtained from the temperature sensors 6. Through the feedback control, the temperature of the heaters 5 is adjusted to a target temperature previously set by a user. This target temperature is hereinafter referred to as a set target temperature. The control device 7 also controls a drive device (not shown) that drives the screw 4 to rotate and move it forward and backward. In this embodiment, a detailed description of the control of the screw 4 is omitted.

[0013] The feedback control of the heater 5 by the control device 7 according to this embodiment will be described in detail below. Fig. 2 is a block diagram for explaining a control system related to feedback control for the heater of the injection molding machine according to this embodiment.

[0014] (Control system of control device 7) As shown in Fig. 2, the control device 7 according to this embodiment controls the heater 5 by PID control as feedback control based on the deviation (control deviation) ε. In this embodiment, the PID control is a speed-type PID control. By this control, the heater 5 is heated to a set target temperature T sv The deviation ε is adjusted so that the set target temperature T sv and the measured temperature T of the heater 5 obtained from the temperature sensor 6. pv In this embodiment, the set target temperature T sv is corrected to the filter target temperature and used for PID control and reinforcement learning. The filter target temperature will be described in detail later.

[0015] Heater 5 target temperature Tsv The adjustment to is performed by calculating an operation amount u as the output of PID control and the result of reinforcement learning described later, and inputting a control signal based on the operation amount u to the heater 5. This PID control is performed individually for each heater 5. In PID control, it is necessary to adjust parameters every time the characteristics of a control target such as a heater are changed. The parameters to be adjusted include a proportional band (or proportional gain), an integral gain, and a differential gain. Hereinafter, these gains are referred to as control parameters. The control device 7 adjusts these control parameters using reinforcement learning based on the deviation ε. Through reinforcement learning, a learning parameter described later for adjusting the control parameter is adjusted, and the operation amount u of the PID control is output as the action a. This eliminates the need for the user to adjust the control parameter by trial and error. In other words, it is possible to significantly reduce the work time.

[0016] (Reinforcement Learning) A brief explanation of reinforcement learning is given below. It is known that reinforcement learning is constructed from an agent that learns and makes decisions, and an environment other than the agent that is influenced by the agent. For example, the agent can be thought of as AI, and the environment can be thought of as the heater in an injection molding machine. The agent determines action a, such as control parameters or manipulated variables, from state s and reward r, according to policy π. State s is a value, such as temperature, that comes from the environment. Reward r is a numerical value or goal that the agent wants to maximize. Policy π defines how action a behaves.

[0017] If the time step indicating each control cycle individually is t, and the action a determined at this time step is a[t], then the action a[t] acts on the environment, and the environment transitions to a new state s[t+1] at the next time step. A new reward r[t+1] is observed due to the state s[t+1]. The agent determines the action a[t+1] for the next time step so that a higher new reward r[t+1] is obtained. In reinforcement learning, learning progresses by repeating such processes so that a better action a is output.

[0018] Fig. 3 is a block diagram for explaining reinforcement learning by the control device according to the present embodiment. Fig. 4 is a diagram showing a state value table according to the present embodiment.

[0019] The control device 7 according to this embodiment uses the Actor-Critic method as shown in FIG. 3 for reinforcement learning. In the Actor-Critic method, a control device 7 is constructed from an agent 81 and an environment 82. The agent 81 has an actor (behavior) 811 and a critic (evaluator) 812. In this embodiment, the agent 81 corresponds to the control device 7, the environment 82 corresponds to the heater 5, and the actor 811 and critic 812 correspond to the program of the control device 7. The actor 811 adjusts control parameters such as the proportional band, integral gain, and differential gain, updates a policy based on the adjusted various control parameters, and outputs an operation amount as action a according to the policy.

[0020] The heater 5 is controlled by the control device 7, and the measured temperature T pv Therefore, the control device 7 does not obtain the state s and reward r from the heater 5. The state s and reward r are calculated based on the measured temperature T pv The control device 7 calculates the control parameters based on the above. Similarly, the control parameters are calculated by the control device 7 and used to update the strategy. Also, the manipulated variable is calculated according to the strategy. The calculated manipulated variable is input to the heater 5 as a control signal.

[0021] The actor 811 selects an action a according to a policy π. The policy π here is a probabilistic policy that indicates a probability distribution (normal distribution) of the probability that the action a is selected depending on the state s. The critic 812 evaluates the new state s after performing the action a, and judges whether the execution result of the action a is better than expected. The evaluation result is expressed in the form of a TD (Temporal Difference) error δ. In other words, the TD error is a value that evaluates the goodness or badness of a certain action a. The critic 812 outputs the TD error δ using a state value function V, which is a function that expresses how good the state s is. The output TD error δ is used to update the policy π, as well as to update the state value function V and the learning parameters described later.

[0022] The state value function V is stored in advance in the control device 7, and can be expressed in the form of a table as shown in Fig. 4. Hereinafter, this table will be referred to as a state value table. In the state value table, a numerical value indicating the value (state value) of the state s is associated with each of a plurality of states s. In other words, the state value function V is determined depending on which state in the state value table the output state s corresponds to.

[0023] In this embodiment, the deviation ε is treated as the state s. For this reason, -ε is written in the center of the state value table. th ≦ε≦ε th A section (hereinafter referred to as a threshold section) is provided. th indicates a predetermined deviation threshold. The state value table is divided into 2N+1 states by dividing the state value table into the left and right of the threshold interval by the deviation value n. Here, n>ε thThe deviation indicated by the state s is considered to be in the best state when it is within the threshold interval, and the highest state value is calculated (updated) based on the state value, reward, and TD error. In general, the value is set so that the state value decreases as the interval moves away from the threshold interval, that is, the interval is located to the left or right in FIG. 4 of the state value table. However, the state value does not necessarily decrease as the interval moves away from the threshold interval. Even if the interval moves away from the threshold interval, an interval may be set in which the state value is higher than the adjacent interval. It has been explained that the highest state value is calculated when the deviation indicated by the state s is within the threshold interval, but this is in the middle to late stages of learning, and in the early stages of learning, such as the first few times of learning, the threshold interval may not be the highest state value.

[0024] In this embodiment, the state value function V is updated for all states s using the eligibility trace. By updating the state value function V for all states, it is possible to collectively update related state values ​​going back to the past. This allows efficient learning, specifically, adjustment of the learning parameter θ for calculating the control parameters. Details of the calculation of the TD error, the calculation of the eligibility trace, the update of the state value function V, and the learning parameter θ described above will be described later.

[0025] (Hardware configuration of the control device 7) Fig. 5 is a block diagram showing a hardware configuration of the control device according to the present embodiment. As shown in Fig. 5, the control device 7 according to the present embodiment includes a CPU (Central Processing Unit) 71, a RAM (Random Access Memory) 72, a storage device 73, a user I / F (Interface) 74, and an input / output I / F 75.

[0026] The CPU 71 executes various programs, such as the BIOS (Basic Input / Output System), OS (Operating System), general-purpose applications, and a control program for the learning feedback control processing described later, which are deployed on the RAM 72, which is a volatile storage area, and controls the control device 7.

[0027] The storage device 73 is a non-volatile storage area including a read only memory (ROM) and the like. The storage device 73 may include a storage device such as a hard disk drive (HDD) or a solid state drive (SSD). The storage device 73 stores various parameters used in the learning feedback control process, past measured data, data of each calculated physical quantity, and the like. The storage device 73 may also store a log indicating the learning results of the reinforcement learning.

[0028] The user I / F 74 presents an input screen to a user who uses the injection molding machine 1 and accepts input from the user. The user I / F 74 also presents a log to the user as a result of the learning feedback control process in a visible manner. The log includes, for example, a graph showing the relationship between the heater temperature and the elapsed time. The user I / F 74 includes, for example, a display, a pointing device such as a mouse for specifying a specific position on the display, and a keyboard on which characters and keys to which specific functions are assigned are arranged. The user I / F 74 may be configured as a touch panel display.

[0029] The input / output I / F 75 inputs and outputs signals and data related to the control device 7. For example, the input / output I / F 75 includes a board that supplies power as a command value to the heater 5 by inputting a PWM (Pulse Width Modulation) signal to an SSR based on a control signal from the control device 7. The board obtains a measured temperature from a thermoelectromotive force returned from the connected temperature sensor 6, and outputs this to the control device 7. The input / output I / F 75 also transmits an ON / OFF control signal to a heater power supply (not shown), which is the power source for the heater 5.

[0030] (Functional configuration of the control device 7) Fig. 6 is a block diagram showing the functional configuration of the control device according to this embodiment. As shown in Fig. 6, the control device 7 according to this embodiment has the functions of display 701, acquisition 702, target temperature calculation 703, deviation calculation 704, restriction 705, learning 706, operation amount calculation 707, output 708, and judgment 709. These functions are realized by cooperation of the above-mentioned hardware resources such as the CPU 71 and RAM 72, and a learning feedback control process described later is executed by these functions.

[0031] The display 701 displays a control method for the heater 5 on a display as the user I / F 74 so that the user can select the control method. The display 701 displays a selection screen on the display including a selection button for selecting whether to execute AI-PID control (hereinafter referred to as AI control: first control law) that calculates an operation amount using machine learning such as reinforcement learning in the learning feedback control process according to the present embodiment or standard PID control (non-AI control: second control law) that calculates an operation amount without using machine learning such as reinforcement learning, and a start button for the selected process. The standard PID control will be described later. The selection screen may include an ON / OFF button for a heater power supply that supplies power to the heater 5. The display 701 also displays a log of the results of the learning feedback control process on the display. The display 701 also displays a screen for notifying the user whether the current control status is under AI control or under standard PID control as a user notification process.

[0032] Acquisition 702 acquires the control method for the heater 5 selected by the user, the set target temperature input by the user, and the measured temperature from the temperature sensor 6. Target temperature calculation 703 calculates a filter target temperature based on the set target temperature, which can suppress the control deviation calculated by comparing with the set target temperature. Deviation calculation 704 calculates the difference between the filter target temperature and the measured temperature as a deviation (control deviation) ε. As described above, the filter target temperature is based on the set target temperature. Therefore, in other words, deviation calculation 704 calculates deviation ε based on the measured temperature and the set target temperature.

[0033] The limit 705 limits the deviation ε used for reinforcement learning within a predetermined range. The learning 706 performs reinforcement learning using the above-mentioned actor-critic method, updates the learning parameters and calculates the control parameters for each control period. The operation amount calculation 707 calculates the operation amount. The output 708 outputs the calculated operation amount as a control signal to the heater 5 to control the temperature of the heater 5. The decision 709 performs various decision processes related to the learning feedback control process. Furthermore, the decision 709 selects either AI control or non-AI control to be executed by the learning 706 and / or the operation amount calculation 707 based on the result of the decision process. A detailed description of these various functions will be given in the description of the learning feedback control process and the user notification process described below.

[0034] (Learning feedback control processing) Next, the learning feedback control process executed by the control device 7 will be described in detail. Fig. 7 is a flowchart showing the learning feedback control process according to this embodiment. The learning feedback control process is triggered by the selection of the learning feedback control process on a selection screen for selecting a control method for the heater 5 displayed by a display 701 and the pressing of a start button. When the learning feedback control process starts, the AI ​​control is in advance selected by a decision 709.

[0035] As shown in Fig. 7, first, in decision 709, it is determined whether or not the heater power supply is ON in order to execute the learning feedback control process (S101). If it is determined that the heater power supply is ON (S101, YES), the operation amount calculation process is executed (S102). The operation amount calculation process will be described later. On the other hand, if it is determined that the heater power supply is OFF (S101, NO), it is determined again whether or not the heater power supply is ON after a predetermined time.

[0036] After the operation amount calculation process, a decision 709 decides whether or not a stop signal for stopping the learning feedback control process has been acquired by an acquisition 702 (S103). The stop signal is acquired, for example, when a stop button for executing stopping of the learning feedback control process is constantly displayed on the display by a display 701 during the learning feedback control process and the button is selected.

[0037] If it is determined that the acquire stop signal 702 has been acquired (S103, YES), the learning feedback control process ends. After the learning feedback control process, if the stop button is selected, the control of the heater 5 is stopped. On the other hand, if it is determined that the acquire stop signal 702 has not been acquired (S103, NO), the process of step S101 is executed. The processes of S101 to S103 represent one control cycle, and are repeated thereafter until a stop signal is acquired. One control cycle is, for example, 125 ms.

[0038] (Operation amount calculation process) The above-mentioned operation amount calculation process will be described. FIG. 8 is a flowchart showing the operation amount calculation process according to this embodiment. As shown in FIG. 8, first, in the judgment 709, it is determined whether or not the above-mentioned learning parameter θ learned by reinforcement learning has been initialized (S201). If it is determined that the learning parameter θ has not been initialized (S201, NO), the judgment 709 initializes the learning parameter θ (S202) and sets the set target temperature T sv It is determined whether there has been a change (S203).

[0039] On the other hand, if it is determined that the learning parameter θ has been initialized (S201, YES), the process of step S203 is executed. By initializing the learning parameter θ when it has not been initialized in this way, an incorrect learning parameter θ will not be reused when a new learning feedback control process is executed. Note that the reinforcement learning result, including various numerical values ​​such as the state value function V updated in the reinforcement learning, may be initialized.

[0040] Set target temperature T svIf it is determined that there has been no change in temperature T (S203, NO), the acquisition 702 acquires the measured temperature T pv The measured temperature T pv After obtaining the deviation, the deviation calculation 704 calculates the filter target temperature T dsv and the measured temperature T pv The deviation ε is calculated from the difference between the filter target temperature T dsv If not calculated, the measured temperature T pv may also be used.

[0041] After calculating the deviation ε, a decision step 709 decides whether or not the current control cycle is the second step or later, in other words, whether or not there was a control cycle before the current control cycle (S206). If it is decided that the current control cycle is the second step or later (S206, YES), a decision step 709 decides whether or not to stop the reinforcement learning (S207). The decision to stop the reinforcement learning is made based on whether or not a predetermined reinforcement learning stop condition related to the deviation is satisfied. For example, if a predetermined switching time has elapsed since the following formula was satisfied, it is decided that the reinforcement learning stop condition is satisfied.

[0042]

number

[0043] T in the above formula (1) sv is the set target temperature, T pv is the measurement temperature, ε th is the deviation threshold. In other words, the absolute value of the deviation ε is the deviation threshold ε th If it is less than ε, learning will stop after a certain switching time has elapsed. th <T sv -T pv <ε th In other words, the deviation threshold ε th Less than or equal to the negative deviation threshold ε thLearning is stopped after the switching time has elapsed when the deviation ε reaches a higher numerical range (allowable threshold range). If immediate learning is stopped without waiting for the switching time to elapse, there is a possibility that learning to deal with overshooting cannot be performed. For this reason, it is preferable that learning be continued until a predetermined switching time has elapsed after satisfying formula (1). Note that the deviation threshold ε in formula (1) th The deviation threshold value in the state value table may be used, or a value different from the deviation threshold value may be used. For example, the deviation threshold value ε th is 0.3. The absolute value of the deviation ε is the deviation threshold ε th After the measured temperature T pv is the set target temperature T sv It is preferable that the learning is performed to a degree that slightly exceeds ε. Therefore, it is preferable that the predetermined switching time is set to a time that allows the learning to be realized. For example, the predetermined switching time is 2 minutes. In other words, when the absolute value of the deviation ε is less than the deviation threshold ε th Once the time limit is reached, decision 709 starts timing, and AI control will continue until two minutes have elapsed.

[0044] If it is determined that the reinforcement learning is not to be stopped because the reinforcement learning stopping condition is not satisfied (S207, NO), a reinforcement learning process is executed in which learning parameters are updated and various control parameters are calculated (S208). The reinforcement learning process will be described later.

[0045] After the reinforcement learning process, the operation amount calculation 707 calculates a tentative first operation amount μ in the current control cycle based on the various control parameters and the deviation ε calculated by the reinforcement learning process (S209). The tentative first operation amount μ is calculated by the following formula.

[0046]

number

[0047] In the above formula (2), P b , K i , and K dindicates the proportional band, integral gain, and differential gain, which are control parameters calculated by the reinforcement learning process. Δt indicates the control period (sampling period). [t] indicates the time step that uniquely indicates the control period. Note that in the formula (2) according to this embodiment, the proportional band P b Using 100 / P b Instead, a proportional gain K p Similarly, the integral gain K i Alternatively, an integral time 1 / Ti may be incorporated.

[0048] After calculating the tentative first manipulated variable μ, the manipulated variable calculation 707 calculates (updates) a normal distribution based on the calculated tentative first manipulated variable μ (S210). The normal distribution according to this embodiment is formed from the tentative first manipulated variable μ with a mean μ and a standard deviation σ calculated from the learning parameters. This normal distribution can be expressed by the following equation.

[0049]

number

[0050] In the above formula (3), Δu is an arbitrary value, and π is the constant pi. exp is a mathematical symbol synonymous with Napier's constant e, for example, e = approximately 2.7. According to the above formula (3), by substituting a certain value for Δu, the corresponding probability of being selected f(Δu) can be found. The closer the values ​​of Δu and the average μ are, the higher the probability of being selected. The standard deviation σ can be calculated using the following formula.

[0051]

number

[0052] In the above formula (4), σ H , σ L indicate the maximum and minimum values ​​that the standard deviation σ can take, respectively. σis a type of learning parameter θ calculated and updated by the reinforcement learning process, and indicates a parameter corresponding to the standard deviation σ, which is the spread of the normal distribution. In other words, the normal distribution is a tentative first operation amount μ based on the deviation and a learning parameter θ σ It can be said that it is based on the learning parameter θ σ In this embodiment, the learning parameter θ is a part of the parameters for updating the policy, and is adjusted by reinforcement learning as described later. σ The calculation method will be described later.

[0053] After calculating the normal distribution, the manipulated variable calculation 707 selects one provisional second manipulated variable Δu using the normal distribution (S211). As described above, the provisional second manipulated variable Δu has the highest probability of being selected as the average μ. The selected provisional second manipulated variable Δu does not correspond to the manipulated variable of the heater 5, but corresponds to the amount of change in the manipulated variable. Therefore, after selecting the provisional second manipulated variable Δu, the manipulated variable calculation 707 calculates the manipulated variable u[t] of the current control cycle by adding the provisional second manipulated variable Δu to the manipulated variable u[t-1] of the control cycle one control cycle before, that is, the control cycle immediately before the current control cycle (S212). That is, the manipulated variable u is calculated by the following formula:

[0054]

number

[0055] In the above formula (5), u[t-1] indicates the manipulated variable for one control cycle before. If the manipulated variable for one control cycle before has not been calculated, the provisional second manipulated variable Δu becomes the manipulated variable u[t] for the current control cycle.

[0056] After the operation amount u[t] is calculated, the output 708 outputs the operation amount u[t] as a control signal to the heater 5 via the input / output I / F, and controls the temperature of the heater 5 according to the operation amount u[t] (S213). After controlling the temperature of the heater 5, the learning 706 updates parameters related to reinforcement learning (S214). The parameters are parameters calculated in the reinforcement learning process in the current control cycle, such as the control parameters and the policy π, and parameters required for calculating learning parameters such as the operation amount u, the tentative second operation amount Δu, and the tentative first operation amount μ. The operation amount u is treated as the operation amount u[t-1] in the next control cycle.

[0057] After updating the parameters related to reinforcement learning, a filter target temperature calculation process is executed by the target temperature calculation 703 (S215). The filter target temperature calculation process will be described later. After the filter target temperature calculation process, the operation amount calculation 707 updates the parameters related to heater control (S216). The parameters are the deviation ε used for PID control and the deviation ε used for reinforcement learning, both calculated in the current control cycle. The difference between these deviations will be described later. After updating the parameters related to heater control, the operation amount calculation process ends.

[0058] In the process of step S207, if it is determined that the reinforcement learning is to be stopped because the reinforcement learning stopping condition is satisfied (S207, YES), non-AI control is selected by decision 709, and the learning stopping operation amount calculation process is executed (S217). The learning stopping operation amount calculation process will be described later.

[0059] In addition, in the process of step S206, if it is determined that the current control cycle is not the second step or later (S206, NO), the process of step S215 is executed. The reason for waiting two control cycles here is that, as shown in the above formula (2), in the speed-type PID control, the derivative term is a second-order derivative.

[0060] Furthermore, in step S203, if it is determined that the set target temperature has been changed (S203, YES), the target temperature calculation 703 resets all the filter target temperatures calculated by the filter target temperature calculation process in the previous control cycle (S218), and the process of step S204 is executed. When the filter target temperatures are reset, the various calculation formulas used in the filter target temperature calculation are also reset.

[0061] (Reinforcement learning processing) The above-mentioned reinforcement learning process will be described. FIG. 9 is a flowchart showing the reinforcement learning process according to this embodiment. As shown in FIG. 9, first, the learning 706 calculates the state s in the Actor-Critic method (S301). Note that the state s is the deviation in the current control cycle, that is, the filter target temperature T dsv and the measured temperature T pv The state number is managed based on the deviation from the state value function V. The state number uniquely indicates each threshold interval in the state value table described above. Therefore, if the state s has a deviation of, for example, 1.5 or 1.4, it is treated as state number 2, and if the deviation is 2.5, it is treated as state number 1, and is involved in the determination of the state value function V.

[0062] After calculating the state s, the learning 706 calculates the reward r (S302). The reward r is calculated by the following formula.

[0063]

number

[0064] In the above formula (6), ε is the deviation in the current control cycle, ε th is a predetermined deviation threshold value. This deviation threshold value may be the deviation threshold value in the state value table, the deviation threshold value in formula (1), or a value different from these. According to formula (6) above, when the absolute value of the deviation is the deviation threshold value ε thIf it is equal to or less than this, a good reward (maximum reward) is given, otherwise a bad reward (minimum reward) is given. In this embodiment, the reward r is determined by the deviation. Therefore, the reward r can be said to be a reward for the result of the control of the heater 5. Note that the reward r may be freely implemented according to the goal that the user or the designer of the control device 7 wants to achieve.

[0065] After the reward r is calculated, the decision 709 decides whether or not one control cycle has passed since the start of learning (S303). The purpose of this decision is to wait in order to request the state s of one control cycle before in the update of the parameter θ. If it is decided that one control cycle has passed since the start of learning (S303, YES), a learning parameter update process is executed (S304). The learning parameter update process will be described later. After the learning parameter update process is executed, the learning 706 calculates various control parameters based on the various learning parameters θ updated (calculated) by the learning parameter update process (S305), and the reinforcement learning process ends.

[0066] The method of calculating the various control parameters will now be described. The control parameters calculated here are the proportional band P b , integral gain K i , and the differential gain K d The respective control parameters are calculated using the following equations (7) to (9).

[0067]

number

number

number

[0068] In the above formulas (7) to (9), P bmax , K imax , K dmaxindicates the maximum value that each control parameter can take, in other words, the maximum value of the manipulated variable that each control parameter can output. exp is a mathematical symbol that is synonymous with Napier's constant e, for example, e = 2.7. θ Pb , θ Ki , θ Kd is a learning parameter θ for calculating each control parameter.

[0069] Specifically, the learning parameter θ Pb is the learning parameter corresponding to the proportional band. Ki is a parameter corresponding to the integral gain. Kd is a parameter corresponding to the differential gain. These learning parameters θ are calculated by the learning parameter update process described later. ε indicates the deviation in reinforcement learning.

[0070] In this embodiment, the deviation ε used in the PID control and the deviation ε used in the reinforcement learning may have different values. This is because the deviation ε used to update the learning parameter θ in the reinforcement learning is limited by a limiting process as described later.

[0071] On the other hand, if it is determined that one control cycle has not elapsed since the start of learning (S303, NO), the process of step S305 is executed. The learning parameter θ used in calculating the various control parameters here may be calculated by an existing PID control method.

[0072] (Learning parameter update process) The learning parameter update process will now be described with reference to a flowchart of FIG 10, which shows the learning parameter update process according to this embodiment.

[0073] 10, first, the learning 706 calculates the TD error δ (S401). The TD error δ is calculated by the following formula.

[0074]

number

[0075] In the above formula (10), r indicates the reward value mentioned above. γ indicates the discount rate. V(s) indicates the state value function selected from the state value table shown in Figure 4 based on the set state s. t indicates each time step in the control period.

[0076] After calculating the TD error δ, the learning 706 calculates the eligibility trace D v Update the eligibility trace D (S402). v For example, a known cumulative trace or a replacement update trace can be used.

[0077] Eligibility Trace D v After updating, the learning 706 updates the state value function V (S403). The state value function V is updated using the following equation.

[0078]

number

[0079] In the above formula (11), V(s) on the right side indicates the state value function one control cycle ago. α indicates the learning rate in the critic 812. Also, s in formula (11) indicates all states s, and according to formula (11), all states are updated in one step.

[0080] After updating the state value function V, the limit 705 performs a limiting process to appropriately limit the value of the deviation ε used to update the learning parameter θ (S404). In the limiting process, a predetermined maximum threshold and a predetermined minimum threshold are used, which are stored in advance in the storage device 73 or the like. The maximum threshold is set to, for example, 2, and the minimum threshold is set to, for example, -2, and these settings can be changed as appropriate. If the deviation ε is a value that exceeds the maximum threshold, the limit 705 changes the deviation ε to the maximum threshold. Similarly, if the deviation ε is a value that is less than the minimum threshold, the limit 705 changes the deviation ε to the minimum threshold. With these changes, the deviation ε is limited to a range between the maximum threshold and the minimum threshold. For the sake of explanation, even if the deviation ε is not changed, the deviation ε is referred to as the "deviation after the limiting process."

[0081] After the limiting process, the learning 706 calculates and updates various learning parameters θ (S405). The calculation method of the various learning parameters θ will be described. The learning parameters θ are the above-mentioned θ Pb , θ Ki , θ Kd and θ σ If these are not distinguished, the learning parameter θ * It is expressed as follows. Note that the subscript * is used when the corresponding control parameters are not distinguished. * is calculated by the following formula:

[0082]

number

[0083] In the above formula (12), θ on the right side * indicates the learning parameters one control cycle ago. α * is the learning rate for actor 811, and the learning parameter θ * The learning rate α is a fixed number that indicates how much to move the * can be changed. This is also true for the learning rate α in the critic 812 described above. δ indicates the TD error. D * shows the eligibility trace for actor 811. Eligibility trace D *is calculated by the following formula:

[0084]

number

[0085] In the above formula (13), D on the right side * is the qualification trace D from one control cycle ago * γ denotes the discount rate. λ θ denotes the trace decay parameter for actor 811. * indicates the degree of eligibility. θ It is preferable that the value of is around 0.9 to 0.8. Therefore, the eligibility trace D * γλ for each control period θ The value is reduced by the amount of eligibility. * is calculated by the following formula:

[0086]

number

[0087] In the above formula (14), π is a policy and indicates a normal distribution. The policy π here is the above-mentioned normal distribution based on the tentative first manipulated variable μ (average μ) and the standard deviation σ as shown in formula (4). For example, if this policy π is expressed as a formula, it becomes an equation in which the left side of formula (3) is replaced with π(Δu,ε,θ) instead of f(Δu) (however, θ=θ Pb , θ Ki , θ Kd , θ σ (Note that even in this equation with the left side replaced, the π on the right side is still the circular constant, just like in equation (3). * The calculation method for the learning parameter θ differs depending on the learning parameter. Pb , θ Ki , θ Kd , θ σ The eligibility of each of the above can be calculated using the following formulas (15) to (18), respectively.

[0088]

number

number

number

number

[0089] In the above formulas (15) to (18), the deviation ε indicates the deviation after the limiting process, Δu indicates the provisional second manipulated variable one control cycle before, and μ indicates the provisional first manipulated variable one control cycle before.

[0090] The term (Δu-μ) in the above equations (15) to (18) can take positive or negative values ​​depending on the time step. Therefore, the term (Δu-μ) is determined by the learning parameter θ * In other words, the positive / negative correction value for the update direction of the learning parameter θ * The positive / negative correction values ​​for the positive / negative of are the TD error δ and the deviation ε.

[0091] After the four learning parameters θ are calculated and updated by the above formulas (15) to (18), the learning parameter update process ends. These four learning parameters θ are updated every control period, and the reinforcement learning progresses. The calculated four learning parameters θ are used to calculate the control parameters in the process of step S305.

[0092] In addition, eligibility level e * can be obtained by partial differentiation as shown in equation (14). Therefore, it is not limited to equations (15) to (18). Even if the proportional band in these equations is set as the proportional gain, the degree of suitability e * can be obtained by performing partial differentiation. Similarly, the integral gain may be expressed as the integral time.

[0093] In addition, the value σ included in equation (18) Hmay be omitted. This is because the presence or absence of this affects only the magnitude of the parameter change, not the direction (positive or negative) of the parameter change. In this case, the magnitude of the parameter change is determined by the learning rate α * However, it is not advisable to omit the value "2" in "2ε[t-1]" in equation (17), because omitting it will affect the direction of parameter change.

[0094] (Learning stop operation amount calculation process) The operation amount calculation process for stopping learning will be described. Fig. 11 is a flowchart showing the operation amount calculation process for stopping learning according to this embodiment. In this calculation process, the control parameters and the operation amount are calculated without using the learning parameter θ learned by reinforcement learning.

[0095] 11, first, in decision 709, it is determined whether or not the integral value has been initialized (S501). The integral value is calculated by the following formula.

[0096]

number

[0097] Here, ε indicates a deviation that has not been limited. If it is determined that the integral value has not been initialized (S501, NO), a decision step 709 initializes the stored integral value (S502). After the initialization, a manipulated variable calculation step 707 calculates an integral initial value (S503). The integral initial value is determined by setting the manipulated variable during reinforcement learning (the manipulated variable one control cycle ago)=(standard PID formula) and solving this for the integral value of the standard PID.

[0098] For example, the manipulated variable calculated by AI control (the manipulated variable one control cycle ago) is u AI , the manipulated variable calculated by standard PID control is u PID Then, the following equation (20) is obtained, and by solving this, the initial integral value can be calculated as in equation (21).

[0099]

number

number

[0100] K shown in the above formulas (20) and (21) p ', K i ', K d ' are pre-stored control parameters, and indicate the proportional gain, integral gain, and differential gain in standard PID control. In addition, in equation (21), the control parameter K i ' and the control period Δt are brought to the right-hand side. However, the control parameter K i The integral initial value may be multiplied by K' and / or the control period Δt. p ' to 100 / P b ', that is, the proportional gain may be made the proportional band.

[0101] As another method, for example, the initial integral value may be calculated as in equation (23) by solving the following equation (22).

[0102]

number

number

[0103] K shown in the above formulas (22) and (23) p ', T i ', T d ' are pre-stored control parameters, and indicate the proportional gain, integral time, and derivative time in standard PID control.

[0104] After calculating the integral initial value, the manipulated variable calculation 707 transfers the manipulated variable of one control cycle before as the manipulated variable of the current control cycle to the output 708 (S504). Then, the process of calculating the manipulated variable for stopping learning is completed.

[0105] On the other hand, if it is determined that the integral value has been initialized (S501, YES), the manipulated variable calculation 707 calculates the manipulated variable for standard PID control using the integral initial value that has already been calculated (S505). Calculating the integral initial value and calculating the manipulated variable using the integral initial value enables bumpless switching from speed-type PID to standard PID (position-type PID). The manipulated variable u can be calculated, for example, by the following formula. Note that the following formula is the same as the formula for the manipulated variable used when solving the equations used in the above-mentioned formulas (20) and (21).

[0106]

number

[0107] In the above equation (24), K p ’ , T i ', T d ' are pre-stored control parameters, and indicate the proportional gain, integral time, and differential time in standard PID control. In standard PID control, if the control target, starting state (state of heater 5 when switching to standard PID control), and PID gains are the same, the calculated manipulated variable and temperature change will basically be the same. In other words, the temperature of heater 5 will change with the same tendency. After the manipulated variable is calculated, the process of calculating the manipulated variable for stopping learning is terminated. The PID control law may be changed as appropriate according to the characteristics of the process.

[0108] After the learning stop operation amount calculation process, that is, after step S217 in the flowchart shown in Fig. 8, step S214 is omitted in the current control cycle and subsequent control cycles. This is because reinforcement learning is not performed after the learning stop operation amount calculation process, that is, the operation amount is calculated by standard PID control. The filter target temperature calculation process in S215, which will be described in detail next, may also be omitted.

[0109] (Filter target temperature calculation process) Before describing the filter target temperature calculation process, the filter target temperature will be described. FIG. 12 is a diagram for explaining the filter target temperature according to the present embodiment. FIG. 13 is a diagram for explaining the effect of the filter target temperature. T in FIG. 12 and FIG. 13 sv is the set target temperature, T pv is the measured temperature, T dsv is the filter target temperature, t is time, and T is temperature.

[0110] Filter target temperature T dsv is the set target temperature T sv is corrected to a value appropriate for the variation of the deviation. dsv is calculated so as to be expressed by a straight line or a curve, as shown in Fig. 12. Specifically, the filter target temperature varies in the form of a first straight line state 91 in which the temperature rises in a first straight line shape, a curved line state 92 in which the temperature rises in a curved line shape, and a second straight line state 93 in which the temperature rises in a second straight line shape that is smaller in temperature rise than the first straight line state 91. The filter target temperature T dsv By calculating the manipulated variable u using the above, the measured temperature T pv and filter target temperature T dsv Set the deviation from the target temperature T sv The feedback control can be continued with the deviation smaller than the deviation at the start of the control and the deviation at the end of the control. The deviation is, for example, about 10 or 20. Therefore, there is not a large difference in the deviation value between when the control starts and when the control ends.

[0111] On the other hand, the filter target temperature T dsv If the temperature control is not used, the deviation will be extremely large, for example, around 160 to 180° C. at the beginning of control. Such a large deviation may lead to overshooting of the heater 5. This is because the temperature of the heater 5 is room temperature at the beginning of control, and the set target temperature is set at a high temperature, such as 180 to 200° C.

[0112] The filter target temperature T dsv 14 is a flowchart showing a filter target temperature calculation process according to this embodiment.

[0113] As shown in FIG. 14, first, a decision 709 is made as to whether the filter target temperature T dsv It is determined whether the measured temperature T pv The first temperature threshold is determined based on whether the set target temperature T sv The set target temperature T sv If the measured temperature T is 200°C and the first judgment temperature is 30°C, the first temperature threshold is 170°C. pv If is less than 170°C, the filter target temperature T dsv is judged as the first straight line state 91. It is preferable that the first judgment temperature is appropriately changed according to the performance of the injection molding machine 1, particularly the heater 5, and the like.

[0114] Filter target temperature T dsvis in the first linear state 91 (S601, YES), the target temperature calculation 703 calculates the slope Sa (corrected slope value) (S602). The slope Sa is calculated using actual measurement data previously stored in the storage device 73. The actual measurement data includes the temperature of the heater 5 when the operation amount is a predetermined % (for example, 70 to 100%) after the heater 5 is heated from the normal temperature, and the time data. The actual measurement data also includes the target temperature when PID control is performed with an appropriate gain. Note that the actual measurement data is preferably obtained using this injection molding machine 1 or a similar injection molding machine. Also, although the operation amount has been described as a predetermined %, this ratio may be appropriately set according to the performance of the injection molding machine 1, particularly the heater 5, and the like.

[0115] The target temperature calculation 703 first obtains the value of the slope a of y=ax+b by the least squares method based on the measured data. This value of slope a is a numerical value that indicates the degree of temperature rise per unit time. The target temperature calculation 703 performs a process to obtain the value of the slope a for all heaters 5, and adopts the one with the smallest value of slope a. Note that the one with the smallest value of slope a is preferable, but a value based on the slope a, such as the average value of the slope a, may also be used. After adoption, the target temperature calculation 703 calculates the slope Sa based on the slope a. The slope Sa is calculated based on the filter target temperature T dsv The filter target temperature T in the current control period is increased so that dsv The slope Sa can be calculated using an equation that has a term or coefficient that corrects the slope a used. The correction can be a coefficient that simply multiplies the slope a by a constant, or a coefficient that simply multiplies the slope a by the set target temperature T sv The coefficient is determined by a function based on the difference between the set target temperature T sv and the target temperature set when calculating the slope a, or a function proportional to the square of the difference.

[0116]

number

[0117] In the above formula (25), I sv indicates the target temperature (target temperature included in the measured data) used to obtain the slope a. Xa indicates a coefficient for relaxing the slope a. For example, a value of 0.1 to 1.0 is substituted for Xa. Ya indicates a coefficient for relaxing the slope a by a predetermined percentage per 100°C. For example, 0.15 / 100 is substituted for Ya in order to relax the slope a by 15% per 100°C. Note that the rate at which the slope a is relaxed is not limited to 15%, and may be set appropriately, for example, 5% to 50%, depending on the performance of the injection molding machine 1, particularly the heater 5. Also, Sa, once calculated, can be reused as long as the coefficients in the formula (25) do not change.

[0118] After the slope Sa is calculated, the target temperature calculation 703 calculates the filter target temperature T dsv The filter target temperature T dsv (S603). Note that the filter target temperature T dsv If the target temperature has not been calculated, the target temperature calculation 703 uses the measured temperature T of the heater 5 at the start of control as the initial value. pv The increased measurement temperature T pv Filter target temperature T dsv After the temperature rises, the filter target temperature calculation process ends.

[0119] On the other hand, the filter target temperature T dsv If it is determined that the filter target temperature T dsv It is determined whether the measured temperature T pv The second temperature threshold is determined based on whether the set target temperature T sv The set target temperature T sv If the measured temperature T is 200°C and the second judgment temperature is 0.5°C, the second temperature threshold is 199.5°C. pv If is equal to or greater than the first temperature threshold and less than 199.5°C, the filter target temperature T dsvis judged to be in the curve state 92. The second judgment temperature is not limited to 0.5° C., and it is preferable to change it appropriately depending on the performance of the heater 5 and the like.

[0120] Filter target temperature T dsv If it is determined that the temperature is in the curve state 92 (S604, YES), the target temperature calculation 703 calculates the filtered target temperature T dsv The filter target temperature T dsv In other words, in this embodiment, the curve state 92 is the filter target temperature T dsv becomes the output of the first-order lag system. After the increase, the filter target temperature calculation process ends. In the first-order lag formula, K indicates the gain and T indicates the time constant. The gain K is the set target temperature T sv The time constant T can be calculated as the first judgment temperature / gradient Sa. The input of this first-order lag is a step input (always 1). As described above, the first-order lag uses a step response, but is not limited to a step input, and the input may be changed as appropriate depending on the performance of the heater 5, etc. Also, it is not limited to a first-order lag, and may be a second-order lag.

[0121] On the other hand, the filter target temperature T dsv If it is determined that the filter target temperature T dsv When it is determined that the filter target temperature T is in the second linear state 93 (S604, NO), the target temperature calculation 703 calculates the filter target temperature T so that the filter target temperature T is increased according to the temperature increase condition (temperature change condition). dsv is corrected (S606). The temperature rise conditions specify a predetermined temperature rise in a predetermined number of seconds, that is, a temperature rise per unit time. For example, a condition is defined such that the temperature rise is 0 to 0.9°C in 70 to 100 seconds. The temperature rise conditions are not limited to these values, and are preferably changed as appropriate depending on the performance of the injection molding machine 1, particularly the heater 5. Filter target temperature T dsv After the correction, the filter target temperature calculation process is completed.

[0122] (Comparison between Comparative Example and This Embodiment) FIG. 15 is a diagram for explaining the relationship between the magnitude of the parameter θ and the elapsed time and the relationship between the measured temperature and the elapsed time obtained as a result of the first feedback control using reinforcement learning according to the comparative example. FIG. 16 is a diagram for explaining the relationship between the magnitude of the parameter θ and the elapsed time and the relationship between the measured temperature and the elapsed time obtained as a result of the tenth feedback control using reinforcement learning according to the comparative example. FIG. 17 is a diagram for explaining the relationship between the magnitude of the parameter θ and the elapsed time and the relationship between the measured temperature and the elapsed time obtained as a result of the nth feedback control using reinforcement learning according to the comparative example. FIG. 18 is a diagram for explaining the relationship between the magnitude of the learning parameter θ and the elapsed time and the relationship between the measured temperature and the elapsed time obtained as a result of the first learning feedback control process according to the present embodiment. FIG. 19 is a diagram for explaining the relationship between the magnitude of the learning parameter θ and the elapsed time and the relationship between the measured temperature and the elapsed time obtained as a result of the tenth learning feedback control process according to the present embodiment. FIG. 20 is a diagram for explaining the relationship between the magnitude of the learning parameter θ and the elapsed time and the relationship between the measured temperature and the elapsed time obtained as a result of the nth learning feedback control process according to the present embodiment.

[0123] 15 to 17, the upper graphs show the change over time of a certain parameter θ for calculating a certain gain. Similarly, in Figs. 18 to 20, the upper graphs show the change over time of a certain learning parameter θ for calculating a certain control parameter. In these graphs, the horizontal axis shows time, and the vertical axis shows the magnitude of the parameter θ. Also, in Figs. 15 to 20, the lower graphs show the change over time of a certain learning parameter θ for calculating a certain control parameter. pv In the graph, the horizontal axis represents time and the vertical axis represents temperature.

[0124] In the feedback control using reinforcement learning according to the comparative example, a parameter θ for calculating various gains is calculated by known reinforcement learning, and various gains are calculated based on the parameter θ. Based on the calculated gains, the heater 5 is controlled by known PID+feedforward control.

[0125] When a control different from the learning feedback control process according to the present embodiment is performed, the parameter θ takes only positive values, and as a result increases unilaterally, as shown in Figs. 15 to 17. This becomes more noticeable as the number of learning times increases. Due to the increase in the parameter θ, the measured temperature T pv The larger the target temperature T sv An overshoot exceeding the limit has occurred.

[0126] On the other hand, as shown in Figs. 18 to 20, when the learning feedback control process according to this embodiment is performed, the value of the learning parameter θ is not only positive but also negative as appropriate in the first and tenth times. Therefore, the learning parameter θ does not increase excessively. The learning parameter θ becomes a stable value in the nth time. As a result, in the first, tenth, and nth times, the measured temperature T pv is the set target temperature T sv The measured temperature T pv is the set target temperature T sv It can be seen that the time to reach a steady state and the time to reach a steady state are faster than those in the comparative example. pv The change over time is stored as the above-mentioned log in the storage device 73. The log can be displayed on a display serving as the user I / F 74 by the display 701. By displaying the log on the display, the user can easily understand the learning results.

[0127] (User notification process) Next, the user notification process will be described in detail. Fig. 21 is a flowchart showing the user notification process according to this embodiment. The learning feedback control process is executed for each heater 5, but the user notification process is not executed for each heater 5. The user notification process is triggered by the execution of the learning feedback control process for all heaters 5, and is executed in parallel with these processes.

[0128] First, the display 701 displays a screen indicating that AI control is in progress on the display included in the user I / F 74 (S701). The screen displayed here is a screen showing the temperature and control status of the cylinder 2, for example, a screen 741 as shown in FIG. 22. In this screen 741, the measured temperatures x1 to x4 acquired from each temperature sensor 6 are shown corresponding to their installation positions on an object 742 that typically shows the cylinder 2. It is preferable that the corresponding set target temperatures y1 to y4 are shown below the measured temperatures so that they can be changed by the user. In addition, the screen 741 displays a control status window 743 below the object 742, which shows the current control status, that is, whether AI control is in progress or standard PID control is in progress. At the stage of step S701, the control status window 743 shows that AI control is in progress.

[0129] After the display, the decision 709 decides whether or not formula (1) has been satisfied at least once for each of all heaters 5 (S702). In order to make this decision, the decision 709 may store the switching information in the storage device 73 each time each heater 5 satisfies formula (1). For example, a table in which whether or not formula (1) has been satisfied for each heater 5 corresponds to information on whether or not the reinforcement learning stop condition has been satisfied (or the time until the reinforcement learning stop condition is satisfied) may be stored in the storage device 73 as switching information, and the table may be rewritten as appropriate.

[0130] If it is determined that formula (1) is not satisfied at least once for all of the heaters 5 (S702, NO), then after a predetermined time, it is determined again whether formula (1) is satisfied at least once for all of the heaters 5. For this reason, even if some of the heaters 5 have switched from AI control to standard PID control, the control status window 743 shows that they are under AI control.

[0131] On the other hand, if it is determined that formula (1) is satisfied at least once for all heaters 5 (S702, YES), then in step 709, it is determined for the heater 5 for which formula (1) was satisfied last time whether a predetermined switching time has elapsed since formula (1) was satisfied (S703). th It is determined whether or not the time measurement, which starts after the time limit falls below the threshold, exceeds a preset switching time, such as two minutes.

[0132] If it is determined that the predetermined switching time has not elapsed since formula (1) is satisfied (S703, NO), the display 701 displays a countdown of the remaining time until switching to standard PID control in the control status window 743 as shown in Fig. 23 (S704). After the display, the process of step S703 is executed. The remaining time displayed here is the time obtained by subtracting the time of timing executed by decision 709 from the switching time. It is preferable that the remaining time be displayed in a flashing manner to attract the user's attention, or in a color different from the color of the numbers, words, and other characters displayed on the screen 741.

[0133] On the other hand, if it is determined that the predetermined switching time has elapsed since formula (1) was satisfied (S703, YES), it can be determined that all heaters 5 have been switched to standard PID control. Therefore, as shown in Fig. 24, the display 701 switches the display of the control status window 743 to content indicating that standard PID control is being performed, and displays the standard control screen (S705), and the user notification process ends. Note that the standard control screen continues until the end of injection molding is input by the user, such as when a stop signal for the learning feedback control process for any of the heaters 5 is obtained.

[0134] In addition, the display 701 may change the color of at least one of the text, frame, and background of screen 741 depending on whether the control status window 743 is an AI control screen that displays "AI control in progress" or a standard control screen that displays "standard control in progress," in other words, whether the control status window 743 is "AI control in progress" or "standard PID control in progress."

[0135] Also, on the standard control screen, for example, an execute button 744 for starting injection molding may be displayed below the control status window 743 in a selectable manner by a GUI (Graphical User Interface) or the like. In other words, injection molding may not be executed unless all heaters 5 are switched to standard PID control. Also, if the execute button for starting injection molding is provided as a pressable switch in the injection molding machine 1 included in the user I / F 74, injection molding may not be started even if the execute button is pressed unless all heaters 5 are switched to standard PID control, that is, no reaction may be generated. Also, if the execute button is pressed in a state in which all heaters 5 are not switched to standard PID control, an error message or the like may be displayed on the screen 741 to notify the user that injection molding cannot be executed yet.

[0136] 25, the display 701 may be configured to display all of the control statuses for each temperature measured by the temperature sensor 6 displayed on the object 742, that is, for each heater 5. In this case, the user notification process is executed individually for each heater 5. In this case, in the user notification process, it is determined whether or not formula (1) is satisfied only for the heater 5 being processed in step S702. The display 701 may also display the measured temperatures of the temperature sensor 6 displayed on the object 742 selectable by a GUI or the like, and when a measured temperature is selected, only the control status of the heater 5 uniquely associated with that measured temperature is displayed as a control status window 743.

[0137] Furthermore, when the control mode is switched from AI to non-AI, the display 701 may automatically notify the user of the switch without the user's operation. Examples of the notification method include a method of opening a separate window on the screen 741 to display a pop-up, or a method of using the speaker to emit at least one of a sound (sound effect) and a voice if the user I / F 74 includes a speaker.

[0138] According to the present embodiment described above, the occurrence of overshoot can be reduced, and better feedback control than before is possible. In particular, the use of normal distribution in the calculation of the operation amount u and the update of the learning parameter θ is highly effective. Specifically, a temporary second operation amount Δu based on normal distribution is used in the calculation of the operation amount u. In addition, the term Δu-μ, that is, a term indicating the effect of normal distribution, is used in the calculation of the learning parameter θ in reinforcement learning. As a result, the learning parameter θ moves in the positive direction or the negative direction as appropriate.

[0139] As shown in Figures 15 to 17, overshooting occurs with a high probability when reinforcement learning continues in which the parameter θ for gain calculation moves only in the positive direction. On the other hand, according to the above-mentioned features, it is possible to reduce the occurrence of overshooting by appropriately moving the learning parameter θ in the positive direction and the negative direction. Since the occurrence of overshooting can be reduced, it is possible to shorten the time it takes for the injection molding machine 1 to reach a state in which the resin material can be melted and kneaded.

[0140] In addition, for example, in the case of feedback control using AI that performs machine learning, which is different from the learning feedback control process according to the present embodiment, as shown in FIG. 26, during the period from the timing of the start of injection molding indicated by the dashed line in the figure to the period during molding indicated by the arrow Mo in the figure, the measured temperature T pvThe fluctuation of the temperature may occur as a disturbance. In such a case, as shown in FIG. 27, a large fluctuation occurs in the parameter Pa such as a control parameter. This means that the influence of the disturbance has entered the parameter Pa, which is the learning result. In other words, a behavior different from the behavior of the measured temperature that should be learned has been learned. Examples of disturbances include changes in the outside air temperature, contact of cold air with the heater 5, injection of resin material into the hopper 3, and temperature rise due to high-speed movement of molten resin during injection molding. If such a fluctuation of the parameter Pa occurs, the way in which the temperature of the heater 5 changes also changes, which may affect the quality of the molded product obtained by molding. Note that the vertical axis in FIG. 26 indicates temperature and the horizontal axis indicates time, while the vertical axis in FIG. 27 indicates the parameter value and the horizontal axis indicates time.

[0141] On the other hand, according to this embodiment, when the reinforcement learning stop condition is satisfied, it is possible to switch from AI control to non-AI control, and if injection molding is performed after switching, it is possible to prevent learning a behavior different from the behavior of the measured temperature that should be learned. Therefore, even if a disturbance such as that shown in FIG. 26 occurs during non-AI control, as shown in FIG. 28, there is no fluctuation in the parameter Pa. Therefore, the measured temperature always changes with the same tendency, that is, the way the temperature changes is stable, and the quality of the molded product can be maintained. Therefore, according to this embodiment, it is possible to perform efficient temperature control of the temperature-adjusted part such as the cylinder 2 using machine learning.

[0142] Furthermore, according to this embodiment, when switching from AI control, which is a speed-type PID control, to non-AI control, which is a position-type PID control, the manipulated variable is calculated using the integral initial value, which makes it possible to prevent a sudden change in the manipulated variable when switching from the speed type to the position type.

[0143] Furthermore, according to this embodiment, the user notification process can notify the user whether the current control is AI control or standard PID control, which allows the user to easily recognize the start timing of injection molding.

[0144] Moreover, according to this embodiment, it is possible to limit the magnitude of the deviation used to update the learning parameter θ in reinforcement learning. This limit makes it possible to prevent an extreme difference from occurring between the values ​​at the time of updating the learning parameter θ at the start of control and near the target temperature. This can therefore provide an effect of further reducing the occurrence of overshoot. This effect is made even stronger by the feature that the reward r is calculated so as to give the maximum reward if the deviation is equal to or less than the deviation threshold, and the minimum reward otherwise.

[0145] In addition, according to this embodiment, the set target temperature T sv Filter target temperature T based on dsv The manipulated variable u is calculated and the learning parameter θ is updated using the above formula. This makes it possible to prevent a deviation that easily exceeds 100 from being calculated at the start of control, and to cause the sign of the deviation to change before the set target temperature is reached. Therefore, an excessive increase in the learning parameter θ is avoided, and the effect of reducing the occurrence of overshoot can be achieved.

[0146] Furthermore, according to this embodiment, after or during the learning feedback control process, the parameter θ and the relationship between the measured temperature and the elapsed time shown in Fig. 18 to Fig. 20 can be displayed as a log on the display 701. By viewing the log, the user can easily visually confirm, for example, that no overshoot has occurred.

[0147] In this embodiment, the subject of feedback control has been described as the heater 5 provided in the cylinder 2. However, for example, a nozzle portion as a temperature controlled portion may be provided at the tip of the cylinder 2, and a nozzle heater as a temperature control device provided around the nozzle portion may be the subject of control. Any heater may be the subject of control as long as the measured temperature can be acquired.

[0148] Further, examples of the temperature-controlled parts include a mold, a tie bar, and the like, in addition to the cylinder 2 and the nozzle. There are two types of molds: one in which the temperature is controlled by the injection molding machine 1, and one in which the temperature is controlled outside the injection molding machine 1. The learning feedback control process according to this embodiment is also applicable to these molds. An example of the type in which the temperature is controlled by the injection molding machine 1 is a hot runner mold. For example, the learning feedback control process controls a heater attached to the hot runner mold based on the measurement results of a sensor also attached to the mold so that the temperature becomes the same as that of the nozzle part of the cylinder 2. In this case, the target temperature becomes the temperature of the nozzle part.

[0149] On the other hand, in the type where the temperature is controlled outside the injection molding machine 1, there is one where the entire mold is heated by a fluid such as oil. In this case, the input / output I / F 75 of the control device 7 is communicably connected to an external temperature regulator, and the temperature of the mold is controlled by controlling the temperature regulator through learning feedback control processing. The temperature regulator has functions of heating the fluid, supplying and discharging fluid to and from the mold, and measuring the temperature of the mold. The control device 7 inputs the operation amount calculated by the learning feedback control processing to the temperature regulator, and acquires the measured temperature from the temperature regulator.

[0150] In addition, when the temperature-controlled part is a tie bar, the temperature of the mold can be controlled via the tie bar. FIG. 29 is a schematic vertical cross-sectional view showing the tie bar and the mold according to an application example of this embodiment. As shown in FIG. 29, the tie bar 21 is a long member extending in the conveying direction of the resin material, and is mainly made of a metal having high thermal conductivity. Four tie bars 21 are provided here, and one end of each is connected to the four corners of the surface perpendicular to the conveying direction of the fixed platen 22 to which one of the two molds 24 is attached. In addition, the four tie bars 21 are inserted into the four corners of the surface perpendicular to the conveying direction of the movable platen 23 to which the other of the two molds 24 is attached. The movable platen 23 is configured to be slidable along the tie bar 21. The mold is clamped when the movable platen 23 approaches the fixed platen 22 and the molds 24 are in contact with each other, and the molded product is obtained by filling the mold 24 with resin or the like. The movable platen 23 is moved by, for example, a toggle type, a direct pressure type, a direct lock type, or the like.

[0151] Each of the four tie bars 21 is provided with a pair of a heater 5 and a temperature sensor 6, and the mold 24 can be heated via the tie bars 21, the fixed platen 22, and the movable platen 23. Therefore, even with such tie bars 21, the heater 5 and the temperature sensor 6 are provided, so that the learning feedback control process according to this embodiment is possible.

[0152] In the present embodiment, the learning feedback control process has been described as being targeted at a heating device such as a heater. However, the learning feedback control process may also be targeted at a cooling device. An example of a cooling device is one that cools the lower part of the hopper 3 that is in a heated state by heat transfer from the cylinder 2 heated by the heater 5. Such a cooling device mainly cools the lower part of the hopper 3 using a fluid such as water. The amount of operation for the cooling device corresponds to the opening degree of a valve body such as a solenoid valve that adjusts the flow rate of the fluid.

[0153] In this case, the learning feedback control process cools the lower part of the hopper 3 so that undershoot does not occur, and the measured temperature Tpv is the set target temperature T sv That is, the process is such that the measured temperature T pv Since the aim is to reduce the temperature, it can be said to be the exact opposite process to the learning feedback control process targeted at the heater 5. The cooling device is not limited to one that cools the lower part of the hopper 3, and may be any type that can cool the main parts, such as a cooling device that cools a mold.

[0154] In this case, the target value filter may be calculated as follows. In the first linear state, that is, in the process corresponding to step S604, the gradient Sa after correction is converted to a negative direction (negative value). In addition to this method of converting to a negative direction, the controlled object may be controlled to decrease in temperature by PID control from a temperature (not necessarily normal temperature) higher than the target temperature, and the gradient a (negative value) may be calculated from the temperature of the controlled object when the manipulated variable is a predetermined % (for example, 70 to 100%) and the time data using the least squares method, and the gradient Sa may be calculated from the gradient a. The above-mentioned method may be used as a method of calculating the gradient Sa from the gradient a. The conversion to a negative direction is similar in the second linear state, that is, in the process corresponding to step S606, and the gradient is converted to a negative direction. Specifically, the temperature decrease condition (temperature change condition) is defined as decreasing the temperature by a predetermined temperature in a predetermined number of seconds. In addition, in the curve state, that is, in the process corresponding to step S605, the gain of the first-order lag may be the same as in the case of increasing the temperature, but the time constant is the first judgment temperature / gradient Sa (absolute value).

[0155] Also, although it has been described that the control parameters are calculated by the reinforcement learning process in the learning 706, the calculation of the control parameters may be performed by the manipulated variable calculation 707. In other words, the calculation of the control parameters does not have to be incorporated into the reinforcement learning.

[0156] In this embodiment, the display 701 displays a selection button for the feedback control process according to this embodiment and a selection button for the standard PID control process on the display as a selection screen. If the selection button for the standard PID control process is selected, the manipulated variable calculation in the standard PID control, which is the process in step S505, may be performed. In this case, the user can input, for example, a proportional band or proportional gain value, an integral gain value, a differential gain value, and a target temperature. If various gain values ​​are input, the manipulated variable is calculated based on the gains input in the standard PID control.

[0157] In addition, in this embodiment, it has been described that the AI ​​control is switched to the non-AI control. The AI ​​control here includes not only machine learning but also an expert system. The non-AI control also includes a control in which a parameter is changed or adjusted, such as a control in which a PID gain is adjusted using adaptive control. Other examples of the non-AI control include state feedback control such as PID+feedforward control and LQ optimal control, model predictive control, and data-driven control. Note that, since FIG. 24 and FIG. 25 illustrate the case in which the non-AI control is the standard PID control, "standard PID control in progress" is displayed on the screen 741. Therefore, when the above-mentioned various non-AI controls are executed instead of the standard PID control, the control name is displayed on the screen 741. Naturally, it may simply be displayed as "non-AI control in progress".

[0158] In the non-AI control exemplified above, when non-PID control that does not use PID control is incorporated as non-AI control into the learning feedback control process, in order to realize bumpless switching of the manipulated variable, new terms such as correction terms and initial value terms may be provided in the calculation formula of the manipulated variable without using the integral initial value. For example, as shown in FIG. 31, the manipulated variable u calculated under non-AI control at the beginning of switching is chg For the operation amount correction value u H The sum of these is the operation amount u calculated by the last AI control. AI So, the operation amount correction value u H is calculated and this is used as the operation amount uchg In FIG. 31, the vertical axis indicates the value of the manipulated variable, the horizontal axis indicates time, and each bar graph indicates the manipulated variable for one control period. In FIG. 31, the manipulated variable correction value u H is shown as a fixed value. However, if the deviation of the measured temperature from the set target temperature does not disappear due to the influence of this correction term, the manipulated variable correction value u H It is preferable to decrease it so that it eventually reaches zero.

[0159] <Second embodiment> In this embodiment, unlike the first embodiment, a control device in which PID control incorporated in AI control is of position type rather than speed type, and its processing operation will be described.

[0160] 32 is a block diagram showing the functional configuration of the control device according to this embodiment. The control device 7A according to this embodiment is different from the control device 7 according to the first embodiment in that it has learning 706A instead of learning 706 and has manipulated variable calculation 707A instead of manipulated variable calculation 707. Learning 706A is different from learning 706 according to the first embodiment in that it performs reinforcement learning to calculate and update a policy π, learning parameter θ, and control parameters related to position-type PID control. Manipulated variable calculation 707A is different from manipulated variable calculation 707 according to the first embodiment in that it performs a manipulated variable calculation process to calculate a manipulated variable related to position-type PID control.

[0161] 33 is a flowchart showing the operation amount calculation process according to this embodiment. Operation amount calculation 707A is similar to operation amount calculation 707 in that it calculates a tentative first operation amount μ in the current control cycle based on various control parameters and deviation ε calculated by the reinforcement learning process in step S209. However, operation amount calculation 707A calculates the tentative first operation amount μ by the following equation.

[0162]

number

[0163] Each control parameter in the above formula (26) can be obtained from the above formula (4) and formulas (7) to (9) also in the reinforcement learning executed by learning 706A according to this embodiment.

[0164] In addition, in the speed-type PID control, a provisional second manipulated variable Δu is selected from a normal distribution based on the provisional first manipulated variable μ, etc., but in the position-type PID control, the Δu becomes the manipulated variable u. In other words, the value selected from the normal distribution becomes the manipulated variable u.

[0165] For this reason, the manipulated variable calculation process according to this embodiment differs from the manipulated variable calculation process according to the first embodiment in that step S212A is incorporated instead of step S212. In step S212A, after selecting a tentative second manipulated variable Δu, the manipulated variable calculation 707A determines the selected tentative second manipulated variable Δu as the manipulated variable u[t] of the current control cycle.

[0166] In addition, in the reinforcement learning executed by learning 706A according to this embodiment, the policy π is defined as follows:

[0167]

number

[0168] In the above equation (27), π on the right side represents the circular constant. * , Eligibility Trace D * , eligibility e * The parameter rules such as above can be obtained by the above formulas (12) to (14).

[0169] Learning 706A is a learning parameter θ Pb , θ Ki , θ Kd , θ σ The eligibility of each of the above is calculated using the following formulas (28) to (31).

[0170]

number

number

number

number

[0171] In addition, 100 / P included in the above formulas (28) to (30) bmax Or K imax Elements that are not related to the direction of the gradient (the direction of change in the parameter, whether positive or negative), such as Δt, may be omitted. This also applies to the above equations (15) to (17).

[0172] According to the present embodiment described above, even position-type PID control can be incorporated into the learning feedback control process.

[0173] The AI ​​control in the learning feedback control process described as the first and second embodiments incorporates PID control. However, P control, PI control, or PD control may be incorporated instead of PID control. In these cases, reinforcement learning is not performed for gains not used in each control (for example, derivative gains when PI control is performed). In addition, position-type PID control can be incorporated into the learning feedback control process as its derivative, proportional-differential-leading PID control or derivative-leading PID. In these cases, deviation is usually used to calculate the manipulated variable, but the process is the same as the learning feedback control process according to the second embodiment, except that the measured temperature is used for the leading proportional and derivative terms. Non-AI control is not limited to PID control as long as a control law corresponding to AI control is used.

[0174] The learning feedback control process may be a process in which feedforward control is added to any of P, PI, PD, and PID control. When feedforward control is incorporated into the learning feedback control process, the control parameters to be adjusted include a feedforward term. That is, the feedforward term u ff The tentative first manipulated variable is calculated by adding the feedforward term u ff is the learning parameter θ calculated and updated by reinforcement learning. uff The learning parameter θ uff Eligibility to calculate and update e uff can be obtained by partial differentiation as shown in equation (14). In this case, for example, in a screen 741 shown in Fig. 22, a plurality of selection buttons for selecting the learning feedback control process to be performed by any one of P, PI, PD, PID, and PID+feedforward control may be displayed selectably as a GUI. By pressing any one of the selection buttons, the learning feedback control process according to the control corresponding to the selection button, that is, AI control and non-AI control (standard P control, standard PI control, standard PID control, standard PID+feedforward control, etc.) is executed.

[0175] In addition, various screen displays on the display 701 may be transmitted to a device communicatively connected to the control device 7 via wire or wirelessly, such as a tablet terminal typified by a smartphone, and may be viewable by that device.

[0176] In the above-mentioned embodiments, the program (control program for the injection molding machine) for realizing various functions of the control device 7 has been described as being pre-installed in the control device 7. However, the program may be provided by recording the program in a computer-readable recording medium in an installable or executable format file or the like. The storage medium includes all media that can be read and executed by the computer serving as the control device 7, such as magnetic tape, magnetic disk (hard disk drive, etc.), optical disk (CD-ROM, DVD disk, etc.), magneto-optical disk (MO, etc.), flash memory, etc., which are removable from the control device 7, and further, media that can be transmitted via a network. Note that, as a medium that can be transmitted via a network, there is one stored on a computer such as an external server connected to a network such as the Internet, and this is provided via the network. Note that at least some of the functions of the control device 7 (e.g., learning 706, etc.) may be constructed by an external device of the injection molding machine 1, such as a server or tablet terminal, and data (operation amount u, etc. in the case of learning 706) may be provided to and operated by the injection molding machine 1 from the external device through communication via the network.

[0177] Although several embodiments of the invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope and spirit of the invention, and are included in the scope of the invention and its equivalents described in the claims. [Explanation of symbols]

[0178] 1 injection molding machine 2 cylinder (temperature controlled part) 5 Heater (temperature control device) 6 Temperature Sensor 7 Control device 74 User I / F (display) 701 Display (notification function) 704 Deviation calculation (function to calculate control deviation) 706,706A Learning (function capable of executing first and second control laws) 707, 707A Calculation of operation amount (function capable of executing and correcting the first and second control laws) 709 Judgment (function to select either the first or second control law, function to judge)

Claims

1. Injection molding machines, including: Temperature control device; A control device for controlling the temperature control device; and A temperature-controlled part whose temperature is controlled by the temperature control device; Here, the control device is A function of calculating a control deviation based on the measured temperature and the set target temperature of the temperature-controlled part; A function capable of executing a first control law that calculates an operation amount of the temperature control device as a result of machine learning based on the control deviation, and a second control law that calculates an operation amount of the temperature control device based on the control deviation without using machine learning; and selecting to implement either the first control law or the second control law; has.

2. The control device has a function of notifying whether the temperature adjustment device is controlled using the first control law or the second control law.

2. The injection molding machine of claim 1, further comprising:

3. the injection molding machine has a display; The control device displays on the display a screen indicating which of the first and second control laws is being used for control.

3. The injection molding machine according to claim 2.

4. The control device displays information indicating that the first control law is being used on the screen, and when the first control law is switched to the second control law, switches the information to information indicating that the second control law is being used.

4. The injection molding machine according to claim 3.

5. The control device displays, on the screen, a countdown of a time until the control law is switched from the first control law to the second control law.

4. The injection molding machine according to claim 3.

6. The control device flashes a time until the control law is switched from the first control law to the second control law.

6. The injection molding machine according to claim 5.

7. The control device displays the time until the control law is switched from the first control law to the second control law in a color different from the color of the characters displayed on the screen.

6. The injection molding machine according to claim 5.

8. The control device changes at least one color of a character, a frame, and a background of the screen depending on whether the first control law is used or the second control law is used.

4. The injection molding machine according to claim 3.

9. A plurality of the temperature control devices are provided in the injection molding machine, The control device notifies each of the temperature adjustment devices that the first control law has been switched to the second control law.

3. The injection molding machine according to claim 2.

10. the injection molding machine has a display; The control device displays, on the display, selectable buttons each uniquely indicating one of the plurality of temperature adjustment devices, and when any one of the plurality of buttons is selected, displays a screen indicating whether the temperature adjustment device corresponding to the selected button is controlled using the first or second control law.

10. The injection molding machine according to claim 9.

11. The control device includes: Further, the control method further includes a function of determining whether or not to switch from the first control law to the second control law based on the measured temperature, When it is determined that the first control law should be switched to the second control law, the control method switches from the first control law to the second control law to control the temperature adjustment device.

2. The injection molding machine according to claim 1.

12. The control device determines to switch from the first control law to the second control law when the control deviation reaches a preset allowable threshold range.

12. The injection molding machine according to claim 11.

13. When the control deviation reaches the allowable threshold range, the control device determines to switch from the first control law to the second control law after a preset switching time has elapsed.

13. The injection molding machine according to claim 12.

14. the control device, when switching from the first control law to the second control law, corrects the second control law immediately after the switching based on the first manipulated variable so that a first manipulated variable calculated using the first control law immediately before the switching and a second manipulated variable calculated using the second control law immediately after the switching are substantially equal to each other; 2. The injection molding machine of claim 1, further comprising:

15. the second control law is a position-type PID control law, the control device calculates an initial integral value by solving an integral term of the second control law when the first manipulated variable and the second manipulated variable are assumed to be identical, and corrects the integral term of the second control law to the initial integral value; Immediately after the switching, the temperature control device is controlled using the modified second control law.

15. The injection molding machine according to claim 14.

16. The first control law is a control law of speed-type PID control, and the second control law is a control law of position-type PID control. The injection molding machine according to any one of claims 1 to 15.

17. When the control law is switched from the first control law to the second control law, the control device automatically notifies a user of the switching.

3. The injection molding machine according to claim 2.

18. the injection molding machine has a speaker; When the control law is switched from the first control law to the second control law, the control device automatically emits at least one of a sound and a voice, by the speaker, indicating that the control law has been switched.

18. The injection molding machine of claim 17.

19. A method for controlling an injection molding machine, comprising the steps of: (a) setting a target temperature of a temperature-controlled part of the injection molding machine; (b) measuring the temperature of the temperature controlled part; (c) after steps (a) and (b), selecting either a first control law that calculates an operation amount of the temperature control device as a result of machine learning based on a deviation between a measured temperature of the temperature controllable part and a set target temperature, or a second control law that calculates an operation amount of the temperature control device based on the deviation without using machine learning; and (d) adjusting the temperature of the temperature-adjustable part based on the control law selected in (c).

20. A control program for an injection molding machine to enable a computer to achieve the following functions: (a) a function of selecting either a first control law that calculates an operation amount of the temperature control device as a result of machine learning based on a deviation between a measured temperature of a temperature controllable part of the injection molding machine and a set target temperature, or a second control law that calculates an operation amount of the temperature control device based on the deviation without using machine learning; and (b) A function of adjusting the temperature of the temperature-controlled part based on the control law selected in (a).

Images