Engine control and engine control methods

Abstract

Engine control (1000), including: a drive control unit (4) to drive a motor (1) on the basis of a control command (104), to operate a control target (2000) consisting of the motor and a mechanical load (3) mechanically connected to the motor, and to perform an initialization operation to put the control target into an initial state and an evaluation operation starting from the initial state; a learning unit (7) to learn the control command used in the evaluation operation and a state sensor signal (101) in conjunction with each other, wherein the state sensor signal has detected a state of the control target at a time point in the evaluation operation, and to determine, based on a learning outcome, the control command to be used in the evaluation operation to be performed after the evaluation operation in which the state sensor signal was detected; and a hiring management unit (9) to determine, based on a time at which a first process is to be carried out, a time at which a second process is to be carried out, wherein the first process is a learning operation, an initialization operation and an evaluation operation, wherein the learning operation is an operation of the learning unit, and wherein the second process is a learning operation, an initialization operation and an evaluation operation.

Description

Area

[0001] The present invention relates to a motor controller that automatically sets a control command for controlling a motor. background

[0002] Electronic component assembly facilities, semiconductor manufacturing facilities, etc., perform positioning control, which drives a motor to move a machine, such as an assembly head, to a target length. To shorten positioning time and improve facility productivity, positioning control performs, for example, setting and adjusting parameters that specify a positioning trajectory and control system parameters included in control signals for driving the motor.

[0003] Adjusting these parameters, which sometimes requires a trial-and-error approach, requires time and effort. An additional problem is that the time required for the adjustment work and the results of the adjustment work depend on the knowledge and experience of the operator. As a solution to the problems described above, a technique that automates the parameter adjustment work has been proposed.

[0004] A control parameter adjustment device described in Patent Literature 1 includes a model update unit that updates a control target model using data when the control target is operated. The device also includes a first search unit that searches for a control parameter within a first range to extract candidates for an optimal value by repeating simulations using the updated control target model. The device further includes a second search unit that allows the control target to be repeatedly operated within a second range that is narrower than the first range and that acquires the operation results.

[0005] A machine learning device described in Patent Literature 2 includes a state observation unit that observes state variables of a motor driven and controlled by a motor controller. The device further includes a learning unit that learns conditions associated with correction amounts used to correct commands of the motor controller in accordance with a training data set consisting of the state variables. Patent Literature 3 relates to a machine learning device that performs machine learning of optimizing coefficients of a filter provided in a motor control device that controls the rotation of a motor, a machine tool, a robot, or an industrial machine, and relates to a control system including the machine learning device and a machine learning method. List of cited documentsPatent literature Patent literature 1: JP 2017 - 102 619 A Patent literature 2: JP 2017 - 102 613 A Patent literature 3: DE 10 2019 212 814 A1 Brief descriptionTechnical problem

[0006] Both the device and apparatus described in Patent Literature 1 and Patent Literature 2 automate the parameter setting work by alternately repeating a single evaluation operation of acquiring a sensor value when the motor is driven and a single calculation process using the sensor value acquired in the evaluation operation. The calculation process is a simulation, a learning process, or the like. When the setting is performed by repeating the evaluation operation provided by driving the motor and the calculation process as described above, in some cases, an initialization operation is necessary in which the motor, etc., is brought into an initial state before starting the evaluation operation.In such a case, the problem arises that it is difficult to shorten the time required for automatic adjustment to set the control command to control the motor by repeating the initialization operation, the evaluation operation and the learning operation when performing automatic adjustment.

[0007] The present invention was developed in view of the above. An object of the present invention is to provide a motor controller capable of shortening the time required for automatic tuning to set a control command for controlling a motor by repeating an initialization operation, an evaluation operation, and a learning operation when performing automatic tuning. Solution to the problem

[0008] A motor controller according to the present invention, comprising: a drive control unit for driving a motor based on a control command to operate a control target consisting of the motor and a mechanical load mechanically connected to the motor, and for performing an initialization operation for setting the control target to an initial state and an evaluation operation from the initial state; a learning unit for learning the control command used in the evaluation operation and a state sensor signal in association with each other, the state sensor signal having detected a state of the control target at a time of the evaluation operation, and for determining, based on a learning result, the control command to be used in the evaluation operation to be performed after the evaluation operation in which the state sensor signal was detected;and a setting management unit for determining, based on a timing at which a first process is to be performed, a timing at which a second process is to be performed, wherein the first process is one of a learning operation, the initialization operation, and the evaluation operation, wherein the learning operation is an operation of the learning unit, wherein the second process is one of the learning operation, the initialization operation, and the evaluation operation; Advantageous effects of the invention

[0009] The present invention can provide the motor controller capable of shortening the time required for the automatic adjustment to set the control command for controlling the motor by repeating the initialization operation, the evaluation operation, and the learning operation in performing the automatic adjustment. Short description of the drawings Fig. 1 is a block diagram showing an example of the configuration of a motor controller according to a first embodiment. Fig. 2 is a diagram showing an example of operation times in the engine controller according to the first embodiment. Fig. 3 is a flowchart showing an example of the operation of a setting management unit according to the first embodiment. Fig. 4 is a diagram showing an example of a command pattern according to the first embodiment. Fig. 5 is a block diagram showing an example of the configuration of a learning unit according to the first embodiment. Fig. 6 is a diagram showing an example of deviating time responses according to the first embodiment. Fig. 7 is a diagram showing a configuration example in the case where the processing circuits included in the motor controller according to the first embodiment are composed of a processor and a memory. Fig. 8 is a diagram showing a configuration example in the case where the processing circuits included in the motor controller according to the first embodiment are provided by dedicated hardware. Fig. 9 is a block diagram showing an example of the configuration of a motor controller according to a second embodiment. Fig. 10 is a diagram showing an example of operation times in the engine controller according to the second embodiment. Fig. 11 is a flowchart showing an example of the operation of a setting management unit according to the second embodiment. Fig. 12 is a block diagram showing an example of the configuration of a motor controller according to a third embodiment. Fig. 13 is a diagram showing an example of operation times in the engine controller according to the third embodiment. Fig. 14 is a block diagram showing an example of the configuration of a motor controller according to a fourth embodiment. Fig. 15 is a diagram showing an example of operation times in the engine controller according to the fourth embodiment. Fig. 16 is a flowchart showing an example of the operation of a setting management unit according to the fourth embodiment. Description of embodiments

[0010] Embodiments will be described in detail below with reference to the drawings. Please note that the embodiments described below are examples. The embodiments may be combined if appropriate for implementation. First embodiment

[0011] Fig. 1 is a block diagram showing an example of the configuration of a motor controller 1000 according to a first embodiment. The motor controller 1000 includes a drive control unit 4 and a command generation unit 2. The drive control unit 4 drives a motor 1 such that the motor 1 follows a command signal 103. The command generation unit 2 acquires a command parameter 104 and generates the command signal 103. The motor controller 1000 also includes a learning unit 7. The learning unit 7 acquires a learning start signal 106 and a state sensor signal 101, and determines a learning completion signal 107 and the command parameter 104. The motor controller 1000 further includes a setting management unit 9. The setting management unit 9 acquires the learning completion signal 107 and determines the learning start signal 106 and a command start signal 105.

[0012] The motor 1 generates torque, thrust, or the like with the drive power E output from the drive control unit 4. Examples of the motor 1 include a servo rotary motor, a linear motor, and a stepping motor. A mechanical load 3 is mechanically connected to the motor 1 and driven by the motor 1. The motor 1 and the mechanical load 3 are referred to as a control target 2000. The mechanical load 3 may be any selected device operated, for example, by the torque or thrust generated by the motor 1. The mechanical load 3 may be a device that performs positioning control. Examples of the mechanical load 3 include electronic component mounting equipment and semiconductor manufacturing equipment.

[0013] The drive control unit 4, based on the command signal 103, supplies the drive power E to the motor 1 to drive the motor 1, allowing the motor 1 to follow the command signal 103 to operate the control target 2000, thereby performing an evaluation operation and an initialization operation. The command signal 103 may be at least one of the position, speed, acceleration, current, torque, or thrust of the motor 1. The initialization operation is an operation in which the control target 2000 is set to an initial state. The evaluation operation is an operation starting from the initial state. The state sensor signal 101 acquired in the evaluation operation is used in a learning operation, as described later.The drive control unit 4 may be configured to allow the position of the motor 1 to follow the command signal 103. For example, a feedback control system may be used that calculates the torque or current of the motor 1 based on PID control so that the difference between the detected position of the motor 1 and the command signal 103 becomes small. The drive control unit 4 may use a two-degree-of-freedom control system in which feedforward control is added to the feedback control to drive the motor 1 such that the detected position of the mechanical load 3 follows the command signal 103.

[0014] The command generation unit 2 generates the command signal 103 based on the command parameter 104. The command generation unit 2 generates the command signal 103 in accordance with a timing specified by the command start signal 105. The motor 1 starts an operation at the time the command generation unit 2 generates the command signal 103. In this way, the motor 1 starts an operation in accordance with a timing specified by the command start signal 105. That is, the motor 1 starts an operation in accordance with the command start signal 105. The evaluation operation or the initialization operation is referred to herein as an operation. The initialization operation and the evaluation operation are performed according to their control signals 103.The control signals 103 for the initialization operation and the evaluation operation are generated based on the instruction parameters 104 used in the respective operations. An example of operation of the instruction generation unit 2 will be described later with reference to FIG. Fig. 4 described.

[0015] A state sensor 5 outputs, as a state sensor signal 101, a state variable of at least one of the motor 1 or the mechanical load 3, ie, a result of detecting a state variable of the control target 2000. Examples of the state variable include the position, speed, acceleration, current, torque, and thrust of the motor 1. Examples of the state variable also include the position, speed, and acceleration of the mechanical load 3. Examples of the state sensor 5 include an encoder, a laser displacement sensor, a gyroscope sensor, an acceleration sensor, a current sensor, and a force sensor. The state sensor 5 of Fig. 1 is described as an encoder that detects the position of motor 1 as a state variable.

[0016] The learning unit 7 learns the command parameter 104 used in the evaluation operation in conjunction with the state sensor signal 101 that detected the state of the control target 2000 at the time of the evaluation operation. The learning unit 7 then determines the command parameter 104 to be used in the subsequent evaluation operation to be performed after the evaluation operation during which the learning unit 7 detected the state sensor signal 101. The operation in the learning unit 7 from the start of learning to the determination of the command parameter 104 is called a learning operation. The learning unit 7 starts learning in accordance with the learning start signal 106. The learning start signal 106 is a signal indicating the start time of the learning operation and is determined by the setting management unit 9, as described later. The learning unit 7 further determines the learning completion signal 107.The learning completion signal 107 indicates the time at which the learning unit 7 determines the command parameter 104, that is, the learning completion signal 107 indicates the completion time of the learning operation. The detailed operation of the learning unit 7 will be explained later with reference to FIG. Fig. 5 and Fig. 6 described.

[0017] The setting management unit 9 determines the value of the command start signal 105 indicating the start time of the evaluation operation based on the learning completion signal 107, thereby determining the start time of the evaluation operation based on the completion time of the learning operation. In an operation example of Fig. 2, the setting management unit 9 determines the learning start signal 106 indicating the start time of the learning operation and the command start signal 105 indicating the start time of the initialization operation based on the completion time of the evaluation operation. As will be described later, the setting management unit 9 can detect the completion time of the evaluation operation when the setting management unit 9 detects the elapse of a predetermined period of time since the start time of the evaluation operation. In other words, the setting management unit 9 determines the start times of the learning operation and the initialization operation based on the completion time of the evaluation operation.

[0018] Fig. Fig. 2 is a diagram showing an example of operating times in the engine controller 1000 according to the first embodiment. The horizontal axes in Fig. 2(a) to 2(e) represent time, and the vertical axes in Fig. 2(a) to 2(e) illustrate the learning operation, the operation processing (the initialization operation and the evaluation operation), the learning start signal 106, the learning completion signal 107, and the instruction start signal 105, respectively.

[0019] Now, the relationships between the values ​​of the command start signal 105, the learning start signal 106, and the learning completion signal 107 will be described, and the information indicated by the signals will be described. Fig. 2, the motor 1 starts the operations at the timings when the value of the command start signal 105 becomes 1. The learning unit 7 starts the learning operation at the timings when the value of the learning start signal 106 becomes 1. The learning unit 7 determines that the value of the learning completion signal 107 is 1 at the timings when the learning operation is completed. The signal values ​​of the command start signal 105, the learning start signal 106, and the learning completion signal 107 become 1, after which these values ​​can each be reset to 0 before a command to execute the next operation is given. Each of the above signals is not limited to the above signals, as long as the signals indicate, for example, the start timing and completion timing of the operation.

[0020] The evaluation operation, the initialization operation, and the learning operation are called processes. A cycle, called a learning cycle, includes each process, that is, each of the initialization operation, the evaluation operation, and the learning operation, at least once and is repeated periodically. The learning cycle of Fig. 2 includes each process once. The learning cycle of Fig. 2 includes a single initialization operation, a single evaluation operation, and a single learning operation. The command parameter 104 can be updated in each learning cycle. The motor controller 1000 repeats the learning cycle, thus continuing the learning process. A tuning operation in which the learning cycle is repeatedly performed to search for the command parameter 104 that enables optimal operation of the control target 2000 is hereinafter referred to as automatic tuning.

[0021] Fig. 3 is a flowchart showing an example of the operation of the setting management unit 9 according to the first embodiment. The operation of the engine controller 1000 will be described with reference to Fig. 2 and Fig. 3. After the start of the automatic adjustment, the setting management unit 9 determines in step S101 that the value of the learning start signal 106 is 1 at a time TL111, and determines the start time of a learning operation L11. The learning unit 7 starts the learning operation L11 at the time TL111 in accordance with the learning start signal 106. If the learning unit 7 starts a learning operation, such as the learning operation L11, without detecting the state sensor signal 101 at the time of the evaluation operation after the start of the automatic adjustment, the learning unit 7 may randomly determine the command parameters 104. Alternatively, the determination may be made based on a previous setting. In the case of random determination, an action value function Q, as described later, may be initialized with a random number, and the command parameter 104 specifying an action a t can be determined randomly.

[0022] In step S102, the setting management unit 9 determines that the value of the command start signal 105 is 1 at the time TL111 and determines the start time of an initialization operation IN11. The engine 1 starts the initialization operation IN11 at the time TL111 in accordance with the command start signal 105. The initialization operation IN11 is performed in parallel with the learning operation L11. The term "performed in parallel" hereinafter refers to a state in which two processes are performed at least partially in a temporally overlapping manner. The time required for the initialization operation IN11 is shorter than the time required for the learning operation L11. Therefore, the setting management unit 9 can delay the start time of the initialization operation IN11 beyond the start time of the learning operation L11 so that the waiting time is not prolonged, that is,to the extent that the completion of the initialization operation IN11 occurs no later than the completion of the learning operation L11. The motor 1 completes the initialization operation IN11 at a time TL112 and enters a standby state after the completion of the initialization operation IN11. In the standby state, the motor 1 can be controlled or stopped within a predetermined position range. In addition, the power supply can be interrupted. Next, the learning unit 7 determines that the value of the learning completion signal 107 is 1 at a time TL113, ie, the completion time of the learning operation.

[0023] In step S103, the setting management unit 9 detects the time at which the value of the learning completion signal 107 becomes 1 and detects the time TL113 as the completion time of the learning operation L11. In the operation of step S103, the setting management unit 9 only needs to detect the completion time of the learning operation; for example, it may detect the time at which the learning unit 7 outputs the command parameter 104. In step S104, based on the time TL113, the completion time of the learning operation, the setting management unit 9 determines that the value of the command start signal 105 is 1 at the time TL113, and determines the start time of an evaluation operation EV11 (a first evaluation operation). The engine 1 starts the evaluation operation EV11 at the time TL113 in accordance with the command start signal 105.When the evaluation operation EV11 is completed at a time TL114, the motor 1 enters a standby state.

[0024] In step S105, the setting management unit 9 detects the lapse of a predetermined period of time from the start time of the evaluation operation EV11 and detects a time TL121 as the completion time of the evaluation operation EV11. Here, the predetermined period of time is a period equal to or longer than an estimated value of the time required for the evaluation operation EV11. Note that, in the present embodiment, the time detected by the setting management unit 9 as the completion time of the evaluation operation EV11 is different from the time at which the evaluation operation EV11 is completed and the engine 1 stops. In step S106, the setting management unit 9 determines whether or not to continue the automatic adjustment. If the setting management unit 9 determines to continue the automatic adjustment, the process proceeds to step S107.When the setting management unit 9 determines not to continue the automatic setting, the process proceeds to step S108.

[0025] In one determination method, for example, in step S106, the setting management unit 9 may determine to continue the automatic adjustment when the number of learning cycles performed during the automatic adjustment is less than a predetermined number. The setting management unit 9 may determine not to continue the automatic adjustment when the number is equal to the predetermined number. Further, the setting management unit 9 may determine not to continue the automatic adjustment when the state sensor signal 101 detected in the evaluation operation immediately before step S106 meets a predetermined criterion, and the setting management unit 9 may determine to continue the automatic adjustment when the predetermined criterion is not met.For example, the criterion of the condition sensor signal 101 may require that the convergence time of a positioning operation, which will be described later with reference to . Fig. 6 is less than or equal to a predetermined period of time.

[0026] In step S106, which is performed at time TL121, the setting management unit 9 determines to continue the automatic setting and proceeds to step S107. In step S107, the setting management unit 9 determines, based on time TL121, that is, the completion time of the evaluation operation EV11, that the values ​​of the learning start signal 106 and the command start signal 105 are 1 at time TL121. This operation determines each of the start times of a learning operation L12 (a first learning operation) and an initialization operation IN12 (a first initialization operation). The learning unit 7 and the motor 1 start the learning operation L12 and the initialization operation IN12 at time TL121 in accordance with the learning start signal 106 and the command start signal 105, respectively. The period from time TL111 to time TL121 is referred to as the learning cycle CYC11.

[0027] Thereafter, steps S103 to S107 are repeatedly performed until the setting management unit 9 determines in step S106 not to continue the automatic setting. Subsequently, in step S103, the setting management unit 9 detects a time TL123 in a learning cycle CYC12 as the completion time of the learning operation L12. Subsequently, in step S104, in the learning cycle CYC12, the setting management unit 9 determines, based on the detected completion time of the learning operation L12, that the start time of an evaluation operation EV12 (a second evaluation operation) is time TL123.

[0028] At a time TL1X1, the setting management unit 9 executes step S106 in a learning cycle CYC1X. The setting management unit 9 determines not to continue the automatic setting and proceeds to step S108. In step S108, the setting management unit 9 determines that the value of the learning start signal at time TL1X1 is a value greater than 1 and instructs the learning unit 7 to execute termination processing T1. The instruction to execute termination processing T1 only needs to indicate to the learning unit 7 the start time of termination processing. For example, the value of the learning start signal 106 may be set to a value other than 0 and 1 at the time of issuing an instruction to execute termination processing, or a different signal may be output to the learning unit 7 at the time of issuing an instruction to execute termination processing.

[0029] The learning unit 7 detects the start time of the termination processing T1 and executes the termination processing T1.

[0030] In the termination processing T1, the learning unit 7 can determine, based on the learning operation repeatedly performed in the automatic adjustment, the command parameter 104 that enables the control target 2000 to perform the best operation, that is, it can determine the optimal command parameter 104. The termination processing T1 will be described using an example in which the evaluation operation is a positioning operation in which the control target 2000 is moved by a target length. First, among the command parameters 104 used in the evaluation operations of all learning cycles, those in which a deviation, that is, the difference between the position of the motor 1 and the target travel length, has once fallen within a predetermined allowable range and has subsequently not fallen out of the allowable range are selected in the evaluation operations.The command parameters 104 used in these evaluation operations are defined as candidates for the optimal command parameter 104. From the candidate command parameters 104, the command parameter 104 that enabled the evaluation operation, with the deviation falling within the allowable range in the shortest period of time since the start of the evaluation operation, can then be further selected and defined as the optimal command parameter 104. The deviation will be described later with reference to . Fig. 4 described.

[0031] The learning unit 7 can determine that a command parameter 104 that was not used in the evaluation operations is the optimal command parameter 104. For example, from the command parameters 104 that were used in the evaluation operations in all learning cycles, those command parameters 104 are selected that enabled the operations in which the deviation fell within the allowable range within a predetermined period of time. Subsequently, an average value of the selected command parameters 104 can be determined as the optimal command parameter 104. If the learning unit 7 terminates the processing T1 at a time TL1Y1 in Fig. 2, the automatic adjustment is completed. Note that the termination processing T1 can be omitted. For example, the instruction parameter 104 used in the evaluation operation EV1X can be determined as the optimal instruction parameter 104.

[0032] Each of the evaluation operation, the initialization operation, or the learning operation is a first process and a second process. The setting management unit 9 can determine the timing at which the second process should be performed based on the timing at which the first process should be performed. The timing at which each of the first process and the second process should be performed may be the start time of this process or the completion time of the other process, or it may be a time shifted from the start time or the completion time by a predetermined period.By determining the timing at which the second process is to be performed based on the timing at which the first process is to be performed, the interval between the two processes can be set to be short, thereby shortening a waiting time until the motor 1 or the learning unit 7 starts the process.

[0033] Now the relationships between the processes in the operational example of Fig. 2. In the operating example of Fig. 2, the next evaluation operation is performed using the command parameter 104 determined in the learning operation, and the next learning operation is performed using the state sensor signal 101 obtained as a result of the evaluation operation. Thus, the learning operation and the evaluation operation are not performed in parallel. Furthermore, the evaluation operation and the initialization operation are not performed in parallel because the single control target 2000 performs the evaluation operation and the initialization operation. On the other hand, the initialization operation and the learning operation do not interfere with each other and can therefore be performed in parallel. Furthermore, in the Fig. 2, the time required for the learning operation is longer than the time required for the initialization operation.

[0034] In the operational example of Fig. 2, the setting management unit 9 determines the learning start signal 106 indicating the start time of the learning operation and the command start signal 105 indicating the start time of the initialization operation based on the completion time of the evaluation operation. The learning operation L12 and the initialization operation IN12 start at the completion time of the evaluation operation EV11 detected by the setting management unit 9, and the evaluation operation EV12 starts at the completion time of the learning operation L11. The present embodiment is not limited to this operation.

[0035] For example, the evaluation operation EV11 (the first evaluation operation), which is one of the evaluation operations, may be performed, the learning operation L12 may be performed using the state sensor signal 101 acquired at the time of the evaluation operation EV11, and further, the initialization operation IN12 may be performed in parallel with the learning operation L12. Based on the command parameter 104 (a control command) determined in the learning operation L12, the evaluation operation EV12 (the second evaluation operation), which is the evaluation operation following the evaluation operation EV11, may then be performed from the initial state determined by the initialization operation IN12.Performing the processes as described above makes it possible to perform the initialization operation IN12 and the learning operation L11 in parallel, adjust the timing between the processes, and shorten the waiting time. The motor controller 1000 or the motor control method can be provided in this way.

[0036] Furthermore, the setting management unit 9 can, for example, detect the completion time of the evaluation operation EV11, determine the start time of the learning operation L12 and the start time of the initialization operation IN12 based on the detected completion time of the evaluation operation EV11, adjust the times between the processes, and shorten the waiting time. Furthermore, the setting management unit 9 can, for example, determine that the start time of one of the learning operation L12 and the initialization operation IN12, where one requires a longer time, is the same as or earlier than the start time of the other, and shorten the waiting time.Furthermore, the setting management unit 9 can detect the completion time of one of the learning operation L12 or the initialization operation IN12, one of which completes at the same time as or later than the other, determine the start time of the evaluation operation EV12 based on the detected completion time, and shorten the waiting time. In the above-described operation examples, when the start time of a next process is determined based on the completion time of one process, it is preferable to shorten the interval between the completion time of the previous process and the start time of the next process as much as possible. Even more preferable is to determine that the completion time and the start time are the same or substantially the same.

[0037] The setting management unit 9 detects the completion time of the learning operation L11 by detecting the lapse of a predetermined period of time since the start time of the learning operation L11. However, the present embodiment is not limited to this mode. For example, there is a case where the first process and the second process, that is, two processes, are performed, and an intermediate process including at least one of the initialization operation, the evaluation operation, or the learning operation is performed between the completion of the first process and the start of the second process.In this case, the setting management unit 9 may estimate the time required for the intermediate process in advance and determine that the start time of the second process follows the time at which the estimated time required to perform the intermediate process has elapsed since the completion time of the first process. Through this operation, the start time of the second process can be set with the estimated time required for the intermediate process as a guide, shortening the waiting time, and thereby shortening the time required for automatic adjustment. Furthermore, the setting management unit 9 may, as shown in the embodiment described with reference to FIG. Fig. 2, the completion time of the learning operation can be more accurately determined using the learning completion signal 107 and the exact time at which the next process should be started. In this way, the waiting time can be shortened.

[0038] Next, the operation of the command generation unit 2 for generating the command signal 103 based on the command parameter 104 will be described. Fig. 4 is a diagram showing an example of a command pattern according to the first embodiment. The command pattern is a pattern indicating the command value of the motor 1 in a time series. The command value of the command pattern is one of the position, speed, acceleration, or jerk of the motor 1. The command value may be equal to the value of the command signal 103. In the operation example of Fig. 4, the command signal 103 shown in a time series is the command pattern.

[0039] In the evaluation operation, the command parameter 104 specifies a command pattern together with an operating condition. In other words, when the command parameter 104 and the operating condition are specified, a command pattern is uniquely determined. The operating condition is a requirement for the operation of the engine 1 at the time of the evaluation operation, and it is constant in the evaluation operation that is repeatedly performed during the automatic tuning. On the other hand, the command parameter 104 can be updated in each learning cycle during the automatic tuning. In the engine controller 1000 of Fig. 1, the command generation unit 2 generates the command signal 103 based on the command parameter 104. As a result, the drive control unit 4 drives the motor 1 based on the command parameter 104. Furthermore, the drive control unit 4 can drive the motor 1 based on the command pattern. As described above, when the command signal 103, the command parameter 104, or the command pattern is defined as a control command, that is, a command to control the motor 1, the drive control unit 4 drives the motor 1 based on the control command.

[0040] The horizontal axes in Fig. 4(a) to 4(d) represent time. The vertical axes in Fig. 4(a) to 4(d) indicate the position, speed, acceleration, and jerk of motor 1, respectively, which represent the command signal 103. The speed, acceleration, and jerk are a first derivative, a second derivative, and a third derivative of the position of motor 1, respectively. The intersection points of the horizontal axes and the vertical axes represent a time 0, that is, a command start time at which the evaluation operation on the horizontal axes begins. The operating condition in the operating example of Fig. 4 is that the target path length is D. That is, the position of motor 1 at the start time 0 of the evaluation operation is 0, and the position of motor 1 at a time t=T1+T2+T3+T4+T5+T6+T7, that is, at the end time, is D.

[0041] The command pattern of Fig. 4 is divided from time 0, that is, from the command start time, to the end time, in this order, into a first section to a seventh section. If n is a natural number from 1 to 7, the time period of an n-th section is called the n-th time period Tn. In the operating example of Fig. 4, seven parameters from a first time period T1 to a seventh time period T7 are the command parameters 104. The acceleration magnitudes in the second and sixth sections are Aa and Ad, respectively. These accelerations are constant within the sections. Note that the acceleration magnitude Aa and the acceleration magnitude Ad are dependent variables of the command parameters 104 and have no degree of freedom in their setting.

[0042] The command signal 103 at a time t (0≤t <T1) im ersten Abschnitt kann wie folgt berechnet werden. Eine Beschleunigung A1, eine Geschwindigkeit V1 und eine Position P1 werden erhalten, indem der Ruck, die Beschleunigung A1 bzw. die Geschwindigkeit V1 zwischen dem Zeitpunkt 0 des ersten Abschnitts und dem Zeitpunkt t im ersten Abschnitt in Bezug auf die Zeit integriert werden. Im ersten Abschnitt nimmt die Beschleunigung mit einer konstanten Rate zu und erreicht zum Zeitpunkt T1 den Beschleunigungsbetrag Aa. Auf diese Weise ist der Ruck im ersten Abschnitt ein Wert, der erhalten wird, indem der Beschleunigungsbetrag Aa durch T1 dividiert wird. Auf diese Weise können die Beschleunigung A1, die Geschwindigkeit V1 bzw. die Position P1 wie in den Formeln (1) bis (3) angegeben berechnet werden. [Formula 1] A1(t)=∫0tAaT1dτ [Formula 2] V1(t)=∫0tA1(τ)dτ [Formula 3] P1(t)=∫0tV1(τ)dτ

[0043] The command signal 103 at a time t in the second section (T1≤t <T1+T2), d. h. eine Beschleunigung A2, eine Geschwindigkeit V2 und eine Position P2, können wie jene im ersten Abschnitt wie in den Formeln (4) bis (6) angegeben berechnet werden. [Formula 4] A2(t)=Aa [Formula 5] V2(t)=V1(T1)+∫T1tA2(τ)dτ [Formula 6] P2(t)=P1(T1)+∫T1tV2(τ)dτ

[0044] The command signal 103 at a time t in the third section (T1+T2≤t <T1+T2+T3), d. h. eine Beschleunigung A3, eine Geschwindigkeit V3 und eine Position P3, können wie jene im ersten Abschnitt wie in den Formeln (7) bis (9) angegeben berechnet werden. [Formula 7] A3(t)=Aa+∫T1+T2t−AaT3dτ [Formula 8] V3(t)=V2(T1+T2)+∫T1+T2tA3(τ)dτ [Formula 9] P3(t)=P2(T1+T2)+∫T1+T2tV3(τ)dτ

[0045] The command signal 103 at a time t in the fourth section (T1+T2+T3≤t <T1+T2+T3+T4), d. h. eine Beschleunigung A4, eine Geschwindigkeit V4 und eine Position P4, können wie jene im ersten Abschnitt wie in den Formeln (10) bis (12) angegeben berechnet werden. [Formula 10] A4(t)=0 [Formula 11] V4(t)=V3(T1+T2+T3)+∫T1+T2+T3tA4(τ)dτ [Formula 12] P3(t)=P3(T1+T2+T3)+∫T1+T2+T3tV4(τ)dτ

[0046] The command signal 103 at a time t in the fifth section (T1+T2+T3+T4≤t <T1+T2+T3+T4+T5), d. h. eine Beschleunigung A5, eine Geschwindigkeit V5 und eine Position P5, können wie jene im ersten Abschnitt wie in den Formeln (13) bis (15) angegeben berechnet werden. [Formula 13] A5(t)=∫T1+T2+T3+T4t−AdT5dτ [Formula 14] V5(t)=V4(T1+T2+T3+T4)+∫T1+T2+T3+T4tA5(τ)dτ [Formula 15] P5(t)=P4(T1+T2+T3+T4)+∫T1+T2+T3+T4tV5(τ)dτ

[0047] The command signal 103 at a time t in the sixth section (T1+T2+T3+T4+T5≤t <T1+T2+T3+T4+T5+T6), d. h. eine Beschleunigung A6, eine Geschwindigkeit V6 und eine Position P6, können wie jene im ersten Abschnitt wie in den Formeln (16) bis (18) angegeben berechnet werden. [Formula 16] A6(t)=−Ad [Formula 17] V6(t)=V5(T1+T2+T3+T4+T5)+∫T1+T2+T3+T4+T5tA6(τ)dτ [Formula 18] P6(t)=P5(T1+T2+T3+T4+T5)+∫T1+T2+T3+T4+T5tV6(τ)dτ

[0048] The command signal 103 at a time t in the seventh section (T1+T2+T3+T4+T5+T6≤t≤T1+T2+T3+T4+T5+T6+T7), ie, an acceleration A7, a velocity V7 and a position P7, can be calculated like those in the first section as given in formulas (19) to (21). [Formula 19] A7(t)=−Ad+∫T1+T2+T3+T4+T5+T6tAdT7dτ [Formula 20] V7(t)=V6(T1+T2+T3+T4+T5+T6)+∫T1+T2+T3+T4+T5+T6tA7(τ)dτ [Formula 21] P7(t)=P6(T1+T2+T3+T4+T5+T6)+∫T1+T2+T3+T4+T5+T6tV7(τ)dτ

[0049] At time t = T1 + T2 + T3 + T4 + T5 + T6 + T7, i.e., at the final time, the velocity V7 is 0, and furthermore, the position P7 is equal to the target path length D. Thus, formulas (22) and (23) apply at the final time. The acceleration magnitude Aa in the second section and the acceleration magnitude Ad in the sixth section can be determined from formulas (22) and (23). [Formula 22] V7=0 [Formula 23] P7=D

[0050] The above explanations are an example of the operation of the command generation unit 2, which generates the command signal 103 based on the command parameter 104 and the operating condition. In the first section, the third section, the fifth section, and the seventh section, the jerk has a constant value other than zero. That is, the first period T1, the third period T3, the fifth period T5, and the seventh period T7 each indicate a period in which the jerk has the constant value other than zero. The constant value other than zero means a constant value greater than 0 or a constant value less than 0. In these sections, the amount of jerk may be used as the command parameter 104 instead of the time period Tn. For example, if the amount of jerk in the first section is defined as J1, the first period T1 and the jerk J1 have the relationship shown in Formula (24). [Formula 24] J1=AaT1

[0051] Determining that the duration of a section in which the jerk has a constant non-zero value is the command parameter 104 is equivalent to determining that the magnitude of the jerk in the section in which the jerk has the constant non-zero value is the command parameter 104. As in the above example, the command parameter 104 only needs to determine the command pattern in combination with the operating condition. As in the example described here, there may be a variety of options for how the command parameter 104 can be selected even under the same operating condition. The manner of selecting the command parameter 104 is not limited to the manner described in the present embodiment.

[0052] Now learning unit 7 is described. Fig. 5 is a block diagram showing an example of the configuration of the learning unit 7 according to the first embodiment. The learning unit 7 includes a reward calculation unit 71, a value function update unit 72, a decision-making unit 73, a learning completion signal determination unit 74, a command parameter determination unit 75, and an evaluation sensor signal determination unit 76. The reward calculation unit 71 calculates a reward r for the command parameter 104 used in the evaluation operation based on an evaluation sensor signal 102. The value function update unit 72 updates an action value function in accordance with the reward r. The decision-making unit 73 uses the action value function updated by the value function update unit 72 to determine an evaluation candidate parameter 108, that is,a candidate for the command parameter 104 to be used in the evaluation operation. Based on the evaluation candidate parameter 108, the command parameter determination unit 75 determines the command parameter 104 to be used in the evaluation operation. The evaluation sensor signal determination unit 76 determines the evaluation sensor signal 102 from the state sensor signal 101 at the time of the evaluation operation. The decision-making unit 73 may determine the command parameter 104 instead of the evaluation candidate parameter 108; in this case, the command parameter determination unit 75 may be omitted from the learning unit 7.

[0053] The learning unit 7 can learn the command signal 103 or the command pattern instead of the command parameter 104. In this way, the learning unit 7 can learn the control command. In this case, the learning unit 7 includes a control command determination unit instead of the command parameter determination unit 75. The control command determination unit determines the control command to be used in the evaluation operation based on the evaluation candidate parameter 108. While the command pattern and the command signal 103 each specify the movement of the motor 1, a combination of the command parameter 104 and the operating condition specifies the movement of the motor 1. In this way, the amount of data is smaller when the learning unit 7 learns the command parameter 104 than when the learning unit 7 learns the command pattern or the command signal 103.Therefore, when the learning unit 7 learns the command parameter 104, the amount of calculation and the calculation time of the learning unit 7 can be reduced. That is, when the command parameter 104 is learned, the learning operation can be performed efficiently.

[0054] The evaluation sensor signal determination unit 76 can derive the evaluation sensor signal 102 by subjecting the state sensor signal 101 to calculation processing such as extraction, conversion, calibration, and filtering. For example, a signal obtained by temporally extracting the state sensor signal 101 at the time of the evaluation operation from the entire state sensor signal 101 can be used as the evaluation sensor signal 102. In this case, the state sensor signal 101 can be extracted between the start and completion of the evaluation operation. Furthermore, the state sensor signal 101 can be extracted from the completion of the evaluation operation until a predetermined period of time has elapsed to assess the influence of vibration immediately after the completion of the evaluation operation.To determine the evaluation sensor signal 102, the evaluation sensor signal determining unit 76 may be configured to calibrate the acquired state sensor signal 101 to thereby remove any offset. The evaluation sensor signal determining unit 76 may be configured to provide a low-pass filter through which the state sensor signal 101 can pass, thereby removing noise. Through these parts of the signal processing, the accuracy of the learning operation can be improved. The reward calculation unit 71 may be configured to calculate the reward r based on the state sensor signal 101, omitting the evaluation sensor signal determining unit 76.

[0055] The learning unit 7 can perform the learning process using various learning algorithms. The present embodiment describes a case where reinforcement learning is applied as an example. In reinforcement learning, an agent observes a current state in a certain environment and determines an action to be taken. The agent selects an action and receives a reward from the environment. Through a series of actions, the agent learns a strategy that can achieve the greatest reward. Q-learning, TD-learning, etc. are known as typical reinforcement learning methods. In Q-learning, for example, a typical update formula of an action value function Q(s, a) is expressed by formula (25). The update formula can be expressed by an action value table. [Formula 25] Q(st,at)←Q(st,at)+α(rt+1+γ maxa Q(st+1,a)−Q(st,at))

[0056] In formula (25) s tfor an environment at a time t, and a t represents an action at time t. The action a t changes the environment to s t+1 . r t+1 represents a reward given due to the environmental change, γ represents a penalty factor, and α represents a learning rate. The value of the penalty factor γ is in a range of more than 0 and 1 or less (0<γ≤1), and the value of the learning rate α is in a range of more than 0 and 1 or less (0<α≤1). When Q-learning is applied, the action a t the determination of the command parameter 104. In practice, an action that determines the evaluation candidate parameter 108 may be the action a t be. The environment s t includes the operating condition, the initial position of motor 1, etc.

[0057] The operation of the reward calculation unit 71 will be explained with reference to Fig. 6 described. Fig. Fig. 6 is a diagram showing an example of deviating time responses according to the first embodiment. The deviation in Fig. 6 is the difference between the target path length and the position of motor 1 when motor 1 is operating in the evaluation operation. In Fig. 6, the horizontal axes represent time, and the vertical axes represent deviation. The intersection points of the vertical axes and the horizontal axes represent a time 0 at which the deviation on the vertical axes is 0 and which is the starting time of the evaluation operation on the horizontal axes. In Fig. 6, IMP represents a limit value of the permissible range of deviation and indicates the amount of error at the operating accuracy permissible for the mechanical load 3.

[0058] The deviation in Fig. 6(a) falls within the permissible range until a time Tst1 has elapsed since the start of the evaluation operation, after which the deviation fluctuatingly converges within the permissible range. The deviation in Fig. 6(b) falls within the permissible range until a time Tst2 has elapsed since the start of the evaluation operation, after which the deviation is temporarily outside the permissible range and subsequently falls back into the permissible range. The deviation in Fig. 6(c) falls within the permissible range until a time Tst3 has elapsed since the start of the evaluation operation, after which the deviation fluctuatingly converges within the permissible range. The time Tst1, the time Tst2, and the time Tst3 have the relationships (Tst1>Tst3>Tst2), which indicates that the value of the time Tst2 is smaller than the value of the time Tst3, and the value of the time Tst3 is smaller than the value of the time Tst1. The deviation in Fig. 6(c) converges faster than the deviations in Fig. 6(a) and Fig. 6(b).

[0059] Changing the way in which the reward calculation unit 71 calculates the reward r makes it possible to select the optimal characteristic of the command parameter 104 obtained as a learning result. For example, to learn the command parameter 104 that allows the deviation to converge at a high speed, the reward calculation unit 71 may give a high reward r when the period from the start of the operation to the time the deviation falls within the allowable range is less than or equal to a predetermined period. The shorter the period from the start of the operation to the time the deviation falls within the allowable range, the larger the reward r can be. The reward calculation unit 71 may calculate the inverse of the period from the start of the evaluation operation to the time the deviation falls within the allowable range as the reward r.If the deviation that fell within the allowable range falls out of the allowable range as shown in . Fig. 3(b), a small reward r can be given so that the command parameter 104 that does not cause vibration of the mechanical load 3 is learned. This is the explanation of the example of the operation of the reward calculation unit 71 shown in Fig. 6 is shown.

[0060] When the reward r is calculated, the value function updating unit 72 updates the action value function Q in accordance with the reward r. The decision-making unit 73 determines the action a as the evaluation candidate parameter 108. t , which results in the largest updated action value function Q, ie the command parameter 104, which results in the largest updated action value function Q.

[0061] The description of the Fig. The motor controller 1000 shown in FIG. 1 is described using an example in which the learning algorithm used by the learning unit 7 is reinforcement learning. In the present embodiment, the learning algorithm is not limited to reinforcement learning. Learning algorithms such as supervised learning, unsupervised learning, or semi-supervised learning can also be used. Furthermore, deep learning for learning feature extraction itself can be used as the learning algorithm. Furthermore, machine learning can be performed using another method, such as a neural network, genetic programming, functional logic programming, a support vector machine, or Bayesian optimization.

[0062] Fig. 7 is a diagram showing a configuration example when the processing circuits included in the motor controller 1000 according to the first embodiment consist of a processor 10001 and a memory 10002. When the processing circuits consist of the processor 10001 and the memory 10002, the functions of the processing circuits of the motor controller 1000 are implemented by software, firmware, or a combination of software and firmware. The software or firmware is described as a program and stored in the memory 10002. In the processing circuits, the functions are implemented by the processor 10001, which reads and executes the programs stored in the memory 10002. That is, the processing circuits include the memory 10002 for storing the programs that lead to the execution of processing in the motor controller 1000.It can be said that these programs cause a computer to perform procedures and processes in the engine control 1000.

[0063] The processor 10001 may be a central processing unit (CPU), a processing unit, an arithmetic unit, a microprocessor, a microcomputer, a digital signal processor (DSP), or the like. The memory 10002 may be a non-volatile or volatile semiconductor memory, such as a random access memory (RAM), a read-only memory (ROM), a flash memory, an erasable programmable ROM (EPROM), or an electrically erasable EPROM (EEPROM) (registered trademark). The memory 10002 may be a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a digital versatile disk (DVD), or the like.

[0064] Fig. Fig. 8 is a diagram showing a configuration example in the case where dedicated hardware provides processing circuits included in the motor controller 1000 according to the first embodiment. When dedicated hardware provides the processing circuits, the Fig. For example, the processing circuits 10003 shown in Figure 8 may be a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or a combination thereof. The functions of the motor controller 1000 may be implemented by the processing circuits 10003 on a single-function basis, or two or more of the functions may be implemented collectively by the processing circuits 10003. The motor controller 1000 and the control target 2000 may be connected to each other via a network. The motor controller 1000 may be located on a cloud server.

[0065] A plurality of control objectives similar to the control objective 2000 may be provided, wherein the evaluation operations may be performed in parallel by the plurality of control objectives to efficiently advance the learning. For example, within the period of the evaluation operation EV11 of Fig. 2 Evaluation operations are performed in parallel by the plurality of control targets to acquire data including a plurality of sets of the command parameter and the evaluation sensor signal. Next, within the learning operation period L12, the action value function Q is updated multiple times using the data acquired within the evaluation operation period EV11 to determine a plurality of command parameters. Further, within the evaluation operation period EV12, evaluation operations are performed by the plurality of control targets using the plurality of command parameters determined within the learning operation period L12. By performing the learning cycle in this way, a plurality of evaluation operations can be performed in parallel.For operating the learning unit to determine a plurality of command parameters, a method described later in a fourth embodiment may be used. While the learning cycle is repeated, some or all of the plurality of control targets may be changed, or the number of control targets constituting the plurality of control targets may be increased or decreased.

[0066] The motor controller 1000 that has performed learning using data acquired from the control target 2000 can be connected to another control object and perform further learning using data acquired from the other control object. The motor controller can be configured using a trained learning device that includes the results of the learning according to the present embodiment. The trained learning device can be implemented by a learned program that determines the command parameter 104 using the action value function Q updated by learning. Furthermore, the trained learning device can be implemented by learned data that includes the result of setting the command parameter 104.By using the motor controller with the trained learning device, it is possible to provide a motor controller that can utilize learning results in a short time. The method described in the present embodiment makes it possible to automatically adjust the command parameter 104 of the motor controller or to establish the motor controller. The automatic adjustment according to the present embodiment only needs to automate at least part of the adjustment work and does not preclude human operation or human participation.

[0067] As described above, the motor controller 1000 according to the present embodiment includes the drive control unit 4, the learning unit 7, and the setting management unit 9. The drive control unit 4 drives the motor 1 based on the command parameter 104 (the control command) to operate the control target 2000 consisting of the motor 1 and the mechanical load 3 mechanically connected to the motor 1. Subsequently, the drive control unit 4 performs the initialization operation of setting the control target 2000 to the initial state and the evaluation operation starting from the initial state. The learning unit 7 learns the command parameter 104 (the control command) and the state sensor signal 101 in conjunction with each other, where the command parameter 104 is used in the evaluation operation, and the state sensor signal 101 has detected the state of the control target 2000 at the time of the evaluation operation.Subsequently, based on the learning result, the learning unit 7 determines the command parameter 104 (the control command) to be used in the evaluation operation to be performed after the evaluation operation in which the state sensor signal 101 was detected. Based on the timing at which the first process, which is one of the initialization operation, the evaluation operation, and the learning operation, is to be performed, the setting management unit 9 determines the timing at which the second process, which is one of the initialization operation, the evaluation operation, and the learning operation, is to be performed. In this way, the timing at which the first process and the second process are performed can be adjusted to shorten the waiting time and thus efficiently perform the setting of the command parameter 104 (the control command).

[0068] The motor control method according to the present embodiment drives the motor 1 based on the command parameter 104 (the control command) to operate the control target 2000 consisting of the motor 1 and the mechanical load 3 mechanically connected to the motor 1. Subsequently, the method performs the initialization operation of setting the control target 2000 to the initial state and the evaluation operation starting from the initial state.The method then performs the following learning operation: learning the command parameter 104 and the state sensor signal 101 in conjunction with each other, wherein the command parameter 104 is used in the evaluation operation and wherein the state sensor signal 101 has detected the state of the control target 2000 at the time of the evaluation operation; and, based on the learning result, determining the command parameter 104 to be used in the evaluation operation to be performed after the evaluation operation in which the state sensor signal 101 was detected. The learning operation is an operation between the start of learning and the determination of the command parameter 104.Subsequently, based on the timing at which the first process, which is one of the learning operation, the initialization operation, and the evaluation operation, is to be performed, the timing at which the second process, which is one of the learning operation, the initialization operation, and the evaluation operation, is to be performed, is determined. Thus, a motor control method capable of efficiently performing automatic adjustment can be provided.

[0069] The timing at which the second process is to be performed may be the same as or subsequent to the timing at which the first process is to be performed. As a result, the timing at which the detected first process is to be performed can be used to determine the timing at which the second process is to be performed, thereby shortening the interval between processes more reliably. Even if the time required for the first process changes, for example, the timing at which the second process is to be performed can be adjusted in response to the change. It is preferable to shorten the interval between the completion time of the first process and the start time of the second process as much as possible.Even more preferably, it is determined that the completion time of the first process and the start time of the second process are the same or substantially the same.

[0070] In this way, the present embodiment can provide a motor controller capable of shortening the time required for automatic adjustment to set the control command for controlling the motor by repeating the initialization operation, the evaluation operation, and the learning operation in performing automatic adjustment. Second embodiment

[0071] Fig. 9 is a block diagram showing an example of the configuration of a motor controller 1000a according to a second embodiment. Fig. 9(a) shows a configuration example of the entire motor controller 1000a. Fig. 9(b) shows a configuration example of a learning unit 7a. The motor controller 1000a includes the learning unit 7a instead of the Fig. 1, and includes a setting management unit 9a instead of the setting management unit 9 of Fig. 1. The configuration of the learning unit 7a is obtained by omitting the learning completion signal determination unit 74 from the configuration of the learning unit 7. Furthermore, the setting management unit 9a of Fig. 9 the completion times of the evaluation operation and the initialization operation based on the state sensor signal 101. The setting management unit 9a of Fig. 9 uses the completion time of the initialization operation to determine the start time of the evaluation operation. When describing the Fig. 9 shown engine control 1000a, components which are identical to those of Fig. 1 are identical or correspond to them, are designated by the same reference numerals.

[0072] Fig. Fig. 10 is a diagram showing an example of operating times in the motor controller 1000a according to the second embodiment. The horizontal axes in Fig. 10(a) to 10(d) represent time, and the vertical axes in Fig. 10(a) to 10(d) illustrate the learning operation, the operation processing (the initialization operation and the evaluation operation), the learning start signal 106 and the instruction start signal 105, respectively. The relationships between the signal values ​​of the instruction start signal 105 and the learning start signal 106 and the information indicated by the signals are the same as those shown in the Fig. 2 will be described.

[0073] In the operational example of Fig. 10, the time required for the initialization operation is longer than the time required for the learning operation. Furthermore, the initialization operation is completed after the learning operation. Therefore, the start time of the evaluation operation is determined based on the completion time of the initialization operation instead of the completion time of the learning operation. The completion times of the initialization operation and the evaluation operation are detected based on the state sensor signal 101. These aspects are differences from the operation example of Fig. 2.

[0074] Fig. Fig. 11 is a flowchart showing an example of the operation of the setting management unit 9a according to the second embodiment. The operation of the engine controller 1000a will be described with reference to Fig. 10 and Fig. 11. After starting the automatic adjustment, the adjustment management unit 9a determines in step S201 that the value of the command start signal 105 is 1 at time TL211, and determines that the start time of an initialization operation IN21 is time TL211. The motor 1 starts the initialization operation IN21 at time TL211 in accordance with the command start signal 105. Thereafter, the initialization operation IN21 is completed at time TL213.

[0075] In step S202, the setting management unit 9a determines that the value of the learning start signal 106 is 1 at time TL211, and determines that the start time of a learning operation L21 is time TL211. The learning unit 7a starts the learning operation L21 at time TL211 in accordance with the learning start signal 106. Thereafter, the learning operation L21 is completed at time TL212. As with the learning operation L11 of Fig. 2, the learning unit 7a can determine the command parameter 104 based on a previous setting or randomly during the learning operation L21. The initialization operation IN21 and the learning operation L21 are performed in parallel. Since the time required for the initialization operation IN21 is longer than the time required for the learning operation L21, the time TL213 is a time following the time TL212. As in the operation example of Fig. 2, the start time of the learning operation L21 can be delayed beyond the start time of the initialization operation IN21 so that the waiting time is not extended.

[0076] In step S203, the setting management unit 9a detects, based on the state sensor signal 101, the time TL213 as the completion time of the initialization operation IN21. In step S204, the setting management unit 9a determines, based on the detected completion time of the initialization operation IN21, that the value of the command start signal 105 is 1 at time TL213, and determines the start time of an evaluation operation EV21 (a first evaluation operation). The engine 1 starts the evaluation operation EV21 at time TL213 in accordance with the command start signal 105. Thereafter, the evaluation operation EV21 is completed at time TL221.

[0077] In step S205, the setting management unit 9a detects, based on the state sensor signal 101, the time TL221 as the completion time of the evaluation operation EV21. As in step S106 of Fig. 3, the setting management unit 9a then determines in step S206 whether or not to continue automatic adjustment. In step S206, which is performed at time TL221, the setting management unit 9a determines to continue automatic adjustment and proceeds to step S207. The period between time TL211 and time TL221 is referred to as the learning cycle CYC21.

[0078] In step S207, the setting management unit 9a determines, based on the completion timing of the evaluation operation EV21, that the values ​​of the command start signal 105 and the learning start signal 106 are 1 at time TL221. This operation determines time TL221 as the start timing of an initialization operation IN22 (a first initialization operation) and a learning operation L22 (a first learning operation). The engine 1 and the learning unit 7a start the initialization operation IN22 and the learning operation L22 in accordance with the command start signal 105 and the learning start signal 106, respectively. The initialization operation IN22 and the learning operation L22 are performed in parallel.

[0079] Thereafter, steps S203 to S207 are repeatedly performed until the setting management unit 9a determines not to continue automatic adjustment in step S206. Subsequently, in step S204, the setting management unit 9a determines, in a learning cycle CYC22, based on TL223, i.e., the completion time of the initialization operation IN22, that the value of the command start signal 105 is 1 at a time TL223. This operation determines the time TL223 as the start time of an evaluation operation EV22 (a second evaluation operation). The engine 1 starts the evaluation operation EV22 at the time TL223 in accordance with the command start signal 105.

[0080] In step S205, the setting management unit 9a detects a time TL2X2 as the completion time of an evaluation operation EV2X in a learning cycle CYC2X, which is a final learning cycle. Subsequently, in step S206, the setting management unit 9a determines not to continue the automatic setting, and the process proceeds to step S208. In step S208, the setting management unit 9a instructs the learning unit 7 to execute termination processing T2 as in step S108 of Fig. 3. The learning unit 7a carries out the termination processing T2 in the same way as the termination processing T1 of Fig. 2. In the present embodiment, as in the first embodiment, a plurality of control targets similar to the control target 2000 can be allowed to perform evaluation operations in parallel to efficiently perform automatic adjustment. The motor controller can be configured using a trained learning device that incorporates the results of the learning according to the present embodiment. Through the learning according to the present embodiment, the automatic adjustment of the control command for controlling the motor can be performed, or the production of the motor controller can be performed.

[0081] Upon detecting the completion of the operation in step S203 or step S205, the setting management unit 9a may detect the completion of the operation by detecting that the deviation, i.e., the difference between the state sensor signal 101 indicating the position of the motor 1, and the target travel length, has become less than or equal to a predetermined reference value. If the setting management unit 9a detects that the deviation has not exceeded the reference value for a predetermined period of time, in addition to determining that the deviation has become less than or equal to the reference value, the setting management unit 9a may further determine that the operation is completed.The setting management unit 9a is not limited to using the state sensor signal 101, but may also use a signal that has detected the state of the control target 2000 to detect the completion time of the operation. Furthermore, the command signal 103 may be used to detect the completion time of the operation.

[0082] The present embodiment can provide a motor controller capable of shortening the time required for automatic adjustment to set the control command for controlling the motor by repeating the initialization operation, the evaluation operation, and the learning operation in performing automatic adjustment.

[0083] The evaluation operation EV21 (the first evaluation operation), which is one of the evaluation operations, may be performed, and the learning operation L22 (the first learning operation) may be performed using the state sensor signal 101 acquired at the time of the evaluation operation EV21. Subsequently, the initialization operation IN22 (the first initialization operation) may be performed in parallel with the learning operation L22, and the evaluation operation EV22 (the second evaluation operation), which is an evaluation operation after the evaluation operation EV21, may be performed from the initial state set by the initialization operation IN22 based on the command parameter 104 (the control command) determined in the learning operation L22.This process allows the learning operation L22 and the initialization operation IN22 to be performed in parallel to shorten the time required for automatic tuning. Thus, it is possible to provide the motor controller 1000a or motor control method capable of efficiently performing automatic tuning.

[0084] The setting management unit 9a can detect the completion time of the evaluation operation EV21 and, based on the detected completion time, determine the start time of the learning operation L22 and the start time of the initialization operation IN22, thereby shortening the waiting time between the processes. The setting management unit 9a can determine that the start time of one of the learning operation L22 and the initialization operation IN22, where one requires a longer time, is the same as or earlier than the start time of the other, thereby shortening the waiting time between the processes.The setting management unit 9a can detect the completion time of one of the initialization operation IN22 and the learning operation L22, one of which completes at the same time as or later than the other, determine the start time of the evaluation operation EV22 based on the detected completion time, and shorten the waiting time between the processes. When two continuously executed processes are referred to as a previous process and a subsequent process, it is preferable to shorten the interval between the completion time of the previous process and the start time of the subsequent process as much as possible, and it is even more preferable to determine that the completion time of the previous process and the start time of the subsequent process are the same or substantially the same.Furthermore, the drive control unit 4 can drive the motor 1 such that the motor 1 follows the command signal 103, that is, a command value for controlling the motor 1, where the command value is a command value of position, speed, acceleration, current, torque, or thrust, and can detect the completion time of the evaluation operation or the initialization operation using a signal that has detected the state of the control target 2000 or the command signal 103 to accurately detect the completion time of the operation. Even if the time required for one operation changes, the time required for automatic adjustment can be shortened by utilizing the fact that the start time of the next process can be accurately determined. The motor controller 1000a or the motor control method can be provided.that is capable of efficiently performing the automatic adjustment as described above. Third embodiment

[0085] Fig. 12 is a block diagram showing an example of the configuration of a motor controller 1000b according to a third embodiment. Fig. 12(a) shows a configuration example of the entire motor controller 1000b. Fig. 12(b) shows a configuration example of a learning unit 7b. The configuration of the motor controller 1000b is the same as that of the Fig. 9, except that the learning unit 7b is included instead of the learning unit 7a. As for the Fig. 12, components of the present embodiment are concerned, components which are identical to those shown in Fig. 9 are identical or correspond to the components of the second embodiment, are designated by the same reference numerals.

[0086] Learning Unit 7b includes, in addition to the components of Learning Unit 7a, Fig. 9(b) a learning limit time determination unit 77. The learning limit time determination unit 77 calculates an estimated value of the time required for the initialization operation as an estimated required initialization operation time. Then, based on the estimated required initialization operation time, the upper limit of a learning time, that is, a period during which the learning unit 7b performs the learning operation, is determined as a learning limit time TLIM1. The learning limit time determination unit 77 may determine that the learning limit time TLIM1 is a period equal to or shorter than the estimated required initialization operation time. Then, the learning unit 7b may perform the learning operation for a period equal to or shorter than the learning limit time TLIM1. By performing the learning operation in this way, the learning operation can be completed before the completion of the initialization operation.The learning unit 7b can externally acquire the estimated required initialization operation time. For example, the learning unit 7b can obtain a practical measurement value for the time spent on the initialization operation from the state sensor signal 101 and the command signal 103, and estimate or update the estimated required initialization operation time using the practical measurement value.

[0087] The learning limit time determination unit 77 may further predetermine a basic learning time TSL1. The basic learning time TSL1 is the lower limit of the learning time. The learning unit 7b may perform the learning operation for a period equal to or longer than the basic learning time TSL1. For example, the basic learning time TSL1 may be set as the minimum time period for determining the command parameter 104, or it may be set as the minimum time period for determining the command parameter 104 with a desired accuracy. The learning limit time determination unit 77 may further determine an additional learning time TAD1 based on the basic learning time TSL1 and the learning limit time TLIM1 such that the sum of the basic learning time TSL1 and the additional learning time TAD1 does not exceed the learning limit time TLIM1. This condition is expressed by formula (26). The learning limit time TLIM1 is set to be longer than the basic learning time TSL1. [Formula 26] TSL1+TAD1 <TLIM1

[0088] The learning unit 7b performs a learning operation during the base learning time TSL1. Then, the learning operation may be further performed during the additional learning time TAD1 to improve the accuracy of the command parameter 104. The learning unit 7b may perform the learning operation during the learning time set in advance using the base learning time TSL1 as the lower limit. The learning limit time TLIM1 may be set alone, without setting the base learning time TSL1 and the additional learning time TAD1. The learning limit time determination unit 77 may store the estimated required initialization operation time, the learning limit time TLIM1, the base calculation time TSL1, the additional learning time TAD1, etc., in a storage device.

[0089] Next, the relationship between the learning time and the accuracy of the command parameter determined in the learning operation is described. For example, when Q-learning is used as the learning algorithm, the decision-making unit 73 selects an action a tthat increases the value of the action value function Q as the evaluation candidate parameter 108. When performing this selection, when the number of action value acquisitions Q is a continuous function, the decision-making unit 73 may, for example, perform iterative calculation. In such a case where iterative calculation is performed during the learning operation, the decision-making unit 73 can improve the calculation accuracy by lengthening the calculation time and increasing the number of calculation steps. Therefore, when the learning operation includes iterative calculation, the effects of the present embodiment can be further enhanced. Examples of iterative calculation include a numerical gradient determination method such as the steepest descent method or Newton's method, and a method using stochastic elements such as a Monte Carlo method.

[0090] Fig. Fig. 13 is a diagram showing an example of operating times in the motor controller 1000b according to the third embodiment. The horizontal axes in Fig. 13(a) to 13(d) represent time, and the vertical axes in Fig. 13(a) to 13(d) illustrate the learning operation, the operation processing (the initialization operation and the evaluation operation), the learning start signal 106 and the instruction start signal 105, respectively. The relationships between the signal values ​​of the instruction start signal 105 and the learning start signal 106 and the values ​​generated by the signals of Fig. 13 operating hours are the same as in Fig. 2 of the first embodiment. The Fig. The operation of the engine control 1000b shown in Figure 13 is the same as that shown in Fig. 10, with the exception of the operation of learning unit 7b. In Fig. 13 operations, learning processes, learning cycles, times, etc., which are identical to those of Fig. 10 or correspond to them, with the same reference numerals as in Fig. 10. A flowchart of the operation of the setting management unit 9a in the operation example of Fig. 13 is the same as that of the second embodiment in Fig. 11. An example of operation of the engine control 1000b is described with reference to Fig. 11 and Fig. 13 described.

[0091] In the operational example of Fig. 13, the learning limit time determination unit 77 calculates the estimated required initialization operation time based on a practical measurement of the time spent on the initialization operation IN21. Then, the learning limit time TLIM1 is determined as a period equal to or shorter than the estimated required initialization operation time. Furthermore, the learning limit time determination unit 77 determines the base learning time TSL1 as the lower limit of the learning time and sets the difference between the learning limit time TLIM1 and the base learning time TSL1 as the additional learning time TAD1.

[0092] In the operational example of Fig. 13 only the operation of learning unit 7b differs from that of Fig. 10 of the second embodiment. Therefore, the operation of the learning unit 7b will be described using the learning cycle CYC22 as an example. The learning unit 7b starts the learning operation L22 (the first learning operation) at time TL221 in accordance with the learning start signal 106 determined in step S202 in the learning cycle CYC22. The learning unit 7b performs a partial learning operation L221 and a partial learning operation L222 as the learning operation L22. The length of the partial learning operation L221 is the basic learning time TSL1. The length of the partial learning operation L222 is the additional learning time TAD1. Further, the learning unit 7b completes the learning operation L22 at time TL222, which is the time at which the basic calculation time TSL1 and the additional learning time TAD1 have elapsed since the time TL221.The value of time TL222 is equal to the sum of three values, namely the value of time TL221, the basic calculation time TSL1 and the additional learning time TAD1, where the relationship of formula (27) holds. [Formula 27] TL222=TL221+TSL1+TAD1

[0093] In the operational example of Fig. 13, the start time of the initialization operation and the start time of the learning operation are the same. If the time required for the initialization operation is longer than the time required for the learning operation, the learning operation may start later than the initialization operation. The learning limit time determination unit 77 may determine the learning limit time TLIM1 such that the time at which the estimated required initialization operation time has elapsed since the start time of the initialization operation IN22 follows the time at which the learning limit time TLIM1 has elapsed since the start time of the learning operation L22 (the first learning operation). Subsequently, the learning unit 7b may perform the learning operation L22 for a period equal to or shorter than the learning limit time TLIM1.This makes it possible to complete the learning operation L22 before the completion of the initialization operation IN22, even if the start time of the learning operation L22 is later than the start time of the initialization operation IN22. Under these circumstances, the evaluation operation EV22 can start immediately after the completion of the initialization operation IN22 without having to wait for the completion of the learning operation L22. Consequently, there is no increase in the delay time due to waiting for the completion of the learning operation L22. In this way, the time required for automatic adjustment can be shortened. In this way, it is possible to provide the motor controller 1000a or the motor control method capable of efficiently performing the automatic adjustment.

[0094] The learning limit time determination unit 77 may determine the basic learning time TSL1, the lower limit of the learning time, in addition to the learning limit time TLIM1. Subsequently, the learning unit 7b may perform the learning operation L22 for a period equal to or longer than the basic learning time TSL1 and equal to or shorter than the learning limit time TLIM1. Performing the learning operation in this way makes it possible to secure the learning time previously set as the lower limit using the learning limit time TLIM1. For example, if the basic learning time TSL1 is set as the minimum time required to obtain the command parameter 104, the command parameter 104 can be calculated with a higher probability in each learning cycle. The motor controller 1000a or the motor control method capable of efficiently performing the automatic adjustment as described above may be provided.

[0095] The present embodiment can provide a motor controller capable of shortening the time required for automatic adjustment to set the command parameter 104 (the control command) for controlling the motor 1 by repeating the initialization operation, the evaluation operation, and the learning operation in performing the automatic adjustment. Fourth embodiment

[0096] Fig. 14 is a block diagram showing an example of the configuration of a motor controller 1000c according to a fourth embodiment. Fig. 14(a) shows a configuration example of the entire motor controller 1000c. Fig. 14(b) shows an exemplary configuration of a learning unit 7c. The Fig. 14 includes the learning unit 7c instead of the learning unit 7 of the engine control 1000 according to the Fig. 1 and includes a setting management unit 9b instead of the setting management unit 9. Furthermore, the engine controller 1000c includes, in addition to the components of the engine controller 1000 of Fig. 1 a learning time estimation unit 10. When describing the Fig. 14, components are used which are identical to those of the Fig. 1 or Fig. 5 are identical or correspond to the first embodiment shown, are designated by the same reference numerals.

[0097] Although various learning algorithms can be applied for learning in the present embodiment, a case in which reinforcement learning based on Q-learning is used will be described. Fig. Learning unit 7c shown in Figure 14 includes a decision-making unit 73a instead of the decision-making unit 73 of learning unit 7 of the Fig. 5 illustrated first embodiment. The learning unit 7 of Fig. 5 acquires a set of the command parameter 104 used in the evaluation operation and the state sensor signal 101 at the time of the evaluation operation in a single learning operation, and determines the command parameter 104 once. The learning unit 7c, on the other hand, acquires a plurality of sets in a single learning cycle. Subsequently, the reward calculation unit 71 and the value function update unit 72 calculate the reward r and update the action value function Q for each of the acquired sets based on the calculated reward r. As a result, the learning unit 7c calculates the reward r and updates the action value function Q multiple times in a single learning cycle.

[0098] The decision-making unit 73a determines a plurality of evaluation candidate parameters 108 based on the action value function Q updated several times and the plurality of data sets used in the updates. Subsequently, the command parameter determination unit 75 determines the command parameter 104 to be used in the evaluation operation based on the determined evaluation candidate parameters 108 after the learning operation is performed.

[0099] The operation of the decision-making unit 73a will now be described. The decision-making unit 73a acquires the action value function Q(s t, at) in the formula (25) updated by the value function updating unit 72. Subsequently, the decision-making unit 73a calculates the values ​​of the action value function Q corresponding to the plurality of actions at, that is, the plurality of command parameters 104 included in the plurality of data sets. If the action a t (the command parameter 104) is selected, a value of the action value function Q(s t , at). In this case, the action at (the command parameter 104) and the value of the action value function Q(s t , a t) each other. Further, the decision-making unit 73a selects a predetermined number of values ​​of the action value function Q in descending order from the plurality of calculated values ​​of the action value function Q. Subsequently, the decision-making unit 73a determines that the command parameters 104 corresponding to the selected values ​​of the action value function Q are the evaluation candidate parameters 108. The above is an example of the operation of the decision-making unit 73a. The number of command parameters 104 determined by the command parameter determination unit 75 may be equal to the number of evaluation operations to be performed in a learning cycle following the learning operation just performed.

[0100] Next, the learning time estimation unit 10 will be described. The learning time estimation unit 10 calculates an estimated learning time of the learning operation to be performed as the estimated learning time and outputs a learning time estimation signal 109 indicating the estimated learning time. The learning time estimation unit 10 may acquire the learning start signal 106 and the learning completion signal 107 for the learning operation being performed and obtain a practical measurement value of the learning time from the difference between the learning start time and the learning completion time. Then, based on the acquired practical measurement value of the learning time, the learning time estimation unit 10 may calculate an estimated learning time of the learning operation to be performed as the estimated learning time. The learning time estimation unit 10 may acquire the estimated learning time through an external input or update the estimated learning time based on an actual measurement value of the learning time.

[0101] Next, the setting management unit 9b will be described. The setting management unit 9b determines the learning start signal 106 based on the learning completion signal 107, thereby determining the start time of the next learning operation based on the completion time of the learning operation. Furthermore, the setting management unit 9b predetermines a required initialization operation time, i.e., the time required for the initialization operation, and a required evaluation operation time, i.e., the time required for the evaluation operation. By detecting the lapse of the required initialization operation time and the required evaluation operation time since the start times of the initialization operation and the evaluation operation, the setting management unit 9b detects the respective completion times of the initialization operation and the evaluation operation.Based on the detected completion times of the initialization operation and the evaluation operation, the setting management unit 9b determines the respective start times of the evaluation operation and the initialization operation to be performed next. Like the setting management unit 9a of the second embodiment, the setting management unit 9b can accurately detect the completion times of the initialization operation and the evaluation operation based on a signal that has detected the state of the control target 2000 or the command signal 103. The operation of the motor 1 consisting of the initialization operation and the evaluation operation starting from the initial state determined by the initialization operation is referred to as an evaluation operation cycle.The setting management unit 9b determines whether or not to complete the evaluation operation cycle at each evaluation operation completion time. The evaluation operation completion time is hereinafter often referred to as the determination time.

[0102] Fig. Fig. 15 is a diagram showing an example of operating times in the motor controller 1000c according to the fourth embodiment. The horizontal axes in Fig. 15(a) to 15(e) represent time, and the vertical axes in Fig. 15(a) to 15(e) illustrate the learning operation, the operation processing (the initialization operation and the evaluation operation), the learning start signal 106, the learning completion signal 107, and the instruction start signal 105, respectively. The relationships between the values ​​of the learning start signal 106, the learning completion signal 107, and the instruction start signal 105, and the timings of the learning operation or the operations indicated by the signals are the same as those shown in the Fig. 2 will be described. Fig. 16 is a flowchart showing an example of the operation of the setting management unit 9b according to the fourth embodiment. In Fig. 15, a single learning operation is performed in a single learning cycle, and two evaluation operation cycles are performed in parallel with the learning operation. However, the number of evaluation operation cycles performed in parallel with the learning operation can also be three or more.

[0103] The operation of the engine control 1000c is described with reference to Fig. 15 and Fig. 16. After starting the automatic adjustment, the adjustment management unit 9b determines in step S401 that the value of the learning start signal 106 is 1 at a time TL411 and designates the time TL411 as the start time of a learning operation L41 (a third learning operation). The learning unit 7c starts the learning operation L41 at the time TL411 in accordance with the learning start signal 106. In step S402, the adjustment management unit 9b determines, based on the start time of the learning operation L41, that the value of the command start signal 105 is 1 at the time TL411 and designates the time TL411 as the start time of an initialization operation IN41. The engine 1 starts the initialization operation IN41 at the time TL411 in accordance with the command start signal 105.Subsequently, the engine 1 completes the initialization operation IN41 at a time TL412 and enters a standby state after the completion of the initialization operation IN41. In step S402, the setting management unit 9b determines the start timing of a first evaluation operation cycle ECYC1 (a first evaluation operation cycle) by determining the start timing of the initialization operation IN41.

[0104] In step S403, the setting management unit 9b detects that the required initialization operation time has elapsed since time TL411 and detects time TL413 as the completion time of the initialization operation IN41. In step S404, the setting management unit 9b determines, based on the detected completion time of the initialization operation IN41, that the value of the command start signal 105 is 1 at time TL413 and determines time TL413 as the start time of an evaluation operation EV41. The motor 1 starts the evaluation operation EV41 at time TL413 in accordance with the command start signal 105. Thereafter, the motor 1 completes the evaluation operation EV41 at time TL414 and enters a standby state after the completion of the evaluation operation EV41.

[0105] In step S405, the setting management unit 9b detects that the required evaluation operation time has elapsed since time TL413 and detects time TL415 as the completion time of the evaluation operation EV41. In step S406, the setting management unit 9b determines whether or not to complete the evaluation operation cycle. If the setting management unit 9b determines not to complete the evaluation operation cycle, the process proceeds to step S407. If the setting management unit 9b determines to complete the evaluation operation cycle, the process proceeds to step S408.

[0106] The determination in step S406 will now be explained. The setting management unit 9b determines in advance an estimated required evaluation operation cycle time, that is, an estimated value of the time required for a single evaluation operation cycle. At the determination time, the setting management unit 9b acquires the learning time estimation signal 109 and calculates a time of elapse of the estimated learning time, that is, the time at which the estimated learning time has elapsed since the start time of the learning operation L41. If the period from the determination time, that is, from the completion time of the evaluation operation, to the elapse time of the estimated learning time is shorter than the estimated required evaluation operation cycle time, the setting management unit 9b further determines to complete the evaluation operation cycle ECYC1.If the period from the determination time to the time of expiration of the estimated learning time is longer than or equal to the estimated required evaluation operation cycle time, the setting management unit 9b determines not to complete the evaluation operation cycle ECYC1. In other words, if it is not possible to perform a single evaluation operation cycle during the remaining time before the time of expiration of the estimated learning time, the setting management unit 9b determines not to complete the evaluation operation cycle ECYC1. If it is possible to perform a single evaluation operation cycle during the remaining time, the setting management unit 9b determines not to complete the evaluation operation cycle ECYC1. The above is an example of the determination in step S406.

[0107] In the determination in step S406 at time TL415, the setting management unit 9b determines not to complete the evaluation operation cycle ECYC1 and proceeds to step S407. In step S407, the setting management unit 9b determines, based on the completion time of the evaluation operation EV41, that the value of the command start signal 105 is 1 at time TL415 and sets time TL415 as the start time of an initialization operation IN42. In accordance with the command start signal 105, the engine 1 starts the initialization operation IN42 at time TL415. Thereafter, the setting management unit 9b repeatedly executes steps S403 to S407 until the setting management unit 9b determines to complete the evaluation operation cycle ECYC1 in step S406.

[0108] At a determination time point TL421, the setting management unit 9b performs the determination in step S406, determining to complete the evaluation operation cycle ECYC1, and proceeds to step S408. In step S408, the setting management unit 9b detects, based on the learning completion signal 107, the time point TL421 as the completion time point of the learning operation L41. As in step S106 of the Fig. In the first embodiment illustrated in Figure 3, the setting management unit 9b next determines whether or not to continue automatic setting in step S409. If the setting management unit 9b determines to continue automatic setting, the process proceeds to step S410. If the setting management unit 9b determines not to continue automatic setting, the process proceeds to step S411. In the determination in step S409 at time TL421, the setting management unit 9b determines to continue automatic setting.

[0109] A learning cycle CYC41 is the time period between time TL411 and time TL421. The evaluation operation cycle ECYC1 starts from a state where no learning operation has been performed. In this way, the evaluation operation EV41 and the evaluation operation EV42 can be performed using the predetermined command parameter 104 or the randomly determined command parameter 104. As with the learning operation L11 of the first embodiment, in the learning operation L41, the command parameter 104 can be randomly determined, or the command parameter 104 can be determined based on a setting.

[0110] In step S410, the setting management unit 9b determines, based on the completion timing of the learning operation L41, that the value of the learning start signal 106 is 1 at time TL421, and designates time TL421 as the start timing of a learning operation L42 (a fourth learning operation). The learning unit 7c starts the learning operation L42 at time TL421 in accordance with the learning start signal 106. The learning operation L42 is performed based on the command parameter 104 used in the evaluation operation cycles ECYC1 and the state sensor signal 101 detected in the evaluation operation cycles ECYC1. Thereafter, the setting management unit 9b repeatedly executes steps S402 to S410 until the setting management unit 9b determines not to continue the automatic setting in step S409.An evaluation operation cycle ECYC2 (a second evaluation operation cycle) is performed using the command parameter 104 determined in the learning operation L41. In step S402, the setting management unit 9b determines the time TL421 as the start time of an initialization operation IN43, thereby determining the time TL421 as the start time of the evaluation operation cycle ECYC2.

[0111] Upon determination in step S409 at a time TL4X3 in a learning cycle CYC4Z, the setting management unit 9b determines not to continue the automatic setting and proceeds to step S411. In step S411, the setting management unit 9b issues an instruction to execute the termination processing T4 as in step S108 of the Fig. 3. Subsequently, the learning unit 7c executes the termination processing T4 in the same manner as the termination processing T1 in the first embodiment shown in Fig. 2 illustrated first embodiment.

[0112] In the present embodiment, as in the first embodiment, a plurality of control targets similar to the control target 2000 can be allowed to perform evaluation operations in parallel to efficiently perform automatic adjustment. For example, if a plurality of control targets are allowed to perform the evaluation operation cycle during the learning operation L41 in Fig. 15 in parallel, more sets of the state sensor signal 101 and the command parameter 104 can be acquired in a single evaluation operation cycle, so that the learning process can be performed efficiently. The engine controller can be configured using a trained learning device that incorporates the results of the learning according to the present embodiment. Furthermore, the learning according to the present embodiment can be performed to thereby perform, for example, the automatic adjustment of the control command for controlling the engine and the establishment of the engine control. Furthermore, an engine control method capable of efficiently performing the automatic adjustment can be provided.

[0113] The learning operation L41 (the third learning operation), which is one of the learning operations, may be performed, and the evaluation operation cycle ECYC1 (the first evaluation operation cycle), which is one of the evaluation operation cycles, may be performed multiple times in parallel with the learning operation L41. Furthermore, the learning operation L42 (the fourth learning operation), which is a learning operation following the learning operation L41, may be performed using the state sensor signal 101 detected at the time of the evaluation operation cycle ECYC1.Subsequently, using the command parameter 104 (control command) determined in the learning operation L41, the evaluation operation cycle ECYC2 (the second evaluation operation cycle), which is an evaluation operation cycle following the evaluation operation cycle ECYC1, can be performed multiple times in parallel with the learning operation L42. In this process, the evaluation operation cycle can be performed multiple times during a single learning operation to efficiently acquire sets from the command parameter 104 and the evaluation sensor signal 102 and shorten the time required for automatic adjustment. In this way, it is possible to provide the motor controller 1000c or the motor control method capable of efficiently performing automatic adjustment.

[0114] The setting management unit 9b can determine the start timing of the learning operation L42 based on the completion timing of the learning operation L41, and can determine the respective start timings of the evaluation operation cycle ECYC1 and the evaluation operation cycle ECYC2 based on the start timings of the learning operation L41 and the learning operation L42. Through this operation, the relationship between the timings for performing two learning operations can be set, and the relationship between the timing at which the learning operation should be performed and the timing at which the evaluation operation cycle should be performed can be set. This can shorten the waiting time. In this way, it is possible to provide the motor controller 1000c or the motor control method capable of efficiently performing the automatic setting.

[0115] The engine controller 1000c further includes the learning time estimation unit 10, which estimates the time required for the learning operation L21 as the estimated learning time. The setting management unit 9b may preliminarily determine an estimated value of the time required to perform the evaluation operation cycle as the estimated required evaluation operation cycle time. Furthermore, the setting management unit 9b may determine to continue the evaluation operation cycle ECYC1 when, at a determination time point (i.e., the time point at which the evaluation operation cycle ECYC1 is completed), the difference between the estimated learning time and a period of time elapsed from the start time point of the learning operation L21 to the determination time point is equal to or longer than the estimated required evaluation operation cycle time.The setting management unit 9b may determine not to continue the evaluation operation cycle ECYC1 if the difference is shorter than the estimated required evaluation operation cycle time. This operation can increase the number of evaluation operation cycles to the point where the evaluation operation cycles can be completed at the completion time of the learning time. When the estimated learning time, the estimated required evaluation operation cycle time, or the like changes, the number of evaluation operation cycles to be performed can be adjusted in response to the change, so that the automatic adjustment can be performed efficiently. In this way, it is possible to provide the engine controller 1000c or the engine control method capable of efficiently performing the automatic adjustment.

[0116] In the operational example of Fig.15, the setting management unit 9b determines the completion time of the initialization operation IN41 based on the start time of the initialization operation IN41 and the required initialization operation time. The present embodiment is not limited to this operation. For example, there is a case where an intermediate process including one of the initialization operation, the evaluation operation, or the learning operation is performed between the completion of the first process, which is one process, and the start of the second process, which is one process.In this case, the setting management unit 9b may estimate the time required to perform the intermediate process in advance and determine that the start time of the second process follows the time at which the estimated time required to perform the intermediate process has elapsed since the completion time of the first process. Through this operation, the start time of the second process can be set with the estimated value of the time required for the intermediate process as a guide, shortening the waiting time, thereby reducing the time required for automatic adjustment. In this way, it is possible to provide the motor controller 1000c or the motor control method capable of efficiently performing automatic adjustment.

[0117] As described above, the present invention can provide a motor controller capable of shortening the time required for the automatic adjustment to set the control command for controlling the motor by repeating the initialization operation, the evaluation operation, and the learning operation in performing the automatic adjustment. List of reference symbols

[0118] 1 Motor; 2 Command generation unit; 3 Mechanical load; 4 Drive control unit; 7, 7a, 7b, 7c Learning unit; 9, 9a, 9b Setting management unit; 10 Learning time estimation unit; 77 Learning limit time determination unit; 101 State sensor signal; 103 Command signal; 1000, 1000a, 1000b, 1000c Motor control; 2000 Control object; ECYC1, ECYC2 Evaluation operation cycle; EV11, EV12, EV21, EV22, EV41, EV42, EV43, EV44 Evaluation operation; IN12, IN22, IN41, IN42, IN43, IN44 Initialization operation; L12, L22, L23, L41, L42 Learning operation; TLIM1 learning limit time; TSL1 baseline learning time.

Claims

[1] Engine control (1000), comprising: a drive control unit (4) for driving a motor (1) based on a control command (104), operating a control target (2000) consisting of the motor and a mechanical load (3) mechanically connected to the motor, and performing an initialization operation for setting the control target to an initial state and an evaluation operation from the initial state; a learning unit (7) for learning the control command used in the evaluation operation and a state sensor signal (101) in conjunction with each other, the state sensor signal having detected a state of the control target at a time of the evaluation operation, and for determining, on the basis of a learning result, the control command to be used in the evaluation operation to be performed after the evaluation operation in which the state sensor signal was detected; and a setting management unit (9) for determining, based on a timing at which a first process is to be performed, a timing at which a second process is to be performed, the first process being one of a learning operation, the initialization operation, and the evaluation operation, the learning operation being an operation of the learning unit, the second process being one of the learning operation, the initialization operation, and the evaluation operation. [2] Engine control according to claim 1, wherein a first evaluation operation (EV11; EV21), which is one of the evaluation operations, is carried out, a first learning operation (L12; L22), which is the learning operation, is performed using the state sensor signal detected at a time of the first evaluation operation, a first initialization operation (IN12; IN22), which is the initialization operation, is carried out in parallel with the first learning operation, and a second evaluation operation (EV12; EV22), which is the evaluation operation following the first evaluation operation, is carried out starting from the initial state determined by the first initialization operation on the basis of the control command determined in the first learning operation. [3] The engine controller according to claim 2, wherein the setting management unit (9; 9a) detects a completion time of the first evaluation operation (EV11; EV21) and determines a start time of the first learning operation (L12; L22) and a start time of the first initialization operation (IN12; IN22) based on the detected completion time of the first evaluation operation. [4] The engine controller according to claim 2 or 3, wherein the setting management unit (9; 9a) determines that a start timing of one of the first learning operation (L12; L22) and the first initialization operation (IN12; IN22), one of which requires a longer time, is identical to or precedes a start timing of the other. [5] The engine controller according to any one of claims 2 to 4, wherein the setting management unit (9; 9a) detects a completion time of one of the first learning operation (L12; L22) or the first initialization operation (IN12; IN22), the one operation being completed at the same time as or later than the other, and determines a start time of the second evaluation operation based on the detected completion time. [6] Engine control according to one of claims 2 to 5, wherein a time required for the first initialization operation is longer than a time required for the first learning operation, and the engine controller (1000b) further comprises a learning limit time determination unit (77) for determining a learning limit time (TLIM1) such that a time point at which an estimated required initialization operation time has elapsed since a start time point of the first initialization operation follows a time point at which the learning limit time has elapsed since a start time point of the first learning operation, wherein the learning limit time is an upper limit of a learning time, i.e., a period during which the learning operation is performed, wherein the estimated required initialization operation time is an estimate of a time required for the initialization operation, and wherein the learning unit performs the first learning operation for a period equal to or shorter than the learning limit time. [7] The engine controller according to claim 6, wherein the learning limit time determining unit (77) further determines a basic learning time (TSL1) which is a lower limit of the learning time and a period shorter than the learning limit time, and wherein the learning unit performs the first learning operation for a period equal to or longer than the basic learning time. [8] Engine control (1000c) according to claim 1, wherein a third learning operation (L41), which is one of the learning operations, is carried out, a first evaluation operation cycle (ECYC1), which is one of the evaluation operation cycles consisting of the initialization operation and the evaluation operation, is performed several times in parallel with the third learning operation, a fourth learning operation (L42), which is the learning operation following the third learning operation, is performed using the state sensor signal detected at a time of the first evaluation operation cycle, and a second evaluation operation cycle (ECYC2), which is the evaluation operation cycle following the first evaluation operation cycle, is performed several times in parallel with the fourth learning operation, using the control command determined in the third learning operation. [9] The engine controller (1000c) according to claim 8, wherein the setting management unit (9b) determines a start time of the fourth learning operation based on a completion time of the third learning operation, and determines start times of the first evaluation operation cycle and the second evaluation operation cycle based on start times of the third learning operation and the fourth learning operation, respectively. [10] Engine control (1000c) according to claim 8 or 9, further comprising: a learning time estimation unit (10) for estimating a time required for the third learning operation as an estimated learning time, wherein the setting management unit (9b) sets in advance an estimated value of a time required to perform each evaluation operation cycle as the estimated required evaluation operation cycle time, and the setting management unit determines, at a determination time point that is a time point at which the first evaluation operation cycle is completed, to continue the first evaluation operation cycle when a difference between the estimated learning time and a time elapsed from a start time point of the third learning operation to the determination time point is equal to or longer than the estimated required evaluation operation cycle time, and determines not to continue the first evaluation operation cycle when the difference is shorter than the estimated required evaluation operation cycle time. [11] Engine control according to one of claims 1 to 10, wherein an intermediate process including at least one of the initialization operation, the evaluation operation, or the learning operation is performed between the completion of the first process and the start of the second process, and the setting management unit (9) estimates in advance a time required for performing the intermediate process and determines that a start time of the second process follows a time at which the estimated time required for performing the intermediate process has elapsed since a completion time of the first process. [12] Engine control according to one of claims 1 to 11, wherein the drive control unit (4) drives the motor such that the motor follows a command signal which is a command value for controlling the motor, the command value being a command value of position, speed, acceleration, current, torque or thrust, and the setting management unit detects a timing at which the evaluation operation or the initialization operation is to be performed based on the command signal or a detection result of the detection of the state of the control target. [13] Engine control method comprising: Driving a motor (1) based on a control command (104), operating a control target (2000) consisting of the motor and a mechanical load (3) mechanically connected to the motor, and performing an initialization operation for setting the control target to an initial state and an evaluation operation starting from the initial state; Learning the control command used in the evaluation operation and a state sensor signal (101) in association with each other, wherein the state sensor signal has detected a state of the control target at a time of the evaluation operation, and performing a learning operation to determine, based on a learning result, the control command to be used in the evaluation operation to be performed after the evaluation operation in which the state sensor signal was detected; and Determining, based on a time at which a first process is to be performed, a time at which a second process is to be performed, wherein the first process is one of the learning operation, the initialization operation, and the evaluation operation, wherein the second process is one of the learning operation, the initialization operation, and the evaluation operation.

Images