Device for machine learning; device for servo control; system for servo control; and method for machine learning

Abstract

A machine learning device (200) for performing machine learning in conjunction with the optimization of key figures of at least two forward processing units (109, 110) with respect to a servo control device (100) to control a servo motor (300) so that a shaft of a machine tool or an industrial machine is driven using the forward control in which the at least two forward processing units form multiple loops, wherein an instruction that is balanced by a forward condition calculated by one of the at least two forward processing units is an instruction on an inner side, as seen from the servo motor, than another instruction that is balanced by a forward condition calculated by the other forward processing unit, and after the machine learning with respect to the optimization of the key figures of one forward processing unit has been performed,Machine learning is performed to optimize the key performance indicators (KPIs) of the other forward computing unit based on the optimized KPIs of the first forward computing unit, which were obtained through machine learning to optimize the KPIs of the first forward computing unit.

Description

BACKGROUND OF THE INVENTION Area of ​​the invention

[0001] The present invention relates to a machine learning device that performs machine learning with respect to a servo control device that uses forward control in which at least two forward computation units form multiple loops, a servo control device and a servo control system that includes the machine learning device, and a machine learning method. Related state of the art

[0002] A servo control device that uses forward control is disclosed, for example, in patent documents 1 to 4. A servo control device as disclosed in patent document 1 includes a neural network that calculates the forward condition of a speed command from a position setpoint and adds the forward condition to a speed command issued by a position control unit, and a neural network that calculates a forward condition of a torque command from a speed setpoint and adds the forward condition to the torque command issued by a speed control unit. The neural networks learn variations in the moment of inertia of a drive system, as well as resonance characteristics and similar features of the drive system, to calculate an optimal forward condition.

[0003] A forward control device as disclosed in patent document 2 includes a position forward calculation unit that calculates a forward condition of a velocity command from a position setpoint and adds the forward condition to a velocity command issued by a position control, and a velocity forward calculation unit that calculates a forward condition of the torque command from a position setpoint and adds the forward condition to a torque command issued by the velocity control.The forward control unit as disclosed in patent document 2 also includes a learning control which learns an increase in the attitude forward calculation unit based on an attitude error which is a difference between the attitude setpoint and the feedback attitude detection value, and a learning control which learns an increase in the attitude forward calculation unit based on an attitude error or a velocity error which is a difference between the velocity setpoint and the feedback attitude detection value.

[0004] An optimal command generation device, as disclosed in patent document 3, receives a command value, generates an ideal operation command with which a control target can perform a desired operation, and outputs an operation command to the servo control unit that controls the control target. The optimal command generation device includes a control target model and a learning control unit that executes the learning control, or a predictive control unit that executes the predictive control, so that the control target model can perform a desired operation.

[0005] A servo control device as disclosed in patent document 4 includes a forward control system including a speed forward generation unit that generates a speed forward signal based on a position command, a torque forward generation unit that generates a torque forward signal based on a position command, and a speed forward change unit that generates a speed forward change signal based on a speed forward signal and a torque forward signal. Patent document 1: Unexamined Japanese patent application, publication no. H4-084303 Patent document 2: Unexamined Japanese patent application, publication no. H2-085902 Patent document 3: Unexamined Japanese patent application, publication no. 2003-084804 Patent document 4: Unexamined Japanese patent application, publication no. 2010-033172 OVERVIEW OF THE INVENTION

[0006] In patent document 2, the servo control device performs position-forward learning by simultaneously using a position-forward learning controller and a velocity-forward learning controller. However, the amount of information processed for learning increases when the servo control device performs position-forward learning and velocity-forward learning simultaneously. Even if one learning controller modifies a forward condition of a velocity command based on a position error to reduce the position error, if the other learning controller modifies a forward condition of a torque-velocity command based on the position error, the position error will still change due to the effect of the modification.Therefore, the learning operations of the two learning controllers interfere with each other, and the amount of information processed for the learning operations of the two learning controllers increases.

[0007] An objective of the present invention is to provide a machine learning device that performs machine learning with respect to a servo control device that uses forward control in which at least two forward computation units form multiple loops, the servo control device is capable of reducing the amount of information processed for machine learning in order to shorten the machine learning setup time and suppressing variation in positional error in order to achieve high accuracy, and to provide a servo control device and a servo control system including the machine learning device, and a machine learning method. (1) A machine learning device according to the present invention is a machine learning device (e.g. a machine learning device).200 , which will be described later) for performing machine learning in conjunction with the optimization of key performance indicators of at least two forward calculation units (e.g., a location forward calculation unit). 109 and a speed forward calculation unit 110 , which will be described later) regarding a servo control device (e.g. a servo control device) 100 , which will be described later), to power a servo motor (e.g. a servo motor) 300, which will be described later) to control a shaft of a machine tool or an industrial machine using forward control in which the at least two forward calculation units form multiple loops, wherein, if an instruction balanced by a forward condition calculated by one of the at least two forward calculation units is an instruction on an inner side as seen from the servo motor, then another instruction balanced by a forward condition calculated by the other forward calculation unit, after machine learning with respect to optimization of the key figures of one forward calculation unit, is executed, machine learning with respect to optimization of the key figures of the other forward calculation unit is performed on the basis of the optimized key figures of one forward calculation unit,which were obtained through machine learning regarding the optimization of the key performance indicators of a forward calculation unit. (2) In the machine learning device according to (1) the at least two forward calculation units can be at least two forward calculation units under a position forward calculation unit (e.g. a position forward calculation unit). 109 , which will be described later) to calculate a first forward condition of a velocity command based on a position command, a velocity forward calculation unit (e.g. a velocity forward calculation unit) 110 , which will be described later) for calculating a second forward condition of a torque command based on a position command, and a current forward calculation unit (e.g., a current forward calculation unit). 114, which will be described later) to calculate a third forward condition of a current command based on a position command, One command and the other command can be two commands under the speed command, the torque command, and the current command, and The servo motor can be driven according to the torque command or the current command. (3) In the machine learning device according to (2) the first forward calculation unit may be the velocity forward calculation unit and the other forward calculation unit may be the position forward calculation unit. (4) In the machine learning device according to (2), the servo control device may include the position forward calculation unit, the velocity forward calculation unit, and the current forward calculation unit, and one forward calculation unit may be the velocity forward calculation unit or the current forward calculation unit, and the other forward calculation unit may be the position forward calculation unit. (5) In the machine learning device according to any of (1) to (4), the default values ​​of the transfer function indicators of the other forward computing unit may be the same values ​​as the default values ​​of the transfer function indicators of the one forward computing unit. (6) The machine learning device according to any of (1) to (5) may further include: a state information acquisition unit (e.g. a state information acquisition unit) 201 , which will be described later) to obtain, from the servo control device, the state information including a servo state which includes at least one position error and a linkage of the transfer function key figures of one or the other forward calculation unit, by the servo control device executing the predetermined machining program; an action information output unit (e.g., an action information output unit) 203 , which will be described later) for outputting action information including adjustment information for the linking of the key figures included in the status information of the servo control device; a reward dispensing unit (e.g. a reward dispensing unit) 2021 , which will be described later) to output a reward value for enhanced learning, based on the positional error included in the state information; and a value function update unit (e.g., a value function update unit) 2022 , which will be described later) to update a value function based on the reward value output by the reward output unit, the state information, and the action information. (7) In the machine learning device according to (6) the reward output unit can output the reward value based on an amount of positional error. (8) The machine learning device according to (6) or (7) may further include: an optimization action information output unit (e.g. an optimization action information output unit) 205, which will be described later) to generate and output a link of the key figures of the transfer function of the at least two forward calculation units based on the value function that has been updated by the value function update unit. (9) A servo control system according to the present invention is a servo control system including: the machine learning device according to one of (1) to (8); and a servo control device for controlling a servo motor that drives a shaft of a machine tool or an industrial machine by means of forward control in which at least two forward calculation units form several loops. (10) A servo control device according to the present invention is a servo control device including: the machine learning device according to one of (1) to (8); and at least two forward calculation units, wherein the servo control device controls a servo motor which drives a shaft of a machine tool or an industrial machine by means of forward control in which the at least two forward calculation units form several loops. (11) A machine learning method according to the present invention is a machine learning method of a machine learning device for performing machine learning with respect to the optimization of the key figures of at least two forward computation units with respect to a servo control device for controlling a servo motor which drives a shaft of a machine tool or an industrial machine by means of forward control in which the at least two forward computation units form several loops, wherein, if an instruction that is balanced by a forward condition calculated by one of the at least two forward computation units is an instruction on an inner side as seen from the servo motor, then another instruction that is balanced by a forward condition calculated by the other forward computation unit,After machine learning has been performed to optimize the key performance indicators (KPIs) of one forward computing unit, machine learning to optimize the KPIs of the other forward computing unit is performed based on the optimized KPIs of the first forward computing unit, which were obtained through machine learning to optimize the KPIs of the first forward computing unit.

[0008] According to the present invention, it is possible to provide a machine learning device that performs machine learning with respect to a servo control device, using forward control in which at least two forward calculation units form several loops, the servo control device is able to shorten the machine learning setup time and suppress fluctuations in positional error in order to achieve high accuracy. List of characters

[0009] Fig. Figure 1 is a block diagram showing a servo control system according to a first embodiment of the present invention. Fig. Figure 2 is a block diagram showing a pair consisting of the servo control device and a machine learning device for the servo control system according to the first embodiment of the present invention and a motor. Fig. Figure 3 is a block diagram showing an area of ​​a machine tool including a motor, which serves as an example object of a control target of the device for servo control. Fig. Figure 4 is a diagram to describe an engine operation when one machining shape is an octagon. Fig. Figure 5 is a diagram for describing an engine operation when the machining shape is a shape in which the corners of an octagon are alternately replaced by arcs. Fig. Figure 6 is a block diagram showing a machine learning device according to the first embodiment. Fig. Figure 7 is a flowchart describing the operation of the machine learning device according to the first embodiment. Fig. Figure 8 is a flowchart describing the operation of an optimization action information output unit of the machine learning device according to the first embodiment. Fig. Figure 9 is a block diagram showing an area of ​​a servo control device according to a second embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment of the present invention is described below with reference to the drawings. (First embodiment)

[0011] Fig. Figure 1 is a block diagram showing a servo control system according to a first embodiment of the present invention. As shown in Fig. Shown in 1, this includes a servo control system. 10 n devices for servo control 100-1 until 100-n , n devices for machine learning 200-1 until 200-n , and a network 400 one. Here, n is any natural number. It is assumed that the devices for machine learning 200-1 until 200-n The first embodiment features enhanced learning with respect to the key figures of a position forward calculation unit. 109 and a speed forward calculation unit 110 , as will be described later as an example of machine learning. The position forward calculation unit 109 and the speed forward calculation unit 110 form in the devices for servo control 100-1 until 100-nmultiple loops. The present invention is not limited to machine learning with respect to the key figures of the position forward calculation unit. 109 and the speed forward calculation unit 110 Therefore, the present invention can also be applied to machine learning with respect to a forward calculation unit that forms multiple loops, unlike the position forward calculation unit. 109 and the speed forward calculation unit 110 Furthermore, the machine learning of the present invention is not limited to augmented learning, but the present invention can also be applied to a case in which another machine learning is performed (e.g. supervised learning).

[0012] The device for servo control 100-1 and the device for machine learning 200-1are paired in a one-to-one relationship and communicatively connected to each other. The servo control devices 100-2 until 100-n and the devices for machine learning 200-2 until 200-n are similar to the device for servo control 100-1 and the device for machine learning 200-1 connected. Although n pairs of servo control devices 100-1 until 100-n and the devices for machine learning 200-1 until 200-n via the network 400 in Fig. 1 are connected, the n pairs of devices can be used for servo control. 100-1 until 100-n and the devices for machine learning 200-1 until 200-n Each pair is directly connected via interfaces. A large number of the n pairs of servo control devices are used. 100-1 until 100-n and the devices for machine learning 200-1 until 200-nIt can be provided for in the same plant, for example, and it can be provided for in different plants.

[0013] The network 400 A local area network (LAN) is a network built, for example, in a facility, the internet, a public telephone network, or a combination thereof. It uses a specific communication scheme for the network. 400 Whether the network uses wired or wireless connections, and the like, is not specifically defined.

[0014] Fig. Figure 2 is a block diagram showing a pair of components, consisting of the servo control device and a machine learning device, of the servo control system according to the first embodiment of the present invention and a motor. The servo control device 100 and the device for machine learning 200 as in Fig. 2 shown, correspond, for example, to the device for servo control 100-1and the machine learning device 200-1 as in Fig. 2 shown. A servo motor 300 is part of a control target (e.g. a machine tool, a robot, or an industrial machine) of the device for servo control 100 The device for servo control 100 can be used as part of a machine tool, a robot, an industrial machine, or similar, together with the servo motor 300 be provided for.

[0015] First, the device for servo control is 100 described. The device for servo control 100 includes a position command creation unit 101 , a substrate 102 , a position control unit 103 , an adder 104 , a substrate 105 , a speed control unit 106 , an adder 107 , an integrator 108 , a position forward calculation unit 109, and a velocity forward calculation unit 110 one. The position forward calculation unit 109 includes a differentiator 1091 and a position forward processing unit 1092 one. The velocity forward calculation unit 110 includes a double differential 1101 and a speed forward processing unit 1102 one. The situation command creation unit 101 generates a target position value and passes the generated target position value to the subtractor. 102 , the position forward calculation unit 109 , the speed calculation unit 110 and the machine learning device 200 out. The subtractor 102 calculates a difference between the target position and a feedback detection position and outputs the difference to the attitude control unit. 103 and the machine learning device 200 as a positional error.

[0016] The position command creation unit 101 generates a target position value based on a program for operating a servo motor. 300 The servo motor 300 For example, it is enclosed in a machine tool. In a machine tool, when a table that has a workpiece (a job) mounted on it moves in an x-axis direction and in a y-axis direction, the device for servo control is 100 and the servo motor 300 as in Fig. Figure 2 shows the device for movement in the x-axis and y-axis directions, respectively. If the table is moved in the directions of three or more axes, the servo control device is required. 100 and the servo motor 300 provided in the corresponding directions. The position command creation unit 101 It sets a feed rate and generates a target position value, so that a machining shape determined by a machining program is obtained.

[0017] The attitude control unit 103 gives a value, obtained by multiplying a positional increment Kp by the positional error, to the adder. 104 as a speed command. The differentiator 1091 the position forward calculation unit 109 distinguishes the position setpoint and multiplies a distinction result by a constant β, and the position forward processing unit 1092 This leads to a position-forward process, represented by a transfer function G(s) in equation 1 (shown by Math. 1 below), to the output of the differentiator. 1091 and outputs the processing result to the adder. 104 as a forward condition. Key figures a i and b j (m≥ i ≥ 0, n≥ j ≥ 0) in equation 1 are parameters of the transfer function of the position forward processing unit 1092 m and n are of course numbers. G ( s ) = b 0 + b 1 s + b 2 s 2 + ⋯ + b x s n a 0 + a 1 s + a 2 s 2 + ⋯ + a x s m

[0018] The adder 104 adds the target velocity value and the output value (the position forward condition) of the position forward calculation unit. 109 and returns an addition result to the subtractor 105 as a forward-controlled speed setpoint. The subtractor 105 calculates a difference between the output of the adder 104 and a feedback speed detection value and gives the difference of the speed control unit 106 as a speed error further.

[0019] The speed control unit 106 It adds a value obtained by multiplying and integrating an integral increment K1v with the velocity error and a value obtained by multiplying a proportional increment K2v with the velocity error, and returns the result of the addition to the adder. 107 as a target torque value.

[0020] The double differential 1101 the speed forward calculation unit 110 It differentiates the position setpoint twice and multiplies the differentiation result by a constant α, and the speed forward processing unit 1102 introduces a velocity-forward process, represented by a transfer function F(s) in equation 2 (shown by Math. 2 below), to the output of the double differentiator. 1101 and outputs the processing result to the adder. 107 as a forward velocity condition. Key figures c and d j (m≥ i ≥ 0, n≥ j ≥ 0) in equation 2 are parameters of the transfer function of the velocity forward processing unit 1102 m and n are natural numbers. The natural numbers m and n in equation 2 can be the same as, or different from, the natural numbers m and n in equation 1. F ( s ) = d 0 + d 1 s + d 2 s 2 + ⋯ + d x s n c 0 + c 1 s + c 2 s 2 + ⋯ + c x s m

[0021] The adder 107 adds the torque setpoint and an output value (the forward velocity condition) of the forward velocity calculation unit. 110 and sends the addition result to the servo motor. 300 as a forward-controlled torque setpoint to drive the servo motor 300 to drive.

[0022] A rotational angle position of the servo motor 300 is determined by a rotary encoder, which is connected to the servo motor as a position determination unit. 300 serves, and a speed determination value is entered into the subtractor. 105 The speed value is entered as a speed feedback. The speed determination value is then processed by the integrator. 108 integrated to become a location-determining value, and the location-determining value is given to the subtractor 102 added as position feedback. The servo control device100 is designed in this way.

[0023] Next, a tax target will be set. 500 including the servo motor 300 , which is controlled by the servo control device 100 how it is controlled is described. Fig. 3 is a block diagram showing an area of ​​a machine tool including a motor, which serves as an example object of a control target. 500 the device for servo control 100 It serves its purpose, as shown. The device for servo control 100 forces the servo motor 300 the table 303 using a coupling mechanism 302 to move a workpiece (a job) that is on the table 303 It is attached, to be processed. The coupling mechanism 302 closes a coupling 3021 , which are connected to the servo motor 300 is coupled and has a ball screw drive 3023 , which is attached to the clutch 3021 is attached, and a nut3022 is in the ball screw drive 3023 screwed. With rotation of the servo motor. 300 The ball screw drive moves 3023 screwed screw nut 3022 in the axial direction of the ball screw drive 3023 . By moving the nut 3022 does the table move 303 .

[0024] A rotational angle position of the servo motor 300 is powered by a rotary encoder 301 determined to be a position determination unit connected to the servo motor 300 It serves this purpose. As described above, the detected signal is used as velocity feedback. The detected signal is processed by the integrator. 108 Integrated and used as position feedback. An output of a linear scale, located at the end of the ball screw drive. 3023 is appropriate to achieve a movement distance of the ball screw drive 3023Determining this can be used as position feedback. Furthermore, position feedback can be generated using an accelerometer. <Vorrichtung für Maschinelles Lernen 200>

[0025] The device for machine learning 200 Machine learning (hereinafter referred to as learning) is practiced, for example, on a key figure of a transfer function of the positional forward processing unit. 1092 and a key figure of a transfer function of the speed forward processing unit 1102 As described above in connection with patent document 2, the two learning methods interfere when learning on a position-forward condition and learning on a velocity-forward condition are performed simultaneously, increasing the amount of processed information for learning the position-forward control parameter and learning the velocity-forward control parameter. Therefore, the machine learning device200 In this embodiment, learning is based on the key figure of the transfer function of the velocity forward calculation unit. 110 separate from learning on the key figure of the transfer function of the position forward calculation unit 109 and performs learning on the key figure of the transfer function of the velocity forward calculation unit. 110 on the inner side (the inner loop) as the position-forward calculation unit 109 earlier than learning the key figure of the transfer function of the position forward calculation unit 109 out. In particular, the device for machine learning brings 200 the key figure of the transfer function of the position forward processing unit 1092 the position forward calculation unit 109 in order and learning the optimal value of the key figure of the transfer function of the speed forward processing unit 1102 the speed forward calculation unit 110Then the machine learning device 200 the key figure of the transfer function of the speed forward processing unit 1102 to an optimal value, obtained through learning, and learns the key figures of the transfer function of the positional forward processing unit. 1092 .

[0026] The reason why the device is for machine learning 200 the key figure of the transfer function of the speed forward processing unit 1102 rather learns than the key figure of the transfer function of the position forward processing unit 1092 will be related to Fig. 2 described. From the servo motor 300As seen, once a torque instruction is generated using a velocity instruction, the torque instruction is an instruction on the inner side, unlike the velocity instruction. Therefore, the calculation of the velocity forward condition, which is included in the torque instruction, is a process that is on the inner side, unlike the calculation of the position forward condition, which is included in the velocity instruction. Specifically, the output (the position forward condition) of the position forward calculation unit 109 is added to the adder 104 given, and the output (the velocity forward condition) of the velocity forward calculation unit 110 is added to the adder 107 given. The adder 104 is with the servo motor 300 via the subtractor 105 , the speed control unit 106 , and the adder 107connected. When learning regarding the optimization of the transfer function metric of the positional forward processing unit. 1092 earlier than learning regarding the optimization of the transfer function metric of the speed forward processing unit 1102 The speed forward condition is modified by learning with respect to the optimization of the measure of the transfer function of the speed forward processing unit. 1102 This will be carried out later. To adequately suppress positional errors, the machine learning device must be... 200 Learning regarding the optimization of the transfer function metric of the position forward processing unit 1092 Run again under the conditions of the changed speed condition. In contrast, if learning is performed with regard to the optimization of the transfer function metric of the speed forward processing unit. 1102earlier than learning regarding the optimization of the key figure of the transfer function of the position forward processing unit 1092 is, the device for machine learning 200 Learning regarding the optimization of the transfer function of the position forward processing unit 1092 under the condition of an optimized forward speed condition is performed through learning, and fluctuations in positional error are suppressed. Therefore, the machine learning device performs 200 Learning the key figure of the transfer function of the speed forward processing unit 1102 in the inner side (inner loop), as the position-forward calculation unit 109 earlier than learning the key figure of the transfer function of the position forward processing unit 1092 As a result, fluctuations in positional error are suppressed and high accuracy is achieved.

[0027] The device for machine learning200 learns the key figure of the transfer function of the position forward processing unit. 1092 the position forward calculation unit 109 and the transfer function indicator of the speed forward processing unit 1102 the speed forward calculation unit 110 by executing a predetermined machining program (hereinafter also referred to as the "learning machining program"). Here, a machining shape defined by the learning machining program is an octagon or a shape in which the corners are alternately replaced by arcs.

[0028] Fig. Figure 4 is a diagram to describe an engine operation when one machining shape is an octagon. Fig. Figure 5 is a diagram for describing engine operation when the machining shape is a shape in which the corners of an octagon are alternately replaced by arcs. Fig. 4 and Fig. 5. It is assumed that a table is moved in the x-axis and y-axis directions so that a workpiece (a job) is machined clockwise.

[0029] If the processing shape is an octagon, as in Fig. As shown in Figure 4, the rotational speed of a motor that moves the table in the y-axis direction decreases at the corner position. A1 , while the rotational speed of a motor moving the table in the x-axis direction increases. A rotational direction of the motor moving the table in the y-axis direction is achieved at the corner position. A2 Conversely, the table moves as if it had been linearly reversed in the y-axis direction. Furthermore, the motor that rotates the table in the x-axis direction rotates at the same speed and in the same direction from its position. A1 to the position A2 and from position A2 to the position A3The rotational speed of the motor that moves the table in the y-axis direction increases in a corner position. A3 on, while the rotational speed of a motor moving the table in the x-axis direction decreases. The rotational direction of a motor moving the table in the x-axis direction is determined at the corner position. A4 Conversely, the table moves linearly in the opposite direction along the x-axis. The motor that moves the table along the y-axis continues to rotate at the same speed in the same direction from its current position. A3 to the position A4 and from position A4 to the next corner position.

[0030] If the processing shape is one in which the corners of an octagon have been alternately replaced with arcs, as in Fig. As shown in Figure 5, the rotational speed of a motor moving the table in the y-axis direction decreases at the corner position. B1, while the rotational speed of a motor moving the table in the x-axis direction increases. A rotational direction of the motor moving the table in the y-axis direction is determined at the corner position. B2 Conversely, the table moves linearly in the opposite direction along the y-axis. Furthermore, the motor that moves the table along the x-axis rotates at the same speed in the same direction of rotation from its position. B1 by position B3 In contrast to the case where the processing shape is an octagon, as in Fig. As shown in Figure 4, the rotational speed of the motor that moves the table in the y-axis direction gradually decreases as the position is approached. B2 The rotation stops when the position is left. B2 , so that a processing mode of a bow before and after the position B2 is formed. The rotational speed of the motor that moves the table in the y-axis direction increases at the corner position.B3 , while the rotational speed of a motor moving the table in the x-axis direction decreases. The rotational direction of the motor moving the table in the x-axis direction is determined at the corner position. B4 Conversely, the table moves linearly in the opposite direction along the x-axis. The motor that moves the table along the y-axis continues to rotate at the same speed and in the same direction from its position. B3 to the position B4 and from position B4 to the next corner position. The rotational speed of the motor that moves the table in the x-axis direction gradually decreases as the position is approached. B4 , the rotation stops at the position B4 , and the rotational speed gradually increases as the position is left behind. B4 , so that a processing mode of a bow before and after the position B4 is formed.

[0031] In this embodiment, the machine learning device performs 200 The machine learning of the key figures is carried out by examining vibrations when a rotational speed is applied during linear control at the positions. A1 and A3 and the positions B1 and B3 The machining shape, determined by the machine learning program, is modified, and the influence on a positional error is examined. Although not used in the present embodiment, the machine learning device can be used 200 Investigate the phenomenon of idling (running due to inertia) that occurs when there is a rotational speed at the positions A2 and A4 and the positions B2 and B4 The processing method is reversed, and the influence of a positional error is examined.

[0032] The following section describes the device for machine learning. 200further described in detail. The following description, although it describes a case in which the device is used for machine learning, 200 enhanced learning is the learning that is performed by the machine learning device. 200 The invention is not expressly limited to enhanced learning, but can equally well be applied to a case in which the device is used for machine learning. 200 for example, supervised learning.

[0033] Before the respective functional blocks that are in the machine learning device 200 Included are described, first a basic mechanism of enhanced learning is described. A representative (corresponding to the machine learning device) 200(of the present invention) observes an environmental condition and selects a specific action. Then, the environment is modified based on the action. A specific reward is presented according to the environmental change, and the agent learns to select (decide on) a better action. While supervised learning presents a completely correct answer, in reinforcement learning, the reward often presents a fragmentary value based on the change in one area of ​​the environment. Consequently, the agent learns to select an action in such a way as to maximize the total future reward.

[0034] In this way, enhanced learning acquires a method of learning a suitable action based on the reciprocal effect of an action on the environment (which is an action to maximize the reward to be achieved in the future) by learning an action. In the present invention, this means obtaining such an action, for example, an action that influences the future, such as an action for selecting action information to reduce a positional error.

[0035] Although any learning method is used here as reinforcement learning, Q-learning, a method for learning a value Q(S,A) by choosing an action A under a given environmental state S, is described as an example in the following description. One goal of Q-learning is to choose an action A that has the highest value Q(S,A) as an optimal action among the actions A that can be taken under a given state S.

[0036] However, at the initial stage when Q-learning begins, the correct value of Q(S,A) is not known for all combinations of state S and action A. Therefore, the mediator learns the correct value of Q(S,A) by choosing different actions A under a given state S and selecting a better action based on the rewards given for the chosen actions A.

[0037] Since the goal is to maximize the total reward in the future, the aim is to ultimately establish a relationship of Q(S,A) = E[Σ(γ) t )r t to obtain ]. Here, E[] denotes an expected value, t denotes time, γ is a parameter called a discount factor, which will be described later, and Σ is the sum at time t. In this expression, the expected value is the value at which the state has changed according to an optimal action. However, since it is unclear which action is optimal in the Q-learning process, enhanced learning is performed while searching for an optimal action by performing several different actions. An update expression of such a value Q(S,A) can be represented by the expression below (Math. 3). Q ( S t + 1 , A t + 1 ) ← Q ( S t , A t ) + α ( r t + 1 + γ m a x A Q ( S t + 1 , A ) − Q ( S t , A t ) )

[0038] In expression 3, S denotes tan environmental state at a time t, and A t denotes an action at a time t. Through the action A t The state changes to S t+1 . r t+1 denotes a reward obtained by changing the state. Furthermore, the expression with max represents a multiplication of the value Q by γ when an action A with the highest value Q known at that time is performed under state S. t+1 was chosen. Here, γ is a parameter of 0 < γ ≤ 1 and is called a discount factor. Furthermore, α is a learning coefficient and lies in the range 0 < α ≤ 1.

[0039] Expression 3 denotes a procedure for updating a value Q(St,At) of an action A t in a state S t based on a recurring reward r t+1 , if the action A t is executed. This update function refers to what happens when the value maxa Q(S) t+1, A) the best action in the nearest state S t+1 associated with action A t is greater than the value Q(S) t ,A t ) of an action A t In state St, Q(St,At) is increased, and otherwise, Q(S) t , A t The value of a particular action in a given state approaches the value of the best action in the next state associated with that action. However, this difference between the values ​​varies depending on the discount factor γ and the reward r. t+1 , the update equation has a structure such that the value of the best action in a given state basically spreads to the value of an action in a state prior to that state.

[0040] A Q-learning method for generating a value function Q(S,A) table for all state-action pairs (S,A) in order to perform learning is known. However, setting up Q-learning can take a considerably long time if the values ​​of the value functions Q(S,A) of all state-action pairs need to be calculated once the number of states becomes too large.

[0041] Therefore, Q-learning can utilize an existing technique called a deep Q-network (DQN). Specifically, the facilitator can determine the value of the values ​​Q(S,A) by constructing a value function. Q using a suitable neural network, calculate and the value function Q The appropriate neural network can be used to approximate the desired outcome by adapting the neural network process. Using DQN, it is possible to reduce the time required to establish Q-learning. Details of DQN are disclosed in the non-patent document, such as the one below. <nicht-patent-dokument>

[0042] “Human-level control through deep reinforcement learning”, Volodymyr Mnihl [online], [accessed on January 17, 2017], Internet<URL:http: / / files.davidqiu.com / research / nature14236.pdf>

[0043] The device for machine learning 200 It performs the Q-learning described above. The machine learning device 200 In particular, learns a value function Q for choosing an action A to adjust the key figures a i and b j the transfer function of the position forward processing unit 1092 or the key figures c i and d j the transfer function of the speed forward processing unit 1102 , associated with a servo state S, such as the values ​​of the key figures a i and b j (i and j ≥ 0) of the transfer function of the position forward processing unit 1092 or the values ​​of the key figures c i and d j (i and j ≥ 0) of the transfer function of the speed forward processing unit 1102 the device for servo control 100 , and commands, and feedback. The command includes a position command, the feedback includes position error information from the device for servo control. 100 one, which is obtained by running the machine learning program. First, the machine learning device learns 200 the values ​​of the key figures c i and d j (i and j ≥ 0) of the transfer function of the speed forward processing unit 1102 , and then, she learns the values ​​of the key figures a i and b j (i and j ≥ 0) of the transfer function of the position forward processing unit 1092 In the following description, although learning the values ​​of the key figures c i and d j (i and j ≥ 0) of the transfer function of the speed forward processing unit 1102 As described, learning the values ​​of the key figures is a i and b j (i and j ≥ 0) of the transfer function of the position forward processing unit 1092 executed in a similar manner.

[0044] The device for machine learning 200 observes the status information S including a servo state such as commands and feedback including the position command and position error information of the servo control device. 100 at the positions A1 and A3 and the positions B1 and B3 the processing method by executing the machine learning program based on the key figures c i and d j the transfer function of the speed forward processing unit 1102 to determine action A. The machine learning device 200 It returns a reward every time action A is performed. The machine learning device 200 It searches for the optimal action A such that a future total reward is maximized through trial-and-error learning. This allows the machine learning device to... 200 Choose an optimal action A (namely, the optimal key figures c). i and d j the transfer function of the speed forward processing unit 1102 ) regarding a state S including the servo state such as commands and feedback including the position command and position error information of the servo control device 100 , obtained by basing the machine learning program on the key figures c i and d j the transfer function of the speed forward processing unit 1102 The operation is carried out. The rotation direction of the servo motor in the x-axis and y-axis directions does not change at the positions. A1 and A3 and the positions B1 and B3 , and the machine learning device 200 can the key figures c i and d j the transfer function of the speed forward processing unit 1102 Learning during linear operation.

[0045] Therefore, the device can be used for machine learning. 200 such an action A (namely the key figures c) i and d j the speed forward processing unit 1102 ) select an action A that reduces the positional error obtained by executing the machine learning program by choosing an action A that maximizes the value Q among the actions A applied to the measures c i and d j the transfer function of the speed forward processing unit 1102 associated with a specific state S based on the learned value function Q.

[0046] Fig. Figure 6 is a block diagram showing the machine learning device. 200 as shown in the first embodiment of the present invention. As shown in Fig. Figure 6 shows the device for machine learning. 200 , a status information acquisition unit 201 , a learning unit 202 , an action information output unit 203 , a value function storage unit 204 , and an optimization action information output unit 205 one, in order to carry out enhanced learning. The learning unit 202 includes a reward dispensing unit 2021 , a value function update unit 2022 and an action information generation unit 2023 a.

[0047] The status information acquisition unit 201 detected by the device for servo control 100 , the state S including a servo state such as commands and feedback including the position command and position error information of the servo control device 100 , obtained by basing the machine learning program on the key figures c i and d j the transfer function of the speed forward processing unit 1102 the device for servo control 100 , was executed. The state information S corresponds to an environmental state S during Q-learning. The state information acquisition unit 201 provides the recorded status information S of the learning unit 202 out of.

[0048] The key figures c i and d j the speed forward processing unit 1102 At the point in time when Q-learning initially starts, the initial values ​​are generated in advance by a user. In the present embodiment, the initial setting values ​​of the key figures c are i and d j the speed forward processing unit 1102 , generated by the user, adapted to optimal values ​​through enhanced learning. The coefficient α of the double differentiator 1101 the speed forward calculation unit 110 is set to a fixed value (e.g., α = 1). The initial setting values ​​of the key figures c i and d j the speed forward processing unit 1102 In equation 2, the variables are chosen such that c0 = 1, c1 = 0, c2 = 0, ..., and c m = 0, and d0 = 1, d1 = 0, d2 = 0, ..., and d n = 0. The dimensions m and n of the key figures c i and d j are predetermined. Namely, 0 ≤ i ≤ m for c i , and 0 ≤ j ≤ n for d j The coefficient β of the differentiator 1091 the forward position calculation unit 109 is set to a fixed value (e.g., β = 1). The initial setting values ​​of the key figures a i and b j the position forward processing unit 1092 In equation 1, the values ​​are chosen such that a0 = 1, a1 = 0, a2 = 0, ..., and am = 0, and b0 = 1, b1 = 0, b2 = 0, ..., and b n = 0. The dimensions m and n of the key figures a i and b j are predetermined. Namely, 0 ≤ i ≤ m for a i and 0 ≤ j ≤ n for b j The same values ​​as the initial setting values ​​of the key figures c i and d j the transfer function of the speed forward processing unit 1102 can be used for the initial values ​​of the key figures a i and b j can be applied. If a machine tool is modified by an operator, machine learning can be performed, using the modified values ​​as the initial values ​​of the key performance indicators (KPIs). i and b j and the key figures c i and d j .

[0049] The learning unit 202 is a unit that learns the value Q(S,A) when a specific action A is chosen under a specific environmental state S.

[0050] The reward dispensing unit 2021 is a unit that calculates a reward when action A is chosen under a specific state S. This involves a series (a positional error series) of positional errors, the state variables of state S are denoted by PD(S), and a positional error series, the state variables with respect to the state information S', modified from state S due to the action information A (correction of the key figures c). i and d j (i and j are 0 or positive integers) of the speed forward processing unit 1102 The value representing the positional error in state S is denoted by PD(S'). Furthermore, the weighting function value of the positional error in state S is a value calculated based on a predetermined weighting function f(PD(S)). If e is a positional error, the following functions, for example, can be used as the weighting function f: A function that calculates an integrated value of the magnitude of a positional error. ∫ | e | dt

[0051] A function that calculates an integrated value by weighting the magnitude of a positional error over time. ∫ t | e | dt

[0052] A function that calculates an integrated value of a 2n-th power (n is a natural number) of the magnitude of a positional error. ∫ e 2 n dt ( n ist eine nat u ¨ rliche Zahl )

[0053] A function that calculates the maximum value of the magnitude of a positional error. Max { | e | }

[0054] f(PD(S')) is an evaluation function value of the positional error of the servo control device 100 , which are based on the velocity forward calculation unit 110 is operated, after improvement regarding the state information S', compensated by the action information A, and f(PD(S)) is an evaluation function value of the position error of the device for servo control. 100 , which are based on the velocity forward calculation unit 110 is operated before improvements are made to the state information S, before these are compensated for by the action information A. In this case, the reward output unit represents 2021 The value of a reward becomes negative if the evaluation function value f(PD(S')) is greater than the evaluation function value f(PD(S)).

[0055] On the other hand, the reward output unit represents 2021 The value of a reward is set to a positive value if the evaluation function value f(PD(S')) is less than the evaluation function value f(PD(S)). The reward output unit 2021 sets the value of a reward to zero if the evaluation function value f(PD(S')) is equal to the evaluation function value f(PD(S)).

[0056] Furthermore, the reward dispensing unit 2021 The negative value increases according to a ratio as soon as the evaluation function value f(PD(S')) of the positional error in state S' after execution of action A is greater than the evaluation function value f(PD(S)) of the positional error in the previous state S. That is, the negative value can increase according to the degree of increase in the evaluation function value of the positional error. In contrast, the reward output unit 2021 The positive value decreases according to a ratio as soon as the evaluation function value f(PD(S')) of the positional error in state S' after execution of action A is smaller than the evaluation function value f(PD(S)) of the positional error in the previous state S. That is, the positive value can increase according to the degree of reduction in the evaluation function value of the positional error.

[0057] The value function value update unit 2022 updates the value function memory unit 204 The stored value function Q is updated by performing Q-learning based on the state S, the action A, the state S' when the action A is applied to state S, and the reward value calculated in this way. Updating the value function Q can be performed through online learning, batch learning, or mini-batch learning. Online learning is a learning method where a specific action A is applied to a current state S, and the value function Q is updated immediately as soon as the current state S transitions to a new state S'. Batch learning is a learning method where a specific action A is applied to a current state S and repeated when the transition from state S to a new state S' is reached, learning data is collected, and the value function Q is updated using all the collected learning data.Mini-batch learning is a learning method that lies between online learning and batch learning and involves updating the value function Q whenever a certain amount of learning data has been collected.

[0058] The action information generation unit 2023 The action A collects information about the current state S during the Q-learning process. The action information generation unit 2023 generates action information A and outputs the generated action information A to the action information output unit. 203 to initiate a process (corresponding to action A of Q-learning) to correct the key figures c i and d j the speed forward processing unit 1102 the device for servo control 100 to be carried out in the process of Q-learning. More precisely, the action information generation unit adds or subtracts. 2023 the key figures c i and d j the speed forward processing unit 1102 , which are included in action A, stepwise (e.g. with a step of about 0.01) with respect to the characteristics of the speed forward processing unit, which is included, for example, in state S.

[0059] As soon as the key figures c i and d j the speed forward processing unit 1102 The action information generation unit can be activated if the value is increased or decreased, if state S changes to state S', and a positive reward is returned. 2023 Choose a guideline such that an action A' that further reduces the value of the positional error, such as gradually increasing or decreasing the key figures c, is possible. i and d j the speed forward processing unit 1102 similar to the previous action, when the next action A' is chosen.

[0060] In contrast, the action information generation unit 2023 , as soon as a negative reward is returned, choose a policy such that an action A' further reduces the value of the location error so that it becomes smaller than the previous value, such as gradually increasing or decreasing the measures c i and d j the speed forward processing unit in the opposite direction to the previous action, for example when the next action A' is chosen.

[0061] The action information generation unit 2023 A guideline for choosing an action A' can correspond to a known method, such as a greedy procedure for choosing an action A' with the highest value Q(S,A) among the currently estimated actions A, and an ε-greedy procedure for randomly choosing an action A' with a certain small probability ε, and choosing an action A' with the highest value Q(S,A) in other cases.

[0062] The action information output unit 203 is a unit that contains the action information A, which is provided by the learning unit 202 is output to the servo control device 100 sends. As described above, the device is suitable for servo control. 100 the current state S (i.e., the current set of indicators c) i and d j the speed forward processing unit 1102 ) based on the action information in order to transition to the next state S' (i.e., the compensated metrics of the velocity forward processing unit) 1102 ).

[0063] The value function storage unit 204 is a storage device that stores the value function Q. The value function can be stored as a table (hereinafter referred to as an action value table) for, for example, each state S and each action A. The value function Q, which is stored in the value function storage unit 204 The value function update unit is used to store the stored value. 2022 updated. In addition, the value function Q, which is stored in the value function memory unit, can be updated. 204 is stored, with other devices for machine learning 200 to be shared. Once the value function Q is combined with a variety of machine learning devices. 200 When shared, it is possible because enhanced learning involves a kind of distribution across machine learning devices. 200 This can be done to improve the efficiency of enhanced learning.

[0064] The optimization action information output unit 205 generates an action information A (hereinafter referred to as "optimization action information") which the speed forward calculation unit 110 prompts the execution of a procedure to maximize the value Q(S,A) based on the value function Q defined by the value function update unit. 2022 is updated through the application of Q-learning. In particular, the optimization action information output unit acquires 205 the value function Q, which is stored in the value function memory unit 204 is stored. As described above, the value function Q is stored by the value function update unit. 2022 Updated through the application of Q-learning. The optimization action information output unit. 205 generates the action information based on the value function Q and passes the generated action information to the servo control device. 100 (the speed forward processing unit) 1102 the speed forward calculation unit 110 ). The optimization action information includes information that defines the key performance indicators (KPIs) c. i and d j the speed forward processing unit 1102 similar to the action information that the action information output unit 203 Correct the output generated during the Q-learning process.

[0065] In the device for servo control 100 are the key figures c i and d j the speed forward processing unit 1102 Compensation is performed based on action information. The device can be used for machine learning with the method described above. 200 Learning and optimization of key performance indicators a i and b j the position forward processing unit 1092 similar to learning and optimizing the key performance indicators of the speed forward processing unit 1102 after executing the optimization of the key figures c i and d j the speed forward processing unit 1102 Perform the process and work in such a way as to reduce the positional error value. As described above, it is possible to achieve this by using the machine learning device. 200 According to the present embodiment, the adjustment of the parameters of the speed forward calculation unit 110 and the position forward calculation unit 109 the device for servo control 100 to simplify.

[0066] The inventors present optimized the key performance indicators (KPIs) c i and d j the speed forward processing unit 1102 using the machine learning device 200 carried out, which utilizes enhanced learning and uses an octagon as an editing format determined by a learning editing program, and optimization of key performance indicators. i and b j the position forward processing unit 1092 The procedure was carried out and a range of variation of the positional error was examined. In addition, the inventors present optimized the key performance indicators for comparison. i , and b j the position forward processing unit 1092 using the machine learning device 200 carried out, which utilizes enhanced learning and uses an octagon as an editing format determined by a learning editing program, and optimization of key performance indicators (KPIs) c. i and d j the speed forward processing unit 1102 The study was conducted and examined a range of positional error variations. The results showed that the machine learning setup time could be reduced, variations in positional error could be suppressed more effectively, and higher accuracy could be achieved by optimizing the key performance indicators (KPIs). i and b j the position forward processing unit 1092 after optimizing the key figures Ci and d j the speed forward processing unit 1102 was carried out.

[0067] In each of the devices for servo control 100 and the machine learning device 200 An arithmetic processing unit reads an application or operating system from the additional storage device and executes the read application software or operating system in the main storage device to perform arithmetic processing based on the read application software or operating system. The arithmetic processing unit also controls various types of hardware in each device based on the computational result. In this way, the functional blocks of the present embodiment are realized. That is, the present embodiment can be realized through the cooperation of hardware and software.

[0068] Since the device is for machine learning 200 Since machine learning involves a large amount of computation, graphics processing units (GPUs) can be attached to a personal computer and used for arithmetic processing related to machine learning using a technique called "general-purpose computing on graphics processing units" (GPGPUs). This allows the machine learning device to... 200 To perform even higher speed processing, a computer cluster can be built using a large number of computers equipped with such GPUs, and the large number of computers included in the computer cluster can perform parallel processing.

[0069] Next, an operating procedure for the machine learning device will be described. 200 during Q-learning according to the present embodiment with reference to the flowchart of the Fig. 7 described.

[0070] In step S11 The status information acquisition unit captures 201 the status information S from the device to the servo control 100 The captured state information is sent to the value function update unit. 2022 and the action information generation unit 2023 output. As described above, the state information S is information regarding the state of Q-learning and includes the key figures c. i and d j the speed forward processing unit 1102 at the time of the step S11 one. In this way, the status information acquisition unit records 201 a position error set PD(S) corresponding to a predetermined feed rate and a machining shape of a circle, provided that the characteristic values ​​of the speed forward calculation unit 110 These are initial values.

[0071] As described above, the key figures c i and d j the speed forward processing unit 1102 In the initial state S0, the values ​​are set such that, for example, c0 = 1, c1 = 0, c2 = 0, ..., and c m = 0, and d0 = 0, d1 = 0, d2 = 0, ..., and d n = 0.

[0072] The positional error value PD(S0) in state S0 of the subtractor 102 at a point in time when Q-learning initially starts, is obtained by the machine learning device 100 is operated according to a learning processing program. The situation command creation unit 101 It issues position commands sequentially according to a predetermined processing pattern (e.g., a processing pattern of an octagon) determined by the processing program. For example, a target position value is set by the position command creation unit according to the processing pattern of an octagon. 101 output, and the target position value is sent to the subtractor. 102 , the speed forward calculation unit 110 , and the machine learning device 200 output. The subtractor 102 There is a difference between the target position value and the position determination output from the integrator. 108 at the positions A1 and A3 and the positions B1 and B3 the processing method to the device for machine learning 200 The position error PD(S0) is expressed as the difference between the target position value and the position determination output from the integrator. 108 at the positions A2 and A4 and the positions B2 and B4 the processing method in the machine learning device 200 can be taken as the positional error PD(S0).

[0073] In step S12 generates the action information generation unit 2023 new action information A and gives the generated new action information A to the servo control device 100 via the action information output unit 203 Next. The action information generation unit 2023 Outputs new action information A based on the guideline described above. The servo control device 100 , which has received this action information A, drives a machine tool including a servo motor 300 according to the state S', obtained by correcting the key figures c i and d j the speed forward processing unit 1102 in connection with state S based on the received action information. As described above, the action information corresponds to action A in Q-learning.

[0074] In step S13 The status information acquisition unit captures 201 the position error PD(S') in the new state S' of the subtractor 102 and records the key figures c i and d j from the speed forward processing unit 1102 In this way, the status information acquisition unit captures 201 the positional error PD(S') according to the processing form of an octagon (specifically, the positions A1 and A3 and the positions B1 and B3 the processing method) and the key figures c i and d j in state S' of the speed forward processing unit 1102 The recorded status information is sent to the reward dispensing unit. 2021 issued.

[0075] In step S14 determines the reward output unit 2021 an order of magnitude between the evaluation function value f(PD(S')) of the positional error in state S' and the evaluation function value f(PD(S)) of the positional error in state S, and sets a reward to a negative value in step S15 The reward output unit is set as soon as f(PD(S')) > f(PD(S)). As soon as f(PD(S')) < f(PD(S)), the reward output unit is set. 2021 the reward from a positive value in step S16 one. As soon as f(PD(S')) = f(PD(S)), the reward output unit represents 2021 the reward to zero in step S17 one. The reward dispensing unit 2021 can apply a weighting to the negative and positive reward values.

[0076] As soon as one of the steps S15 , S16 and S17 When it ends, the value function update unit is updated. 2022 the value function Q, which is stored in the value function memory unit 204 The value is stored based on the reward value calculated in one of the steps in S18. Afterwards, the history returns to step S11 The process returns, and the process described above is repeated, with the value function Q setting itself to a suitable value. The process can end with the condition that the process described above is repeated for a predetermined period. Although online updates occur in step S18 To illustrate, batch updates or mini-batch updates can be performed instead of online updates.

[0077] In the present embodiment, the device provides machine learning 200 thanks to the operating procedure, which relates to Fig. Section 7 describes the advantages of being able to use a suitable value function to adjust the key figures c. i and d j the speed forward processing unit 1102 to obtain and optimize key performance indicators c i and d j the speed forward processing unit 1102 to simplify. Next, an operating procedure is implemented during the generation of the optimization action information by the optimization action information output unit. 205 with reference to the flowchart of the Fig. 8 described. First, the optimization action information output unit captures 205 in step S21 the value function Q, which is in the value function storage unit 204 is stored. As described above, the value function Q is stored by the value function update unit. 2022 , which operates Q-Learning, updated.

[0078] In step S22 generates the optimization action information output unit 205 The optimization action information is based on the value function Q and provides the generated optimization action information and the speed of forward processing. 1102 the device for servo control 100 out. The machine learning device 200 optimizes the key performance indicators c i and d j the speed forward processing unit 1102 using the operating procedure described above, and then performs learning and optimization of the key performance indicators. i , and b j the position forward processing unit 1092 similar procedures.

[0079] In the present embodiment, the machine learning device generates 200 thanks to the operating procedure, which relates to Fig. 8 describes the optimization action information based on the value function Q obtained through learning, and the device for servo control. 100 Can the adjustment of the currently set key figures c i and d j the speed forward processing unit 1102 Based on the optimization action information, simplify and reduce the value of the positional error. Furthermore, the key figures of the forward processing unit's speed are... 1102 set to the initial values ​​for higher dimensions, and the machine learning device 200 Performs learning, which can further reduce the positional error value. To adjust the key figures a i and b j the position forward processing unit 1092 The positional error value can be adjusted similarly to the adjustment of the key figures c. i and d j the speed forward processing unit 1102 will be reduced.

[0080] In the first embodiment, the reward output unit calculates 2021 The reward value is determined by comparing the evaluation function value f(PD(S)) of the positional error in state S, calculated based on a predetermined evaluation function f(PD(S)) using the positional error PD(S) in state S as input, with the evaluation function value f(PD(S')) of the positional error in state S', calculated based on an evaluation function f(PD(S')) using the positional error PD(S') in state S' as input. However, the reward output unit 2021 It's also possible to add an element other than the positional error when calculating the reward value. For example, the machine learning device can 200 Add at least one position-forward controlled speed command, which is controlled by the adder. 104 was output, a difference between a velocity feedback and a position-forward controlled velocity command, or a position-forward controlled torque command, which was issued by the adder 107 in addition to the positional error output by the subtractor 102 , was issued. (Second embodiment)

[0081] In the first embodiment, a device for machine learning was incorporated into a servo control device including the position forward calculation unit. 109 and the speed forward calculation unit 110 The present embodiment describes a device for machine learning of a servo control device, including a current forward calculation unit in addition to the position forward calculation unit and the velocity forward calculation unit.

[0082] Fig. Figure 9 is a block diagram showing part of a servo control device according to the present embodiment. As shown in Fig. As shown in 9, the servo control device of the present embodiment further includes a subtractor 111 , a power control unit 112 , an adder 113 , and a current forward calculation unit 114 one, which is defined by the dashed line region in Fig. 9 are indicated, in addition to the components of the device for servo control. 100 as in Fig. 1 shown. The subtractor 111 Calculates the difference between a speed-forward controlled torque setpoint output by the adder. 107 and a feedback current detection value, and indicates the difference to the current control unit 112 as a power fault. The power control unit 112 It calculates a current setpoint based on the current error and passes the current setpoint to the adder. 113 The current forward calculation unit 114 It calculates a current setpoint based on the position setpoint and gives the current setpoint to the adder. 113 next. The adder 113 adds the current setpoint and the output value of the current forward calculation unit. 114 , sends the added value to the servo motor 300 as a forward-controlled current setpoint, and drives the servo motor 300 on. The machine learning device 200 learns the key figures of the transfer function of the current forward calculation unit 114 similar to key figures c i and d j the speed forward processing unit 1102 .

[0083] In the present embodiment, the current command is, if from the servo motor 300 It appears to be a command on the inner side instead of the torque command, and the torque command is a command on the inner side instead of the speed command. Provided the servo motor... 300 As seen, the current forward control, velocity forward control, and position forward control are arranged such that the sequence runs from the inner side to the outer side. Similar to the first embodiment, it is therefore desirable that learning regarding the optimization of the velocity forward control's parameters is performed earlier than learning regarding the optimization of the position forward control's parameters. Since the current forward control is located on the inner side instead of the velocity forward control, it is further desirable that learning regarding the optimization of the current forward control's parameters is performed earlier than learning regarding the optimization of the velocity forward control's parameters. However, if the current forward control has a small influence on the position error, the machine learning device can be used. 200 Perform learning regarding the optimization of the velocity forward calculation unit's key figures and learning regarding the current forward calculation unit's key figures, and then perform learning regarding the optimization of the attitude forward calculation unit's key figures. This case is an example of a situation where speed forward learning is performed before attitude forward learning.

[0084] In the embodiment described above, the device was used for machine learning. 200 described as performing learning with respect to the optimization of the key figures of the position forward calculation unit and the velocity forward calculation unit during linear operation, where the direction of rotation of the servo motor in the x-axis and y-axis directions is not changed, and learning with respect to the optimization of the key figures of the position forward calculation unit, the velocity forward calculation unit, and the current forward calculation unit. However, the present invention is not limited to learning during linear operation, but can be applied to learning during nonlinear operation. For example, if the machine learning device 200 The machine learning device can perform learning regarding the optimization of the key figures of the position forward calculation unit and the velocity forward calculation unit, or learning regarding the optimization of the position forward calculation unit, the velocity forward calculation unit, and the current forward calculation unit, and learning regarding the optimization of the key figures of the forward calculation unit, so that feedback is improved. 200 a difference between the target position and the actual position, output by the integrator 108 at the positions A2 and A4 and the positions B2 and B4 The processing method can be interpreted as a positional error, and enhanced learning can be implemented by using these positional errors as identification information. At the positions A2 and A4 and the positions B2 and B4 is the direction of rotation of the servo motor 300 in the y-axis direction or vice versa in the x-axis direction, resulting in nonlinear operation and feedback effects. In this case, the machine learning device can perform the learning of the transfer function parameters of the forward processing unit.

[0085] The servo control unit of the servo control device as described above and the components included in the machine learning device can be implemented using hardware, software, or a combination of both. The servo control method, executed through the cooperation of the components included in the servo control device as described above, can be implemented using hardware, software, or a combination of both. Here, "implemented by software" means implementation when a computer reads and executes a program.

[0086] The program can be stored on various types of seamless, computer-readable media and made available to a computer. Seamless computer-readable media include various types of physical storage media. Examples of seamless computer-readable media include magnetic recording media (e.g., a flexible floppy disk and a hard disk drive), magneto-optical storage media (e.g., a magneto-optical floppy disk), CD-ROM (read-only memory), CD-R, CD-R / W, semiconductor memory (e.g., a mask ROM, a PROM (programmable ROM), an EPROM (erasable PROM), a flash ROM, and RAM (random access memory)). Furthermore, programs can be made available to a computer using various types of temporary computer-readable media.

[0087] The embodiment described above is a preferred embodiment of the present invention. However, the scope of the present invention is not limited to this embodiment; rather, the present invention can be implemented in numerous variations without departing from the spirit of the present invention. <Abwandlung, in der die Vorrichtung zur Servosteuerung die Vorrichtung für maschinelles Lernen beinhaltet>

[0088] Although the device for machine learning 200 is designed to allow the device to be used for servo control 100 In the embodiments described above, the separate device may include some or all of the machine learning functions. 200 through the servo control device 100 to be realized. <Freiheiten in der Anlagenkonfiguration>

[0089] In the embodiment described above, the device for machine learning 200 and the device for servo control 100 They communicate with each other in a one-to-one interaction. However, for example, a machine learning device can... 200 and a variety of servo control devices 100 communicating with each other via the network 400 be connected and the machine learning of each device for servo control 100 This can be carried out. In this case, a widely used processing system can be applied, in which the respective functions of the machine learning device are employed. 200 distributed to a large number of suitable servers. The machine learning device's functions 200 This can be achieved by using a virtual server function or similar in a cloud. If there are a variety of machine learning devices available... 200 - 1 until 200-n corresponding to a variety of respective devices for servo control 100 - 1 until 100-n Devices with the same name type, the same specification, or the same series can be used for machine learning. 200 - 1 until 200-n be designed to enhance the learning outcomes of the machine learning devices 200 - 1 until 200-n to share. This allows for the creation of a better optimized model. Reference symbol list 10: Servo control system 100: Device for servo control 101: Situation Command Recognition Unit 102: Subtractor 103: Position control unit 104: Adder 105: Subtractor 106: Speed ​​control unit 107: Adder 108: Integrator 109: Position forward calculation unit 110: Speed ​​forward calculation unit 200: Device for machine learning 201: Status Information Acquisition Unit 202: Learning Unit 203: Action Information Output Unit 204: Value function memory unit 205: Optimization Action Information Output Unit 300: Engine 400: Network

Claims

[] What is claimed is: [1] A machine learning device (200) for performing machine learning in conjunction with the optimization of key figures of at least two forward computation units (109, 110) with respect to a servo control device (100) to control a servo motor (300) so that a shaft of a machine tool or an industrial machine is driven using the forward control in which the at least two forward computation units form multiple loops, wherein an instruction that is balanced by a forward condition calculated by one of the at least two forward computation units is an instruction on an inner side, as seen from the servo motor, than another instruction that is balanced by a forward condition calculated by the other forward computation unit, and after the machine learning with respect to the optimization of the key figures of one forward computation unit has been performed,Machine learning is performed to optimize the key performance indicators (KPIs) of the other forward computing unit based on the optimized KPIs of the first forward computing unit, which were obtained through machine learning to optimize the KPIs of the first forward computing unit. [2] The machine learning device according to claim 1, wherein the at least two forward calculation units are at least two forward calculation units under a position forward calculation unit (109) for calculating a first forward condition of a velocity command based on a position command, a velocity forward calculation unit (110) for calculating a second forward condition of a torque command based on a position command, and a current forward calculation unit (114) for calculating a third forward condition of a current command based on a position command, The one command and the other command are two commands under the speed command, the torque command, and the current command, and the servo motor is driven according to the torque command or the current command. [3] The machine learning device according to claim 2, wherein the first forward calculation unit is the velocity forward calculation unit and the other forward calculation unit is the position forward calculation unit. [4] The machine learning device according to claim 2, wherein the servo control device includes the position forward calculation unit, the velocity forward calculation unit, and the current forward calculation unit, and one forward calculation unit is the velocity forward calculation unit or the current forward calculation unit, and the other forward calculation unit is the position forward calculation unit. [5] The machine learning device according to any one of claims 1 to 4, wherein the basic setting values ​​of the transfer function indicators of the other forward calculation unit are the same values ​​as the basic setting values ​​of the transfer function indicators of the one forward calculation unit. [6] The machine learning device according to any one of claims 1 to 5 further comprising: a state information acquisition unit (201) for obtaining, from the servo control device, the state information including a servo state which includes at least one position error and a linkage of the transfer function indicators of one or the other forward calculation unit, by the servo control device executing the predetermined machining program, an action information output unit (203) for outputting action information including adaptation information of the linking of the key figures included in the status information of the servo control device; a reward output unit (2021) for outputting a reward value for enhanced learning, based on the positional error included in the state information; and a value function update unit (2022) for updating a value function based on the reward value output by the reward output unit, the state information, and the action information. [7] The machine learning device according to claim 6, wherein the reward output unit outputs the reward value based on an amount of positional error. [8] The machine learning device according to claim 6 or 7 further comprising: an optimization action information output unit (205) for generating and outputting a link of the transfer function indicators of one or the other forward calculation unit based on the value function updated by the value function update unit. [9] A servo control system comprising: the machine learning device according to any one of claims 1 to 8; and a device for servo control for controlling a servo motor, for driving a shaft of a machine tool or an industrial machine using forward control, in which at least two forward calculation units form several loops. [10] A servo control device comprising: the machine learning device according to any one of claims 1 to 8; and at least two forward calculation units, wherein the servo control device controls a servo motor, for driving a shaft of a machine tool or an industrial machine by means of forward control in which the at least two forward calculation units form several loops. [11] A method for machine learning of a machine learning device (200) for performing machine learning with respect to the optimization of the key figures of at least two forward computation units (109, 110) with respect to a servo control device (100) for controlling a servo motor (300) which drives a shaft of a machine tool or an industrial machine by means of forward control in which the at least two forward computation units form several loops, wherein, if an instruction that is balanced by a forward condition calculated by one of the at least two forward calculation units is an instruction on an inner side, as seen from the servo motor, than another instruction that is balanced by a forward condition calculated by the other forward calculation unit, After the machine learning regarding the optimization of the key figures of one forward computing unit has been performed, the machine learning regarding the optimization of the key figures of the other forward computing unit is performed based on the optimized key figures of one forward computing unit, which were obtained by machine learning regarding the optimization of the key figures of one forward computing unit.

Images