Output device, control device and method for outputting assessment functions and machine learning results

Abstract

Output device (200, 200A), comprising: an output unit (206, 209) that outputs a variety of evaluation functions used by a machine learning device (100) that performs machine learning of parameters of components of a servo control device (300) that controls a servo motor (400) for driving an axis of a machine tool, robot or industrial machine, and outputs a machine learning result of each of the evaluation functions, and an information retention unit (201, 207) that receives the machine learning result from the servo control device (300) and / or the machine learning device (100).

Description

BACKGROUND OF THE INVENTIONField of invention

[0001] The present invention relates to an output device, a control device, and a method for outputting evaluation functions and machine learning results. More specifically, the present invention relates to an output device that performs machine learning using a plurality of evaluation functions, respectively, by means of a machine learning device that performs machine learning using an evaluation function with respect to a servo control device that controls a servo motor for driving an axis of a machine tool, a robot, or an industrial machine, and outputs the plurality of evaluation functions and the machine learning results of the corresponding evaluation functions, a control device including the output device, and a method for outputting evaluation functions and machine learning results. State of the art

[0002] As a related technique to the present invention, for example, Patent Document 1 discloses a control device including: a feedback control unit that inputs an error between a reference command and a control amount output by an external control target to a controller, which controls the error using a predetermined control parameter to control the control target with an operation amount output by the controller; and a setting unit that sets the control parameter based on a learning result obtained through a machine learning process. An example of a machine control system including a control device is a system that controls a linear motion of a slider by controlling the drive of a rotary motor.Examples of control parameters include a position loop gain, a velocity loop gain, a velocity loop integration time constant, a torque command filter time constant, a notch filter frequency, and the like. It is also stated that reinforcement learning can be used as the machine learning process.

[0003] Patent Document 1: International Application Publication No. WO 2018 / 151215A1. DE 10 2018 000 342 A1 discloses a numerical controller having a machine learning device that performs machine learning of adjusting a setting value used in override control. The machine learning device acquires state data indicating states of the numerical controller and a machine, sets reward conditions, calculates a reward based on the state data and the reward conditions, performs machine learning of adjusting the setting value used in override control, and determines the adjustment of the setting value used in override control based on a machine learning result and the state data. SUMMARY OF THE INVENTION

[0004] In Patent Document 1, the control parameter is adjusted based on the learning result of the machine learning process using reinforcement learning. Because a user such as an operator cannot be involved in the machine learning, the error obtained by adjusting the control parameter may not be suppressed as desired by the user. For example, a certain user may find it important to suppress a high-frequency vibration even though the error is not sufficiently small, while another user may find it important to suppress the error to a sufficiently small value even though a high-frequency vibration remains.However, when parameters of components of a servo control device that controls a servo motor for driving an axis of a machine tool, a robot, or an industrial machine are machine-learned by a machine learning device, because the evaluation function used by the machine learning device is generally not displayed, an operator cannot understand what learning effect is obtained when a specific evaluation function is used.

[0005] It is an object of the present invention to provide an output device that outputs a plurality of evaluation functions and a machine learning result of each of the evaluation functions so that a relationship between the evaluation function and the learning result can be determined, and further to provide a control device including the output device and a method for outputting evaluation functions and machine learning results.

[0006] This object is achieved by an output device having the features of patent claim 1 and by a control device having the features of patent claim 11. The object is further achieved by an output method having the features of patent claim 13.

[0007] (1) An output device according to the present invention is an output device (for example, the output device 200, 200A described below) that includes: an output unit (for example, the control unit 205 and the display unit 209 or the control unit 205 and the storage unit 206, each described below) that outputs a plurality of evaluation functions used by a machine learning device (for example, the machine learning device 100 described below) that performs machine learning of parameters of components of a servo control device (for example, the servo control device 300 described below) that controls a servo motor (for example, the servo motor 400 described below) for driving an axis of a machine tool, a robot, or an industrial machine, and outputs a machine learning result of each of the evaluation functions;and an information obtaining unit (for example, the learning information obtaining unit 201 or the servo information obtaining unit 207, each described below) that obtains the machine learning result from the servo control device and / or the machine learning device;

[0008] (2) In the output device according to (1), the output unit may include a display unit (for example, the output unit 209 described below) that displays the plurality of evaluation functions and the machine learning result obtained for each of the evaluation functions on a display screen.

[0009] (3) The output device according to (1) or (2) may further include an information output unit (for example, the servo information output unit 208 described below) that outputs a parameter selected based on the machine learning result of each of the plurality of evaluation functions from a plurality of parameters machine-learned using the plurality of evaluation functions to the servo control device.

[0010] (4) The output device according to any one of (1) to (3) may further include an information output unit (for example, the learning information output unit 202 described below) that outputs a change command for an evaluation function selected from the plurality of evaluation functions or an evaluation function different from the plurality of evaluation functions to the machine learning device.

[0011] (5) In the output device according to any one of (1) to (4), the machine learning result may be an error between a control command of the servo control device and feedback information when the servo motor was operated based on the control command.

[0012] (6) In the output device according to (4) or (5), the machine learning device may store the parameters of the components of the servo control device acquired during machine learning using the evaluation function and the machine learning result obtained by operating the servo control device using the parameters, and obtain an evaluation function value based on a changed evaluation function using the stored machine learning result when the machine learning is performed while changing the evaluation function based on the change command and the same parameters are selected as the parameters.

[0013] (7) In the output device according to any one of (4) to (6), the different evaluation function may be an evaluation function whose weighting coefficient is different from that of the plurality of evaluation functions.

[0014] (8) In the output device according to any one of (1) to (7), the plurality of evaluation functions may include an evaluation function consisting of a weighted sum of a function in which a position error is a variable, a function in which a difference value of a position error is a variable, and a function in which a difference value of a torque is a variable.

[0015] (9) In the output device according to any one of (3) to (7), the parameters of the components of the servo control device may include parameters of a numerical expression model or a filter.

[0016] (10) In the output device according to (9), the numerical expression model or the filter may be included in a velocity feedforward processing unit or a position feedforward processing unit, and the parameters include coefficients of a transfer function of the filter.

[0017] (11) A control device according to the present invention is a control device including: the dispensing device according to any one of (1) to (10); a servo control device that controls a servo motor for driving an axis of a machine tool, a robot, or an industrial machine; and a machine learning device that performs machine learning with respect to the servo control device.

[0018] (12) In the control device according to (11), the output device may be included in the servo control device or the machine learning device.

[0019] (13) An output method according to the invention, by which an output device can output an evaluation function and a machine learning result, comprises: outputting a machine learning result of each of the evaluation functions obtained by performing machine learning using a plurality of evaluation functions, respectively, by means of a machine learning device that performs machine learning using the evaluation function with respect to a servo control device that controls a servo motor for driving an axis of a machine tool, a robot, or an industrial machine, and

[0020] Obtaining the machine learning result from the servo control device and / or the machine learning device.

[0021] According to the present invention, by outputting a plurality of evaluation functions and a machine learning result of each of the evaluation functions, a relationship between the evaluation function and the machine learning result can be determined. Thus, by an output device issuing a command to a machine learning device, an evaluation function can be selected from a plurality of evaluation functions and changed to another evaluation function. BRIEF DESCRIPTION OF THE DRAWINGS Fig. 1 is a block diagram showing a configuration example of a control device according to a first embodiment of the present invention. Fig. 2 is a block diagram showing a configuration example of an output device in the control apparatus according to the first embodiment of the present invention. Fig. 3 is a schematic view showing an example of a display screen for displaying a plurality of evaluation functions set by a weighting coefficient and a curve obtained by superimposing time responses of position errors based on parameters learned with these evaluation functions. Fig. 4 is a schematic view showing an example of a display screen for displaying, in parallel, screen contents relating to a plurality of evaluation functions and a curve indicating a time response of position errors based on parameters learned from these evaluation functions. Fig. 5 is a schematic view showing a part of a machining path when a machining shape of a workpiece is an octagon. Fig. 6A is a schematic view showing an example of a characteristic diagram showing a relationship between a machining trajectory and a time response of a position error. Fig. 6B is a schematic view showing an example of a time response characteristic diagram of a machining trajectory and a time response of a torque. Fig. 7A is a schematic view showing an example of a characteristic diagram showing a relationship between a machining trajectory and a time response of a position error. Fig. 7B is a schematic view showing an example of a time response characteristic diagram of a machining trajectory and a time response of a torque. Fig. 8 is a schematic view showing an example of a display screen that displays a curve obtained by superimposing time response curves of position errors obtained based on parameters learned with a plurality of evaluation functions. Fig. 9 is a schematic view showing an example of a display screen that displays a curve obtained by superimposing time response curves of position errors obtained based on parameters learned with a plurality of evaluation functions. Fig. 10A is a schematic view showing an example of a time response characteristic diagram of a position error. Fig. 10B is a schematic view showing an example of a frequency response characteristic diagram of a position error. Fig. 11A is a schematic view showing an example of a time response characteristic diagram of a position error. Fig. 11B is a schematic view showing an example of a frequency response characteristic diagram of a position error. Fig. 12 is a diagram showing an example of a curve obtained by superimposing a plurality of frequency characteristic curves. Fig. 13 is a flowchart showing an operation of a control device, mainly an output device, from the start of machine learning to the end of machine learning. Fig. 14 is a block diagram showing a configuration example of a servo control device in the control device according to the first embodiment of the present invention. Fig. 15 is a schematic view showing an operation of a motor when a machining shape is an octagon. Fig. 16 is a schematic view showing an operation of a motor when a machining shape is a shape in which the corners of an octagon are alternately replaced by arcs. Fig. 17 is a block diagram showing a machine learning apparatus according to the first embodiment of the present invention. Fig. 18 is a block diagram showing a configuration example of a control device according to a second embodiment of the present invention. Fig. 19 is a block diagram showing a configuration example of a control device according to a third embodiment of the present invention. Fig. 20 is a block diagram showing a control device according to another embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0022] An embodiment of the present invention will be described below with reference to the drawings. (First embodiment)

[0023] Fig. 1 is a block diagram showing a configuration example of a control device according to a first embodiment of the present invention. Fig. The control device 10 shown in Figure 1 includes a machine learning device 100, an output device 200, a servo control device 300, and a servo motor 400. The control device 10 operates a machine tool, a robot, or an industrial machine. The control device 10 may be provided separately from a machine tool, a robot, or an industrial machine, or may be included in a machine tool, a robot, or an industrial machine.

[0024] The servo control device 300 outputs a torque command based on a control command such as a position command and a speed command, and controls the rotation of the servo motor 400. The servo control device 300 includes components such as a speed feedforward processing unit represented by a transfer function including coefficients learned by the machine learning device 100. The component of the servo control device 300 that is machine-learned by the machine learning device 100 is not limited to the speed feedforward processing unit and may also be, for example, a notch filter.The velocity feedforward processing unit performs a process specified by a transfer function F(s) represented by Expression 1 (see Equation 1) with respect to an input position command and adds the processing result to a torque command as a velocity feedforward term. The optimal values ​​of the coefficients a. i and b j (0≤i≤m, 0≤j≤n, where m and n are natural numbers) of Expression 1 are machine-learned by the machine learning device 100. Details of the configuration of the servo control device 300 will be described below. F(s)=b0+b1s+bss2+⋯+bnsna0+a1s+ass2+⋯+amsm

[0025] The servo motor 400 drives an axis of a machine tool, a robot, or an industrial machine. The servo motor 400 is included, for example, in a machine tool, a robot, or an industrial machine. The servo motor 400 outputs a detected position and / or a detected speed to the servo controller 300 as feedback information. The servo controller 300 receives a difference between a position command and the detected position as a position error and receives a difference between a speed command and the detected speed as a speed error.

[0026] The output device 200 receives a control command such as a position command, a speed command, and a torque command, as well as servo information including a position error, a speed error, or the like, output from the servo control device 300, and outputs the received command and information to the machine learning device 100. Furthermore, the output device receives the coefficients (parameters) of a transfer function of a speed feedforward processing unit of the servo control device from the servo control device 300 and outputs the coefficients to the machine learning device 100.

[0027] The machine learning device 100 finely adjusts the coefficients of the transfer function of the speed feedforward processing unit so that the coefficients (parameters) of the transfer function of the speed feedforward processing unit approach optimal values ​​to obtain correction parameters by performing machine learning based on a predetermined evaluation function, and outputs the correction parameters to the servo control device 300 via the output device 200.

[0028] The servo control device 300 changes the coefficients of a transfer function of the velocity feedforward processing unit based on the correction parameters for driving the servo motor 400. Furthermore, the output device 200 receives a control command such as a position command, a velocity command, and a torque command, as well as servo information including a position error and a velocity error based on the correction parameters from the servo control device 300, and outputs the received command and information to the machine learning device 100. The control device 10 repeatedly performs the above-described operations, whereby the machine learning device 100 obtains the optimal values ​​of the coefficients of the transfer function of the velocity feedforward processing unit.

[0029] The operations described above are the basic operations of the control device 10 for obtaining the optimal values ​​of the transfer function coefficients of the velocity feedforward processing unit. In this embodiment, the output device 200 outputs a plurality of evaluation functions and learning results, such as a time response or a frequency response of a position error, obtained by operating the servo control device using the parameters obtained by machine learning based on each of the evaluation functions.When outputting learning results such as a time response or a frequency response of a position error using a display device, a plurality of learning results may be displayed in parallel on a display screen for corresponding evaluation functions, or the plurality of learning results may be displayed superimposed on a display screen. In this way, a user can easily compare the learning results of the corresponding evaluation functions. The plurality of evaluation functions output by the output device 200 may be set by a user, or a machine tool manufacturer may set the plurality of evaluation functions in advance. As described below, some or all of the plurality of evaluation functions may be changed to other evaluation functions by changing a weighting coefficient.Examples of a method for outputting information from the output device 200 include displaying on a liquid crystal display screen, printing on a sheet using a printer or the like, storing in a storage unit such as a memory, and outputting a signal to an external device via a communication unit.

[0030] When an evaluation function with which a learning result desired by a user is obtained is selected, the output device 200 outputs parameters obtained by the evaluation function with which a desired learning result is obtained to the servo control device 300. In this way, a predetermined evaluation function can be selected from a plurality of evaluation functions, so that a learning result desired by the user is obtained. The output device 200 outputs evaluation function command information of the selected evaluation function to the machine learning device 100. The evaluation function command information includes a weighting coefficient value of the evaluation function, the identification information of the evaluation function, and the like. The machine learning device 100 sets the selected evaluation function as an evaluation function to be used when performing machine learning.If a learning result desired by the user is not obtained from the learning result, such as a time response or a frequency response of the position error, from the output device 200, the output device 200 changes part or all of the plurality of evaluation functions based on a user's operation. Changing the evaluation function includes, for example, changing a weighting coefficient of an evaluation function to a different value or changing the type of an evaluation function. The output device 200 outputs evaluation function command information to the machine learning device 100 to set or change the weighting coefficient of an evaluation function or the type of evaluation function to be used by the machine learning device 100.

[0031] The machine learning device 100 sets or changes the evaluation function based on the evaluation function command information. The machine learning device 100 learns by outputting correction parameters obtained by fine-tuning the transfer function coefficients using the set or changed evaluation function to the servo control device 300 via the output device 200, so that, for example, the transfer function coefficients (parameters) of the speed feedforward processing unit approach optimal values.

[0032] The output device 200 receives the servo information of the position error from the servo control device 300, which operates based on the correction parameters, and outputs information including the set or changed evaluation function and a learning result such as a time response or a frequency response of the position error. In this way, the user can set or change the evaluation function, thereby obtaining a learning result desired by the user.

[0033] A time response or a frequency response of the position error is described here as an example of the machine learning result, but the machine learning result is not limited to this. For example, a frequency response obtained from the transfer function of the velocity feedforward processing unit set to the machine-learned coefficient can be used as the learning result. Software is known that can analyze a frequency response from the transfer function; for example, the following software can be used: https: / / jp.mathworks.com / help / signal / ug / frequency-renponse.html https: / / jp.mathworks.com / help / signal / ref / freqz.html https: / / docs.scipy.org / doc / scipy-0.19.1 / reference / generated / scipy.signal.freqz.html https: / / wiki.octave.org / Control_package

[0034] A damping center frequency fc, a bandwidth fw, and a damping coefficient R can be obtained from the frequency response. The machine learning result can be presented using a curve representing the time response or the frequency response of a position error and using a table. When the machine learning result is the time response or the frequency response of a position error, the output device 200 can obtain the position error from the machine learning device 100 or the servo control device 300. When the machine learning result is a frequency response obtained from the transfer function of a velocity feedforward processing unit, the output device 200 can obtain the transfer function from the servo control device 300.

[0035] In the following, the respective components of the control device 10 according to the first embodiment will be described in more detail.

[0036] In the following description, the output device 200 will be described first, and then the machine learning device and the servo control device 300 will be described. <Ausgabevorrichtung 200>

[0037] The output device 200 has an output function for outputting a plurality of evaluation functions and a learning result, such as a time response of a position error, corresponding to each evaluation function. Furthermore, the output device 200 has a setting function for performing control (for example, an evaluation function change command related to a machine learning device and a program activation command related to a machine learning device) for the operation of the machine learning device 100.Furthermore, the output device 200 also has a relay function for relaying information (for example, a control command such as a position command, a speed command and a torque command, a position error and the coefficients of the speed feedforward processing unit) from the servo control device 300 to the machine learning device 100 and information (for example, the corrected coefficients a. i and b j The speed feedforward processing unit (VF) from the machine learning device 100 to the servo control device 300. Control of operations and forwarding of information are performed via a learning information obtaining unit 201, a servo information obtaining unit 207, a learning information outputting unit 202, and a servo information outputting unit 208, each of which will be described below.

[0038] Fig. 2 is a block diagram showing a configuration example of the output device 200 in the control device 10 according to the first embodiment of the present invention. As shown in Fig. 2, the output device 200 includes a learning information obtaining unit 201, a learning information outputting unit 202, a graphics generating unit 203, an operating unit 204, a control unit 205, a storage unit 206, a servo information obtaining unit 207, a servo information outputting unit 208, and a display unit 209.

[0039] The learning information obtaining unit 201 obtains the correction parameters (for example, the corrected coefficients a i and b jthe speed feedforward processing unit 308) from the machine learning device 100. The learning information output unit 202 outputs servo information indicating a servo state including a control command such as a position command, a speed command, and a torque command, and feedback such as a position error, a speed error, and a current error, and parameters (for example, the corrected coefficients a i and b jthe velocity feedforward processing unit 308) to the machine learning device 100. The graphics generation unit 203 reads a set of position errors stored in the storage unit 206 to generate image information of a time-response characteristic diagram or a frequency-response characteristic diagram for display on the display unit 209. The operation unit 204 is a mouse, a keyboard, or the like, and inputs commands and data to the control unit 205. The servo information obtaining unit 207 obtains servo information indicating a servo state including a control command such as a position command, a velocity command, and a torque command, and feedback such as a position error, a velocity error, and a current error, and parameters (for example, the corrected coefficients a i and b jthe speed feedforward processing unit 308) of the servo control device 300. The servo information output unit 208 outputs the correction parameters output from the machine learning device 100 to the speed feedforward processing unit 308 of the servo control device 300. The display unit 209 is a liquid crystal display, a printer, or the like, and outputs information including an evaluation function and a time-response characteristic diagram of a position error. The control unit 205 and the display unit 209 serve as an output unit for outputting an evaluation function and a diagram representing a position error obtained by operating the servo control device 300 based on parameters learned with the evaluation function (for example, the coefficients a i and b jThe output comprises storing information in the storage unit 206, in which case the output unit is constituted by the control unit 205 and the storage unit 206. Examples of an evaluation function that can be displayed on the display unit 209 of the output device 200 can be used by the machine learning device 100 as follows. For example, the following functions can be used as the evaluation function f when e is a position error and Tm is a torque. min ∫(a|e|2+β|dedt|2+γ|dTmdt|2)dt min ∫(α|e|2+γ|dTmdt|2)dt min ∫t0t1|e(t)|2dt min ∫|e(t)|2dt min max{e(t)}

[0040] In Expressions 2 to 6, α, β, and γ are weighting coefficients, min indicates the smallest value, and max indicates the largest value. In addition to these functions, the following functions can be used.

[0041] A function that calculates an integrated value of an absolute value of a position error: ∫|e|dt

[0042] A function that calculates an integrated value by weighting an absolute value of a position error with time: ∫t|e|dt

[0043] A function that calculates an integrated value of a 2n-th power (n is a natural number) of an absolute value of a position error: ∫ e2ndt(n is a natural number)

[0044] A function that calculates a maximum value of an absolute value of a position error: Max{|e|}

[0045] The evaluation function of Expression 2 is an evaluation function consisting of a weighted sum of a function using a position error as a variable, a function using a difference value of the position error as a variable, and a function using a difference value of a torque as a variable. The control unit 205 outputs part or all of the evaluation functions and a learning result such as a time response of a position error corresponding to each evaluation function to the display unit 209. The control unit 205 outputs evaluation function command information for setting or changing a weighting coefficient of an evaluation function to be used by the machine learning device or the type of an evaluation function to the machine learning device 100 in accordance with a command from a user.

[0046] In the following, the output device 200 is described in more detail using specific examples 1 to 5. (Specific Example 1: Example of a variety of evaluation functions with a variety of different weighting coefficients)

[0047] Fig. 3 is a schematic diagram showing an example of a display screen that displays a plurality of evaluation functions set by a weighting coefficient and a curve obtained by superimposing time responses of position errors based on parameters learned with these evaluation functions. In this specific example, an operation will be described in which an output device 200 outputs evaluation functions (hereinafter referred to as "evaluation functions of weighting factors W1 to W3") in which weighting factor values ​​W1 to W3 are set to the weighting coefficients α, β, and γ of the evaluation function of Expression 2, and a curve obtained by superimposing time responses of three position errors obtained by operating the servo control device 300 based on the parameters learned with the evaluation functions of the weighting factors W1 to W3.The number of evaluation functions is not limited to three, but two or four or more evaluation functions can also be used.

[0048] As in Fig. 3, the display screen P includes sections P1, P2, and P3. In section P1, for example, selection options such as a selection axis, a test parameter, a test and processing program, machine learning, and a termination determination are displayed. In section P2, for example, a setting target such as speed feedforward, a state such as data acquisition, and the cumulative number of trials relative to a predetermined number of trials (hereinafter also referred to as the "largest number of trials") until the end of machine learning, as well as a button for selecting a learning suspension, are displayed.For example, in section P3, the evaluation function represented in Expression 2, a table with three setting examples (weighting factors W1 to W3) of the weighting coefficients α, β, and γ of the evaluation function, and a curve obtained by superimposing the time responses obtained based on the parameters learned with the evaluation function for the corresponding weighting factors W1 to W3 of the position error are displayed. A curve representing a time response of a position error is displayed only after performing learning.

[0049] As described below, the control unit 205 displays the evaluation function represented in Expression 2, a table with three setting examples (weighting factors W1 to W3) of the weighting coefficients α, β and γ of the evaluation function, and a graph representing the time responses for the corresponding weighting factors W1 to W3 of the position error obtained based on the parameters learned with the evaluation function, on the display unit 209.

[0050] If a user such as an operator uses a “machine learning” in section P1 of the Fig. 3 of the display screen P of the display unit 209 and further selects the column of the weighting factor W1 of the weighting coefficients α, β, and γ in the table of the section P3, the control unit 205 indicates the column of the weighting factor W1 with a bold rectangle so that the weighting factors α, β, and γ in the column can be edited. Then, the user inputs, for example, α=1, β=0, and γ=0 and selects "Select" in the section P3 of the display screen P. In this example, the weighting coefficients α, β, and γ are input by the user, but the weighting coefficients can also be set beforehand.The control unit 205 sends evaluation function command information including the identification information of the evaluation function of Expression 2 and the weighting coefficients (α=1, β=0, and γ=0) of the weighting factor W1 to the machine learning device 100 via the learning information output unit 202.

[0051] The machine learning device 100 changes the evaluation function to the evaluation function of Expression 2 of the weighting coefficients (α=1, β=0, and γ=0) based on the evaluation function command information and starts machine learning based on the evaluation function. The machine learning device 100 sends the correction parameters to be used in the machine learning (trial) and the number of trials to the output device 200 (the learning information obtaining unit 201) to perform the machine learning (trial), while fine-tuning the coefficients of the transfer function of the speed feedforward processing unit so that the coefficients (parameters) of the transfer function of the speed feedforward processing unit approach optimal values.The output device 200 (the servo information output unit 208) outputs the correction parameters received from the machine learning device 100 to the servo control device 300.

[0052] The servo control device 300 changes the coefficients of the transfer function of the speed feedforward processing unit based on the correction parameters for operating the servo motor 400 received from the output device 200 (the servo information output unit 208) and performs machine learning (a trial). When the trial is completed, the output device 200 (the servo information obtaining unit 207) obtains servo information including a control command such as a position command, a speed command, and a torque command during the trial, as well as a position error, a speed error, and a current error based on the correction parameters from the servo control device 300, and stores the position error, the speed error, and the like in the storage unit 206.Furthermore, the output device 200 (the learning information output unit 202) outputs the obtained servo information to the machine learning device 100. When the learning (experiment) is performed, the machine learning device 100 stores the coefficients (parameters) of the transfer function and the servo information, as well as the learning result (a time response or a frequency response of the position error) obtained by operating the servo control device 300 using the parameters, in a storage unit (not shown).Therefore, as described below, when the evaluation function is changed based on the evaluation function command information (a change command) to relearn the coefficients (the parameters) of the transfer function, the evaluation function value with the changed evaluation function can be obtained using the stored servo state and the learning result without operating the servo motor 400 if the same parameters as those stored in the storage unit are stored during machine learning (the experiment). In this way, an operation for operating the servo control device 300 with the same parameters for obtaining the servo state and the learning result can be omitted, thus improving the efficiency of machine learning.

[0053] The control unit 205 displays a speed feedforward in the setting target field of the section P2 of the display screen P of Fig. 3 and displays a data acquisition in the status field of section P2 if the number of attempts has not yet reached a predetermined number of attempts. Furthermore, the control unit 205 displays the ratio between the number of attempts and the largest number of attempts in the "Number of Attempts" field of section P2. The control unit 205 continues the above-described operation if the number of attempts has not yet reached the predetermined number of attempts.

[0054] The machine learning device 100 performs learning with the evaluation function represented by Expression 2 of the weighting coefficient (α=1, β=0, and γ=0) with a predetermined number of trials, and the control unit 205 passes control to the graph generation unit 203 when the output device 200 (the control unit 205) determines that the number of trials has reached the predetermined number of trials based on the information on the number of trials from the machine learning device 100.

[0055] The graphics generation unit 203 reads the set of position errors stored in the storage unit 206 to generate image information of a time-response characteristic diagram, stores the image information in the storage unit 206, and passes control to the control unit 205. The control unit 205 displays the time-response characteristic diagram of the position error in the section P3 of the display screen P as shown in Fig. 3 shown.

[0056] The control unit 205 shows a characteristic diagram with the time response curve of the weighting factor W1 in the section P3 of the Fig. 3. And then, when the column of the weighting factor W2 in the table of the section P3 of the display screen P is selected by a user, the control unit 205 moves the part indicated by the bold rectangle in the table from the column of the weighting factor W1 to the column of the weighting factor W2, so that the weighting coefficients α, β, and γ of the column of the weighting factor W2 can be edited.

[0057] For example, when a user inputs α=1, β=1, and γ=1 and selects “Select” in the section P3 of the display screen P, the control unit 205 sends evaluation function command information including the identification information of the evaluation function of Expression 2 and the weighting coefficients (α=1, β=1, and γ=1) of the weighting factor W2 of the evaluation function to the machine learning device 100 via the learning information output unit 202. The operations of the machine learning device 100 after receiving the evaluation function command information and the output device 200 after sending the evaluation function command information are similar to the operations for obtaining the characteristic diagram with the time response curve of the weighting factor W1, but the machine learning device 100 performs learning with the weighting coefficient expressed by Expression 2. (α=1,β=1 and γ=1) for a predetermined number of trials, the graphic generation unit 203 superimposes the time-response curve of the weighting factor W2 on the characteristic diagram with the response curve of the weighting factor W1 to generate image information of a new characteristic diagram after the machine learning has been completed with the predetermined number of trials, and the control unit 205 displays the characteristic diagram with the time-response curve of the weighting factor W1 and the time-response curve of the weighting factor W2 in the section P3 of the display screen P.

[0058] The control unit 205 displays a characteristic diagram with the time response curves of the weighting factors W1 and W2 in the section P3 of the display screen P of Fig. 3. Subsequently, when the weighting factor W3 column in the table of section P3 of the display screen P is selected by a user, the control unit 205 moves the part in the table indicated by the bold rectangle from the weighting factor W2 column to the weighting factor W3 column, so that the weighting factors α, β, and γ of the weighting factor W3 column can be edited. For example, when a user inputs α=1, β=0.3, and γ=0.5 and selects "Select" in section P3 of the display screen P, the control unit 205 sends evaluation function command information including the identification information of the evaluation function of Expression 2 and the weighting coefficients (α=1, β=0.3, and y=0).5) the weighting factor W3 of the evaluation function to the machine learning device 100 via the learning information output unit 202. The operations of the machine learning device 100 after receiving the evaluation function command information and the output device 200 after transmitting the evaluation function command information are similar to the operations for obtaining the characteristic diagram with the time response curve of the weighting factor W1, but the machine learning device 100 performs learning with the evaluation function represented by Expression 2 of the weighting coefficient (α=1, β=0.3, and y=0.5) for a predetermined number of trials, the graphic generation unit 203 superimposes the time response curve of the weighting factor W3 on the characteristic diagram with the time response curves of the weighting factors W1 and W2 to generate image information of a new characteristic diagram,after the machine learning has been completed with the predetermined number of trials, and the control unit 205 displays the characteristic diagram with the time response curves of the weighting factors W1, W2 and W3 in the section P3 of the display screen P.,

[0059] In this way, the characteristic diagram with the time response curves of the weighting factors W1, W2, and W3 is displayed in section P3 of the display screen P, and an output function and a forwarding function are fulfilled. The user can select and set the weighting factor of the evaluation function by observing the time responses of the position errors corresponding to the plurality of evaluation functions.

[0060] In the display screen P of Fig. 3, selecting the "Select" button may be performed after completing the input of the weighting coefficients α, β, and γ of W1, W2, and W3 in the table. In this case, for example, the machine learning device 100 performs machine learning up to the predetermined number of trials sequentially using the evaluation functions of the weighting factors W1, W2, and W3, and performs machine learning by changing the evaluation function when the machine learning with the corresponding evaluation functions has reached the predetermined number of trials. The output device 200 displays the characteristic diagram including the time-response curves of the position errors with respect to the weighting factors W1, W2, and W3 in the section P3 of the display screen P when the machine learning of all the evaluation functions of the weighting factors W1, W2, and W3 is completed.

[0061] For example, the following properties can be seen from the time response curves of the weighting factors W1, W2 and W3 displayed in section P3 of the display screen P. As shown in Fig. 3, the time response curve of the position error with respect to the weighting factor W1 exhibits a low-frequency position error that is suppressed compared to the time response curves of the position errors with respect to the weighting factors W2 and W3, with the position error area of ​​the characteristic diagram being the smallest within the time response curves of the position errors with respect to the weighting factors W1 to W3. Furthermore, it can be seen that the high-frequency vibration of the position error remains. Furthermore, it should be noted that the time response curve of the position error with respect to the weighting factor W2 exhibits a high-frequency position error that is suppressed compared to the time response curves of the position errors with respect to the weighting factors W1 and W3, with the position error area of ​​the characteristic diagram being the largest within the time response curves of the position errors with respect to the weighting factors W1 to W3.Furthermore, it can be seen that the curve is smooth and shocks are well suppressed. Furthermore, it can be seen that the time response curve of the position error with respect to the weighting factor W3 exhibits a medium-frequency position error that is suppressed compared to the time response curves of the position errors with respect to the weighting factors W1 and W2, where the curve is smooth and shocks are reduced compared to the time response curve of the position error with respect to the weighting factor W1. Furthermore, it can be seen that the position error area is slightly enlarged.

[0062] In this way, a user can select a weighting factor of the evaluation function by observing the time responses of the position errors corresponding to a plurality of evaluation functions. The user's operation for selecting one of the weighting factors W1 to W3 is similar to the operation for selecting one of the weighting factors W1 to W3 for displaying the characteristic diagram, except that "Determine" is selected in the section P3 of the display screen P. For example, if the weighting factor W1 is selected and the user selects the column of the weighting factor W1 in the table of the section P3 of the display screen P, the control unit 205 indicates the column of the weighting factor W1 with a bold rectangle.When the user selects "Determine" in section P3 of the display screen P, the control unit 205 sends the parameters of the velocity feedforward processing unit obtained by performing machine learning with the evaluation function of Expression 2 of the weighting factor W1 of the weighting factor coefficients α=1, β=0, and γ=0. This allows the user to obtain the servo characteristics (a time response or the like of a position error) when the servo control device 300 subsequently performs servo control in the velocity feedforward processing unit using the parameters. Furthermore, the control unit 205 sends the identification information of the evaluation function of the weighting factor W1 to the machine learning device 100.When the machine learning device 100 sets the evaluation function to the evaluation function of the weighting factor W1, the machine learning device 100 can use the evaluation function of the weighting factor W1, for example, when performing relearning. [Relearning with changed weighting coefficient]

[0063] The following describes a process for changing the weighting factor values ​​to other weighting factor values ​​than those shown in the section P3 of the display screen P of Fig. 3 for performing relearning. When the user selects the column of the weighting factor W1 in the table of section P3 of the display screen P, the control unit 205 indicates the column of the weighting factor W1 with a bold rectangle so that the weighting coefficients α, β, and γ in the column can be edited. For example, when the user selects α=1, β=0.1, and γ=0.2 and selects “Select” in the section P3 of the display screen P, the control unit 205 sends evaluation function command information (serving as a change command) including the identification information of the evaluation function of Expression 2 and the weighting coefficients (α=1, β=0.1, and γ=0.2) of the weighting factor W1 of the evaluation function to the machine learning device 100 via the learning information output unit 202.The machine learning device 100 changes the evaluation function to the evaluation function of Expression 2 of the weighting coefficient (α=1, β=0.1, and γ=0.2) based on the evaluation function command information and starts machine learning based on the evaluation function. The following processes are similar to the operations for obtaining the characteristic diagram with the time-response curve of the weighting factor W1, so their description is omitted here. In this way, the machine learning device 100 performs relearning. (Specific Example 2: Example of a variety of evaluation functions with different weighting factors and types)

[0064] In Specific Example 1, an example was described in which an evaluation function is selected from a plurality of weighting functions set by a plurality of weighting coefficients or a weighting coefficient is corrected. In Specific Example 2, an example is described in which an evaluation function is selected from two evaluation functions with different weighting factors and types. In Specific Example 2, with reference to Fig. 4 describes an operation in which the output device 200 displays, in parallel, a screen content indicating the evaluation function of Expression 2 and a graph indicating the time response of a position error obtained based on the parameters learned with the evaluation function, and a screen content indicating the evaluation function of Expression 3 and a graph indicating the time response of a position error obtained based on the parameters learned with the evaluation function.

[0065] In this embodiment, a servo control device is operated using a learning machining program in which a machining shape of a workpiece as shown in Fig. 5. When a workpiece is subjected to the machining shape of Fig. 5, a servo motor and a servo controller that drive a table in the X-axis direction, and a servo motor and a servo controller that drive the table in the Y-axis direction, are required. In this embodiment, the machine learning device 100 machine-learns the coefficients of the velocity feedforward processing unit of the servo controller for driving the table in the Y-axis direction, and the output device 200 displays two evaluation functions to be used for machine learning and a graph representing a time response of a position error obtained based on the parameters learned with the corresponding evaluation functions, as shown in Fig. 4 shown. In Fig. 5, the servo motor that drives the table in the Y-axis direction rotates at a constant speed in a linear part L1, the speed is reduced at a point P1, and the servo motor rotates at the constant reduced speed in a linear part L2. Fig. 4 is a schematic view of an exemplary display screen that displays, in parallel, a screen including the evaluation function of Expression 2 and a graph representing a time response of a position error obtained based on the parameters learned with this evaluation function, and a screen including the evaluation function of Expression 3 and a graph representing a time response of a position error obtained based on the parameters learned with this evaluation function.

[0066] As in Fig. 4, the display screen P includes a section P4 and a section P5. The control unit 205 displays a graph representing the evaluation function of Expression 3, the weighting coefficients α and γ of the evaluation function, and a time response of a position error obtained based on the parameters learned with the evaluation function in the section P4. Furthermore, the control unit 205 displays a graph representing the evaluation function of Expression 2, the weighting coefficients α, β, and γ of the evaluation function, and a time response of a position error obtained based on the parameters learned with the evaluation function in the section P5. A graph representing a time response of a position error is displayed in the sections P4 and P5 only after learning is performed.

[0067] When a user such as an operator enters a frame of section P4 of the Fig. 4 using the operation unit 204, the control unit 205 indicates the frame of section P4 with a bold rectangle so that the weighting coefficients α and γ in section P4 can be edited. For example, the user inputs α=100 and γ=0 and selects "Select" in section P4 of the display screen P. The weighting coefficients α and γ are input here by the user, although the weighting coefficients can also be set beforehand.

[0068] Thereafter, similarly to Specific Example 1, the control unit 205 sends evaluation function command information including the identification information of the evaluation function of Expression 3 and the weighting coefficients (α=100 and γ=0) of the weighting factor W1 of the evaluation function to the machine learning device 100 via the learning information output unit 202. The machine learning device 100 performs machine learning using the evaluation function of Expression 3 and the weighting coefficients (α=100 and γ=0).

[0069] The servo control device 300 changes the coefficients of the transfer function of the velocity feedforward processing unit based on the correction parameters output from the machine learning device 100 during machine learning for operating the servo motor 400. Furthermore, the output device 200 receives servo information including a control command such as a position command, a velocity command, and a torque command, and a position error, a velocity error, or the like based on the correction parameters from the servo control device 300, outputs it to the machine learning device 100, and stores the position error, the velocity error, and the like in the storage unit 206.When the learning (experiment) is performed, the machine learning device 100 stores the coefficients (parameters) of the transfer function and the servo information, as well as the learning result (a time response or a frequency response of the position error) obtained by operating the servo control device 300 using the parameters, in a storage unit (not shown).

[0070] When the machine learning device 100 performs learning with a predetermined number of trials, the output device 200 performs similar processes as in Specific Example 1 and displays the evaluation function of Expression 3, the weighting coefficients α and γ of the evaluation function, and a graph representing a time response of a position error obtained with the parameters learned based on the evaluation function, in the section P4 of the display screen P of Fig. 4 on.

[0071] If a user then enters the frame of section P5 of the Fig. 4 using the operation unit 204, the control unit 205 indicates the frame of section P5 with a bold rectangle so that the weighting coefficients α, β, and γ in section P5 can be edited. For example, the user inputs α=100, β=10, and γ=40 and selects "Select" in section P5 of the display screen P. The weighting coefficients α, β, and γ are input here by the user, but they can also be set beforehand.

[0072] The following operations of the output device 200 and the machine learning device 100 are similar to the above-described operations for displaying information such as the evaluation function of the expression 3, the weighting coefficients α and γ of the evaluation function, and the diagram representing the time response of the position error in the section P4 of the display screen P, but the evaluation function of the expression 2, the weighting coefficients α, β, and γ of the evaluation function, and a diagram representing a time response of a position error obtained with the parameters learned based on the evaluation function are displayed in the section P5 of the display screen P of Fig. 4 are displayed.

[0073] The following is a diagram shown in section P4 of the display screen P of Fig. 4 displayed time response curve of the position error with reference to Fig. 5 and Fig. 6A and Fig. 6B. Fig. 6A and Fig. 6B show a time response characteristic of a position error and a time response characteristic of a torque shown in the section P4 of the display screen P of Fig. 4. L1, P1 and L2 in Fig. 6A and Fig. 6B correspond respectively to the linear part L1, the point P1 and the linear part L2 of Fig. 5. Because the weighting coefficients of the evaluation function of expression 3 shown in the section P4 of the display screen P are α=100 and γ=0, a strong weighting factor is applied to a position error and a value obtained at the point P1 of Fig. 5 occurring large position error quickly in the linear part L2 as in Fig. 6A shown. However, because it is as in Fig. 6B, a torque may increase or decrease abruptly in the linear part L2, a high-frequency vibration remains in the position error. And because the weighting coefficients of the evaluation function of Expression 2 shown in section P5 of the display screen P are α=100, β=10, and γ=40, a weighting factor is applied to a torque difference value, the torque in the linear part L2 is calculated as shown in Fig. 7B shown smooth and reduces a high frequency signal of the position error in the linear part L2 as in Fig. 7A, however, time convergence of a position error becomes smooth.

[0074] The user can select the type of evaluation function and a weighting factor by viewing the time responses of the position errors corresponding to the two evaluation functions. The user's operation to select the evaluation function type and the weighting factor from the evaluation functions of Fig. 4 is similar to the selection operation of Specific Example 1. For example, when a user selects the evaluation function of expression 3 by selecting the frame of section P4 of the display screen P using the operation unit 204, the control unit 205 represents the frame of section P4 with a bold rectangle. When the user selects "Determine" in section P4 of the display screen P, the evaluation function of expression 3 is selected. When the user changes the weighting coefficients α=100 and γ=0, the user selects "Determine" in section P4 of the display screen P after changing the weighting coefficients.

[0075] When the user selects "Set" in the section P4 of the display screen P, the control unit 205 sends the parameters of the velocity feedforward processing unit obtained by performing machine learning based on the evaluation function of Expression 3 of the weighting coefficients α=100 and γ=0 to the servo control device 300. Servo characteristics desired by the user (a time response or the like of a position error) can be obtained when the servo control device 300 performs servo control in the velocity feedforward processing unit with the set parameters.

[0076] When the user selects “Determine” in the section P4 of the display screen P, the control unit 205 sends the weighting coefficients α=100 and γ=0 and the identification information of the evaluation function of Expression 3 to the machine learning device 100. When the machine learning device 100 sets the evaluation function to the evaluation function of Expression 3 of the weighting coefficients α=100 and γ=0, and the user then performs machine learning in the machine learning device 100, the machine learning device 100 can perform machine learning with the evaluation function of Expression 3 of the weighting coefficients α=100 and γ=0. [Relearning]

[0077] A user may want to change the weighting factor of an evaluation function to perform relearning by observing the time responses of the position errors corresponding to the evaluation functions of the weighting coefficients in the sections P4 and P5 of the display screen P. When a user changes the weighting factor of an evaluation function to perform relearning of the time response of the position error corresponding to the evaluation function of the weighting coefficients in the section P4 of the display screen P, and the user selects, for example, the frame of the table in the section P4 of the display screen P, the control unit 205 indicates the frame by a bold rectangle so that the weighting coefficients α and γ in the frame can be edited.For example, when the user inputs α=100 and γ=20 and selects “Select” in the section P4 of the display screen, the control unit 205 sends evaluation function command information (a change command) including the identification information of the evaluation function of Expression 3 and the weighting coefficients (α=100 and γ=20) of the evaluation function to the machine learning device 100 via the learning information output unit 202.The machine learning device 100 changes the evaluation function to the evaluation function of Expression 3 of the weighting coefficients (α=100 and γ=20) based on the evaluation function command information, performs machine learning based on the evaluation function to fine-tune the coefficients of the transfer function of the speed feedforward processing unit so that the coefficients (parameters) of the transfer function of the speed feedforward processing unit approach optimal values ​​for obtaining correction parameters, and sends the correction parameters and the number of trials to the output device 200. The control unit 205 outputs the correction parameters to the servo control device 300. In this way, the machine learning device 100 performs relearning. (Specific Example 3: Example of a variety of evaluation functions of different types)

[0078] In Specific Example 2, an example of selecting one of two evaluation functions with different weighting factors and types was described. In Specific Example 3, an evaluation function of Expression 5 that minimizes the square of the absolute value of a position error or an evaluation function of Expression 6 that minimizes the largest value of position errors is selected using the evaluation functions of Expressions 5 and 6. Fig. Fig. 8 is a schematic view showing an example of a display screen that superimposes the evaluation function (Type 1) of Expression 5 and the evaluation function (Type 2) of Expression 6, as well as a graph representing the time response curves of the position errors obtained based on the parameters learned with each of the evaluation functions. The evaluation function is selected such that when a user places a test mark in a Fig. 8 using the operation unit 204, the control unit 205 determines that the evaluation function corresponding to the check mark is selected. The operations of the machine learning device 100, the output device 200, and the servo control device 300 are similar to those of Specific Example 1, so a description thereof will be omitted here. (Specific Example 4: Example of a variety of evaluation functions with different integration time ranges)

[0079] In Specific Example 2, an example of selecting one of two evaluation functions with different weighting factors and types was described. In Specific Example 4, an evaluation function is selected from a plurality of evaluation functions in which a time range (a range from t0 to t1) of a set of squares of the absolute value of the position error is changed using the evaluation function of Expression 4. Fig. 9 is a schematic diagram showing an example of a display screen on which the evaluation function of Expression 4, a table showing the time range t0 and t1 (Case 1 and Case 2) of the evaluation function, and a graph superimposing the time response curves of the position errors obtained based on the parameters learned with the evaluation functions are displayed. The operations of the machine learning device 100, the output device 200, and the servo control device 300 are similar to those of Specific Example 1, so their descriptions are omitted here. (Specific Example 5)

[0080] Specific examples 1 to 4 describe an example of displaying a time response characteristic of a position error on the display screen P, wherein a frequency response characteristic of a position error may be displayed instead of or together with the time response characteristic of the position error. Fig. 10B and Fig. 11B show a frequency response characteristic diagram which is used instead of a time response characteristic diagram of Fig. 10A in accordance with the characteristic diagram in section P4 of Fig. 4 and a time response characteristic diagram of Fig. 11A in accordance with the characteristic diagram in section P5. Fig. 10B is a frequency characteristic diagram of a position error obtained by the servo control device 300 when the machine learning device 100 performs learning using the evaluation function of Expression 3. And Fig. 11B is a frequency characteristic diagram of a position error obtained by the servo control device 300 when the machine learning device 100 performs learning using the evaluation function of Expression 2. The frequency characteristic diagram of Fig. Figure 10B shows that a high-frequency error is large and has peaks around 80 Hz and 140 Hz. The Fig. The frequency characteristic diagram shown in Figure 11B shows that a low-frequency error is large and has peaks around 25 Hz and 75 Hz. Furthermore, the Fig. 10B shows the frequency characteristic curve over the Fig. 11B shown frequency characteristic curve can be displayed in storage. Fig. 12 is a schematic view showing the Fig. 10B and the frequency characteristic curve shown in Fig. 11B shows the frequency characteristic curve superimposed.

[0081] Examples of the screen content to be displayed on the display screen P of the display unit 209 have been described above. The screen contents described in specific examples 1 to 5 or the characteristic diagrams displayed in the screen contents are examples, and the present invention is not limited thereto. Information other than the items described here may also be displayed. Furthermore, the displayed information of some of the described items may be omitted.In the above description, the control unit 205 stores the position error obtained from the servo control device 300 in the storage unit 206 and displays the evaluation function and the chart representing the time response of the position error on the display unit 209, or the like, when the number of trials of the machine learning device 100 has reached the predetermined number of trials, but the invention is not limited thereto. For example, the following examples can be used as a configuration example in which the information is displayed regardless of the number of trials. Configuration Example 1: An evaluation function and a chart representing a time response of a position error, or the like, are displayed when a user such as an operator issues a display command.Configuration example 2: An evaluation function and a graph representing a time response of a position error or the like are displayed when machine learning is suspended.

[0082] In the following, an operation of the output device 200 will be described with reference to Fig. 13 described. Fig. 13 is a flowchart showing an operation of a control device, mainly an output device, from the start of machine learning to the end of machine learning. When a user selects "Activate Program" in step S31 in the section P1 of the display screen P of the display unit 209 of Fig. 3 using, for example, the operation unit 204 such as a mouse or a keyboard, the control unit 205 (the output device 200) outputs a program activation command to the machine learning device 100 via a learning information output unit 202. Furthermore, the output device 200 outputs a learning program activation command notification to the servo control device 300, informing that a learning program activation command has been issued to the machine learning device 100. In step S32, the output device 200 outputs a learning machining program activation command to a host device, which outputs a learning machining program to the servo control device 300. Step S32 may be performed before step S31 or simultaneously with step S31.After receiving the learning machining program activation command, the host device generates a positioning command and outputs the positioning command to the servo control device 300. In step S33, the output device 200 sends evaluation function command information including a weighting coefficient value of an evaluation function and the identification information of the evaluation function to be used to the machine learning device 100. In step S21, the machine learning device 100 starts machine learning after receiving the program activation command. After receiving the evaluation function command information from the output device 200, the machine learning device 100 sets an evaluation function with which machine learning is performed based on the evaluation function, the commanded weighting coefficient, and the type.

[0083] In step S11, the servo control device 300 controls the servo motor 400 to output the servo information including the parameters (the coefficients a i and b j ) of the velocity feedforward processing unit 308, the position command, and the position error to the output device 200. The output device 200 outputs the parameters, the position command, and the position error to the machine learning device 100.

[0084] In step S21, the machine learning device 100 outputs information including an evaluation function value, a maximum number of trials, the number of trials, and the correction information (correction parameters) of the coefficients a i and b jthe transfer function of the speed feedforward processing unit 308 with respect to the state S associated with the number of trials used by the reward output unit 2021 during machine learning using the set evaluation function, to the output device 200. The output device 200 outputs the correction information of the coefficients a i and b j the transfer function of the velocity feedforward processing unit 308 to the servo controller 300. Steps S11 and S21 are repeatedly performed until the predetermined number of attempts.

[0085] In step S34, the output device 200 determines whether machine learning with the plurality of evaluation functions specified by the user has been completed when the number of trials reaches the predetermined number of trials and the machine learning with the set evaluation function is completed. If it is determined that the learning operation with the plurality of evaluation functions has been completed, the output device 200 displays the plurality of evaluation functions and the learning results (for example, time responses of position errors) machine-learned with the corresponding evaluation functions on the display screen P of the display unit 209 in step S35. If it is determined that the learning operation with the plurality of evaluation functions has not been completed, the flow returns to step S33, and the output device 200 outputs an evaluation function command for the next evaluation function to the machine learning device 100.When the user selects an evaluation function in step S36, the output device 200 sends the coefficients a. i and b j the transfer function of the speed feedforward processing unit 308, which has been optimized by machine learning using the selected evaluation function, to the servo control device 300. In step S12, the servo control device 300 sets the optimized coefficients a i and b j as the coefficients (parameters) of the transfer function of the velocity feedforward processing unit 308. The servo control device 300 will be described below.<Servosteuervorrichtung 300>

[0086] Fig. 14 is a block diagram showing a configuration example of a servo control device in the control device according to the first embodiment of the present invention. As shown in Fig. 14, the servo control device 300 includes as components: a subtractor 301, a position control unit 302, an adder 303, a subtractor 304, a speed control unit 305, an adder 306, an integrator 307, a speed feedforward processing unit 308, and a position feedforward processing unit 309.

[0087] A position command is output to the subtractor 301, the velocity feedforward processing unit 308, the position feedforward processing unit 309, and the output device 200. The position command is generated by a host device based on a program operating the servo motor 400. The servo motor 400 is included, for example, in a machine tool. In a machine tool, when a table with a workpiece mounted thereon moves in an X-axis direction and a Y-axis direction, the servo control device 300 and the servo motor 400 are Fig. 14 are provided in the X-axis direction and the Y-axis direction, respectively. When the table is moved in three or more axes, the servo controller 300 and the servo controller 400 are provided in the corresponding axis directions. A feed rate and a position command are set to obtain a machining shape designed by a machining program.

[0088] The subtractor 301 obtains a difference between a position command value and a position feedback detection position and outputs the difference to the position control unit 302 and the output device 200 as a position error. The position control unit 302 outputs a value obtained by multiplying the position error by a position gain Kp to the adder 303 as a speed command value. The adder 303 adds the speed command value and an output value (a position feedforward term) of the position feedforward processing unit 309 and outputs an addition result to the subtractor 304 as a feedforward-controlled speed command value. The subtractor 304 obtains a difference between the output of the adder 303 and the feedback speed detection value and outputs the difference to the speed control unit 305 as a speed error.

[0089] The speed control unit 305 adds a value obtained by multiplying and integrating the speed error by an integral gain K1v and a value obtained by multiplying the speed error by a proportional gain K2v, and outputs the added value to the adder 306 as a torque command value. The adder 306 adds the torque command value and an output value (a speed feedforward term) of the speed feedforward processing unit 308, and outputs the added value to the servomotor 400 via a current control unit (not shown) as a feedforward-controlled torque command value for driving the servomotor 400.

[0090] A rotation angle position of the servo motor 400 is detected by a rotary encoder serving as a position detection unit and associated with the servo motor 400, and the speed detection value is input to the subtractor 304 as speed feedback. The speed detection value is integrated by the integrator 307 as a position detection value, and the position detection value is input to the subtractor 301 as position feedback.

[0091] The velocity feedforward processing unit 308 performs a velocity feedforward process with respect to the position command and outputs a processing result to the adder 306 as a velocity feedforward term. The transfer function of the velocity feedforward processing unit 308 is a transfer function F(s) represented by the aforementioned Expression 1.

[0092] The speed feedforward processing unit 308 can be represented by an inverse characteristic (a transfer function is Js 2 ) of the motor and a notch filter. When the speed feedforward processing unit 308 is represented by an inverse characteristic (a transfer function is Js 2 ) of the motor and a notch filter, the transfer function F(s) represented by Expression 1 is a numerical expression model on the right side of Expression 7 and is represented similarly to the right side of Expression 7 using the inertia J, the center angular frequency ω, the specific bandwidth ζ and the damping coefficient R. b2s2+b3s3+b2s4a0+a1s+a2s2=Js2⋅ω2+2Rςωs+s2ω2+2ςωs2

[0093] The position feedforward processing unit 309 differentiates the position command value, multiplies it by a constant α, and outputs a processing result to the adder 303 as a position feedforward term.

[0094] The servo control device 300 of Fig. 14 outputs the position error to the output device 200. However, when the machine learning device 100 uses a velocity error or a torque command in addition to the position error as the evaluation function, the velocity error, which is the output of the subtractor 304, or the velocity feedforward-controlled torque command, which is the output of the adder 306, is output to the output device 200. The servo control device 300 is configured in this way. The machine learning device 100 will be described below. <Maschinelles-Lernen-Vorrichtung 100>

[0095] The machine learning device 100 learns the coefficients of the transfer function of the velocity feedforward processing unit 308 by executing a predetermined machining program (hereinafter also referred to as a "learning machining program"). Here, a machining shape provided by the learning machining program is, for example, an octagon or a shape in which the corners of an octagon are alternately replaced by arcs. The machining shape provided by the learning machining program is not limited to these machining shapes and may also be other machining shapes.

[0096] Fig. 15 is a schematic view showing an operation of a motor when a machining shape is an octagon. Fig. Fig. 16 is a schematic view showing an operation of a motor when a machining shape is a shape in which the corners of an octagon are alternately replaced by arcs. In Fig. 15 and Fig. 16 assumes that a table is moved in the X and Y axis directions so that a workpiece is machined in the clockwise direction.

[0097] If the machining shape is an octagon, the value decreases as in Fig. 15 shows the rotational speed of a motor that moves the table in the Y-axis direction at corner position A1, while the rotational speed of a motor that moves the table in the X-axis direction increases. The rotational direction of the motor that moves the table in the Y-axis direction is reversed at corner position A2, and the motor that moves the table in the X-axis direction rotates at the same speed in the same rotational direction from position A1 to position A2 and from position A2 to position A3. The rotational speed of the motor that moves the table in the Y-axis direction increases at corner position A3, while the rotational speed of a motor that moves the table in the X-axis direction decreases.The rotation direction of the motor that moves the table in the X-axis direction is reversed at the corner position A4, and the motor that moves the table in the Y-axis direction rotates at an equal speed in the same rotation direction from the position A3 to the position A4 and from the position A4 to the next corner position.

[0098] If the processing form as in Fig. 16 is a shape in which the corners of an octagon are alternately replaced by arcs, the rotational speed of a motor that moves the table in the Y-axis direction decreases at the corner position B1, while the rotational speed of a motor that moves the table in the X-axis direction increases. The rotational direction of the motor that moves the table in the Y-axis direction is reversed at the corner position B2, and the motor that moves the table in the X-axis direction rotates at the same speed in the same rotational direction from position B1 to position B3. In contrast to a case where the machining shape as in Fig. 15 is an octagon, the rotation speed of the motor that moves the table in the Y-axis direction gradually decreases as it approaches the position B2, stops the rotation at the position B2, and gradually increases the rotation speed as it moves away from the position B2, so that a machining shape of an arc is formed before and after the position B2.

[0099] The rotation speed of the motor moving the table in the Y-axis direction decreases at corner position B3, while the rotation speed of a motor moving the table in the X-axis direction decreases. The rotation direction of the motor moving the table in the X-axis direction is reversed at corner position B4, and the table moves with a linear reversal in the X-axis direction. Furthermore, the motor moving the table in the Y-axis direction rotates at the same speed in the same rotation direction from position B3 to position B4 and from position B4 to the next corner position.The rotation speed of the motor that moves the table in the X-axis direction gradually decreases as it approaches the position B4, the rotation stops at the position B4, and the rotation speed gradually increases as it moves away from the position B4, so that a machining shape of an arc is formed before and after the position B4.

[0100] In this embodiment, it is assumed that machine learning related to optimizing the transfer function coefficients of the speed feedforward processing unit 308 is performed by evaluating vibration when the rotational speed is changed during linear control at positions A1 and A3 and positions B1 and B3 of the machining shape specified by the learning machining program, and examining the influence on a position error. Although not used in this embodiment, runout (running due to inertia) that occurs when the rotation direction is reversed at positions A2 and A4 and positions B2 and B4 of the machining shape may be evaluated to examine the influence on a position error.Machine learning related to transfer function coefficient optimization is not limited to a velocity feedforward processing unit, but can also be applied to, for example, a position feedforward processing unit or a current feedforward processing unit when performing current feedforward of a servo controller. In this case, the position feedforward processing unit and the current feedforward processing unit have a transfer function (for example, a transfer function similar to the transfer function F(s) represented by Expression 1) with coefficients similar to those of the velocity feedforward processing unit.

[0101] The machine learning device 100 will be described in more detail below. In the following description, a case where the machine learning device 100 performs reinforcement learning will be described. However, the learning performed by the machine learning device 100 is not limited to reinforcement learning, and the present invention can also be applied, for example, to a case where the machine learning device 100 performs supervised learning.

[0102] Before describing the corresponding functional blocks in the machine learning device 100, the basic mechanism of reinforcement learning will first be described. An agent (corresponding to the machine learning device 100 in this embodiment) observes an environmental state and selects a specific action. Then, the environment changes due to the action. A specific reward is given in accordance with the environmental change, and the agent learns a selection (decision) for a better action. While supervised learning presents a completely correct answer, the reward in reinforcement learning often presents a partial value based on a change in a part of the environment. Therefore, the agent learns to select an action in such a way that the total reward is maximized in the future.

[0103] In this way, reinforcement learning learns a method for learning an appropriate action based on the reciprocal effect of an action on the environment (i.e., an action to maximize the reward received in the future) through action learning. This means that in this embodiment, an action that influences the future, such as an action for selecting action information to reduce a position error, is obtained.

[0104] Any reinforcement learning method is used here. In the following description, Q-learning, which is a method for learning a value function Q(S,A) for selecting an action A in a given environmental state S, is described as an example. One goal of Q-learning is to select an action A with the highest value function Q(S,A) as an optimal action among the actions available in a given state S.

[0105] However, at the time Q-learning initially starts, the correct value of Q(S,A) for a combination of state S and action A is unknown. Therefore, the agent learns the correct value Q(S,A) by selecting different actions A in a given state S and selecting a better action based on the rewards received for the selected actions A.

[0106] Because it is desirable to maximize the sum of rewards received in the future, the goal is to eventually achieve a relationship of Q(S,A)=E[Σ(γ t )r t ] where E[ ] is an expected value, t is the time, γ is a parameter called a reduction factor and is described below, r ta reward at time t, and Σ is the sum at time t. In this expression, the expected value is the expected value when the state has been changed corresponding to an optimal action. However, because it is unclear which action is optimal in the Q-learning process, reinforcement learning is performed while searching for an optimal action by performing various actions. An update expression for such a value Q(S,A) can be expressed by the following expression 8 (Equation 8). Q(St+1,At+1)←Q(St,At)+α(rt+1+γmaxAQ(St+1,A)−Q(St,At))

[0107] In equation 8, St indicates an ambient state at time t and gives A t an action at time t. By the action A t the state changes to S t+1 . r t+1indicates a reward received by changing the state. Furthermore, a term with max is a multiplication of the Q-value by γ when an action A with the highest Q-value known at that time in the state S t+1 was selected. Here, γ is a parameter ranging from 0<γ≤1 and is called the discount rate. Furthermore, α is a learning coefficient and lies in the range 0<α≤1.

[0108] Expression 8 gives a method for updating a value Q(St,At) of an action A t in a state St based on a received reward r t+1 , when carrying out A t This update expression indicates that if the value max a Q(S t+1 ,A) the best action in the next state St+1, which ends with an action A t associated with, greater than the value Q(St,At) of an action A tin the state St, Q(St,At) is increased, while otherwise Q(St,At) is decreased. This means that the value of a particular action in a particular state approaches the value of the best action in the next state associated with the action. Although this difference between the values ​​depends on the discount rate γ and the reward r t+1 differs, the update expression has a structure such that the value of the best action in a given state generally propagates to the value of an action in a state preceding that state.

[0109] A Q-learning method is known for generating a value function table A(S,A) for all state-action pairs (S,A) for performing learning. However, Q-learning can take quite a long time if the values ​​of the value functions Q(S,A) of all state-action pairs must be calculated because the number of states is too large.

[0110] Q-learning can utilize an existing technique called a deep Q-network (DQN). Specifically, an agent can calculate the value Q(S,A) by constructing a value function Q using a corresponding neural network and approximating the value function Q with the corresponding neural network by adjusting the parameters of the neural network. Using DQN, the time required to complete Q-learning can be shortened. Details of DQN are described, for example, in the following non-patent document. <nicht-patentdokument>

[0111] "Human-level control through deep reinforcement learning", Volodymyr Mnihl [online], [searched January 17, 2017], Internet<URL: http: / / files.davidqiu.com / research / nature14236.pdf>

[0112] The machine learning device 100 performs the Q-learning described above. The machine learning device 100 learns a value Q for selecting an action A for adjusting the values ​​of the coefficients a. i and b j the transfer function of the velocity feedforward processing unit 308 associated with a servo state S, such as commands and feedback including the values ​​of the coefficients a i and b j (0≤i≤m, 0≤j≤n, where m and n are natural numbers) of the transfer function of the velocity feedforward processing unit 308 of the servo control device 300 and the position commands and position error of the servo control device 300 obtained by executing the learning machining program.

[0113] The machine learning device 100 observes the state information S including the servo state such as commands and feedbacks including the position commands and the position error information of the servo control device 300 at the positions A1 and A3 and the positions B1 and B3 of the machining shape by executing the learning machining program based on the coefficients a i and b j the transfer function of the velocity feedforward processing unit 308 to determine the action A. The machine learning device 100 outputs a reward when the action A is performed. The machine learning device 100 searches for the optimal action A such that a total future reward is maximized through trial-and-error learning. In this way, the machine learning device 100 can determine an optimal action A (the optimal coefficients a i and b j the speed feedforward processing unit 308) with respect to the state S including the servo state such as commands and feedback including the position commands and the by executing the learning machining program based on the coefficients a i and b j the position error of the servo control device 300 obtained from the transfer function of the velocity feedforward processing unit 308. The rotation direction of the servo motor in the X-axis direction and the Y-axis direction does not change at positions A1 and A3 and positions B1 and B3, and the machine learning device 100 can select the coefficients a i and b j the transfer function of the velocity feedforward processing unit 308 during linear operation.

[0114] The machine learning device 100 can therefore perform an action A (ie the coefficients a i and b j the velocity feedforward processing unit 308) that minimizes the position error obtained by executing the learning machining program by selecting an action A that maximizes the value of the value function Q, among the actions A acting on the coefficients a i and b j the transfer function of the velocity feedforward processing unit 308 with respect to a specific state S, based on the value function Q learned by the machine learning device 100.

[0115] Fig. Fig. 17 is a block diagram showing the machine learning apparatus 100 according to the first embodiment of the present invention. To perform the reinforcement learning described above, as shown in Fig. 17, the machine learning device 100 includes a state information obtaining unit 101, a learning unit 102, an action information outputting unit 103, a value function storing unit 104, and an optimization action information outputting unit 105. The learning unit 102 includes a reward outputting unit 1021, a value function updating unit 1022, and an action information generating unit 1023.

[0116] The state information obtaining unit 101 obtains the state S which represents the position command and the position error of the servo control device 300 obtained by executing the learning machining program based on the coefficients a i and b j the transfer function of the speed feedforward processing unit 308 of the servo control device 300 is obtained from the servo control device 300 via the output device 200. The state information S corresponds to a state S of the environment in the Q-learning. When the reward output unit 1021 uses a speed error and a torque command in addition to the position error as the evaluation function, the speed error and the torque command are obtained from the servo control device 300 via the output device 200. Furthermore, the state information obtaining unit 101 receives command information for changing a weighting coefficient of the evaluation function or the type of the evaluation function from the output device 200. The state information obtaining unit 101 outputs the obtained state information S and the command information to the learning unit 102.

[0117] The coefficients a i and b j of the velocity feedforward processing unit 308 at a time when Q-learning starts are initially generated by a user. In this embodiment, the initial values ​​of the coefficients a generated by the user are i and b j the velocity feedforward processing unit 308 are adjusted to optimal values ​​by reinforcement learning. The initial values ​​of the coefficients a i and b j of the velocity feedforward processing unit 308 are set, for example, such that in the expression 1 a0=1, a1=0, a2=0 ... and a m =0, b0=1, b1=0, b2=0 ... and b n =0. The dimensions m and n of the coefficients a i and b j are set in advance. This means that 0≤i≤m for a i and 0≤j≤n for b j If a machine tool is previously set by an operator, machine learning can be performed using the set values ​​as the initial values ​​of the coefficients a i and b j be performed.

[0118] The learning unit 102 is a unit that learns the value Q(S,A) when a certain action A is selected in a certain environmental state S.

[0119] The reward output unit 1021 is a unit that calculates a reward when the action A is selected in a certain state S. Here, a set (a position error set) of position errors, which are state variables of the state S, is specified by PD(S), and a position error set, which are state variables with respect to state information S', which changes due to the action information A (corrections of the coefficients a i and b j The value of the position error (the velocity feedforward processing unit 308) that has changed from the state S is indicated by PD(S'). Furthermore, the evaluation function value of the position error in the state S is a value calculated based on a predetermined evaluation function f(PD(S)). The evaluation function at a time point when Q-learning initially starts can be generated by a user in advance. For example, the machine learning apparatus 100 stores a plurality of evaluation functions (for example, a plurality of evaluation functions of Expressions 2 to 6), and a user can select one evaluation function from the plurality of stored evaluation functions. Furthermore, the user can set the weighting coefficients α, β, and γ. The weighting coefficients α, β, and γ can be set to default values ​​by a manufacturer of a machine tool at a time point when Q-learning initially starts.When assigning identification information to the plurality of evaluation functions of Expressions 2 to 6, the user selects identification information. As described in Specific Examples 1 to 5, the output device 200 outputs evaluation function command information to the machine learning device 100 to obtain a time response of a position error and set the evaluation function selected by the user when the time response of the position error is to be relearned. The evaluation function command information includes identification information of an evaluation function or identification information and a weighting coefficient of an evaluation function.For example, the evaluation function command information is obtained by the state information obtaining unit 101, and the state information obtaining unit 101 outputs the evaluation function command information to the reward output unit 1021. The reward output unit 1021 sets or changes the weighting coefficient of an evaluation function or the type of the evaluation function based on the evaluation function command information. For example, the reward output unit 1021 changes the weighting coefficients α, β, and γ to commanded values ​​when it receives command information for changing the weighting coefficients α, β, and γ of the evaluation function f in Equation 3.

[0120] The reward output unit 1021 sets the value of a reward to a negative value when the evaluation function value f(PD(S')) of the position error of the servo control device 300 operated based on the speed feedforward processing unit 308 after the correction with respect to the state information S' corrected by the action information A is larger than the evaluation function value f(PD(S)) of the position error of the servo control device 300 operated based on the speed feedforward processing unit 308 before the correction with respect to the state information S before the correction by the action information A.

[0121] Conversely, if the position error evaluation function value f(PD(S')) becomes smaller than the position error evaluation function value f(PD(S)), the reward output unit 1021 sets the reward value to a positive value. If the position error evaluation function value f(PD(S')) is equal to the position error evaluation function value f(PD(S)), the reward output unit 1021 may set the reward value to zero.

[0122] And if the evaluation function value f(PD(S')) of the position error in state S' after the execution of action A becomes larger than the evaluation function value f(PD(S)) of the position error in the previous state S, the negative value can be increased in proportion. Thus, the negative value can be increased in proportion to the degree of increase of the position error value. On the other hand, if the evaluation function value f(PD(S')) of the position error in state S' after the execution of action A becomes smaller than the evaluation function value f(PD(S)) of the position error in the previous state S, the positive value can be increased in proportion. That is, the positive value can be increased in proportion to the degree of decrease of the position error value.

[0123] The value function updating unit 1022 updates the value function A stored in the value function storage unit 104 by performing Q-learning based on the state S, the action A, the state S' when the action A was applied to the state S, and the reward value calculated as described above. Updating the value function Q can be performed through online learning, batch learning, or mini-batch learning. Online learning is a learning method for applying a specific action A to a current state S and updating the value function Q immediately whenever the current state S transitions to a new state S'.Batch learning is a learning method in which the value function Q is updated using all the collected learning data by applying a specific action A to a current state S, repeating the transition of state S to a new state S', and collecting learning data. Mini-batch learning is a learning method that lies between online learning and batch learning and involves updating the value function Q whenever a certain amount of learning data has been collected.

[0124] The action information generation unit 1023 selects the action A in the process of Q-learning with respect to the current state S. The action information generation unit 1023 generates action information A and outputs the generated action information A to the action information output unit 103 to perform an operation (corresponding to the action A of Q-learning) for correcting the coefficients a i and b j the speed feedforward processing unit 308 of the servo control device 300 in the process of Q-learning. For example, the action information generation unit 1023 adds or subtracts the coefficients a i and b j the velocity feedforward processing unit 308 in action A incrementally (for example, with an increment of approximately 0.01) with respect to each of the coefficients of the velocity feedforward processing unit in state S.

[0125] If the coefficients a i and b j the velocity feedforward processing unit 308, the state S transitions to the state S' and a plus reward (a positive reward) is offered, wherein the action information generation unit 1023 may select a procedure in which an action A' that increases or decreases the value of the position error, for example, by incrementally increasing or decreasing the coefficients a i and b j the velocity feedforward processing unit 308 is further reduced similarly to the previous action when the next action A' is selected.

[0126] On the other hand, when a minus reward (a negative reward) is offered, the action information generation unit 1023 may select a procedure in which an action A' with which the position error becomes smaller than the previous value, such as by incrementally decreasing or increasing the coefficients a i and b j the velocity feedforward processing unit 308 in contrast to the previous action, as the next action A' is selected.

[0127] The action information generation unit 1023 may select a procedure in which the action A' is selected in accordance with a known method such as a greedy method in which an action A' having the highest value function Q(S,A) within the values ​​of the currently estimated actions A is selected, or an ε-greedy method in which an action A' is randomly selected with a certain small probability ε and otherwise an action A' having the highest value function Q(S,A) is selected.

[0128] The action information output unit 103 is a unit that outputs the action information A and the evaluation function change information output from the learning unit 102 to the output device 200. As described above, the servo control device 300 sets the current state S (that is, the currently set coefficients a i and b j the speed feedforward processing unit 308) based on the action information obtained via the output device 200 to transition to the next state S' (ie, to the corrected coefficients of the speed feedforward processing unit 308).

[0129] The value function storage unit 104 is a storage device that stores the value function Q. The value function Q can be stored, for example, as a table (hereinafter referred to as an action value table) for each state S and each action A. The value function Q stored in the value function storage unit 104 is updated by the value function updating unit 1022. Furthermore, the value function Q stored in the value function storage unit 104 can be shared with other machine learning devices 100. When the value function Q is shared by a plurality of machine learning devices 100, because reinforcement learning is performed distributedly in the respective machine learning devices 100, the efficiency of reinforcement learning can be improved.

[0130] The optimization action information output unit 105 generates the action information A (hereinafter referred to as "optimization action information"), thereby causing the speed feedforward processing unit 308 to perform an operation for maximizing the value function Q(S,A) based on the value function Q updated by the value function updating unit 1022 performing Q learning. Specifically, the optimization action information output unit 105 obtains the value function Q stored in the value function storage unit 104. As described above, the value function Q is updated by the value function updating unit 1022 performing Q learning. The optimization action information output unit 105 generates the action information based on the value function Q and outputs the generated action information to the output device 200.The optimization action information contains information that determines the coefficients a. i and b j the speed feedforward processing unit 308, and evaluation function change information similar to the action information output by the action information output unit 103 in the process of Q-learning.

[0131] As described above, by using the machine learning device 100 according to this embodiment, the setting of the parameters of the speed feedforward processing unit 308 of the servo control device 300 can be simplified.

[0132] In the above-described embodiment, a case is described in which the machine learning device 100 performs learning regarding optimization of the coefficients of the velocity feedforward processing unit during a linear operation in which the rotation direction of the servo motor is not changed in the X-axis and Y-axis directions. However, the invention is not limited to a learning operation during a linear operation and can also be applied to a learning operation during a non-linear operation.For example, when performing learning related to optimizing the coefficients of the velocity feedforward processing unit to correct backlash, a difference between the position command value and the detected position output from the integrator 108 at positions A2 and A4 and positions B2 and B4 of the above-described machining form can be extracted as a position error, and reinforcement learning can be performed by giving a reward using the position error as determination information. At positions A2 and A4 and positions B2 and B4, the rotation direction of the servo motor is reversed in the Y-axis direction or the X-axis direction, so that nonlinear operation is performed and backlash occurs.In this case, the machine learning device can perform learning of the coefficients of the transfer function of the feedforward processing unit during nonlinear operation. The output device and the control device according to the first embodiment were described above. An output device and a control device according to the second and third embodiments will be described below. (Second embodiment)

[0133] In the first embodiment, the output device 200 is connected to the servo control device 300 and the machine learning device 100 to relay information between the machine learning device 100 and the servo control device 300 and to control the operations of the servo control device 300 and the machine learning device 100. In this embodiment, a case where the output device is connected only to the machine learning device will be described. Fig. 18 is a block diagram showing a configuration example of a control device according to the second embodiment of the present invention. A control device 10A includes the machine learning device 100, an output device 200A, the servo control device 300, and the servo motor 400. Unlike the output device 200, the output device 200A does not include the servo information obtaining unit 207 and the servo information output unit 208.

[0134] Because the output device 200A is not connected to the servo control device 300, the output device 200A does not relay information between the machine learning device 100 and the servo control device 300, and does not send and receive information to and from the servo control device 300. In particular, the output device 200A outputs, as shown in Fig. 13, in step S31, outputs a learning program activation command, in step S33 outputs physical quantities of the parameters, and in step S35 outputs a re-learning command, but the output device 200A performs the other operations (for example, steps S32 and S34) of Fig. 13. Because the output device 200A is not connected to the servo control device 300, the number of operations performed by the output device 200A is smaller and the device configuration can be simplified. (Third embodiment)

[0135] In the first embodiment, the output device 200 is connected to the servo control device 300 and the machine learning device 100. In this embodiment, a case will be described where a setting device is connected to the machine learning device 100 and the servo control device 300, and an output device is connected to the setting device. Fig. Figure 19 is a block diagram showing a configuration example of a control device according to the third embodiment of the present invention. A control device 10B includes a machine learning device 100, an output device 200A, a servo control device 300, and an adjustment device 500. Fig. The output device 200A shown in Figure 19 has the same configuration as that shown in Fig. 18, wherein the learning information obtaining unit 201 and the learning information output unit 202 are connected to the setting device 500 and not to the machine learning device 100. The setting device 500 has a configuration in which the graphics generation unit 203, the operation unit 204, the display unit 209, and the arithmetic unit 2100 of the output device 200 of Fig. 2 away.

[0136] As in Fig. 13, gives the Fig. 19 shown output device 200A similar to that in Fig. 18, the output device 200A of the second embodiment outputs a learning program activation command in step S31, outputs physical quantities of parameters in step S33, outputs a parameter fine adjustment command in step S34, and outputs a relearning command in step S35, these operations being performed via the setting device 500. The setting device 500 relays information between the machine learning device 100 and the servo control device 300. Furthermore, the setting device 500 relays a learning program activation command and the like related to the machine learning device 100, which are performed by the output device 200A, to output an activation command to the machine learning device 100, respectively.At this time, since the functions of the output device 200 are distributed between the output device 200A and the setting device 500, the number of operations performed by the output device 200A can be reduced and the device configuration can be simplified compared to the first embodiment.

[0137] Various embodiments of the present invention have been described above. The servo control device described above and the components included in the machine learning device can be implemented by hardware, software, or a combination thereof. The control method performed by the interaction of the components in the servo control device described above can be implemented by hardware, software, or a combination thereof. Software implementation means that a computer reads and executes a program.

[0138] The programs may be stored on any type of non-transitory, computer-readable media and made available to a computer. Non-transitory, computer-readable media include various types of tangible storage media. Examples of non-transitory, computer-readable media include a magnetic recording medium (e.g., a floppy disk or hard disk drive), a magneto-optical recording medium (e.g., a magneto-optical disk), a CD-ROM, a CD-R, a CD-R / W, or a semiconductor memory (e.g., a mask ROM, a PROM (programmable ROM), an EPROM (erasable ROM), a flash ROM, and a RAM).

[0139] The above-described embodiment is a preferred embodiment of the present invention. However, the scope of the present invention is not limited to the above-described embodiments and examples, which may be modified in various ways without departing from the scope of the invention. <Modifikation, in der die Ausgabevorrichtung in der Servosteuervorrichtung oder der Maschinelles-Lernen-Vorrichtung enthalten ist>

[0140] In the embodiments described above, in the first and second embodiments, the machine learning device 100, the output device 200 or 200A, and the servo control device 300 are configured as the control device 10, and in the third embodiment, the output device 200 is provided in the control device such that it is divided into the output device 200A and the adjustment device 500. In these embodiments, the machine learning device 100, the output device 200 or 200A, the servo control device 300, and the adjustment device 500 are configured as separate devices, but the devices may also be integrated into other single devices. For example, part or all of the functions of the output device 200 or 200A may be realized by the machine learning device 100 or the servo control device 300.Furthermore, the output device 200 or 200A may be realized outside a control device constituted by the machine learning device 100 and the servo control device 300. <Freiheit der Systemkonfiguration>

[0141] Fig. 20 is a block diagram showing a control device according to another embodiment of the present invention. As shown in Fig. 20, a control device 10C includes n machine learning devices 100-1 to 100-n, n output devices 200-1 to 200-n, n servo control devices 300-1 to 300-n, servo motors 400-1 to 400-n, and a network 600. Where n is an arbitrary natural number. Each of the n machine learning devices 100-1 to 100-n corresponds to the machine learning device 100 of Fig. 17. Each of the n output devices 200-1 to 200-n corresponds to the output device 200 of Fig. 2 or the output device 200A of Fig. 18. Each of the n servo control devices 300-1 to 300-n corresponds to the servo control device 300 of Fig. 2. The output device 200A and the setting device 500 of Fig. 19 correspond to the output devices 200-1 to 200-n.

[0142] The output device 200-1 and the servo control device 300-1 are paired in a one-to-one relationship and communicatively connected to each other. The output devices 200-2 to 200-n and the servo control devices 300-2 to 300-n are similarly connected to the output device 200-1 and the servo control device 300-1. Although n pairs of output devices 200-1 to 200-n and the servo control devices 300-1 to 300-n are connected via the network 600 in Fig. 20, the n pairs of output devices 200-1 to 200-n and servo control devices 300-1 to 300-n may be directly connected via connection interfaces, respectively. A plurality of n pairs of output devices 200-1 to 200-n and servo control devices 300-1 to 300-n may be provided in the same facility or in different facilities.

[0143] Network 600 is a LAN within a facility, the Internet, a public switched telephone network, a direct connection via a connection interface, or a combination of these. No specific requirements are made here regarding the specific communication scheme of Network 600, whether the network uses a wired or wireless connection, etc.

[0144] In the control device of Fig. 20, the output devices 200-1 to 200-n and the servo control devices 300-1 to 300-n are communicatively connected in a one-to-one correspondence, wherein an output device 200-1 is communicatively connected to a plurality of servo control devices 300-1 to 300-m (m <n oder m="n)" über das netzwerk 600 verbunden sein kann und eine mit einer ausgabevorrichtung 200-1 verbundene maschinelles-lernen-vorrichtung ein maschinelles lernen der servosteuervorrichtungen 300-1 bis 300-m durchführen kann. in diesem fall verteiltes verarbeitungssystem verwendet werden, in dem entsprechende funktionen 100-1 auf vielzahl von servern verteilt sind. die können unter verwendung virtuellen serverfunktion ähnlichem cloud realisiert werden. wenn maschinelles-lernen-vorrichtungen 100-n entsprechung zu 300-n jeweils gleichen typenbezeichnungen, spezifikationen serien vorgesehen sind, die konfiguriert zum gemeinsamen verwenden lernergebnissen den 100-n. dadurch optimales modell erstellt werden.LIST OF REFERENCE SYMBOLS 10, 10A, 10B, 10C control device 100 Machine Learning Device 200 dispensing device 201 Learning Information Preservation Unit 202 Learning information output unit 203 Graphics generation unit 204 Actuating unit 205 Control unit 206 storage unit 207 Servo Information Preservation Unit 208 Servo information output unit 209 Display unit 300 servo control device 400 servo motor 500 adjustment device 600 network< / n>

Claims

[1] Dispensing device (200, 200A) comprising: an output unit (206, 209) that outputs a plurality of evaluation functions used by a machine learning device (100) that performs machine learning of parameters of components of a servo control device (300) that controls a servo motor (400) for driving an axis of a machine tool, a robot, or an industrial machine, and outputs a machine learning result of each of the evaluation functions, and an information obtaining unit (201, 207) that obtains the machine learning result from the servo control device (300) and / or the machine learning device (100). [2] Dispensing device (200, 200A) according to claim 1, wherein: the output unit (206, 209) includes a display unit (209) that displays the plurality of evaluation functions and the machine learning result obtained for each of the evaluation functions on a display screen. [3] Dispensing device (200, 200A) according to claim 1 or 2, comprising: an information output unit (202) that outputs a parameter selected from a plurality of parameters machine-learned using the plurality of evaluation functions based on the machine learning result of each of the plurality of evaluation functions to the servo control device (300). [4] Dispensing device (200, 200A) according to one of claims 1 to 3, comprising: an information output unit (202) that outputs a change command for an evaluation function selected from the plurality of evaluation functions or an evaluation function different from the plurality of evaluation functions to the machine learning device (100). [5] Dispensing device (200, 200A) according to one of claims 1 to 4, wherein: the machine learning result is an error between a control command of the servo control device (300) and feedback information when the servo motor (400) was operated based on the control command. [6] Dispensing device (200, 200A) according to claim 4 or 5, wherein: the machine learning device (100) stores the parameters of the components of the servo control device (300) retrieved during the machine learning using the evaluation function and the machine learning result obtained by operating the servo control device (300) using the parameters, and obtains an evaluation function value based on a changed evaluation function using the stored machine learning result when the machine learning is performed while changing the evaluation function based on the change command and the same parameters are selected as the parameters. [7] Dispensing device (200, 200A) according to one of claims 4 to 6, wherein: the different evaluation function is an evaluation function whose weighting coefficient is different from that of the plurality of evaluation functions. [8] Dispensing device (200, 200A) according to one of claims 1 to 7, wherein: the plurality of evaluation functions include an evaluation function consisting of a weighted sum of a function in which a position error is a variable, a function in which a difference value of a position error is a variable, and a function in which a difference value of a torque is a variable. [9] Dispensing device (200, 200A) according to one of claims 3 to 8, wherein: the parameters of the components of the servo control device (300) contain parameters of a numerical expression model or a filter. [10] Dispensing device (200, 200A) according to claim 9, wherein: the numerical expression model or the filter is contained in a velocity feedforward processing unit or a position feedforward processing unit and the parameters contain coefficients of a transfer function of the filter. [11] Control device (10) comprising: the output device (200, 200A) according to one of claims 1 to 10, a servo control device (300) that controls a servo motor (400) for driving an axis of a machine tool, a robot or an industrial machine, and a machine learning device (100) that performs machine learning with respect to the servo control device (300). [12] Control device (10) according to claim 11, wherein: the output device (200, 200A) is included in the servo control device (300) or the machine learning device (100). [13] An output method for an output device (200, 200A) to output an evaluation function and a machine learning result, comprising: Outputting a machine learning result of each of the evaluation functions obtained by performing machine learning using a plurality of evaluation functions, respectively, by means of a machine learning device (100) that performs machine learning using the evaluation function with respect to a servo control device (300) that controls a servo motor (400) for driving an axis of a machine tool, a robot, or an industrial machine, and Obtaining the machine learning result from the servo control device (300) and / or the machine learning device (100).

Images