Information processing device and information processing method

Abstract

The present invention relates to an information processing device and an information processing method, which can more easily perform learning related to a device having a driving unit. The information processing device (400) comprises: a storage unit (403) storing a processing program for causing a motor (201, 202) of a machine tool, a robot or an industrial machine to operate; and a generation unit (405) extracting a part of the processing program containing characteristic elements from the processing program stored in the storage unit (403), thereby generating a learning program for learning based on the operating characteristics of the motor (201, 202).

Description

Technical Field

[0001] The present invention relates to an information processing device and an information processing method for performing information processing related to machine learning. Background Art

[0002] Conventionally, the motion characteristics of devices having drive parts such as machine tools, robots, and industrial machinery are learned through machine learning. These devices can be controlled more appropriately by performing control based on the learning results of machine learning.

[0003] For example, a technology related to such machine learning is disclosed in Patent Document 1. In the technology disclosed in Patent Document 1, in a control system for machining a machining shape formed by an arc, a polygon, or a combination thereof, learning control is performed based on the deviation between the position feedback of a servo motor detected in each sampling period and the position command.

[0004] Patent Document 1: Japanese Patent Application Publication No. 2012-58824 Summary of the invention

[0005] Problem that the invention aims to solve

[0006] In addition, for example, it is desirable to perform machine learning based on position deviations and the like as described above using a machining program itself that is actually used in machining a workpiece.

[0007] However, it takes a lot of time to learn by repeatedly operating the machining program actually used in machining the workpiece. In addition, when the amount of data of the machining program actually used in machining the workpiece is large, a large-capacity memory or the like needs to be prepared for learning.

[0008] The present invention has been made in view of such circumstances, and an object of the present invention is to provide an information processing device and an information processing method for more easily performing learning related to a device having a driving unit.

[0009] Solutions for solving problems

[0010] (1) The information processing device involved in the present invention (for example, the program generation device 400 described later) comprises: a storage unit (for example, the processing program storage area 403 described later), which stores a processing program for causing the motor (for example, the motor 201 and the motor 202 described later) of a machine tool, a robot or an industrial machine to operate; and a generation unit (for example, the learning program generation unit 405 described later), which generates a learning program for learning based on the operation characteristics of the motor by extracting a part of the processing program containing characteristic elements from the processing program stored in the storage unit.

[0011] (2) The information processing device described in (1) above may also be configured to further include: a learning unit (for example, the learning unit 302 described later), which causes the motor to operate based on the learning program generated by the generating unit to perform learning based on the operation characteristics of the motor; and an adjustment unit (for example, the optimized action information output unit 305 described later), which adjusts the control parameters of the motor based on the learning results of the learning unit.

[0012] (3) The information processing device described in (1) or (2) above may also be configured such that the generating unit extracts a processing program including at least any one of the following processing programs as a part of the processing program containing the characteristic elements: a processing program for causing the motor to perform an action accompanied by a reversal of the motor speed; a processing program for causing the motor to perform an action accompanied by switching from stop to action and switching from action to stop; and a processing program for causing the motor to perform an action with acceleration change but not accompanied by a reversal of the motor speed.

[0013] (4) The information processing device described in (1) or (2) above may be configured such that the generating unit extracts the following processing program as a part of the processing program including the characteristic element: when the motor is caused to move, the processing program causes the motor to move in such a way that the position deviation exceeds a threshold value.

[0014] (5) In the information processing device described in (4) above, the generating unit may determine whether the position deviation exceeds a threshold value based on a result of operating the motor based on the machining program.

[0015] (6) The information processing device described in (1) or (2) above can also be set up so that the generating unit extracts the following part of the processing program as the part of the processing program containing the characteristic element: the part of the processing program is specified by the user with reference to the information representing the change of the position deviation along the time series or the information of the moving trajectory in the action of the motor.

[0016] (7) The information processing device described in any one of the above (1) to (6) may also be configured to further include a display unit (for example, a user interface unit 406 described later) that displays the portion of the processing program extracted by the generation unit.

[0017] (8) The information processing method involved in the present invention is performed by a computer (for example, the program generation device 400 described later), and the computer performs the following steps: a storage step, storing a processing program for causing a motor (for example, the motor 201 and the motor 202 described later) of a machine tool, a robot or an industrial machine to operate; and a determination step, generating a learning program for learning based on the operation characteristics of the motor by extracting a part of the processing program containing characteristic elements from the processing program stored in the storage step.

[0018] Effects of the Invention

[0019] According to the present invention, it is possible to provide an information processing device and an information processing method for more easily performing learning related to a device having a driving unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a block diagram showing the structure of a CNC device that operates based on a learning program, a motor, a machine learning device, and a program generation device according to an embodiment of the present invention.

[0021] Figure 2 This is a block diagram showing a configuration example of a motor control unit.

[0022] Figure 3 This is a block diagram showing a structural example of a program generation device.

[0023] Figure 4 This is an explanatory diagram showing an example in which the processed shape is a shape formed by an arc (circle).

[0024] Figure 5 This is an explanatory diagram showing an example in which the processed shape is a shape formed of quadrilaterals.

[0025] Figure 6 This is an explanatory diagram showing an example in which the processed shape is a shape formed by a quadrilateral with a rounded corner.

[0026] Figure 7 This is an explanatory diagram showing a first configuration example in which a processed shape is formed of an octagon in which some corners (corners) are rounded.

[0027] Figure 8 is an explanatory diagram showing a second structural example in which the processed shape is a shape formed by a cross, each end of the cross having Figure 7 The first structural example shown has a shape of a half of an octagon in which a part of the corners (corners) are circular arcs.

[0028] Fig. 9 This is an explanatory diagram showing a third structural example in which the processed shape is a shape formed by a cross with a rounded angle.

[0029] Fig.10 This is an explanatory diagram showing a fourth structural example in which the processed shape is a cross-shaped shape composed of a convex circular arc and a concave circular arc.

[0030] Fig.11 It is an explanatory diagram showing a fifth structural example in which the processed shape is a shape formed by a star shape.

[0031] Fig.12 This is a schematic diagram showing an example of highlighting a portion extracted from a machining program.

[0032] Fig.13 This is a block diagram showing a part of the structure of another example of the structure of the CNC device.

[0033] Fig.14 This is a block diagram showing the machine learning device 300 according to the first embodiment.

[0034] Description of Reference Numerals

[0035] 100: CNC (Computerized Numerical Control) device; 101: storage unit; 102: numerical control information processing unit; 103, 104: motor control unit; 201, 202: motor; 300: machine learning device; 400: program generation device; 401: machining program acquisition unit; 402: storage unit; 403: machining program storage area; 404: learning program storage area; 405: learning program generation unit; 406: user interface unit; 407: learning instruction unit. DETAILED DESCRIPTION

[0036] Hereinafter, embodiments of the present invention will be described in detail using the drawings.

[0037] In the present embodiment, a part of a machining program including characteristic elements is extracted from a machining program for operating a motor of a machine tool, a robot, or an industrial machine, thereby generating a learning program for learning based on the operating characteristics of the motor.

[0038] In addition, in the present embodiment, learning is performed using a learning program generated in such a manner as to include characteristic elements, rather than using the machining program itself. For example, a CNC (Computerized Numerical Control) device as a control device operates the learning program to perform feedforward control on the motor, thereby learning control parameters related to the feedforward control.

[0039] In this embodiment, by generating a learning program including characteristic elements in this way for learning, compared with the case of learning using the machining program itself actually used in workpiece machining, more learning can be carried out in a short time. In addition, since a learning program with a smaller data volume than the data volume of the machining program is used, the capacity of the memory required for learning can be reduced.

[0040] That is to say, according to this embodiment, learning related to a device having a drive unit (here, as an example, a motor) can be carried out more simply.

[0041] <System Structure>

[0042] Before explaining the generation of the above learning program, first, the structure of the entire system including the CNC device that operates based on the learning program will be explained.

[0043] Figure 1 It is a block diagram showing the structure of the CNC device 100, the motors 201 and 202, the machine learning device 300, and the program generation device 400 according to an embodiment of the present invention.

[0044] The CNC device 100, the machine learning device 300, and the program generation device 400 are connected in a manner that enables communication with each other. In addition, the CNC device 100 is connected to the motors 201 and 202 in a manner that can output current command values (torque command values) to the motors 201 and 202 as control objects.

[0045] In addition, in this embodiment, the case where the machine learning device 300 adjusts (learns) the control parameters related to velocity feedforward is taken as an example for explanation. However, this is merely an example, and this embodiment can also be applied to the adjustment (learning) of other control parameters. For example, this embodiment can also be applied to the adjustment (learning) of control parameters related to position feedforward, control parameters related to static friction, and control parameters related to backlash during reverse rotation.

[0046] <CNC Device 100, Motor 201, and Motor 202>

[0047] The CNC device 100 includes a storage unit 101, a numerical control information processing unit 102, and motor control units 103 and 104 that constitute a servo control device. The CNC device 100 controls the motor 201 using the motor control unit 103 and controls the motor 202 using the motor control unit 104.

[0048] The CNC device 100 is used for a machine tool, a robot, an industrial machine, or the like. In this embodiment, a case where the CNC device 100 is used for a machine tool will be described as an example.

[0049] The storage unit 101 stores a processing program. The processing program is a program for actually processing a workpiece by cutting, laser beam processing, etc. The designer generates the processing program using CAD (Computer Aided Design) or CAM (Computer Aided Manufacturing), and stores the processing program in the storage unit 101.

[0050] When machining a workpiece, the numerical control information processing unit 102 reads a machining program from the storage unit 101 and executes the read machining program. Specifically, the numerical control information processing unit 102 sets the axial movement distance, feed speed, etc. based on the code included in the machining program and generates position command values ​​for the X-axis and Y-axis to form a machining shape specified by the machining program.

[0051] In addition, during learning, the numerical control information processing unit 102 reads and executes the learning program input from the program generation device 400. Specifically, the numerical control information processing unit 102 sets the axial movement distance, feed speed, etc. based on the learning program, and generates position command values ​​for the X-axis and Y-axis.

[0052] Furthermore, the numerical control information processing unit 102 outputs the generated X-axis position command value to the motor control unit 103 during workpiece machining and learning. Furthermore, the numerical control information processing unit 102 outputs the generated Y-axis position command value to the motor control unit 104.

[0053] The motor control unit 103 creates a current command value (torque command value) based on the X-axis position command value and the position detection value fed back from the motor 201 , and outputs the current command value to the motor 201 .

[0054] The motor control unit 104 creates a current command value (torque command value) based on the Y-axis position command value and the position detection value fed back from the motor 202 , and outputs the current command value to the motor 202 .

[0055] The motor 201 and the motor 202 rotate based on the current command value (torque command value) input from the motor control unit 103 and the motor control unit 104, respectively.

[0056] The motor 201 and the motor 202 are connected to ball screws, respectively. The two ball screws connected to the motor 201 and the motor 202 convert the rotational motion of the motor 201 and the motor 202 into linear motion, thereby driving the table in the X-axis direction and the Y-axis direction.

[0057] Thus, the motor 201 drives the table on which the workpiece to be processed is placed in the X-axis direction. Also, the motor 202 drives the table on which the workpiece to be processed is placed in the Y-axis direction.

[0058] In addition, the case where the controlled axes are two axes, the X axis and the Y axis, is described here, but this is only an example, and the controlled axes may be three or more axes. In the case where there are three or more axes to be controlled, the number of motors corresponding to the number of axes is further provided. In addition, the number of motor control units is further provided corresponding to the number of motors. In addition, as the motor 201 and the motor 202, for example, a servo motor or a spindle motor can be used.

[0059] The rotation angle position of the motors 201 and 202 is detected by a position detection unit (eg, a rotary encoder) associated with the motors 201 and 202, and the detected signal is used as a speed detection value for speed feedback. The speed detection value is integrated and used as a position detection value for position feedback.

[0060] Reference Figure 2 The following describes more specific structures of the motor control unit 103 and the motor control unit 104. The motor control unit 103 is used as an example for description, but the motor control unit 104 also has the same structure. Therefore, repeated description of the motor control unit 104 is omitted.

[0061] Figure 2 1 is a block diagram showing a configuration example of the motor control unit 103. Figure 2 As shown, the motor control unit 103 includes a subtractor 1001 , a position control unit 1002 , an adder 1003 , a subtractor 1004 , a speed control unit 1005 , an adder 1006 , an integrator 1007 , a position feedforward calculation unit 1008 , and a speed feedforward calculation unit 1009 .

[0062] The numerical control information processing unit 102 generates a position command value based on a code included in a machining program or a learning program described later, and outputs the position command value to the subtractor 1001 and the position feedforward calculation unit 1008 .

[0063] The subtractor 1001 obtains the difference between the position command value and the position detection value of the position feedback, and outputs the difference as the position deviation to the position control unit 1002. During learning, the position deviation is also output to the machine learning device 300. After learning based on the learning program, the position deviation is not output to the machine learning device 300 when the machining program is executed.

[0064] Position control unit 1002 outputs the value obtained by multiplying the position deviation by position gain Kp as a speed command value to adder 1003. Position feedforward calculation unit 1008 outputs the value obtained by differentiating the position command value and multiplying it by the feedforward coefficient to adder 1003 and speed feedforward calculation unit 1009.

[0065] The adder 1003 adds the speed command value and the output value of the position feedforward calculation unit 1008, and outputs the speed command value after feedforward control to the subtractor 1004. The subtractor 1004 obtains the difference between the output of the adder 1003 and the speed detection value of the speed feedback, and outputs the difference to the speed control unit 1005 as a speed deviation.

[0066] The speed control unit 1005 adds a value obtained by multiplying the speed deviation by the integral gain K1v and integrating the value obtained by multiplying the speed deviation by the proportional gain K2v, and outputs the result as a torque command value to the adder 1006 .

[0067] The speed feedforward calculation unit 1009 performs speed feedforward calculation processing represented by a transfer function Gf(s), for example, represented by Formula 1 (hereinafter referred to as Formula 1), and outputs the result to the adder 1006 .

[0068]

Number 1

[0069]

[0070] The adder 1006 adds the torque command value and the output value of the speed feedforward calculation unit 1009 and outputs the result to the adder 1006. The adder 1006 outputs the added value to the motor 201 as a current command value (torque command value).

[0071] The rotation angle position of the motor 201 is detected by a rotary encoder as a position detection unit associated with the motor 201, and the detected signal is used as a speed detection value for speed feedback. The speed detection value is integrated by an integrator 1007 and used as a position detection value for position feedback.

[0072] <Machine Learning Device 300>

[0073] In this embodiment, the case where the machine learning device 300 performs reinforcement learning as machine learning is described as an example. However, this is just an example, and the machine learning device 300 can also perform machine learning other than reinforcement learning. For example, machine learning such as supervised learning and semi-supervised learning can also be performed.

[0074] When the machine learning device 300 performs reinforcement learning as machine learning, the dimension of the transfer function Gf(s) is set to a preset value, and each coefficient a is i , b j (i, j ≥ 0) is set to an arbitrary value, and then the CNC device 100 runs the learning program to observe the relationship between the control parameter a and the control parameter a. i , b j The operation characteristics of the motors 201 and 202 operated under the control of the CNC device 100 are obtained by using the machine learning device 300. i , b j The coefficient a that is optimal for the motion characteristics of the machine tool when the machine tool is operated based on the learning program is adjusted (learned) from the set of i , b j .

[0075] Therefore, the machine learning device 300 uses the position detection values ​​fed back from the motors 201 and 202 to learn the control parameter a for feedforward correction. i , b j , and set optimal control parameters for the motor control unit 103 and the motor control unit 104.

[0076] More specifically, the machine learning device 300 observes the following state information S and performs control parameters a related to the state S. i , b j Adjustment (Action A): The state information S contains the adjustment based on the control parameter a i , b j (i, j ≥ 0) is position deviation information as the operation characteristics of the CNC device 100 acquired by executing the learning program. The state information S includes servo states such as commands and feedback.

[0077] When the state information S is modified to the state information S′ by the action information A, based on the modified control parameter a related to the state information S′ i ′、b j The position deviation value of the CNC device 100 that performs the operation is greater than the control parameter a before correction based on the state information S before correction by the action information A. i , b jIn the case of the position deviation of the CNC device 100 that performs the operation, the value of the reward r is set to a negative value. On the other hand, based on the corrected control parameter a related to the state information S′ corrected by the action information A i ′、b j The position deviation value of the CNC device 100 that performs the operation is smaller than the control parameter a before correction based on the state information S before correction by the action information A. i , b j In the case of a position deviation value of the CNC device 100 that is operating, the value of the reward r is set to a positive value.

[0078] The machine learning device 300 searches for the best action A that maximizes the total of the reward r that continues into the future through repeated trials. In this way, the machine learning device 300 can identify the optimal action A that includes the action based on the control parameter a. i , b j The state S of the servo state including the command, feedback, etc. of the position deviation information of the CNC device 100 acquired by executing the learning program is selected as the best action A (that is, the best control parameter a i , b j ).

[0079] Note that the machine learning performed by the machine learning device 300 will be described in more detail in the supplementary description to be described later.

[0080] <Program Generation Device 400>

[0081] Figure 3 4 is a block diagram showing the structure of the program generation device 400. Figure 3 As shown, the program generating device 400 includes a machining program acquiring unit 401 , a storage unit 402 , a machining program storage area 403 , a learning program storage area 404 , a learning program generating unit 405 , a user interface unit 406 , and a learning instructing unit 407 .

[0082] As described above, the program generation device 400 is a device that generates a learning program for learning based on the operation characteristics of the motors 201 and 202 by extracting a part of the machining program including characteristic elements from the machining program.

[0083] The machining program acquisition unit 401 acquires the machining program stored in the storage unit 101 . In addition, the machining program acquisition unit 401 stores the acquired machining program in a machining program storage area 403 provided in the storage unit 402 .

[0084] The storage unit 402 is a storage unit having a machining program storage area 403 and a learning program storage area 404. The storage unit 402 stores the machining program input from the machining program acquisition unit 401 in the machining program storage area 403. In addition, the storage unit 402 stores the learning program generated by the learning program generation unit 405 in the learning program storage area 404.

[0085] The learning program generation unit 405 generates a learning program for learning based on the operation characteristics of the motors 201 and 202 by extracting a portion of the machining program including characteristic elements from the machining program stored in the machining program storage area 403. In addition, the learning program generation unit 405 stores the generated learning program in the learning program storage area 404.

[0086] For details on the generation of the learning program by the learning program generation unit 405, refer to Figures 4 to 11 This will be described later.

[0087] The user interface unit 406 has a function of displaying a user interface for accepting operations from a user who uses the program generation device 400. In addition, the user interface unit 406 also has a function of accepting operations from a user who has referred to the user interface. The user's operation is, for example, an instruction to generate a learning program, an instruction to start machine learning using the generated learning program, etc. The user interface unit 406 outputs the accepted operation content to the learning program generation unit 405 and the learning instruction unit 407. Then, the learning program generation unit 405 and the learning instruction unit 407 perform processing corresponding to the input operation content.

[0088] The learning instruction unit 407 instructs the machine learning device 300 to start learning. Specifically, the learning instruction unit 407 reads the learning program stored in the learning program storage area 404. Then, the learning instruction unit 407 outputs the read learning program to the numerical control information processing unit 102, thereby instructing the machine learning device 300 to start learning.

[0089] The numerical control information processing unit 102 to which the learning program is input generates position command values ​​for each of the X-axis and the Y-axis based on the input learning program that replaces the machining program stored in the storage unit 101. Then, the numerical control information processing unit 102 outputs the generated position command to the motor control unit 103 and the motor control unit 104. As a result, the motors 201 and 202 start to perform actions based on the learning program. In addition, information indicating the action characteristics of the motors 201 and 202 (for example, position deviation in this embodiment) is output to the machine learning device 300. As a result, learning of the machine learning device 300 based on the learning program can be achieved.

[0090] In this way, by generating a learning program based on a portion of the machining program including characteristic elements, it is possible to add a control parameter a for adjusting the feedforward. i , b j Therefore, the control parameter a related to the feedforward can be adjusted (learned) more accurately and efficiently. i , b j .

[0091] <Creation of Learning Program>

[0092] Next, the details of the generation of the learning program by the learning program generation unit 405 will be described.

[0093] The learning program generation unit 405 adjusts (learns) the control parameter a i , b j When the learning program generation unit 405 extracts a portion of the machining program including the characteristic elements from the machining program. For example, the learning program generation unit 405 extracts a machining program including at least any one of the following machining programs as a portion of the machining program including the characteristic elements: a machining program for causing the motor to perform an action accompanied by a reversal of the motor speed; a machining program for causing the motor to perform an action accompanied by switching from a stop to an action and switching from an action to a stop; and a machining program for causing the motor to perform an action with acceleration change without a reversal of the motor speed.

[0094] Such a part of the machining program including characteristic elements is a machining program for machining into a circular arc, a quadrilateral, and a quadrilateral with rounded corners. Here, the quadrilateral with rounded corners refers to a quadrilateral with a circular arc corner.

[0095] Then, the learning program generating unit 405 generates a learning program so as to include the extracted machining program.

[0096] Here, when the learning program can be generated using only the extracted machining program, the learning program generating unit 405 generates the learning program using only the extracted machining program.

[0097] However, the machining program is originally created for machining a workpiece, and is not created with the assumption that a part thereof is extracted and utilized. Therefore, sometimes it is not possible to properly operate the motors 201 and 202 for learning using only the extracted machining program.

[0098] Therefore, in this case, the learning program generation unit 405 generates a learning program by adding a program for performing the following processing to the extracted processing program: the processing required to execute the extracted processing program (hereinafter referred to as "pre-processing"); the processing required after executing the extracted processing program (hereinafter referred to as "post-processing").

[0099] Preprocessing is, for example, driving a table for carrying a workpiece to be processed to a position corresponding to the start position of the extracted processing program along the X-axis direction and the Y-axis direction. In addition, preprocessing is, for example, setting the tool to a state in which the extracted processing program can be started.

[0100] In addition, post-processing is, for example, a process of driving a table for carrying a workpiece to be processed to a position corresponding to the end position of the processing program before extraction along the X-axis direction and the Y-axis direction. In addition, post-processing is, for example, a process of setting a tool to the same state as the end state of the processing program before extraction.

[0101] Next, refer to Figures 4 to 11 , using a specific example, it is explained what kind of machining program the learning program generation unit 405 extracts as a part of the machining program including the characteristic elements. In addition, Figures 4 to 11 The diagram shows a machining shape based on a machining program. The machining shape corresponds to the shape of the movement trajectory of the workpiece as a control object, which moves along the X-axis and Y-axis driven by the motor 201 and the motor 202. In addition, it is assumed that the machining in each figure is performed in the clockwise direction. However, in addition to the clockwise machining, the machining can also be performed in the counterclockwise direction.

[0102] When the machining shape is an arc (circle), for example Figure 4 At point A shown in the figure, the rotation direction of the servo motor that moves the table in the X-axis direction is reversed, and the table moves in a linear reverse manner in the X-axis direction. That is, point A is a characteristic element accompanied by the reversal of the motor speed, so the learning program generation unit 405 extracts a processing program for processing the workpiece into a processing shape including point A (for example, a processing shape represented by a dotted line in the figure) to generate a learning program.

[0103] When the processed shape is a quadrilateral, for example Figure 5The corner (corner) of point B shown in the figure causes the servo motor that moves the table in the Y-axis direction to switch from a stop to a rotational motion, the servo motor that moves the table in the X-axis direction to switch from a rotational motion to a stop, and the table to switch from a linear motion in the X-axis direction to a linear motion in the Y-axis direction. That is, point B is a characteristic element accompanied by switching from a stop to a motion, so the learning program generation unit 405 extracts a machining program for machining the workpiece into a machining shape including point B (for example, a machining shape indicated by a dotted line in the figure) to generate a learning program.

[0104] This is also true when the processed shape is a quadrilateral with R corners, for example Figure 6 At point C1 shown, the servo motor that moves the table along the Y-axis direction changes from a stop to a rotational motion, and the table changes from a linear motion in the X-axis direction to an arc motion. Figure 6 At the point C2 shown, the servo motor that moves the table in the X-axis direction changes from rotation to stop motion, and the table changes from circular motion to linear motion in the Y-axis direction. That is, point C is a characteristic element accompanied by switching from stop to motion and from motion to stop, so the learning program generation unit 405 extracts a machining program for machining the workpiece into a machining shape including point C (for example, a machining shape indicated by a dotted line in the figure) to generate a learning program.

[0105] As described above, it is possible to evaluate the inertia (inertial motion) that occurs when the rotation direction is reversed or the rotation state is stopped according to a designated machining shape, and examine the influence on the position deviation.

[0106] In addition to these processing shapes, the learning program generation unit 405 can also extract a part of the processing program from the processing program corresponding to the following processing shapes: the processing shape can observe the action at the corner where both motors do not reverse, the action in a shape starting from stop and moving in the same direction as before stop, and the characteristics of the action at the corner where both motors reverse.

[0107] For example, you can also Figures 7 to 11 A part of the machining program is extracted from the machining program corresponding to the machining shape shown.

[0108] When the processed shape is an octagon with some corners (corners) rounded, for example Figure 7Point D shown in the figure has a discontinuous change in speed in addition to the action characteristics when the arc is reversed. That is, point D is a characteristic element accompanied by the reversal of the motor speed and the change in acceleration, so the learning program generation unit 405 extracts a machining program for machining the workpiece into a machining shape including point D (for example, the machining shape indicated by the dotted line in the figure) to generate a learning program.

[0109] When the processed shape is a shape formed by the following cross, for example Figure 8 At the point E shown, the motor driving the X-axis performs an action from deceleration to stop, while the motor driving the Y-axis performs an action from stop to rotation, and each end of the cross has a shape of half of an octagon with a part of the angle (corner) set as an arc. That is, the point E is a characteristic element accompanied by acceleration change without the reversal of the motor speed, so the learning program generation unit 405 extracts a processing program for processing the workpiece into a processing shape including the point E (for example, the processing shape indicated by the dotted line in the figure) to generate a learning program.

[0110] In the case where the processed shape is a shape formed by a cross with an R angle, for example Fig. 9 At point F1 shown in FIG. 1 , the motor driving the X-axis starts rotating in the same direction as before the stop. At point F2, the motor driving the Y-axis stops rotating while the motor driving the X-axis is rotating in the same direction as before the stop. That is, point F1 and point F2 are characteristic elements that are accompanied by switching from stop to action and switching from action to stop. Therefore, the learning program generation unit 405 extracts a processing program for processing the workpiece into a processing shape including point F1 and point F2 (for example, a processing shape indicated by a dotted line in the figure) to generate a learning program.

[0111] In the case where the processed shape is a cross-shaped shape composed of a convex arc and a concave arc, for example Fig.10 At the H1 point shown, the motor driving the X-axis starts rotating in the same direction immediately after the rotation stops. Also, at the H2 point, the motor driving the Y-axis starts rotating in the same direction immediately after the rotation stops. That is, the G1 point and the G2 point are characteristic elements accompanied by switching from stop to action and switching from action to stop, so the learning program generation unit 405 extracts a processing program for processing the workpiece into a processing shape including the G1 point and the G2 point (for example, the processing shape indicated by the dotted line in the figure) to generate a learning program.

[0112] In the case where the processed shape is a shape formed by a star, for example Fig.11At point I shown in the figure, the rotation directions of the motor driving the X-axis and the motor driving the Y-axis are reversed. That is, point H is a characteristic element accompanied by the reversal of the motor speed, so the learning program generation unit 405 extracts a machining program for machining the workpiece into a machining shape including point H (for example, the machining shape indicated by the dotted line in the figure) to generate a learning program.

[0113] As described above, the learning program generation unit 405 extracts, for example, a processing program including at least any one of the following processing programs as a part of the processing program containing characteristic elements: a processing program for causing the motor to perform an action accompanied by a reversal of the motor speed; a processing program for causing the motor to perform an action accompanied by switching from stop to action and switching from action to stop; and a processing program for causing the motor to perform an action with acceleration change without a reversal of the motor speed.

[0114] Then, by operating the motors 201 and 202 based on the learning program created based on the characteristic elements, it is possible to obtain the control parameter a for observing and adjusting the speed feedforward. i , b j Therefore, the control parameters a related to speed feedforward can be learned more accurately and efficiently. i , b j .

[0115] In addition, even if learning is performed based on only some characteristic elements in this way, problems such as over-learning will not occur in actual application. The reason for this is that the control structure of the CNC device 100 is configured to be stable in a wide range as a premise.

[0116] As described above, in this embodiment, a part of a machining program including characteristic elements is extracted from a machining program for operating a motor of a machine tool, a robot, or an industrial machine, thereby generating a learning program for learning based on the operating characteristics of the motor.

[0117] In addition, in the present embodiment, learning is performed using a learning program generated in such a manner as to include characteristic elements, rather than using the machining program itself. For example, the CNC device 100 as a control device operates the learning program and performs feedforward control on the motors 201 and 202, thereby learning control parameters related to the feedforward control.

[0118] In this embodiment, a learning program generated in such a manner as to include characteristic elements is generated for learning, so that more learning can be performed in a short time compared to the case where the machining program itself actually used in the machining of the workpiece is used for learning. In addition, since a learning program having a smaller data volume than the machining program is used, the capacity of the memory required for learning can be reduced. Therefore, even a device having a small memory capacity such as an embedded device can implement this embodiment.

[0119] That is, according to the present embodiment, it is possible to more easily perform learning related to a device having a drive unit (here, a motor as an example).

[0120] The above-mentioned embodiments are preferred embodiments of the present invention, but the scope of the present invention is not limited to the above-mentioned embodiments, and can be implemented in a manner obtained by implementing various changes within the scope of the main purpose of the present invention. For example, the above-mentioned embodiments can be implemented in a manner obtained by implementing various changes as in the modified examples described below.

[0121] <First Modification>

[0122] In the above-mentioned embodiment, for example, it is envisaged that Figures 4 to 11 The learning program produced by the various characteristic elements shown in the figure is used for learning. It is not limited to this, and various characteristic elements can also be classified according to the same or similar characteristic elements, and learning is performed for the characteristic elements of each classification. In other words, independent learning models can also be constructed for the characteristic elements of each classification.

[0123] In this case, the number of machine learning devices 300 corresponding to the number of learning models to be constructed is prepared. Alternatively, the number of machine learning devices 300 including learning units corresponding to the number of learning models to be constructed is prepared.

[0124] Then, the learning program generation unit 405 classifies various characteristic elements into the same or similar characteristic elements, and generates a learning program for the characteristic elements of each classification. In addition, the learning program generation unit 405 causes different machine learning devices 300 (or different learning units) to learn based on a plurality of learning programs created for the characteristic elements of each classification.

[0125] Thus, independent learning models are constructed for the characteristic elements of each category.

[0126] In addition, when performing machining based on a machining program and adjusting parameters using a learning model, for example, the numerical control information processing unit 102 pre-reads the machining program before executing it and determines what characteristic elements the machining program contains. Then, the numerical control information processing unit 102 selects a learning model for parameter adjustment based on the determination result.

[0127] Thereby, it is possible to construct learning models corresponding to the characteristic elements of each classification, and to utilize the learning models corresponding to the characteristic elements included in the machining program.

[0128] Furthermore, the numerical control information processing unit 102 may have the function of pre-reading the machining program and selecting the learning model as described above, but the machine learning device 300 and the program generation device 400 may also include a functional module having such a function.

[0129] <Second Modification>

[0130] In the above-mentioned embodiment, the learning program generation unit 405 reads the machining program, and when there is a portion containing characteristic elements in the read machining program, the portion is extracted to generate the learning program. In other words, the learning program is generated by analyzing the machining program. This is not limited to this, and the machining program can also be actually operated, and the learning program can be generated based on the operation characteristics at this time.

[0131] In this case, for example, the learning program generation unit 405 actually causes the machining program to run, and obtains information indicating the motion characteristics at this time (for example, position deviation and speed deviation detected by the CNC device 100). Then, the learning program generation unit 405 generates the learning program by, for example, extracting the machining program corresponding to the motion with large position deviation and speed deviation. In this way, the learning program can be generated with a reference different from the predetermined references such as "with reversal" and "with acceleration change".

[0132] Whether the position deviation or speed deviation is large can be determined by presetting a predetermined threshold and determining that the position deviation or speed deviation is large when the position deviation or speed deviation detected by the CNC device 100 exceeds the predetermined threshold.

[0133] <Third Modification>

[0134] Alternatively, in addition to the processing of the above-mentioned embodiment, the learning program generation unit 405 may output to the user which part of the entire machining program corresponds to the portion extracted from the machining program for generating the learning program. In this case, the output to other than the user may be performed by, for example, displaying it to the user interface unit 406.

[0135] There are various methods for displaying the process, but it is preferable to highlight the part extracted from the processing program so that it can be distinguished from other parts. Fig.12 An example of highlighting is shown in FIG.

[0136] exist Fig.12 In the example shown, the entire code constituting the machining program is displayed, and the code corresponding to the portion extracted from the machining program to generate the learning program is highlighted. By referring to this display, the user can understand which portion of the entire machining program the portion extracted from the machining program corresponds to. Fig.12 The display shown is an example and is not limited to this.

[0137] For example, information indicating the change of the deviation along the time series may be displayed, and in this display, a portion corresponding to the portion extracted from the machining program in order to generate the learning program is highlighted. As information indicating the change of the deviation along the time series, for example, waveform data with time as the horizontal axis and deviation (for example, position deviation, speed deviation) as the vertical axis can be used.

[0138] In addition, for example, it is also possible to display Figures 4 to 11 The information of the moving track of the motor in the operation as shown in FIG. 1 is displayed, and the part corresponding to the part extracted from the machining program for generating the learning program is highlighted in the display. Figures 4 to 11 The dashed portion shown is highlighted.

[0139] In addition, as long as the extracted portion is displayed in a manner distinguishable from other portions, a display other than highlighting may be performed.

[0140] <Fourth Modification>

[0141] In the above-described embodiment, the learning program generation unit 405 determines the portion to be extracted from the machining program, but the portion to be extracted may be determined based on a designation of the extraction portion by the user.

[0142] In this case, for example Fig.12 As in the example shown, the entire code constituting the machining program is displayed, and a user who has referred to the display receives a designation of an extraction portion via the user interface unit 406. Then, the learning program generating unit 405 extracts the machining program corresponding to the designated portion, thereby generating a learning program.

[0143] In this way, a learning program that conforms to the user's intention can be generated.

[0144] Alternatively, the user may specify the extraction portion by another method. For example, information indicating the variation of the deviation along a time series as described in the third modification example may be displayed, and the user may specify the extraction portion by referring to the display via the user interface unit 406.

[0145] In addition, for example, it is also possible to display Figures 4 to 11 The information of the movement trajectory of the motor during operation as shown is displayed, and the user interface unit 406 receives the designation of the extraction portion from the user who has referred to the display.

[0146] <Other Modifications>

[0147] In addition, in this embodiment, machine learning is exemplified, which can efficiently perform control parameter a related to speed feedforward by observing the motion characteristics of the machine tool when the machine tool is operated based on the learning program. i , b j However, the application of the learning program is not limited to the control parameters a related to speed feedforward. i , b j For example, the present invention can also be applied to the adjustment (learning) of control parameters related to position feedforward, control parameters related to static friction, and control parameters related to backlash during reverse rotation.

[0148] The machine learning device 300 observes position deviation as an operation characteristic by causing the CNC device 100 to execute a learning program, but feedback information is not limited to position deviation and may include at least one of position instruction and position feedback (position detection value) in addition to position deviation.

[0149] In addition, as an action instruction, at least one of a speed instruction, a speed feedback, a speed deviation, a current instruction, a current feedback, and a current deviation may be added. Here, the position instruction corresponds to the output of the numerical control information processing unit 102, and the position feedback corresponds to the output of the integrator 1007. The speed instruction corresponds to the output of the adder 1003, the speed feedback (speed detection value) corresponds to the output of the rotary encoder associated with the motor, and the speed deviation corresponds to the output of the subtractor 1004. The current instruction corresponds to the output of the adder 1006, the current feedback corresponds to the detection value of the current flowing in the motor, and the current deviation corresponds to the output of the subtractor 1010.

[0150] In addition, the adjustment of the feedforward is not limited to the adjustment (learning) in the speed feedforward calculation unit 1009. For example, in addition to the speed feedforward calculation unit 1009, the coefficients (control parameters) of the transfer functions of the position feedforward calculation unit 1008 and / or the current feedforward calculation unit 1013 may be adjusted (learned), or instead of the speed feedforward calculation unit 1009, the coefficients (control parameters) of the transfer functions of the position feedforward calculation unit 1008 and / or the current feedforward calculation unit 1013 may be adjusted (learned).

[0151] For example, Fig.13 It is a block diagram showing a part of the configuration of another configuration example of the CNC device 100 . Fig.13 Shown in Fig. 9 The CNC device shown is a CNC device in which a subtractor 1010 , a current control unit 1011 , an adder 1012 , and a current feedforward calculation unit 1013 are further added.

[0152] Alternatively, instead of machine learning, an operator may appropriately input the control parameter a using, for example, a control display panel. i , b j , and run the learning program to observe the action characteristics, thereby adjusting the control parameter a i , b j .

[0153] In addition, the motion characteristics of the machine tool when it is operated by the learning program can be observed to replace the control parameter a related to the feedforward. i , b j Adjustment (learning)) is performed to adjust (learn) the control parameters related to backlash correction and / or backlash acceleration correction. For example, Figure 7 The first structural example shown and Figure 8 The operation characteristics of the reverse rotation at point A in the second configuration example shown, and Fig.11 The observation of the action characteristics during reversal at point I in the fifth structural example shown is particularly effective.

[0154] Alternatively, the control parameter a related to the feedforward may be replaced by observing the motion characteristics of the machine tool when the machine tool is operated based on the learning program. i , b j Adjustment (learning)) is performed to adjust (learn) the control parameters related to static friction correction. For example, Figure 5 The operating characteristics at point C of the second configuration example shown are particularly effective.

[0155] <Supplementary Notes>

[0156] Finally, as a supplementary explanation, the machine learning (reinforcement learning) performed by the machine learning device 300 is described in more detail.

[0157] Fig.14 It is a block diagram showing the machine learning device 300 according to the first embodiment of the present invention.

[0158] In order to carry out the above reinforcement learning, Fig.14 As shown, the machine learning device 300 includes a state information acquisition unit 301, a learning unit 302, an action information output unit 303, a value function storage unit 304, and an optimization action information output unit 305. The learning unit 302 includes a reward output unit 3021, a value function update unit 3022, and an action information generation unit 3023.

[0159] <Agent>

[0160] The intelligent agent (equivalent to the machine learning device 300 in this embodiment) observes the state of the environment, selects an action, and changes the environment based on the action. A certain reward is provided along with the change in the environment, so that the intelligent agent learns better action choices (decision making).

[0161] <Status S>

[0162] The coefficients a of the transfer function G(s) of the velocity feedforward calculation unit 1009 in the CNC device 100 are i , b j The value of (i, j≥0) and the servo state including the command and feedback including the position deviation information of the CNC device 100 acquired by executing the learning program during learning are referred to as the state S. The state information acquisition unit 301 acquires the state S.

[0163] <Action A>

[0164] The control parameter a of the speed feedforward calculation unit 1009 related to the state S is i , b j Adjustment of the control parameter a i , b j Randomly make a micro correction) as action A. In addition, regarding the control parameter a i , b j For example, as the initial setting value, a0=1, a1=0, b0=0, b1=the inertia value of the controlled object in [Equation 1]. In addition, the coefficient a is set in advance. i , b j The action information generating unit 3023 in the learning unit 302 generates action A, and the action information output unit 303 outputs action A in the state S to the CNC device 100 .

[0165] <Reward>

[0166] PD(s) represents the set of position deviations (position deviation set) as state variables in state S, and PD(s′) represents the coefficients a of the velocity feedforward calculation unit as the state variables in state S. i , b j The position deviation set of state variables related to the state information S′ obtained by changing the state S is obtained by modifying the state information S′ obtained by modifying the state S (i, j represent 0 and positive integers). In addition, the value of the position deviation in the state S is set to a value calculated based on a preset evaluation function f(PD(s)).

[0167] As the evaluation function f, for example, the following function can be applied:

[0168] Function to calculate the cumulative value of the absolute value of the position deviation

[0169] ∫|e|dt

[0170] Function that calculates the cumulative value by applying time weight to the absolute value of the position deviation

[0171] ∫t|e|dt

[0172] Function to calculate the cumulative value of the absolute value of the position deviation raised to the power of 2n (n is a natural number)

[0173] ∫e 2n dt (n is a natural number)

[0174] Function to calculate the maximum absolute value of position deviation

[0175] Max{|e|}.

[0176] At this time, when the position deviation value f(PD(s′)) of the CNC device 100 that performs the action based on the corrected speed feedforward calculation unit 1009 related to the state information S′ corrected by the action information A is greater than the position deviation value f(PD(s)) of the CNC device 100 that performs the action based on the speed feedforward calculation unit 1009 before correction related to the state information S before correction by the action information A, the value of the reward r is set to a negative value.

[0177] On the other hand, when the position deviation value f(PD(s′)) of the CNC device 100 that operates based on the corrected speed feedforward calculation unit 1009 with respect to the state information S′ corrected by the action information A is smaller than the position deviation value f(PD(s)) of the CNC device 100 that operates based on the speed feedforward calculation unit 1009 before correction with respect to the state information S before correction by the action information A, the value of the reward r is set to a positive value.

[0178] In addition, when the value f(PD(s′)) of the position deviation of the CNC device 100 that performs the action based on the corrected speed feedforward calculation unit 1009 with respect to the state information S′ corrected by the action information A is equal to the value f(PD(s)) of the position deviation of the CNC device 100 that performs the action based on the speed feedforward calculation unit 1009 before correction with respect to the state information S before correction by the action information A, the value of the reward r is set to zero.

[0179] The reward output unit 3021 in the learning unit 302 calculates a reward r when the action A is applied to the state S.

[0180] <Value Function Q>

[0181] Let Q(S,A) be the value of selecting action A in a certain environment state S. Thus, Q(S,A) is a function that takes state S and action A as input.

[0182] At the beginning of learning, the exact value of the value Q(S,A) for the combination of state S and action A is completely unknown. Therefore, the agent selects various actions A in a certain state S, and makes better action choices based on the rewards provided for action A at that time, thereby gradually learning the exact value Q(S,A).

[0183] For example, the update formula of the value Q(S, A) can be expressed by the following formula 2 (hereinafter referred to as formula 2).

[0184]

Number 2

[0185]

[0186] In the above formula 2, S t represents the state of the environment at time t, A t represents the action at time t. By action A t , the state changes to S t+1 . r t+1 represents the reward obtained by changing the state. In addition, the item with max is the reward for the state S. t+1 The value obtained by multiplying the Q value of action A, which has the highest known Q value at that time, by γ. Here, γ is a parameter of 0<γ≤1, which is called a discount rate. In addition, α is a learning coefficient, which is set in the range of 0<α≤1.

[0187] In learning, there is a method of making a table of Q(S, A) for all state-action pairs (S, A) for learning. The value function updating unit 3022 in the learning unit 302 learns based on the state S, the action A, the state S′ when the action A is applied to the state S, and the value of the reward r calculated by the reward output unit 3021, thereby updating the value function Q stored in the value function storage unit 304.

[0188] As described above, based on the value function Q learned by the machine learning device 300, each coefficient a of the transfer function to the speed feedforward calculation unit 1009 related to a certain state S is selected. i , b j By selecting the action A that maximizes the value of Q(S, A) among the applied actions A, the action A that minimizes the position deviation obtained by executing the learning program (i.e., the coefficient a of the velocity feedforward calculation unit 1009) can be selected. i , b j ).

[0189] The optimization action information output unit 305 generates optimization action information based on the value function Q updated by learning by the value function update unit 3022. The optimization action information is action information A used to cause the speed feedforward calculation unit 1009 to perform an action that maximizes the value Q(S, A).

[0190] The above supplementary explanation of machine learning in this embodiment is an example of using machine learning to observe and adjust (learn) the evaluation performed by the learning program, and the adjustment (learning) is not limited to this. For example, the operator may also use the learning program to adjust the control parameters.

[0191] In the description of the above-mentioned embodiment, the functions of the CNC device 100, the machine learning device 300, and the program generation device 400 are described. In order to realize these functions, the CNC device 100, the machine learning device 300, and the program generation device 400 are respectively provided with a CPU (Central Processing Unit) or other computing processing device. In addition, the CNC device 100, the machine learning device 300, and the program generation device 400 are respectively provided with an auxiliary storage device such as a HDD (Hard Disk Drive) storing various control programs such as application software and OS (Operating System), and a main storage device such as a RAM (Random Access Memory) for temporarily storing required data when the computing processing device executes the program.

[0192] Moreover, the CNC device 100, the machine learning device 300, and the program generation device 400 respectively read the application software and OS from the auxiliary storage device by the operation processing device and expand the read application software and OS in the main storage device, while performing operation processing based on these application software and OS. In addition, various hardware possessed by each device is controlled based on the operation results. In this way, the functional modules of this embodiment can be realized. In other words, this embodiment can be realized by the cooperation of hardware and software.

[0193] In addition, regarding the machine learning device 300, since the amount of computation associated with machine learning is large, it is preferable to install a GPU (Graphics Processing Units) in a personal computer and use the GPU in the computation associated with machine learning through a technology called GPGPU (General-Purpose computing on Graphics Processing Units) to perform high-speed processing. In addition, in order to perform even faster processing, a computer cluster can be constructed using multiple computers equipped with such GPUs, and multiple computers included in the computer cluster can be used for parallel processing.

[0194] In the above-described embodiment, each component included in the CNC device 100, the machine learning device 300, and the program generation device 400 can be realized by hardware, software, or a combination thereof. Here, realization by software means realization by a computer reading and executing a program.

[0195] In this case, various types of non-transitory computer readable media can be used to store the software and provide the software to the computer. Non-transitory computer readable media include recording media (tangible storage medium) having various types of entities. Examples of non-transitory computer readable media include magnetic recording media (e.g., floppy disks, magnetic tapes, hard disk drives), optical magnetic recording media (e.g., magneto-optical disks), CD-ROMs (Read Only Memory), CD-Rs, CD-R / Ws, semiconductor memories (e.g., mask ROMs, PROMs (Programmable ROMs), EPROMs (Erasable PROMs), flash ROMs, RAMs (random access memory)). In addition, the software can also be provided to the computer via various types of transitory computer readable media (transitory computer readable medium). Examples of transitory computer readable media include electrical signals, optical signals, and electromagnetic waves. Transitory computer readable media can provide the program to the computer via wired communication paths such as wires and optical fibers, or wireless communication paths.

Claims

1. An information processing device comprising: a storage unit storing a processing program for operating a motor of a machine tool, a robot, or an industrial machine; and a generating unit for generating a learning program for learning based on the operation characteristics of the motor by extracting a part of the machining program including characteristic elements from the machining program stored in the storage unit, in, The generating unit extracts a machining program including at least any one of the following machining programs as a part of the machining program including the characteristic element: A processing program for causing the motor to perform an action accompanied by a reversal of the motor speed; a processing program for causing the motor to perform an action accompanied by switching from stop to action and switching from action to stop; and a processing program for causing the motor to perform an action with acceleration change without a reversal of the motor speed.

2. The information processing device according to claim 1, characterized in that Also available: a learning unit configured to operate the motor based on the learning program generated by the generating unit to perform learning based on the operating characteristics of the motor; as well as An adjustment unit adjusts a control parameter of the electric motor based on a learning result of the learning unit.

3. The information processing device according to claim 1 or 2, characterized in that: The generating unit extracts, as a part of the machining programs including the characteristic element, a machining program that causes the motor to operate such that a position deviation exceeds a threshold value when the motor is operated.

4. The information processing device according to claim 3, characterized in that: The generating unit determines whether the position deviation exceeds a threshold value based on a result of operating the motor according to the machining program.

5. The information processing device according to claim 1 or 2, characterized in that: The generating unit extracts, as the part of the machining program including the characteristic element, a part of the machining program specified by a user who has referred to information indicating a change in position deviation along a time series or information on a movement trajectory of the motor.

6. The information processing device according to claim 1 or 2, characterized in that: A display unit is further provided, the display unit displaying the part of the machining program extracted by the generating unit.

7. An information processing method, performed by a computer, wherein the computer performs the following steps: a storing step of storing a processing program for operating a motor of a machine tool, a robot, or an industrial machine; and a generating step of generating a learning program for learning based on the operation characteristics of the motor by extracting a part of the machining program including characteristic elements from the machining program stored in the storing step, in, In the generating step, a machining program including at least any one of the following machining programs is extracted as the part of the machining program including the characteristic element: A processing program for causing the motor to perform an action accompanied by a reversal of the motor speed; a processing program for causing the motor to perform an action accompanied by switching from stop to action and switching from action to stop; and a processing program for causing the motor to perform an action with acceleration change without a reversal of the motor speed.

Images