Robot control system, robot controller, and control method

Abstract

A robot control system according to the present disclosure comprises: a robot including a work tool; and a robot controller that controls driving of the robot. The robot controller: acquires teaching information for teaching a position and a posture of the work tool of the robot; inputs the teaching information to a machine learning model in which machine learning has been performed using, as teacher data, driving information of the robot and position / posture information including the position and the posture of the work tool which has moved on the basis of the driving information, thereby acquiring control information of the robot output from the machine learning model; and drives the robot on the basis of the control information.

Description

Robot control system, robot controller, and control method

[0001] The present disclosure relates to a robot control system, a robot controller, and a control method.

[0002] For example, Patent Document 1 discloses a welding system having a robot control device capable of controlling a welding robot equipped with a torch and a welding robot control parameter creation device. In the welding system described in Patent Document 1, the welding robot control parameter creation device acquires controller position information indicating the position of a controller that can be held by an operator and controller attitude information indicating the attitude of the controller. Further, the welding robot control parameter creation device creates welding robot control parameters for causing the operation of the torch to follow the operation of the controller based on the controller position information and the controller attitude information, and transmits the parameters to the robot control device.

[0003] Japanese Patent Application Laid-Open No. 2021-194724

[0004] However, in Patent Document 1, there is room for improvement in terms of improving the position accuracy of the robot.

[0005] The present disclosure provides a robot control system, a robot controller, and a control method capable of improving the position accuracy of industrial robots in general.

[0006] The robot control system of the present disclosure includes a robot including a working tool and a plurality of joints, and a robot controller that controls the driving of the robot. The robot controller acquires teaching information that teaches the position and attitude of the working tool of the robot, and inputs the teaching information to a machine learning model in which machine learning is performed using, as teacher data, drive information of the robot and position and attitude information including the position and attitude of the working tool moved based on the drive information, and acquires control information of the robot output from the machine learning model. The robot is driven based on the control information.

[0007] The robot controller of this disclosure is a robot controller for controlling the drive of a robot including a work tool and a plurality of joints, comprising a processor and a storage unit that stores instructions executed by the processor, wherein the instructions include acquiring teaching information that teaches the position and orientation of the work tool of the robot, acquiring control information for the robot output from a machine learning model by inputting the teaching information into a machine learning model that has been machine-learned using the robot's drive information and position and orientation information including the position and orientation of the work tool moved based on the drive information as training data, and driving the robot based on the control information.

[0008] The control method of the present disclosure is a control method for controlling a robot including a work tool and a plurality of joints with a robot controller, comprising the steps of: acquiring teaching information that teaches the position and orientation of the work tool of the robot; acquiring control information for the robot output from a machine learning model by inputting the teaching information into a machine learning model that has been machine-learned using the robot's drive information and position and orientation information including the position and orientation of the work tool moved based on the drive information as training data; and driving the robot based on the control information.

[0009] According to this disclosure, it is possible to provide a robot control system, a robot controller, and a control method that can improve the positional accuracy of a robot.

[0010] A schematic block diagram illustrating the main configuration of the robot control system in Embodiment 1. A schematic perspective diagram illustrating an example of a robot. A schematic perspective diagram illustrating an example of a simulated teaching device. A schematic diagram illustrating the posture of a work tool. A schematic diagram illustrating the posture of a work tool. A schematic flowchart illustrating an example of the main operation of the robot control system. A schematic flowchart illustrating an example of the operation of the robot control system in Embodiment 1. A schematic flowchart illustrating an example of the operation of the robot control system in Embodiment 1. A schematic flowchart illustrating an example of the operation of the robot control system in Embodiment 2. A schematic flowchart illustrating an example of the operation of the robot control system in Embodiment 2. A schematic perspective diagram illustrating an example of multiple coordinate systems of the robot control system in Embodiment 2. A schematic flowchart illustrating an example of the process of acquiring posture setting values. A schematic diagram illustrating an example of the posture of an ideal simulated teaching device. A simulated teaching device when a user actually teaches. A schematic diagram illustrating an example of the positioning posture. A schematic diagram illustrating the posture of the work tool corrected by the posture setting value. A schematic block diagram illustrating the main configuration of the robot control system in Embodiment 3. A schematic flowchart illustrating an example of robot control in Embodiment 3. A schematic flowchart illustrating an example of robot control in Embodiment 4. A schematic flowchart illustrating an example of robot control in Embodiment 5. A schematic flowchart illustrating an example of robot control in Embodiment 6. A schematic block diagram illustrating the main configuration of the robot control system in Embodiment 7. A schematic flowchart illustrating an example of robot control in Embodiment 7. A schematic diagram illustrating the teaching of the robot control system in Modification 1. A schematic diagram illustrating the teaching of the robot control system in Modification 2. A schematic diagram illustrating the teaching of the robot control system in Modification 3. A schematic diagram illustrating the teaching of the robot control system in Modification 4. A schematic diagram illustrating the simulated teaching device for the robot control system in Modification 5.

[0011] (Background to this disclosure) For example, a system is known that controls the movement of an articulated robot based on position and orientation information taught by a simulated torch such as a controller, as described in Patent Document 1. Specifically, the articulated robot moves in accordance with the position and orientation taught by the simulated torch.

[0012] However, in such systems, the actual position and orientation of the articulated robot may deviate from the position and orientation taught by the simulated torch. This can be caused, for example, by the deflection of the articulated robot's arm due to gravity, or by errors in components such as motors or encoders that move the robot's position and orientation. In addition, the difference between the measurement coordinate system used to measure the position of the simulated torch and the robot coordinate system used to control the robot also contributes to the error. Therefore, in systems that control an articulated robot based on position and orientation information taught by a simulated torch, improvement in the robot's positional accuracy is required.

[0013] Therefore, after diligent research, the present inventors investigated a configuration for a robot control system that can improve the positional accuracy of the robot, leading to this disclosure.

[0014] (Embodiment 1) Hereinafter, Embodiment 1 will be described with reference to the drawings. Embodiment 1 will be described using as an example an operating system in which the target equipment to be operated is operated by an operating device.

[0015] In this specification, terms such as "first," "second," etc., are used solely for illustrative purposes and should not be understood as expressing or implying relative importance or ranking of technical features. Features designated as "first" and "second" express or imply that they include one or more such features.

[0016] [1-1. Configuration of the Robot Control System] An example of the configuration of the robot control system will be described with reference to Figure 1. Figure 1 is a schematic block diagram illustrating the main configuration of the robot control system in Embodiment 1. Note that Figure 1 shows the main configuration of the robot control system 1A, and some elements have been omitted.

[0017] As shown in Figure 1, the robot control system 1A comprises a robot 10, a simulation teaching device 20, and a robot controller 40. The robot control system 1A also comprises a first measuring device 31 and a second measuring device 32.

[0018] In the robot control system 1A, the robot controller 40 generates control information for driving the robot 10 based on the teaching information taught by the simulation teaching device 20, and drives the robot 10 based on the control information.

[0019] <Robot> Robot 10 is a multi-joint robot having multiple joints. Robot 10 comprises a main body 11 and a work tool 12.

[0020] An example of the configuration of the robot 10 will be described with reference to Figure 2. Figure 2 is a schematic perspective view illustrating an example of the robot. Figure 2 shows an example in which the robot 10 welds a workpiece WK1 fixed to a fixed base 2.

[0021] As shown in Figure 2, the main body 11 is composed of multiple movable members. Each of the members has multiple drive shafts and moves by rotating around these drive shafts.

[0022] The main body 11 includes a base 13, a plurality of arms 14, and a plurality of joints 15. The plurality of arms 14 are connected via the plurality of joints 15. Each of the plurality of joints 15 has a drive shaft. The plurality of arms 14 move as the plurality of joints 15 rotate around the drive shaft. In this specification, "rotation" includes movement around the drive shaft at an angle of a predetermined angle or less, such as 360° or 720°, or infinite rotation.

[0023] For example, each of the multiple joints 15 is equipped with a motor and an encoder. The joints 15 rotate around the drive shaft due to the rotation of the motor. The encoder detects the amount of rotation of the motor, drive shaft, etc. Based on the amount of rotation of the motor, drive shaft, etc., the joint angle of the joint 15 can be calculated.

[0024] The work tool 12 is attached to the main body 11. The position and orientation of the work tool 12 change as the main body 11 moves. The work tool 12 is a tool for the robot 10 to perform tasks, and a tool suitable for the purpose of the task can be used. In this embodiment, the work tool 12 is a tool for welding.

[0025] The work tool 12 is attached to the arm 14 located at the very end of the multiple arms 14. The work tool 12 has a tool center point TCP located at its tip, which has information on at least one of its position and orientation. The position and orientation of the work tool 12 can be changed by moving the multiple arms 14 by rotating the multiple joints 15. The position of the work tool 12 refers to the position of the tool center point TCP, and is the coordinate position of the tool center point TCP. For example, the position of the tool center point TCP of the work tool 12 is the position in a reference coordinate system with the base 13 of the main body 11 as the origin. The reference coordinate system has mutually orthogonal X, Y, and Z axes. The orientation of the work tool 12 refers to the inclination angle of the work tool 12, and is the angle of inclination with respect to the X, Y, and Z axes.

[0026] The robot 10 may also include a mechanism for transmitting the rotational force of the motor, a control circuit for controlling the motor and encoder, and the like.

[0027] The operation of the robot 10 is controlled by the robot controller 40. In this embodiment, the robot 10 moves the tool center point TCP of the work tool 12 along the welding portion WP1 of the workpiece WK1 by rotating a plurality of joints 15 of the main body 11 and moving a plurality of arms 14. The operation of the robot 10 is controlled based on information taught by the simulation teaching device 20.

[0028] <Teaching Device> The simulated teaching device 20 is a device that teaches the position and orientation of the work tool 12 of the robot 10. The user teaches the position and orientation of the work tool 12 by operating the simulated teaching device 20.

[0029] An example of the configuration of the simulated teaching device 20 will be described with reference to Figure 3. Figure 3 is a schematic perspective view illustrating an example of the teaching device.

[0030] As shown in Figure 3, the simulated teaching device 20 includes a gripping unit 21, a teaching unit 22, a trigger 23, and a button 24.

[0031] The gripping section 21 is the part that the user grips with their hand. The teaching section 22 is connected to the gripping section 21 and is a part that mimics the shape of the work tool 12. The teaching section 22 has a virtual tool center point (V-TCP) VCP located at the tip of the teaching section 22. The virtual tool center point VCP corresponds to the tool center point TCP of the work tool 12. The trigger 23 is provided on the gripping section 21. The trigger 23 is a lever that controls the start and end of teaching. The button 24 is provided on the gripping section 21. The button 24 is a button for turning the power of the simulated teaching device 20 ON / OFF or switching modes.

[0032] The user teaches the position and orientation of the work tool 12 by grasping and moving the simulation teaching device 20 with their hand. Specifically, the user teaches the tool center point TCP of the work tool 12 by moving the position of the virtual tool center point VCP of the simulation teaching device 20, and teaches the orientation of the work tool 12 by moving the orientation of the teaching unit 22 of the simulation teaching device 20.

[0033] <First measuring device> The first measuring device 31 measures the position and orientation of the robot 10's work tool 12. Specifically, the first measuring device 31 measures the position of the tool center point TCP of the work tool 12, as well as the orientation of the work tool 12.

[0034] For example, the first measuring device 31 includes multiple cameras and multiple reflective markers. The multiple cameras can be, for example, stereo cameras, depth cameras, or RGB-D cameras that combine depth cameras and RGB cameras. The multiple reflective markers are attached to the robot 10. The multiple reflective markers are made of retroreflective material. The first measuring device 31 measures the multiple reflective markers using the multiple cameras and acquires the position of the tool center point TCP of the robot's work tool 12 and the orientation of the work tool 12 in three dimensions.

[0035] <Second Measurement Device> The second measurement device 32 measures the position and orientation of the simulated teaching device 20. Specifically, the second measurement device 32 measures the position of the virtual tool center point VCP of the simulated teaching device 20 and also measures the orientation of the simulated teaching device 20. The second measurement device 32 is the same as the first measurement device 31. In this embodiment, the same measurement device is used for both the first measurement device 31 and the second measurement device 32 because it simplifies the system configuration. However, different measurement devices may be used for the first measurement device 31 and the second measurement device 32 depending on the workpiece WK1.

[0036] <Robot Controller> The robot controller 40 comprehensively controls the components of the robot control system 1.

[0037] Returning to Figure 1, the robot controller 40 includes a processor 41, a storage unit 42, an input interface 43, and an output interface 44.

[0038] The processor 41 can be composed of, for example, a microcontroller, CPU, MPU, GPU, DSU, FPGA, ASIC, etc. The processor 41 may also be composed of dedicated electronic circuits designed to realize predetermined functions.

[0039] The memory unit 42 is a storage medium that stores programs, instructions, and / or data for realizing the functions of the robot controller 40. The memory unit 42 can be implemented, for example, by a hard disk (HDD), SSD, RAM, DRAM, ferroelectric memory, flash memory, magnetic disk, or a combination thereof.

[0040] The storage unit 42 includes a database 51, a machine learning model 52, and an execution program 53.

[0041] The functions required for the robot controller 40 have been described, but other functions may be added. Furthermore, the functions of the robot controller 40 may be divided among multiple computers. For example, a configuration may be provided that includes a first robot controller directly connected to the robot 10 and a second robot controller connected to the first controller. In this case, the second robot controller may be a computer equipped with a GPU.

[0042] The database 51 stores drive information for the robot 10 and position and orientation information, including the position and orientation of the work tool 12 of the robot 10 as it moved based on the drive information. In this embodiment, the drive information for the robot 10 is a command value for driving the robot 10, and includes, for example, the joint angles of a plurality of joints 15, the voltage or current value of the motor, or the value of the encoder. The position and orientation information includes the position and orientation of the work tool 12 measured by the first measuring device 31. Specifically, the position and orientation information includes the position and orientation of the tool center point TCP in an arbitrary coordinate system measured by the first measuring device 31. Note that the position and orientation information may also be the position and orientation of any part of the work tool 12. For example, the position and orientation information may be the position and orientation of any center position (center of gravity) of the work tool 12. Alternatively, the position and orientation information may be the position and orientation of a part whose positional relationship with the tool center point TCP is specified.

[0043] The machine learning model 52 is a model in which machine learning is performed using, as teacher data, the drive information of the robot 10 and the position and orientation information including the position and orientation of the work tool 12 that has moved based on the drive information. The machine learning model 52 outputs control information for the robot 10 by inputting teaching information for teaching the position and orientation of the work tool 12. The teaching information is command information for driving the robot 10 and includes the position and orientation of the simulation teaching device 20 measured by the second measuring device 32. Specifically, the teaching information includes the coordinate position of the virtual tool center point VCP of the simulation teaching device 20 and the tilt angle of the simulation teaching device 20 measured by the second measuring device 32. The control information is information for controlling the drive of the robot 10 so that the deviation between the position and orientation of the work tool 12 with respect to the taught position and orientation is reduced when the robot 10 is moved based on the teaching information. The control information includes, for example, the joint angles of the plurality of joints 15, the voltage value or current value of the motor, or the value of the encoder, etc.

[0044] The machine learning model 52 may include, for example, a neural network, a simple regression model, a logistic regression model, a support vector machine (SVM), a Bayesian model, etc.

[0045] In the present embodiment, the machine learning model 52 is a model that directly outputs control information for the robot 10 when a command for driving the robot 10, which is teaching information, is input.

[0046] The execution program 53 is a program for driving the robot 10.

[0047] The input interface 43 is a device for receiving input from the user. For example, the input interface 43 includes buttons, operation keys, levers, switches, keyboards, microphones, or touch panels that can be operated by the user.

[0048] The output interface 44 is a device for outputting information. For example, the output interface 44 includes a display, a speaker, or a lamp for outputting information.

[0049] The input interface 43 and the output interface 44 may be an integrated input / output interface.

[0050] The robot 10, the simulation teaching device 20, the first measuring device 31, the second measuring device 32, and / or the robot controller 40 may be equipped with a communication device that transmits information via wired or wireless means. The communication device includes a circuit that performs communication in accordance with a predetermined communication standard. The predetermined communication standard includes, for example, LAN, Wi-Fi®, Bluetooth®, USB, HDMI®, CAN (controller area network), and SPI (Serial Peripheral Interface).

[0051] [1-2. Regarding the posture of the work tool] The posture of the work tool 12 will be explained with reference to Figures 4A to 4C. Figures 4A to 4C are schematic diagrams illustrating the posture of the work tool. Figures 4A to 4C show an example of the operation when the work tool 12 is a welding torch and the welding torch moves in the X-axis direction while welding the workpiece WK1.

[0052] As shown in Figures 4A to 4C, the posture of the work tool 12 is a three-dimensional angle, including the tilt angle θt, the forward / backward movement angle θa, and the twist angle θw.

[0053] When the welding direction is set to the positive X-axis side, the tilt angle θt is the angle of the working tool 12 tilted with respect to the Z-axis direction when viewed from the YZ plane. The forward / backward advance angle θa is the angle of the working tool 12 tilted with respect to the Y-axis direction when viewed from the XY plane. The twist angle θw is the angle of the working tool 12 tilted with respect to the X-axis direction when viewed from the XZ plane.

[0054] [1-3. Operation of the Robot Control System] The main operations of the robot control system 1A will be described with reference to Figure 5. Figure 5 is a schematic flowchart illustrating an example of the main operations of the robot control system.

[0055] As shown in Figure 5, the robot control system 1A includes the first phase ST10 to the fifth phase ST50. The first phase ST10 and the second phase ST20 are included in the calibration process ST1. The third phase ST30 to the fifth phase ST50 are included in the task process ST2.

[0056] Calibration process ST1 is a process for performing calibration of the robot 10. Calibration means correcting the deviation of the actual position and orientation of the robot 10 when it is moved relative to the target position and orientation of the robot 10. Specifically, calibration corrects the deviation of the actual position and orientation of the work tool 12 relative to the target position and orientation of the work tool 12.

[0057] The robot 10, which has multiple joints 15, is subject to factors such as deflection due to gravity, encoder origin deviation, and variations in parts or assemblies. Therefore, if calibration is not performed, the actual position and orientation of the work tool 12 moved by the robot 10 will deviate from the theoretical target position and orientation of the work tool 12 calculated by robot forward kinematics from the encoder information.

[0058] By performing the calibration process ST1, the deviation of the actual position and orientation of the work tool 12 from the target position and orientation of the work tool 12 can be reduced.

[0059] Calibration in this disclosure refers to the control of the tool center point TCP measured by the first measuring device 31 and the virtual tool center point VCP of the simulated teaching device 20 measured by the second measuring device 32 to substantially the same position and orientation. This makes it possible to easily perform teaching using the simulated teaching device 20.

[0060] Robots with multiple joints experience deflection due to gravity, encoder origin shifts, and variations in parts or assemblies, meaning the actual robot's tool center point does not move to the theoretical robot coordinate system's position and orientation. This differs from conventional implementations where a measuring device measures the virtual tool center point VCP of a simulated teaching device and controls the robot in a theoretical robot coordinate system.

[0061] The calibration method may involve using a machine learning model to determine various parameters such as deflection due to gravity, encoder origin deviation, and variations in parts or assemblies, so that the actual tool center point TCP of the robot 10 moves to the position and orientation in the theoretical robot coordinate system. Alternatively, the calibration method may involve using a machine learning model to obtain control information that is approximately the same position and orientation as the virtual tool center point VCP of the simulated teaching device 20, by referring to the measurement coordinate system or the fixed base coordinate system without considering the robot coordinate system.

[0062] Here, "approximately identical position and orientation" means a range of positions and orientations that ensure equivalent quality depending on the task being performed, and may include errors. If the task is precision positioning or precision insertion, approximately identical position and orientation may include errors such as a position error of 0.1 mm or less or an angular error of 0.5 degrees or less. If it is a welding task, approximately identical position and orientation may include errors such as a position error of 0.5 mm or less or an angular error of 2 degrees or less. If it is a painting task, approximately identical position and orientation may include errors such as a position error of 5 mm or less or an angular error of 5 degrees or less. If it is a rough pick-and-place task, approximately identical position and orientation may include errors such as a position error of 10 mm or less or an angular error of 15 degrees or less. Note that the above values ​​are examples, and the above errors may be changed depending on the required quality.

[0063] Calibration process ST1 is performed when the robot 10 is introduced or when a work tool 12 is attached to the robot 10, etc. Calibration process ST1 does not need to be performed every time the robot control system 1A is in operation.

[0064] Task process ST2 is a process for generating control information for driving the robot 10. Task process ST2 acquires teaching information for teaching the position and orientation of the work tool 12 and generates control information for the robot 10 based on the teaching information. Task process ST2 is executed when working on a new workpiece WK1 or when driving the robot 10 with a new motion. The control information is information for driving the robot 10, and may include, for example, the joint angles of multiple joints 15, the voltage or current values ​​of the motors, or the values ​​of the encoders.

[0065] Phase 1 ST10 is a data acquisition phase. In Phase 1 ST10, the robot controller 40 collects drive information of the robot 10 and position and orientation information, including the position and orientation of the work tool 12 of the robot 10 as it moved based on the drive information. The drive information of the robot 10 is information of command values ​​for controlling the position and orientation of the work tool 12 of the robot 10. For example, the drive information may include the joint angles of multiple joints 15, the voltage or current values ​​applied to the motors, or the values ​​of the encoders. The position and orientation information is the actual position and orientation of the work tool 12 as it moved based on the drive information.

[0066] Phase 2, ST20, is the phase in which machine learning is performed on a machine learning model using the information acquired in Phase 1, ST10.

[0067] The third phase, ST30, is a phase in which teaching information is acquired to teach the position and orientation of the work tool 12 of the robot 10.

[0068] The fourth phase, ST40, is a phase in which control information for the robot 10 is generated based on the teaching information acquired in the third phase, ST30.

[0069] The fifth phase, ST50, is a phase in which the robot 10 is driven based on the control information generated in the fourth phase, ST40.

[0070] Next, the detailed operation of the robot control system 1A will be described with reference to Figures 6 and 7. Figures 6 and 7 are schematic flowcharts illustrating an example of the operation of the robot control system in Embodiment 1. Figure 6 shows the processing of the first phase ST10 and the second phase ST20 of the calibration process ST1, and Figure 7 shows the processing of the third phase ST30 to the fifth phase ST50 of the task process ST2.

[0071] As shown in Figure 6, the first phase ST10 includes steps ST11 to ST14.

[0072] In step ST11, the robot controller 40 drives the robot 10 based on drive information that drives a predetermined operation. The drive information is information that drives the robot 10 to move the work tool 12 to multiple postures and multiple positions. Multiple postures include, for example, multiple tilt angles θt, multiple forward and backward angles θa, and multiple twist angles θw. Multiple positions include, for example, multiple coordinate positions that move in the X-axis direction, multiple coordinate positions that move in the Y-axis direction, and multiple coordinate positions that move in the Z-axis direction.

[0073] In step ST12, the robot controller 40 acquires position and orientation information, including the position and orientation of the work tool 12 that has moved based on the drive information. Specifically, the first measuring device 31 measures the position and orientation of the tool center point TCP of the work tool 12 that has moved based on the drive information, and transmits the position and orientation information to the robot controller 40. As a result, the robot controller 40 acquires the position and orientation information of the work tool 12.

[0074] In step ST13, the robot controller 40 stores the drive information of the robot 10 and the position and orientation information of the work tool 12 in the storage unit 42 and creates a database 51 that associates the drive information and the position and orientation information. In the database 51, command values ​​for the robot 10 (e.g., joint angles, etc.) are associated with the position and orientation of the work tool 12 that moved according to those command values. The position and orientation of the work tool 12 also includes the position and orientation of the tool center point TCP.

[0075] In step S14, the robot controller 40 determines whether the robot 10 has completed a predetermined operation. If the predetermined operation has been completed, the flow proceeds to the second phase ST20. If the predetermined operation has not been completed, the flow returns to step ST11.

[0076] The second phase ST20 includes steps ST21 and ST22.

[0077] In step ST21, the robot controller 40 reads drive information and position / orientation information from the database 51 of the storage unit 42.

[0078] In step ST22, the robot controller 40 uses the drive information and position / orientation information read from the database 51 as training data to train the machine learning model 52. As a result, when the robot controller 40 receives the teaching information from the simulated teaching device 20, it creates a machine learning model 52 that outputs control information to control the drive of the robot 10 based on the teaching information.

[0079] Next, as shown in Figure 7, in the third phase ST30, the robot controller 40 acquires teaching information for teaching the robot 10 its movements. The teaching information is information for teaching the robot 10 the position and orientation of the work tool 12. For example, the user teaches the robot 10 its movements by grasping the simulated teaching device 20 in their hand and moving it. The second measuring device 32 measures the position and orientation of the simulated teaching device 20 being moved by the user and transmits the teaching information, including the measured position and orientation, to the robot controller 40.

[0080] In the fourth phase ST40, the robot controller 40 acquires control information for the robot 10 by inputting teaching information into the machine learning model 52. The robot controller 40 inputs the position and orientation of the simulated teaching device 20 into the machine learning model 52 and acquires control information for the robot 10 output from the machine learning model 52. The robot controller 40 generates an execution program 53 based on the control information.

[0081] In the fifth phase ST50, the robot controller 40 drives the robot based on the control information. The robot controller 40 executes the execution program 53 that was generated based on the control information.

[0082] [1-4. Machine Learning Models] Next, we will describe an example of machine learning model 52.

[0083] When the machine learning model 52 receives drive information for driving the robot 10, it outputs information for driving the robot 10 as control information for the robot 10.

[0084] When the machine learning model 52 receives measured values ​​of the position and orientation of the work tool 12 stored in the database 51, it outputs estimated drive information for the robot 10 to move the work tool 12 to the input position and orientation. The position and orientation of the work tool 12 also include the position and orientation of the tool's center point TCP. The estimated drive information is, for example, an estimated value of the joint angles of multiple joints 15 of the robot 10, which can be calculated using [Equation 1].

[0085]

[0086] Here, the definition of variables in [Mathematics 1] is shown below.

[0087] Next, the machine learning model 52 calculates a loss function based on the measured values ​​of the robot 10's drive information stored in the database 51 and the estimated drive information calculated using [Equation 1], and performs machine learning to minimize the loss function. The measured values ​​of the drive information are, for example, measured values ​​of joint angles. The loss function can be calculated using [Equation 2].

[0088]

[0089] Here, the definition of variables in [Mathematics 2] is shown below.

[0090] When the target position and target orientation of the work tool 12 are input to the machine learning model 52 described above, the model outputs information to directly drive the robot 10 as control information for moving the work tool 12 to the target position and target orientation. The target position and target orientation of the work tool 12 are the position and orientation taught by the simulation teaching device 20. The control information is, for example, the joint angles of the multiple joints 15 of the robot 10, which can be calculated using [Equation 3].

[0091]

[0092] Here, the definition of a variable in [Mathematics III] is shown below.

[0093] Next, we will explain another example of machine learning model 52.

[0094] In another example, the machine learning model 52 corrects the measured values ​​using robot inverse kinematics.

[0095] When the machine learning model 52 receives measured values ​​of the position and orientation of the work tool 12 stored in the database 51, it outputs estimated correction drive information for the robot 10 to move the work tool 12 to the input position and orientation. The estimated correction drive information is, for example, an estimated correction amount for the joint angles of multiple joints 15 of the robot 10, and can be calculated using [Equation 4].

[0096]

[0097] Here, the definition of variables in [Mathematics 4] is shown below.

[0098] The machine learning model 52 calculates theoretical drive information for the robot 10, which is theoretically calculated from measured values ​​of the position and orientation of the work tool 12 stored in the database 51 using inverse kinematics. The theoretical drive information is, for example, the theoretical values ​​of the joint angles of multiple joints 15 of the robot 10, and can be calculated using [Equation 5].

[0099]

[0100] Here, the definition of variables in [Mathematics 5] is shown below.

[0101] The machine learning model 52 is trained to minimize the loss function, which is based on the sum of the measured values ​​of the robot 10's drive information stored in the database 51, the estimated corrected drive information calculated using [Equation 4], and the theoretical drive information calculated using [Equation 5]. The drive information is, for example, the measured values ​​of the joint angles. The loss function can be calculated using [Equation 6].

[0102]

[0103] Here, the definition of variables in [Mathematics 6] is shown below.

[0104] The machine learning model 52, trained by the machine learning described above, outputs control information for the robot 10 to move the work tool 12 to the target position and target orientation when the target position and target orientation of the work tool 12 are input. The control information is, for example, the joint angles of multiple joints 15 of the robot 10, which can be calculated using [Equation 7].

[0105]

[0106] Here, the definition of the variables in [Equation 7] is shown below.

[0107] Next, we will describe another example of machine learning model 52.

[0108] In yet another example, the machine learning model 52 corrects the measured values ​​using robot forward kinematics and robot inverse kinematics.

[0109] The machine learning model 52 calculates theoretical position and orientation information of the work tool 12 using robot forward kinematics from measured values ​​of the robot 10's drive information stored in the database 51. The measured values ​​of the drive information are, for example, measured values ​​of the joint angles of multiple joints 15 of the robot 10. The theoretical drive information is, for example, theoretical values ​​of the position and orientation of the work tool 12 calculated from the joint angles using inverse kinematics, and can be calculated using [Equation 8].

[0110]

[0111] Here, the definition of the variables in [Equation 8] is shown below.

[0112] The machine learning model 52 calculates error information that shows the difference between the measured position and orientation of the work tool 12 stored in the database 51 and the theoretical position and orientation information calculated by [Equation 8]. The error information is, for example, the amount of error in the joint angles of multiple joints 15 of the robot 10, and can be calculated by [Equation 9].

[0113]

[0114] Here, the definition of the variables in [Mathematics 9] is shown below.

[0115] When the machine learning model 52 receives measured values ​​of the position and orientation of the work tool 12 stored in the database 51, it outputs estimated error information for the position and orientation of the work tool 12. The estimated error information is, for example, the amount of estimated error in joint angles. The estimated error information can be calculated using [Equation 10].

[0116]

[0117] Here, the definition of the variables in [Equation 10] is shown below.

[0118] The machine learning model 52 calculates a loss function based on the error information calculated in [Equation 9] and the estimated error information calculated in [Equation 10]. The loss function can be calculated using [Equation 11].

[0119]

[0120] Here, the definition of the variables in [Equation 11] is shown below.

[0121] When the target position and target orientation of the work tool 12 are input to the machine learning model 52, control information for the robot 10 to move the work tool 12 to the target position and target orientation is output. Specifically, the machine learning model 52 adds the target position and target orientation of the work tool 12 to the amount of error generated in the machine learning model 52 and calculates the control information from inverse kinematics. The control information is, for example, the joint angles of multiple joints 15 of the robot 10, and can be calculated using [Equation 12].

[0122]

[0123] Here, the definition of the variables in [Equation 12] is shown below.

[0124] The machine learning model 52 described above is merely an example and is not limited to it. Furthermore, while the machine learning model 52 described above uses joint angle information, it is not limited to this. For example, the machine learning model 52 may use motor voltage or current values, or encoder values, instead of joint angles.

[0125] [2. Effects, etc.] As described above, the robot control system 1A in this embodiment comprises a robot 10 including a work tool 12, and a robot controller 40 that controls the driving of the robot 10. The robot controller 40 acquires teaching information that teaches the position and orientation of the work tool 12 of the robot 10. The robot controller 40 acquires control information for the robot 10 output from the machine learning model 52 by inputting the teaching information into the machine learning model 52, which has been machine-trained using the driving information of the robot 10 and the position and orientation information including the position and orientation of the work tool 12 moved based on the driving information as training data. The robot controller 40 drives the robot 10 based on the control information.

[0126] This configuration improves the positional accuracy of the robot 10. Specifically, it reduces the discrepancy between the position and orientation of the robot 10's work tool 12, which moves based on the teaching information, and the taught target position and target orientation.

[0127] The robot control system 1A includes a simulation teaching device 20 that teaches the robot 10 the position and orientation of the work tool 12.

[0128] This configuration allows the user to easily teach the robot 10 how to operate.

[0129] In this embodiment, the work tool 12 was described as a tool for welding, but it is not limited to this. For example, the work tool 12 may be a tool for tasks such as screw fastening, picking up, or pressing.

[0130] In this embodiment, an example has been described in which the simulated teaching device 20 includes a gripping unit 21, a teaching unit 22, a trigger 23, and a button 24, but it is not limited to this. For example, the simulated teaching device 20 may be a mockup that mimics the shape of the work tool 12. Alternatively, the simulated teaching device 20 may be a teaching controller equipped with operation keys that can be operated by the user's fingers.

[0131] In this embodiment, an example has been described in which the first measuring device 31 measures the position and orientation of the work tool 12 using a plurality of reflective markers and a plurality of cameras, but it is not limited to this. For example, the first measuring device 31 may measure the position and orientation of the work tool 12 by attaching visible or invisible markers to the work tool 12 and measuring with a camera.

[0132] The first measuring device 31 may have a receiver that receives signals such as infrared rays or sound waves. The receiver may be attached to the work tool 12. The first measuring device 31 may measure the position and orientation of the work tool 12 by receiving signals such as infrared rays or sound waves transmitted from an external device with the receiver.

[0133] The first measuring device 31 may include a light-receiving unit that receives specific light and a tracker that captures the light-receiving unit. The light-receiving unit may be attached to the work tool 12. The first measuring device 31 may measure the position and orientation of the work tool 12 by using specific light and capturing the light-receiving unit with the tracker.

[0134] The first measuring device 31 may measure the position and orientation of the work tool 12 by identifying its shape using pattern recognition or object recognition with one or more cameras.

[0135] The first measuring device 31 may have a configuration that allows the work tool 12 to estimate its own position. For example, a camera or IMU may be attached to the work tool 12, and multiple markers may be attached to the external environment where the robot 10 is placed (e.g., a workpiece fixing table, a factory, etc.). The first measuring device 31 may measure the multiple markers using the camera or IMU attached to the work tool 12 to measure the position and orientation of the work tool 12.

[0136] The first measuring device 31 may directly measure the tool center point TCP of the work tool 12, or it may measure a measurement point located at a predetermined position relative to the center point TCP. If the positional relationship between the tool center point TCP and the measurement point is predetermined, the position and orientation of the tool center point TCP may be calculated from the position and orientation of the measurement point.

[0137] In this embodiment, an example has been described in which the second measuring device 32 has the same configuration as the first measuring device 31, but the embodiment is not limited thereto. The second measuring device 32 can be any device capable of measuring the position and orientation of the simulated teaching device 20.

[0138] In this embodiment, an example has been described in which the robot control system 1A includes a first measuring device 31 and a second measuring device 32, but it is not limited to this. For example, the first measuring device 31 may be used to measure the position and orientation of the robot 10's work tool 12 and the position and orientation of the simulation teaching device 20. In this case, the robot control system 1A does not need to include the second measuring device 32. Alternatively, the second measuring device 32 may be used to measure the position and orientation of the robot 10's work tool 12 and the position and orientation of the simulation teaching device 20. In this case, the robot control system 1A does not need to include the first measuring device 31.

[0139] In this embodiment, an example was described in which the data acquired in the first phase ST10 is used as training data for the machine learning model 52. However, the instruction information acquired in the third phase ST30 may also be used as training data for the machine learning model 52.

[0140] In this embodiment, an example was described in which the base 13 of the robot 10 is used as the origin of the reference coordinate system, but the embodiment is not limited to this. For example, the mounting base for fixing the workpiece WK1 may be used as the origin of the reference coordinate system.

[0141] In this embodiment, the tilt angle θt, forward / backward movement angle θa, and twist angle θw were described as the posture of the work tool 12, but the embodiment is not limited to these. Also, the tilt angle θt, forward / backward movement angle θa, and twist angle θw were defined with the welding direction being the positive X-axis side, but the embodiment is not limited to this. For example, the posture of the work tool 12 can be any posture with one or more axes. Furthermore, the posture of the work tool 12 may be defined in the robot's coordinate system.

[0142] (Embodiment 2) The robot control system in Embodiment 2 will be described with reference to Figures 8 and 9. Figures 8 and 9 are schematic flowcharts illustrating an example of the operation of the robot control system in Embodiment 2. Figure 8 shows the processing of the first phase ST10 and the second phase ST20 of the calibration process ST1, and Figure 9 shows the processing of the third phase ST30 to the fifth phase ST50 of the task process ST2.

[0143] In the robot control system of Embodiment 2, the work tool 12 is calibrated in a predetermined number of poses with intervals between them. Furthermore, the robot control system of Embodiment 2 corrects the pose of the teaching information. Aside from these points and those described below, the robot control system of Embodiment 2 is the same as the robot control system of Embodiment 1.

[0144] Therefore, Embodiment 2 will primarily describe the differences from Embodiment 1.

[0145] The calibration process ST1 of the robot control system 1A will be explained with reference to Figure 8. Note that steps ST12 to ST14 of the first phase ST10 and step ST21 of the second phase ST20 shown in Figure 8 are the same as in Embodiment 1, so their explanation will be omitted.

[0146] In step ST11A of the first phase ST10, the robot controller 40 can limit the calibration range by driving the robot 10 based on drive information that drives it in a plurality of predetermined postures with intervals between them. Here, the "predetermined postures with intervals between them" may be set to a range of postures that can be used depending on the task.

[0147] For example, in a welding task, the tilt angle θt and forward / backward angle θa may be adjusted in increments of 10° or 15° to ensure welding quality. The twist angle θw may be determined by the posture of the robot 10. When the tilt angle θt and forward / backward angle θa are adjusted in increments of 10°, the robot controller 40 may drive the robot 10 in a posture of (i = 10 × n + a)° with respect to the tilt angle θt and forward / backward angle θa. Alternatively, when the tilt angle θt and forward / backward angle θa are adjusted in increments of 15°, the robot controller 40 may drive the robot 10 in a posture of (i = 15 × n + a)° with respect to the tilt angle θt and forward / backward angle θa. Here, i is the interval (interval), n is an arbitrary list of integers (0, 1, 2, 3, ..., m), and a is the initial value.

[0148] For higher quality welding or precise pick-and-place operations, the spacing may be in 5° increments. For calibration purposes, the minimum spacing may be 3°. In other words, in the case of 3° increments, the robot controller 40 may drive the robot 10 in a (3 × n)° position for one or two axes. Here, the axes may be selected from multiple coordinate systems.

[0149] Figure 10 is a schematic perspective view illustrating an example of multiple coordinate systems in the robot control system in Embodiment 2.

[0150] As shown in Figure 10, the robot control system 1A includes a plurality of coordinate systems CS1 to CS5. The plurality of coordinate systems CS1 to CS5 include the robot coordinate system CS1, the tool coordinate system CS2, the object coordinate system CS3, the fixed platform coordinate system CS4, and the measurement coordinate system CS5.

[0151] The robot coordinate system CS1 is a coordinate system based on the robot 10, for example, a coordinate system with the base 13 as the origin. The tool coordinate system CS2 is a coordinate system based on the work tool 12, for example, a coordinate system with the tool center point TCP as the origin. The object coordinate system CS3 is a coordinate system based on the workpiece WK1, for example, a coordinate system with the corner of the workpiece WK1 as the origin. The fixed base coordinate system CS4 is a coordinate system based on the fixed base 2, for example, a coordinate system with the fixed marker 3 provided on the fixed base 2 as the origin. The measurement coordinate system CS5 is a coordinate system based on the first measuring device 31.

[0152] The coordinate systems described above are examples only, and the coordinate system of the robot control system 1A is not limited to these.

[0153] Furthermore, in this embodiment, (15 × n)° = 30°, 45°, 60°, 75°, 90°, defined at regular intervals, may be used to set multiple postures that the robot 10 will drive, by setting the interval, minimum value, and number, or the interval, minimum value, and maximum value. This allows for easy setting of multiple postures. However, the multiple predetermined postures with intervals may also be set by specifying a list, for example, one in which 30°, 40°, 45°, 50°, 75°, 90°, and 120° are set.

[0154] The specified range and interval may vary depending on the task being performed. As mentioned above, when adjusting the orientation in 15° increments for a welding task, i may be 15, with a minimum of 30 deg and a maximum of 120 deg (m=7, a=15). For a welding task involving three planes, including a first plane, a second plane perpendicular to the first plane, and a third plane facing the first plane and perpendicular to the second plane, i may be 90, with a minimum of 0 deg and a maximum of 180 deg (m=2, a=0).

[0155] In step S22A of the second phase ST20, the machine learning model 52 uses a plurality of predetermined poses of the work tool 12, spaced apart, and a plurality of positions in the plurality of predetermined poses, as training data for machine learning. That is, in step ST22A, the machine learning model 52 does not use information on the pose and position of the work tool 12 other than the plurality of predetermined poses spaced apart as training data.

[0156] Furthermore, the machine learning model 52 outputs control information for a robot that is driven while fixed in one or more of a predetermined set of postures spaced apart. For example, if the predetermined set of postures spaced apart are (15 × n)° = 30°, 45°, 60°, 75°, and 90°, the joint angles of multiple joints 15 of the robot 10, calculated to be one or more of the (15 × n)° postures, are output as control information.

[0157] In this way, by limiting the data used as training data for machine learning to a predetermined set of postures for the work tool 12, the amount of data acquired in the calibration process ST1 can be reduced.

[0158] The task process ST2 of the robot control system 1A will be explained with reference to Figure 9. Note that the third phase ST30 and the fifth phase ST50 shown in Figure 9 are the same as in Embodiment 1, so their explanation will be omitted.

[0159] Steps ST41 to ST43 of the fourth phase ST40 show the process by which the robot controller 40 acquires attitude setting values ​​and corrects the attitude of the taught information.

[0160] The teaching information is information about the position and orientation of the simulated teaching device 20, obtained by the user moving the simulated teaching device 20 by hand. Therefore, the orientation in the teaching information may deviate from the orientation intended by the user. For example, if the working tool 12 of the robot 10 is a welding tool, there is an ideal orientation for the working tool 12 that is suitable for welding. Even if the user tries to position the simulated teaching device 20 in the ideal orientation for welding, the orientation of the simulated teaching device 20 may not be ideal due to the effects of hand tremors or other factors.

[0161] Therefore, the robot controller 40 performs steps ST41 to ST43 and corrects the pose of the teaching information according to the pose setting value.

[0162] In step ST41, the robot controller 40 acquires an attitude setting value to correct the attitude of the teaching information. The attitude setting value is selected from a plurality of predetermined attitudes with intervals between them. For example, the attitude setting value may be acquired by receiving input from the user input into the input interface 43.

[0163] An example of the robot controller 40 acquiring attitude settings will be explained with reference to Figure 11. Figure 11 shows a schematic flowchart illustrating an example of the process for acquiring attitude settings.

[0164] As shown in Figure 11, in step S41A, the robot controller 40 outputs one or more pose candidates via the output interface 44. The one or more pose candidates are one or more poses selected from a predetermined set of poses spaced apart. For example, if the output interface 44 is a display, the robot controller 40 displays one or more pose candidates on the display.

[0165] In step S41B, the input interface 43 receives information about the pose candidates entered by the user. For example, the user selects one pose candidate from among several pose candidates and inputs it into the input interface 43. The input interface 43 obtains the information of the pose candidate selected by the user as input information.

[0166] In step ST41C, the robot controller 40 determines the posture setting value based on the posture candidate information. The robot controller 40 obtains the posture candidate information entered by the user from the input interface 43 and determines the posture setting value corresponding to the posture candidate information. For example, if the user selects a posture candidate of 45°, the posture setting value is determined to be 45°.

[0167] In step ST42, the robot controller 40 corrects the pose of the teaching information based on the pose setting value.

[0168] In step ST43, the robot controller 40 acquires control information for the robot 10 based on the corrected teaching information.

[0169] The third phase ST30 and the fifth phase ST50 are the same as in Embodiment 1.

[0170] The concept of operation of the robot control system of Embodiment 2 will be explained with reference to Figures 12A to 12C. Figure 12A is a schematic diagram illustrating an example of the posture of an ideal simulated teaching device. Figure 12B is a schematic diagram illustrating an example of the posture of the simulated teaching device when a user is actually teaching it. Figure 12C is a schematic diagram illustrating the posture of the work tool corrected by the posture setting value. Figures 12A to 12C illustrate an example where the work tool 12 is a welding tool.

[0171] As shown in Figure 12A, when the user operates the simulation teaching device 20 to teach the orientation of the work tool 12, they want to teach it at an ideal angle θs suitable for welding.

[0172] However, as shown in Figure 12B, even if the user intends to manipulate the posture of the simulated teaching device 20 at the ideal angle θs, in reality, they may be unable to manipulate it at the ideal angle θs and may end up manipulating it at an angle θr instead.

[0173] In the robot control system of this embodiment, the posture of the teaching information provided by the user is corrected by the posture setting value. As a result, the posture of the work tool 12 can be moved at an ideal angle θs, as shown in Figure 12C.

[0174] In the first phase, ST10, the posture is narrowed down to construct the first dataset, and then in the second phase, ST20, learning is performed. However, if the posture in the third phase, ST30, is not present in the first dataset, it may be impossible to acquire robot control information in the fourth phase, ST40. This is a problem stemming from the fact that in supervised learning, when input not present in the training data is given during inference, the output data becomes unstable. The position and posture of the simulated teaching device 20 measured in the third phase, ST30, are those of the simulated teaching device 20 held by the operator, so it is easy to foresee that it may take on an unexpected posture.

[0175] Therefore, in the fourth phase ST40, with respect to the position and orientation of the simulated teaching device 20 measured in the third phase ST30, the operator selects or automatically selects one orientation setting value from the orientation candidates set, corrects it to the orientation setting value, and uses it as input for the learned model. At this time, the position of the simulated teaching device 20 does not need to be changed.

[0176] Regarding attitude settings, if they are set for one or more axes, the amount of data collected in the first phase ST10 can be reduced. Setting one axis means, for example, that the pitch is fixed to the attitude setting value, so that the pitch always maintains the attitude setting value, and the roll and yaw maintain a continuous attitude within the set range. The coordinate system may be set to any of the following: robot coordinate system CS1, tool coordinate system CS2, object coordinate system CS3, fixed platform coordinate system CS4, and measurement coordinate system CS5. Alternatively, the direction of gravity may be used as the height axis, and the front-to-back and left-to-right directions of any of the systems may be used as the front-to-back axis and left-to-right axis, respectively.

[0177] Thus, the machine learning model 52 performs machine learning using a plurality of predetermined poses of the work tool 12 with intervals between them, and a plurality of positions in the plurality of predetermined poses, as training data. That is, the training data consists of a plurality of predetermined poses of the work tool 12 with intervals between them, and a plurality of positions in the plurality of predetermined poses. With this configuration, the training data used for machine learning can be reduced by limiting the work tool 12 to a plurality of predetermined poses. That is, the data acquired in the calibration process ST1 can be reduced. This makes it possible to shorten the calibration time.

[0178] The machine learning model 52 outputs control information for the robot 10, which is driven while fixed in one or more of a predetermined set of postures. This configuration allows the posture of the work tool 12 to be stabilized. For example, when a user grasps and operates the simulated teaching device 20, variations in the user's hand movements may prevent the simulated teaching device 20 from achieving an ideal posture. In such cases, the posture of the work tool 12 can be selected and fixed from a predetermined set of postures. Therefore, even if the posture in the teaching information is not the ideal posture, the posture of the work tool 12 can be fixed and stabilized.

[0179] Multiple predetermined poses are set at intervals of 3° or more. This configuration reduces the amount of training data used for machine learning, thereby shortening the calibration time.

[0180] The robot controller 40 acquires attitude setting values ​​to correct the attitude in the teaching information and corrects the attitude in the teaching information based on the attitude setting values. With this configuration, the attitude in the teaching information can be corrected by the attitude setting values.

[0181] The robot control system of this embodiment further includes an output interface 44 capable of outputting information and an input interface 43 capable of receiving user input. The robot controller 40 outputs one or more pose candidates, each representing one or more of a predetermined set of poses, to the output interface 44. The robot controller 40 acquires the pose candidate information input to the input interface 43 and determines a pose setting value based on the pose candidate information. With this configuration, the pose in the teaching information can be corrected to the ideal pose that the user wants to set.

[0182] In this embodiment, an example has been described in which the posture setting value is determined based on posture candidate information entered by the user, but the embodiment is not limited to this. For example, the robot controller 40 may automatically determine the posture based on the posture in the teaching information and a plurality of predetermined postures spaced apart. For example, the robot controller 40 may calculate the difference between the posture in the teaching information and a plurality of predetermined postures spaced apart, and determine the posture setting value from the posture that minimizes this difference.

[0183] In this embodiment, an example was described in which the predetermined multiple postures with intervals are predetermined at equal intervals such as (15 × n)° = 30°, 45°, 60°, 75°, 90°, etc., but the embodiment is not limited to this. For example, the predetermined multiple postures with intervals may be predetermined at different intervals.

[0184] In this embodiment, an example in which the attitude setting value is determined in the fourth phase ST40 has been described, but the embodiment is not limited to this. For example, the attitude setting value may be determined in the third phase ST30. The user may input the attitude setting value in advance via the input interface 43 before teaching the teaching information.

[0185] (Embodiment 3) The robot control system in Embodiment 3 will be described with reference to Figure 13. Figure 13 is a schematic block diagram illustrating the main components of the robot control system in Embodiment 3.

[0186] In the robot control system of Embodiment 3, the robot controller 40 has multiple machine learning models 52. Aside from these points and the points described below, the robot control system of Embodiment 3 is the same as the robot control system of Embodiment 2.

[0187] Therefore, Embodiment 3 will primarily describe the differences from Embodiment 2.

[0188] As shown in Figure 13, in the robot control system 1B of this embodiment, the robot controller 40 stores a plurality of machine learning models 52 in the memory unit 42. The plurality of machine learning models 52 are separated based on conditions. For example, the plurality of machine learning models 52 are separated based on posture.

[0189] In this embodiment, each of the multiple machine learning models 52 corresponds to a predetermined number of poses spaced apart. That is, the robot controller 40 assigns a machine learning model 52 to each of the predetermined number of poses spaced apart, and each of the multiple machine learning models 52 performs machine learning while fixed in a predetermined pose.

[0190] For example, if there are multiple predetermined postures spaced apart at (15 × n)°, the robot controller 40 has first to nth machine learning models 52 corresponding to the multiple postures of (15 × n)°. Each of the first to nth machine learning models 52 is fixed to a posture of (15 × n)° and performs machine learning. Specifically, if there are multiple predetermined postures spaced apart at 30°, 45°, 60°, 75°, and 90°, the robot controller 40 has first to fifth machine learning models 52 corresponding to the postures of 30°, 45°, 60°, 75°, and 90°, respectively.

[0191] The operation of the robot control system of Embodiment 3 will be described with reference to Figure 14. Figure 14 is a schematic flowchart illustrating an example of robot control in Embodiment 3. The third phase ST30 and the fifth phase ST50 shown in Figure 14 are the same as those in Embodiments 1 and 2, so their explanation will be omitted.

[0192] The fourth phase ST40 includes steps ST41D, ST42D, and ST43D.

[0193] In step ST41D, the robot controller 40 acquires a posture setting value. The posture setting value is selected from a plurality of predetermined postures spaced apart.

[0194] In step ST42D, the robot controller 40 selects a machine learning model 52 to use from among multiple machine learning models 52 based on the posture setting value. For example, if the posture setting value is 45°, the robot controller 40 selects a machine learning model 52 that has been trained to assume the posture of the work tool 12 is 45°.

[0195] In step ST43D, the robot controller 40 acquires control information for the robot 10 by inputting teaching information into the selected machine learning model 52.

[0196] Thus, in the robot control system 1B of this embodiment, the machine learning model 52 includes multiple machine learning models 52, each corresponding to a predetermined set of postures. The robot controller 40 acquires posture setting values ​​to correct the posture in the teaching information, and based on the posture setting values, selects the machine learning model 52 to be used from among the multiple machine learning models 52. With this configuration, multiple machine learning models 52 corresponding to various postures can be used, thereby increasing the variety of postures that can be supported.

[0197] In this embodiment, an example has been described in which the robot controller 40 has multiple machine learning models 52, and each of the multiple machine learning models 52 is used according to conditions, but the embodiment is not limited to this. For example, the robot controller 40 may have one machine learning model 52 as a conditional model. The conditional model may be machine-learned using a predetermined set of postures as conditions. When learning as a conditional model, one-hot vectors or condition IDs may be used depending on the discontinuous conditions of the multiple postures.

[0198] For example, the conditional model may be a conditional model that performs machine learning using a predetermined set of poses as conditions. The conditional model may perform machine learning using a predetermined set of poses and a predetermined set of positions as training data, and when teaching information is input, it may output control information for the robot 10 that is driven while fixed in one or more of the predetermined poses.

[0199] (Embodiment 4) The robot control system in Embodiment 4 will be described with reference to Figure 15. Figure 15 is a schematic flowchart illustrating an example of robot control in Embodiment 4.

[0200] In the robot control system of Embodiment 4, the robot controller 40 evaluates the control information output from the machine learning model 52 and controls the driving of the robot 10 based on the evaluation result. Except for these points and the points described below, the robot control system of Embodiment 4 is the same as the robot control system of Embodiment 1.

[0201] Therefore, Embodiment 4 will primarily describe the differences from Embodiment 1.

[0202] As shown in Figure 15, the robot controller 40 evaluates the control information output from the machine learning model 52 by performing steps ST61 to ST64, and controls the drive of the robot 10 based on the evaluation results. Steps ST61 to ST64 are performed, for example, between the fourth phase ST40 and the fifth phase ST50. Steps ST61 to ST64 may also be performed in parallel in the fourth phase ST40 and / or the fifth phase ST50.

[0203] In step ST61, the robot controller 40 calculates the estimated position and orientation of the work tool 12 when it moves based on the control information using robot forward kinematics. In this way, the robot controller 40 estimates the position and orientation of the work tool 12 based on theoretical forward kinematics using the control information.

[0204] In step ST62, the robot controller 40 calculates the difference between the estimated position and the taught position, and the difference between the estimated posture and the taught posture. The taught position is the position of the work tool 12 as indicated by the teaching information, and the taught posture is the posture of the work tool 12 as indicated by the teaching information. The robot controller 40 determines whether the difference between the estimated position and the taught position, or the difference between the estimated posture and the taught posture, exceeds a threshold. If the difference exceeds the threshold, the flow proceeds to step ST63. If the difference does not exceed the threshold, the flow ends.

[0205] In step ST63, the robot controller 40 outputs a warning. For example, the robot controller 40 outputs a warning to the output interface 44 to notify the user. The robot controller 40 also outputs a warning and prompts the user for input. The input from the user may be, for example, correction values ​​for position and / or posture. While the warning is being output, the robot controller 40 stops the robot 10 without driving it.

[0206] In step ST64, the robot controller 40 determines whether or not there has been input from the user. For example, the robot controller 40 determines whether or not there has been input from the user based on the information entered into the input interface 43. If there is input from the user, the flow ends. If there is no input from the user, the flow repeats step ST64.

[0207] Next, we will explain the evaluation of the control information output from the machine learning model 52.

[0208] The robot controller 40 inputs the target position and target orientation of the work tool 12 to the machine learning model 52 and acquires the drive information of the robot 10 output from the machine learning model 52. The target position and target orientation of the work tool 12 are the position and orientation taught by the simulation teaching device 20. The drive information of the robot 10 is, for example, joint angles, which can be calculated using [Equation 13].

[0209]

[0210] Here, the definition of the variables in [Equation 13] is shown below.

[0211] The robot controller 40 calculates the estimated position and estimated orientation of the work tool 12 from the control information calculated by robot forward kinematics using [Equation 13]. The estimated position and estimated orientation are the position and orientation when the work tool 12 moves based on the control information, and are calculated by robot forward kinematics. The estimated position and estimated orientation of the work tool 12 can be calculated using [Equation 14].

[0212]

[0213] Here, the definition of the variables in [Equation 14] is shown below.

[0214] The robot controller 40 calculates the difference between the estimated position and orientation of the work tool 12 calculated using [Equation 14] and the target position and orientation of the work tool 12, and determines whether the difference exceeds a threshold. This determination is represented by [Equation 15].

[0215]

[0216] Here, the definition of the variables in [Equation 15] is shown below.

[0217] The robot controller 40 controls the driving of the robot 10 based on [Equation 15]. For example, if the difference exceeds a threshold, the robot controller 40 outputs a warning from the output interface 44.

[0218] The evaluation of control information is not limited to the examples described above. Control information may be evaluated by any method. Also, although the examples described above described an example where the control information is joint angle, the control information may also be the motor voltage value, current value, or encoder value.

[0219] In this way, the robot controller 40 evaluates the control information output from the machine learning model 52 and controls the movement of the robot 10 based on the evaluation results. With this configuration, safety can be improved because the movement of the robot 10 can be controlled by evaluating the control information output from the machine learning model 52. For example, if the control information output from the machine learning model 52 is an abnormal value that deviates significantly from the target position or target posture, it is possible to suppress the robot 10 from making dangerous movements.

[0220] The robot controller 40 calculates the position and orientation of the work tool 12 when it moves based on control information using robot forward kinematics. If the difference between the calculated position and the position taught by the teaching information, or the difference between the calculated orientation and the orientation taught by the teaching information, exceeds a threshold, it outputs a warning. This configuration allows for user notification and further improves safety.

[0221] (Embodiment 5) The robot control system in Embodiment 5 will be described with reference to Figure 16. Figure 16 is a schematic flowchart illustrating an example of robot control in Embodiment 5.

[0222] In the robot control system of Embodiment 5, the robot controller 40 evaluates the teaching information and controls the driving of the robot 10 based on the evaluation result. Except for these points and the points described below, the robot control system of Embodiment 5 is the same as the robot control system of Embodiment 1.

[0223] Therefore, Embodiment 5 will primarily describe the differences from Embodiment 1.

[0224] As shown in Figure 16, the robot controller 40 evaluates the teaching information by performing steps ST71 to ST73 and controls the driving of the robot 10 based on the evaluation results. Steps ST71 to ST73 are performed from the third phase ST30 onward. For example, steps ST71 to ST73 may be performed between the fourth phase ST40 and the fifth phase. Alternatively, steps ST71 to ST73 may be performed in parallel in the fourth phase ST40 and / or the fifth phase ST50.

[0225] In step ST71, the robot controller 40 determines whether the pose of the taught information exceeds a threshold range. For example, the threshold range is -20° or more and +20° or less. In this case, the robot controller 40 determines whether the pose of the taught information is less than -20° or greater than +20°. If the pose of the taught information exceeds the range of -20° or more and +20° or less, the flow proceeds to step ST72. If the pose of the taught information does not exceed the range of -20° or more and +20° or less, the flow proceeds to step ST74.

[0226] In step ST72, the robot controller 40 outputs a warning. For example, the robot controller 40 outputs a warning to the output interface 44 to notify the user. The robot controller 40 also outputs a warning and prompts the user for input. The input from the user may be, for example, the position and / or attitude setting values. While the warning is being output, the robot controller 40 stops the robot 10 without driving it.

[0227] In step ST73, the robot controller 40 determines whether or not there has been input from the user. For example, the robot controller 40 determines whether or not there has been input from the user based on the information entered into the input interface 43. If there has been input from the user, the flow proceeds to step ST74. If there has been no input from the user, the flow repeats step ST73.

[0228] The machine learning model 52 uses poses within a threshold range as training data for machine learning.

[0229] In this way, the robot controller 40 outputs a warning when the pose of the taught information exceeds a threshold range. With this configuration, by setting the range of poses in the taught information, it is possible to prevent the work tool 12 from adopting an unintended pose. In addition, since the machine learning model 52 uses poses within the threshold range as training data, the amount of training data can be reduced. Therefore, the calibration time can be shortened.

[0230] In this embodiment, an example was described in which the threshold range is -20° or more and +20° or less, but this is not limited to this. The range may be changed depending on the specifications or application of the robot 10.

[0231] (Embodiment 6) The robot control system in Embodiment 6 will be described with reference to Figure 17. Figure 17 is a schematic flowchart illustrating an example of robot control in Embodiment 6.

[0232] In the robot control system of Embodiment 6, the robot controller 40 evaluates the attitude setting value and controls the drive of the robot 10 based on the evaluation result. Except for these points and the points described below, the robot control system of Embodiment 6 is the same as the robot control system of Embodiment 5.

[0233] Therefore, Embodiment 6 will primarily describe the differences from Embodiment 5.

[0234] As shown in Figure 17, the robot controller 40 evaluates the attitude setting value by performing steps ST71A to ST73 and controls the drive of the robot 10 based on the evaluation result. Steps ST71A to ST73 are performed from the third phase ST30 onward. For example, steps ST71A to ST73 may be performed between the fourth phase ST40 and the fifth phase. Alternatively, steps ST71A to ST73 may be performed in parallel in the fourth phase ST40 and / or the fifth phase ST50.

[0235] In step ST71A, the robot controller 40 determines whether the attitude setting value exceeds a threshold range. For example, the threshold range is -20° or more and +20° or less. In this case, the robot controller 40 determines whether the attitude setting value is less than -20° or greater than +20°. If the attitude setting value exceeds the range of -20° or more and +20° or less, the flow proceeds to step ST72. If the attitude of the teaching information does not exceed the range of -20° or more and +20° or less, the flow ends.

[0236] Steps ST72 to ST73 shown in Figure 17 are the same as steps ST72 to ST73 in Embodiment 5, so their explanation is omitted.

[0237] In this way, the robot controller 40 outputs a warning when the posture setting value exceeds a threshold range. With this configuration, by setting a range for the posture setting value, it is possible to prevent the work tool 12 from being corrected to an unintended or impossible posture. In addition, since the machine learning model 52 uses postures within the threshold range as training data, the amount of training data can be reduced. Therefore, the calibration time can be shortened.

[0238] In this embodiment, an example was described in which the threshold range is -20° or more and +20° or less, but this is not limited to this. The range may be changed depending on the specifications or application of the robot 10.

[0239] (Embodiment 7) The robot control system in Embodiment 7 will be described with reference to Figures 18 and 19. Figure 18 is a schematic block diagram illustrating the main configuration of the robot control system in Embodiment 7. Figure 19 is a schematic flowchart illustrating an example of robot control in Embodiment 7.

[0240] In the robot control system of Embodiment 7, a third measuring device 33 is provided to measure the position of the workpiece WK1, and the robot controller 40 detects the positional deviation of the workpiece WK1 and corrects the control information based on the positional deviation of the workpiece WK1. Except for these points and the points described below, the robot control system of Embodiment 6 is the same as the robot control system of Embodiment 1.

[0241] Therefore, Embodiment 7 will primarily describe the differences from Embodiment 1.

[0242] As shown in Figure 18, the robot control system 1C includes a third measuring device 33 for measuring the position of the workpiece WK1. The third measuring device 33 acquires image data, for example, by a camera, and acquires the position of the workpiece WK1 based on the image data. The third measuring device 33 is not particularly limited and can be any device capable of acquiring position information of the workpiece WK1.

[0243] As shown in Figure 19, the robot controller 40 corrects the control information based on the positional displacement of the workpiece WK1 by performing steps ST81 to ST83.

[0244] In step ST81, the third measuring device 33 acquires the position information of the workpiece WK1. The third measuring device 33 transmits the position information of the workpiece WK1 to the robot controller 40.

[0245] In step ST82, the robot controller 40 detects the positional displacement of the workpiece WK1 based on the positional information of the workpiece WK1. For example, the robot controller 40 detects how much the workpiece WK1 has shifted relative to the reference position. That is, the robot controller 40 calculates the amount of displacement of the workpiece WK1 relative to the reference position.

[0246] In step ST83, the robot controller 40 corrects the control information based on the positional displacement of the workpiece WK1. The robot controller 40 corrects the control information based on the positional information of the work tool 12 and the positional displacement information of the workpiece WK1. For example, the robot controller 40 corrects the control information such as joint angles so that the position of the tool center point TCP of the work tool 12 is offset by the amount that the workpiece WK1 has shifted.

[0247] In this way, the robot controller 40 detects the positional displacement of the workpiece WK1 and corrects the control information based on the positional displacement. With this configuration, even if the workpiece WK1 is misaligned, the positional accuracy of the robot can be improved.

[0248] The third measuring device 33 may be, for example, an RGB camera, an RGB-D camera, or a 2D profile sensor, and may acquire information about the workpiece WK1 and its surroundings.

[0249] Alternatively, the robot controller 40 may use a position correction model that has been machine-learned using the position information of the workpiece WK1 as training data.

[0250] For example, if the position information of workpiece WK1 is image data captured by a camera, the position correction model 54 may be a deep learning-based object recognition model, a pre-trained segmentation model such as Meta's SAM (Segment Anything Model), or a 6-axis object pose estimation model such as Foundationpose. Alternatively, OC-NVAE or CFIL may be used.

[0251] OC-NVAE performs machine learning aimed at efficiently developing estimation models that estimate the pose of objects from images. Based on behavioral data, image data, and additional information, OC-NVAE performs machine learning on the object's pose in each of multiple frames of image data to generate a trained estimation model. When an image of an object is input, the estimation model outputs an estimated value of the object's pose.

[0252] In this embodiment, an example in which the robot control system 1C includes a third measuring device 33 has been described, but the system is not limited to this. For example, the robot control system 1C does not need to include a third measuring device 33. In this case, the robot control system 1C may acquire position information of the workpiece WK1 using the first measuring device 31 or the second measuring device 32.

[0253] (Other Embodiments) As described above, the embodiments described above have been presented as examples of the technology disclosed in this application. However, the technology in this disclosure is not limited thereto and can be applied to embodiments that are modified, replaced, added, or omitted as appropriate. Furthermore, these general and specific embodiments may be realized by apparatus, systems, methods, computer programs, computer-readable storage media, or combinations thereof.

[0254] The following are examples of modifications as other embodiments.

[0255] (Modification 1) The robot control system in Modification 1 will be described with reference to Figure 20. Figure 20 is a schematic diagram illustrating the teaching of the robot control system in Modification 1.

[0256] As shown in Figure 20, in the third phase ST30, when the user teaches the position and orientation of the work tool 12 using the simulation teaching device 20, the user teaches the work start point P1 and the work end point P2. For example, the input of the work start point P1 and the work end point P2 may be done using the trigger 23 or button 24 of the simulation teaching device 20. Alternatively, the input of the work start point P1 and the work end point P2 may be done using a voice recognition device such as a microphone.

[0257] The robot controller 40 may automatically recognize when the work tool 12 moves along a trajectory L1 between the work start point P1 and the work end point P2. The trajectory L1 is, for example, a straight line.

[0258] Alternatively, the trajectory L1 between the work start point P1 and the work end point P2 may be input by the user.

[0259] This configuration reduces the amount of data that the user needs to provide.

[0260] In Modification 1, an example was described in which the start point P1 and end point P2 of the work are taught, but the system is not limited to this. For example, the user may teach one or more intermediate points in addition to the start point P1 and end point P2.

[0261] For example, the user may input items of action such as approaching workpiece WK1, executing a task, evacuating, or detailed task parameters via the input interface 43. The user can easily teach the system by setting the start point P1 and end point P2 during approach or evacuation, and by switching to inputting the trajectory during task execution, depending on the required precision.

[0262] (Modification 2) The robot control system in Modification 2 will be described with reference to Figure 21. Figure 21 is a schematic diagram illustrating the teaching of the robot control system in Modification 2. Figure 21 shows an example of teaching when the robot 10 performs a welding operation.

[0263] As shown in Figure 21, the user indicates the position and orientation of the work tool 12 with their hand. The user uses their first finger F1 to indicate triggers such as the start and end of the teaching process. The user uses their second finger F2 to indicate the position and orientation of the work tool 12. For example, the first finger F1 is the thumb, and the second finger F2 is the index finger. The tip of the second finger F2 is the virtual tool center point VCP.

[0264] The user controls the start and end of teaching by moving the first finger F1 left and right. The user teaches the position of the work tool 12 by moving the tip of the second finger F2, and teaches the orientation of the work tool 12 by changing the angle of the second finger F2. This teaching method is called a simulated teaching operation.

[0265] The second measuring device 32 may measure the user's finger movements by photographing reflective markers attached to the user's hand with an infrared camera. Alternatively, the second measuring device 32 may measure the user's finger movements using Human Pose Estimation or Hand Pose Estimation, which acquires the position of key points on the human body or fingers.

[0266] With this configuration, the second measuring device 32, which acquires the position of the human hand performing the simulated movement, can acquire the position and orientation, or multiple positions, thereby enabling the acquisition of teaching information without the need for a simulated teaching device 20.

[0267] (Modification 3) The robot control system in Modification 3 will be described with reference to Figure 22. Figure 22 is a schematic diagram illustrating the simulated teaching device for the robot control system in Modification 3. Figure 22 shows an example of teaching when the robot 10 performs a pick-and-place operation. The working tool 12 of the robot 10 is a robot hand.

[0268] As shown in Figure 22, the simulated teaching device 20A includes a first movable finger MF1 and a second movable finger MF2 arranged with an interval SP1 between them. The simulated teaching device 20A also includes a plurality of retroreflective markers 25.

[0269] The first movable finger MF1 and the second movable finger MF2 are movable so as to change the interval SP1. A virtual tool center point VCP is set between the first movable finger MF1 and the second movable finger MF2. For example, the virtual tool center point VCP is set to a position equidistant from the first movable finger MF1 and the second movable finger MF2. The user may operate the first movable finger MF1 and the second movable finger MF2 by operating them with a separate controller.

[0270] With this configuration, teaching information can be acquired when the work tool 12 is a robot hand.

[0271] Although the example of the simulated teaching device 20A having two movable fingers MF1 and MF2 has been described, it is not limited to this. For example, the simulated teaching device 20A may have two or more movable fingers.

[0272] (Modification 4) The robot control system in Modification 4 will be described with reference to Figure 23. Figure 23 is a schematic diagram illustrating the simulated teaching device for the robot control system in Modification 4. Figure 23 shows an example of teaching when the robot 10 performs a pick-and-place operation. The working tool 12 of the robot 10 is a robot hand.

[0273] As shown in Figure 23, the simulated teaching device 20B includes a simulated hand 26. The simulated hand 26 has, for example, two movable fingers. Note that the number of movable fingers is not limited to two, but should be at least two or more.

[0274] The simulated teaching device 20B is equipped with a second measuring device 32. The second measuring device 32 may be a camera, an IMU, or an RGB-D camera with a built-in IMU. The second measuring device 32 may perform self-position estimation based on the measured information and acquire position and orientation information of the simulated hand 26.

[0275] Furthermore, the second measuring device 32 may also be used as the third measuring device 33 for measuring the position of the workpiece WK1. For example, if a camera or IMU is used as the second measuring device 32, the camera or IMU can also be used as the third measuring device 33. This allows for a reduction in the number of parts by using a single camera or IMU to serve as both the second measuring device 32 and the third measuring device 33.

[0276] Furthermore, the second measuring device 32 may also be used in conjunction with the first measuring device 31 for measuring the position and orientation of the robot's work tool 12. For example, if a camera or IMU is used as the second measuring device 32, the camera or IMU can also be used as the first measuring device 31. The second measuring device 32 may have a configuration that allows it to be attached to and detached from the simulation teaching device 20B and the robot 10 in order to be used in conjunction with the first measuring device 31. In the first phase ST10 of the calibration process ST1, the second measuring device 32 may be removed from the simulation teaching device 20B and attached to the robot's work tool 12. Then, based on the information measured by the second measuring device 32 attached to the work tool 12, the self-position estimation of the work tool 12 may be performed and position and orientation information of the work tool 12 may be obtained. As a result, a database 51 that associates drive information and position and orientation information may be created.

[0277] Thus, the second measuring device 32 can be used interchangeably with the first measuring device 31 and / or the third measuring device 33. In other words, by attaching a camera, an IMU, or an RGB-D camera with a built-in IMU to the simulated teaching device 20B and acquiring its own position, one measuring device can perform the roles of the first to third measuring devices 31 to 33, resulting in an inexpensive system configuration.

[0278] (Modification 5) The robot control system in Modification 5 will be described with reference to Figure 24. Figure 24 is a schematic diagram illustrating the teaching of the robot control system in Modification 5. Figure 24 shows an example of teaching when the robot 10 performs a pick-and-place operation. The working tool 12 of the robot 10 is a robot hand.

[0279] As shown in Figure 24, the user teaches the pick-and-place movements of the robot hand, which is the work tool 12, by changing the distance SP2 between the first finger F1 and the second finger F2. The virtual tool center point VCP is set to, for example, the third finger F3. The user also teaches the posture of the work tool 12 in terms of the positional relationship of the first to third fingers F1 to F3. For example, the first finger F1 is the thumb, the second finger F2 is the index finger, and the third finger F3 is the middle finger.

[0280] With this configuration, teaching information can be acquired even without the provision of a simulated teaching device 20.

[0281] (Outline of Embodiments) (1) The robot control system of the present disclosure comprises a robot including a work tool and a robot controller that controls the driving of the robot, wherein the robot controller acquires teaching information that teaches the position and orientation of the work tool of the robot, inputs the teaching information into a machine learning model that has been machine-learned using the robot's driving information and position and orientation information including the position and orientation of the work tool moved based on the driving information as training data, acquires control information for the robot output from the machine learning model, and drives the robot based on the control information.

[0282] (2) In the robot control system of (1), the teaching information may be obtained by measuring the position and orientation of the simulated teaching device, or by measuring multiple positions.

[0283] (3) In the robot control system of (1) or (2), a measuring device may be provided to measure the position and posture of the human hand performing the simulated teaching action, or multiple positions, in order to acquire the teaching information.

[0284] (4) In any one of the robot control systems described in (1) to (3), the robot controller may evaluate the control information output from the machine learning model and control the robot's drive based on the evaluation result.

[0285] (5) In the robot control system of (4), the robot controller may calculate the position and orientation of the work tool when it moves based on the control information using forward kinematics, and may output a warning if the difference between the calculated position and the position taught by the teaching information, or the difference between the calculated orientation and the orientation taught by the teaching information, exceeds a threshold.

[0286] (6) In any one of the robot control systems of (1) to (5), the training data may consist of a plurality of predetermined postures of the work tool with intervals between them, and a plurality of positions in the plurality of predetermined postures.

[0287] (7) In the robot control system of (6), the machine learning model may output control information for the robot that is driven while fixed in one or more of the predetermined multiple postures.

[0288] (8) In the robot control system of (6) or (7), the machine learning model may include a plurality of machine learning models corresponding to a plurality of predetermined postures, the robot controller may acquire posture setting values ​​to correct the posture of the teaching information, and may select a machine learning model to use from the plurality of machine learning models based on the posture setting values.

[0289] (9) In any one of the robot control systems described in (6) to (8), the predetermined multiple postures may be set at intervals of 3° or more.

[0290] (10) In any one of the robot control systems of (1) to (9), the machine learning model may include a conditional model that performs machine learning on a predetermined set of attitudes, and the conditional model may perform machine learning on the predetermined set of attitudes from the attitudes of the position and attitude information and the predetermined set of positions as training data, and when the teaching information is input, it may output control information for the robot that is driven in a fixed state of one or more of the predetermined set of attitudes.

[0291] (11) In any one of the robot control systems of (1) to (10), the robot controller may acquire an attitude setting value to correct the attitude in the teaching information, and may correct the attitude in the teaching information based on the attitude setting value.

[0292] (12) The robot control system of (11) may further include an output interface capable of outputting information and an input interface capable of receiving user input, and the robot controller may output one or more posture candidates representing one or more of the predetermined plurality of postures to the output interface, may acquire information on posture candidates input to the input interface, and may determine the posture setting value based on the information on the posture candidates.

[0293] (13) In any one of the robot control systems described in (1) to (12), the robot controller may output a warning if the posture of the teaching information exceeds a threshold range.

[0294] (14) In the robot control system of (11) or (12), the robot controller may output a warning when the attitude setting value exceeds a threshold range.

[0295] (15) In any one of the robot control systems described in (1) to (14), the robot controller may detect a misalignment of the workpiece being worked on by the work tool and correct the control information based on the misalignment.

[0296] (16) In any one of the robot control systems of (1) to (15), a measuring device may be further provided that is used for both measuring the teaching information and measuring the position and orientation information of the work tool in the teaching data.

[0297] (17) The robot controller of the present disclosure is a robot controller for controlling the drive of a robot including a work tool, comprising a processor and a storage unit storing instructions executed by the processor, wherein the instructions include acquiring teaching information that teaches the position and orientation of the work tool of the robot, acquiring control information for the robot output from a machine learning model by inputting the teaching information into a machine learning model that has been machine-learned using the robot's drive information and position and orientation information including the position and orientation of the work tool moved based on the drive information as training data, and driving the robot based on the control information.

[0298] (18) A control method of the present disclosure is a control method for controlling a robot including a work tool with a robot controller, comprising the steps of: acquiring teaching information that teaches the position and orientation of the work tool of the robot; acquiring control information of the robot output from a machine learning model by inputting the teaching information into a machine learning model that has been machine-learned using the robot's drive information and position and orientation information including the position and orientation of the work tool moved based on the drive information as training data; and driving the robot based on the control information.

[0299] This disclosure is applicable, for example, to robot control systems that control robots equipped with work tools for performing tasks such as welding, pick-and-place, and screw fastening.

[0300] 1A, 1B, 1C Robot control system 2 Fixed base 3 Fixed marker 10 Robot 11 Main body 12 Working tool 13 Base 14 Arm 15 Joint 20, 20A, 20B Simulated teaching device 21 Gripping unit 22 Teaching unit 23 Trigger 24 Button 25 Marker 26 Simulated hand 31 First measuring device 32 Second measuring device 33 Third measuring device 40 Robot controller 41 Processor 42 Memory unit 43 Input interface 44 Output interface 51 Database 52 Machine learning model 53 Execution program CS1 Robot coordinate system CS2 Tool coordinate system CS3 Work coordinate system CS4 Fixed base coordinate system CS5 Measurement coordinate system F1, F2, F3 Finger L1 Trajectory MF1, MF2 Movable finger P1 Work start point P2 Work end point TCP Tool center point VCP Virtual tool center point WK1 Workpiece WP1 Welding area

Claims

1. A robot control system comprising: a robot including a work tool; and a robot controller for controlling the driving of the robot, wherein the robot controller acquires teaching information that teaches the position and orientation of the work tool of the robot; the teaching information is input to a machine learning model that has been machine-learned using the robot's driving information and position and orientation information including the position and orientation of the work tool moved based on the driving information as training data, and the robot control system acquires control information for the robot output from the machine learning model; and drives the robot based on the control information.

2. The robot control system according to claim 1, wherein the teaching information is obtained by measuring the position and orientation of a simulated teaching device, or by measuring multiple positions.

3. The robot control system according to claim 1, further comprising a measuring device for measuring the position and posture of a human hand performing a simulated teaching action, or multiple positions, in order to acquire the teaching information.

4. The robot controller evaluates the control information output from the machine learning model and controls the driving of the robot based on the evaluation result, according to any one of claims 1 to 3.

5. The robot controller calculates the position and orientation of the work tool when it moves based on the control information using forward kinematics, and outputs a warning when the difference between the calculated position and the position taught by the teaching information, or the difference between the calculated orientation and the orientation taught by the teaching information, exceeds a threshold, the robot control system according to claim 4.

6. The robot control system according to any one of claims 1 to 3, wherein the training data comprises a plurality of predetermined postures spaced apart in the work tool, and a plurality of positions in the plurality of predetermined postures.

7. The robot control system according to claim 6, wherein the machine learning model outputs control information for the robot, which is driven while fixed in one or more of the predetermined multiple postures.

8. The robot control system according to claim 6, wherein the machine learning model includes a plurality of machine learning models corresponding to a plurality of predetermined postures, the robot controller obtains posture setting values ​​to correct the postures in the teaching information, and selects a machine learning model to be used from the plurality of machine learning models based on the posture setting values.

9. The robot control system according to claim 6, wherein the predetermined multiple postures are defined at intervals of 3° or more.

10. The robot control system according to any one of claims 1 to 3, wherein the machine learning model includes a conditional model that performs machine learning on a predetermined set of poses, the conditional model performs machine learning on the predetermined set of poses from the positional pose information and the predetermined set of positions as training data, and when the teaching information is input, it outputs control information for the robot that is driven in a fixed state of one or more of the predetermined set of poses.

11. The robot controller acquires an attitude setting value for correcting the attitude in the teaching information, and corrects the attitude in the teaching information based on the attitude setting value, the robot control system according to any one of claims 1 to 3.

12. A robot control system according to claim 11, further comprising: an output interface capable of outputting information; and an input interface capable of receiving user input, wherein the robot controller outputs one or more posture candidates representing one or more of the predetermined multiple postures to the output interface, acquires information on the posture candidates input to the input interface, and determines the posture setting value based on the information on the posture candidates.

13. The robot control system according to any one of claims 1 to 3, wherein the robot controller outputs a warning when the posture of the teaching information exceeds a threshold range.

14. The robot control system according to claim 11, wherein the robot controller outputs a warning when the attitude setting value exceeds a threshold range.

15. The robot control system according to any one of claims 1 to 3, wherein the robot controller detects a positional displacement of a workpiece being worked on by the work tool and corrects the control information based on the positional displacement.

16. The robot control system according to claim 1 or 2, further comprising a measuring device used for both measuring the teaching information and measuring the position and orientation information of the work tool in the teaching data.

17. A robot controller for controlling the drive of a robot including a work tool, comprising: a processor; a storage unit storing instructions executed by the processor, wherein the instructions include: acquiring teaching information that teaches the position and orientation of the work tool of the robot; acquiring control information for the robot output from a machine learning model by inputting the teaching information into a machine learning model that has been machine-learned using the robot's drive information and position and orientation information including the position and orientation of the work tool moved based on the drive information as training data; and driving the robot based on the control information.

18. A control method for controlling a robot including a work tool using a robot controller, comprising: a step of acquiring teaching information that teaches the position and orientation of the work tool of the robot; a step of acquiring control information for the robot output from a machine learning model by inputting the teaching information into a machine learning model that has been machine-learned using the robot's drive information and position and orientation information including the position and orientation of the work tool moved based on the drive information as training data; and a step of driving the robot based on the control information.

Images