Work instruction method and work instruction system
The method corrects teaching data for robots to ensure accurate high-speed work by capturing and modifying time-series data of object positions and orientations, addressing errors in absolute positioning to enhance teaching precision.
Patent Information
- Application Number
- JP2023549292
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-09-27
- Publication Date
- 2024-12-09
- Estimated Expiration
- 2041-09-27
AI Technical Summary
Robots face challenges in achieving accurate absolute positioning during tasks due to errors between commanded coordinates and real-space positions, which are influenced by factors like robot structure, command path, and object weight, making high-speed work difficult to teach accurately.
A method and device that utilize a gripping part to grasp an object, capture time-series data of its position and orientation, generate and modify teaching data to match a reference path, and correct errors before actual work, using cameras and a control device to ensure precise teaching.
Enables robots to perform high-speed tasks with high precision by correcting errors in teaching data, allowing accurate work execution without requiring continuous ultrasonic distance measurement during operation.
Smart Images

Figure 0007596550000011 
Figure 0007596550000012 
Figure 0007596550000013
Abstract
Description
[Technical field]
[0001] The present invention relates to a work teaching method, a work teaching device, and a work teaching system for teaching a robot a work. [Background technology]
[0002] Patent Document 1 discloses a work teaching method for teaching a robot a work. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2017-217738 A Summary of the Invention [Problem to be solved by the invention]
[0004] Robots generally have high accuracy in repeating tasks at the same position, but in many cases the positioning accuracy of absolute positions in real space (hereafter referred to as absolute positioning accuracy) is not guaranteed. For this reason, absolute positioning accuracy is an issue when teaching a robot a task. An error occurs between the coordinates given to the robot as command values and the position to be positioned in real space. For this reason, measures must be taken to address the effects of this error in order to have the robot perform the task accurately. However, this error changes due to various factors, such as the structure of the robot, the path of the command values given to the robot, and the weight of the object the robot grasps.
[0005] In the work teaching method disclosed in Patent Document 1, the work motion of a worker is measured in three dimensions using an ultrasonic range finder, and the work motion of a multi-axis robot is corrected and controlled using this three-dimensional measurement data. In Patent Document 1, while the multi-axis robot is performing an actual task, the work motion of the multi-axis robot is measured using an ultrasonic range finder, and the work motion of the multi-axis robot is corrected and controlled using the three-dimensional measurement data related to the work motion of the worker obtained in advance.
[0006] In Patent Document 1, an ultrasonic range finder is always required to perform correction control while the multi-axis robot is performing actual work. Also, if the work motion is fast, the correction control during the work cannot keep up, so there is a problem that it is difficult to apply to high-speed work.
[0007] The present invention aims to provide a robot task teaching method and task teaching device that are capable of causing a robot to perform actual tasks with a simple configuration and that can teach a robot performing high-speed tasks the task movements of an instructor with high accuracy. [Means for solving the problem]
[0008] In order to solve the above problems, the robot task teaching method of the present invention is a task teaching method executed by a task teaching device that has a gripping unit that grips an object to be grasped and that teaches a task to a robot that moves the object gripped by the gripping unit, and the task teaching method includes a generation step of outputting, as a reference path, time series data of the position and posture of the object to be grasped when an instructor grasps and moves it, and generating teaching data for the robot corresponding to the reference path, and a correction step of outputting, as a motion path, time series data of the position and posture of the object to be grasped or the gripping unit when the robot is operated using the teaching data, and correcting the teaching data so that the motion path matches the reference path before the robot performs actual task. Effect of the Invention
[0009] According to the present invention, it is possible to provide a robot task teaching method and task teaching device that can cause a robot to perform actual tasks with a simple configuration and can teach a robot performing high-speed tasks the task movements of an instructor with high accuracy. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]
[0010] [Figure 1]1 is a configuration diagram of an operation teaching device according to a first embodiment. [Diagram 2] FIG. 2 is a block diagram showing a hardware configuration of a control device. [Diagram 3] FIG. 1 is a diagram showing the arrangement of cameras on a workbench. [Figure 4] 13 is a diagram showing a state in which a marker plate is attached to an object to be grasped. FIG. [Diagram 5] FIG. 13 is a diagram showing a coordinate system set in a robot hand that grips a test tube. [Figure 6] 1 is a diagram showing the relationship between a coordinate system set in a robot hand that holds a test tube and a coordinate system set in a marker plate. FIG. [Figure 7] 4 is a flowchart showing the operation of the control device. [Figure 8] FIG. 4 is a configuration diagram of an operation teaching device in a generating step. [Figure 9] 10 is a flowchart showing details of a generating step. [Figure 10] FIG. 13 is a configuration diagram of an operation teaching device in a correction step. [Figure 11] 10 is a flowchart showing details of a modification step. [Figure 12] FIG. 2 is a configuration diagram of a task teaching device for causing a robot to perform an actual task. [Figure 13] FIG. 13 is a schematic diagram of a reference path measured in a generation step. [Figure 14] FIG. 13 is a schematic diagram of a motion path measured in a correction step. [Figure 15] FIG. 11 is a diagram showing a coordinate system set in a hand of a robot in the second embodiment. [Figure 16] FIG. 11 is a diagram showing the relationship between a coordinate system set in a robot hand and a coordinate system set in a marker plate in the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0011] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS Hereinafter, an embodiment of an operation teaching method, an operation teaching device, and a robot equipped with an operation teaching device of the present invention will be described with reference to the drawings. <Example 1> In the first embodiment, an example of an operation teaching method and an operation teaching device capable of teaching an operation performed by an instructor to a robot with high accuracy will be described with reference to Figs. 1 to 14.
[0012] (Work teaching device 1) FIG. 1 is a configuration diagram of an operation teaching device of the first embodiment. In the first embodiment, as an example of an operation taught by an instructor 11, an instructor teaches a robot 12 to dispense a reagent into a test tube 320 held by a left hand on a workbench 10 using a micropipette 310 held by a right hand. The operation teaching device 1 includes cameras 201-204 for a motion capture system installed on the workbench 10, a marker plate 311 attached to the micropipette 310, a marker plate 321 attached to the test tube 320, and a control device 100. The cameras 201-204 are an example of the imaging means of the present invention. The robot 12 grasps an object to be grasped (in the first embodiment, the micropipette 310 and the test tube 320) and moves the grasped object. The robot 12 has two arms for grasping and moving the micropipette 310 and the test tube 320, and is capable of moving the object to be grasped to any position within a movable range, and is capable of grasping the object in any posture. The operation of the robot 12 is controlled by a control device 100. In the first embodiment, the control device 100, which is separate from the robot 12, teaches the robot 12 how to perform the work, but the control device 100 may be incorporated in the robot 12. The control device 100 may be on-premise or on cloud.
[0013] (Robot 12) The robot 12 is equipped with a stereo camera 12c installed in the area of its head that corresponds to the eyes. The robot 12 is equipped with one or more joint axes in its neck, and the stereo camera 12c can be directed in any direction of the workbench 10. The robot 12 is installed on an unmanned guided vehicle (not shown), and autonomously moves in front of the workbench 10 by moving within the room using a map of the room created in advance.
[0014] (Software configuration of the control device 100) The control device 100 includes a pose measurement unit 101, a teaching data generation / correction unit 102, and a teaching data execution unit 103. The pose measurement unit 101 outputs time series data of the three-dimensional position and orientation of each of the marker plates 311 and 321 from a plurality of images captured by the cameras 201-204. The pose measurement unit 101 is an example of the output means of the present invention. Hereinafter, the three-dimensional position and orientation of the marker plates 311 and 321 will be referred to as a pose. The pose time series data output by the pose measurement unit 101 is recorded as a reference path 104 or a movement path 105.
[0015] The teaching data generating / modifying unit 102 calculates data that becomes command values for the robot 12 from the reference path 104, and this data is recorded as teaching data 106. The teaching data generating / modifying unit 102 also modifies and records the recorded teaching data 106. The teaching data generating / modifying unit 102 is an example of a generating means of the present invention.
[0016] The teaching data execution unit 103 operates the robot 12 using the teaching data 106 and the corrected teaching data 106. The reference path 104, the motion path 105, and the teaching data 106 will be described in detail later.
[0017] In the first embodiment, an example of teaching a task to a robot 12 having two arms is shown, but one or more single-arm robots may be used. In addition, when using multiple single-arm robots, the single-arm robots may be different types.
[0018] (Hardware configuration of the control device 100) Fig. 2 is a block diagram showing the hardware configuration of the control device. As shown in Fig. 2, the control device 100 includes a processor 120, a communication interface (hereinafter, interface is abbreviated as I / F) 121, a main memory device 122, an auxiliary memory device 123, an input / output I / F 124, and a bus 125 that communicatively connects the above-mentioned units.
[0019] The processor 120 is a central processing unit that controls the operation of each part of the control device 100. The processor 120 is, for example, a central processing unit (CPU), a digital signal processor (DSP), or an application specific integrated circuit (ASIC). The processor 120 deploys a program stored in the auxiliary storage device 123 in an executable manner in a working area of the main storage device 122. The main storage device 122 stores the program executed by the processor 120, data processed by the processor 120, and the like. The processor 120 functions as the pose measurement unit 101, the teaching data generation and correction unit 102, and the teaching data execution unit 103 by executing the program deployed in the main storage device 122. The main storage device 122 is, for example, a flash memory, a random access memory (RAM), a read only memory (ROM), and the like. The auxiliary storage device 123 stores various programs and various data. The auxiliary storage device 123 stores, for example, an OS (Operating System), various programs, various tables, and the like. The auxiliary storage device 123 is a silicon disk including a non-volatile semiconductor memory (flash memory, EPROM (Erasable Programmable ROM)), a solid state drive device, a hard disk (HDD, Hard Disk Drive) device, and the like. Input devices such as a keyboard and a mouse, not shown, and output devices such as a display are connected to the input / output I / F 124. The communication I / F 121 is communicably connected to the cameras 201 to 204 and the robot 12. Images captured by the cameras 201 to 204 are input to the processor 120 via the communication I / F 121. The images input to the processor 120 are processed by the processor 120 and recorded in the auxiliary storage device 123 as the reference path 104 and the motion path 105. In addition, the processor 120 processes the reference path 104 recorded in the auxiliary storage device 123 to generate teaching data 106 , and records the teaching data 106 in the auxiliary storage device 123 .Furthermore, the processor 120 modifies the teaching data 106, and records the modified teaching data 106 in the auxiliary storage device 123. The communication I / F 121 outputs the teaching data 106 and the modified teaching data 106 to the robot 12 in accordance with an instruction from the processor 120.
[0020] FIG. 3 is a diagram showing the arrangement of cameras 201-204 on the workbench 10. Marker plates 311 and 321 are attached to a micropipette 310 and a test tube 320 held by an instructor 11 working on the workbench 10. To output time-series data of the poses of the marker plates 311 and 321, the cameras 201-204 are installed on the opposite side of the workbench 10 to the instructor 11. The cameras 201-204 are installed at intervals from each other so that their fields of view 210 overlap and cover the workbench 10. The motion capture system performs calibration to grasp the relative positional relationship between the cameras 201-204 installed on the workbench 10 and to obtain a workbench coordinate system 2100 (Σ W In the calibration, the cameras 201 to 204 capture images of reflective markers whose positions are known multiple times. The reference path 104, the motion path 105, and the teaching data 106 are aligned in a worktable coordinate system 2100 (Σ W ) is used as a reference. The worktable coordinate system 2100 is a work environment coordinate system that represents the work environment in which the robot 12 performs work. Three markers 12m are placed on the worktable 10. The arrangement of the markers 12m is registered in advance in the control device 100, and when the robot 12 operates in front of the worktable 10, the markers 12m are captured by the stereo camera 12c, thereby determining the relationship between the robot 12 and the worktable coordinate system 2100 (Σ W ) to understand its location in relation to the
[0021] FIG. 4 is a diagram showing a state in which the marker plate is attached to a grasped object. Four reflective markers 311a to 311d and 321a to 321d are arranged on each of the marker plates 311 and 321. The reflective markers 311a to 311d and 321a to 321d are arranged asymmetrically in the vertical and horizontal directions on the marker plates 311 and 321. Each of the marker plates 311 and 321 is associated with a marker plate coordinate system 3100 (Σ RM ) and 3200(Σ LM The pose measurement unit 101 is configured with the arrangement pattern of the reflective markers 311a to 311d and 321a to 321d and the marker plate coordinate system 3100 (Σ RM ) and 3200(Σ LM The positional relationship between the worktable coordinate system 2100 (Σ W ) based on the marker plate coordinate system 3100(Σ RM ) and 3200(Σ LM) is measured. By making the reflective markers 311a to 311d and the reflective markers 321a to 321d have different arrangement patterns, the pose measurement unit 101 can simultaneously measure the poses of the marker plates 311 and 321 while distinguishing between the marker plates 311 and 321. Instead of making the arrangement patterns different, it is also possible to distinguish between the marker plates 311 and 321 by making the size and color of the reflective markers 311a to 311d different from those of the reflective markers 321a to 321d. Also, the number of reflective markers arranged on the marker plates 311 and 321 may be different as long as it is three or more. Also, although the reflective markers are arranged two-dimensionally in FIG. 3, they may be arranged three-dimensionally. The marker plate 311 is attached to the micropipette 310 via the attachment 312. The marker plate 321 is attached to the test tube 320 via the attachment 322. The attachments 312 and 322 are shaped so as not to hinder the instructor 11 from gripping and moving the micropipette 310 and the test tube 320. This allows the motion capture system to easily measure the poses of the marker plates 311 and 321 when the instructor 11 grips and moves the micropipette 310 and the test tube 320.
[0022] (Hand details) Next, the hand of the robot 12 that grasps the object to be grasped will be described in detail. The hand of the robot 12 is an example of a grasping unit of the present invention. Here, the left hand of the robot 12 that grasps the test tube 320 will be described in detail with reference to Figs. 5 and 6.
[0023] FIG. 5 is a diagram showing a coordinate system set in the left hand 620 of the robot 12 gripping the test tube 320, and FIG. 6 is a diagram showing the relationship between the coordinate system set in the left hand 620 and the coordinate system set in the marker plate 321. The left hand 620 has two fingers 620a that open and close, and grips the test tube 320 by pinching it with the recesses at the tips of these fingers 620a. A notch 620b is provided at the tip of the fingers 620a, so that the test tube 320 to which the marker plate 321 is attached can be gripped. The gripping center of the fingers 620a of the left hand 620 is defined as the left gripping part coordinate system 6200(Σ LT ) is set, and the left link tip coordinate system 7200 (Σ LE ) is set. The left link tip coordinate system is 7200 (Σ LE ) to the left grip coordinate system 6200(Σ LT ) is registered in advance in the control device 100 as a link parameter of the robot 12, and a command value to the robot 12 is given in the left gripping unit coordinate system 6200 (Σ LT ) pose. Also, the marker plate coordinate system 3200 (Σ LM ) and the left grip coordinate system 6200(Σ LT The positional relationship between the marker plate 321 and the marker plate coordinate system 3200 (Σ LM ) pose is measured, and the corresponding left grip coordinate system 6200(Σ LT In the first embodiment, the left grip coordinate system 6200 (Σ LT ) is used as teaching data 106. Here, the left hand 620 holding the test tube 320 has been described as an example, but the same applies to the right hand holding the micropipette 310.
[0024] (Operation of the control device 100) FIG. 7 is a flowchart showing the operation of the control device. As shown in FIG. 7, the control device 100 of the first embodiment executes a teaching stage in which the robot 12 is taught a task and an execution stage in which the robot 12 executes an actual task. The teaching stage in which the robot 12 is taught a task includes two steps: a generation step (see FIG. 9) in which the instructor 11 demonstrates the task and generates teaching data 106, and a correction step (see FIG. 11) in which the instructor 11 corrects the teaching data 106. In the generation step, time-series data of the position and posture of the object to be grasped when the instructor 11 grasps and moves the object to be grasped is output as a reference path 104, and teaching data of the robot 12 corresponding to the reference path 104 is generated. In the correction step, time-series data of the position and posture of the object to be grasped when the robot 12 is operated according to the teaching data 106 is output as a motion path 105 before the robot 12 executes the actual task. Then, in the correction step, the teaching data 106 is corrected so that the motion path 105 coincides with the reference path 104. After that, in the execution stage, the control device 100 causes the robot 12 to perform an actual task using the teaching data 106 or the teaching data 106 corrected in the correction step.
[0025] (Generation step) First, the generating step for generating the teaching data 106 will be described with reference to Figs. 8 and 9. Fig. 8 is a configuration diagram of the operation teaching device 1 in the generating step, and Fig. 9 is a flowchart showing the details of the generating step. In the generating step, first, the instructor 11 approaches the work table 10. The instructor 11 demonstrates the operation of gripping and moving a micropipette 310 and a test tube 320 to which marker plates 311 and 321 are attached. The pose measurement unit 101 measures the poses of the marker plates 311 and 321 at a predetermined sampling period (e.g., 10 Hz) and calculates the poses of the marker plate coordinate systems 3100 and 3200 (Σ RM , Σ LM) is output and recorded as a reference path 104 (S901). The time series data of the poses of the marker plates 311 and 321 is an example of a reference path (time series data of the position and posture of the object to be grasped) of the present invention. Next, the teaching data generating / modifying unit 102 generates teaching data 106 for the robot 12 corresponding to the reference path 104. Specifically, the teaching data generating / modifying unit 102 calculates the gripping unit coordinate system 6200 (Σ LT , Σ RT ) is generated as the teaching data 106 (S902). The teaching data generating and correcting unit 102 also checks whether the generated teaching data 106 can be executed (S903). Specifically, the teaching data generating and correcting unit 102 performs inverse kinematic calculations on the teaching data 106 assuming that the robot 12 is installed at a predetermined position in front of the workbench 10. Then, it checks whether a solution for the joint angles of the robot 12 can be obtained, whether the obtained joint angle solution is not discontinuous, and whether the obtained joint angle solution causes interference between the robot 12 itself or the robot 12 and the surrounding environment. If there are no problems and the teaching data 106 can be executed, the teaching data 106 is recorded as data that can be executed by the robot 12 (S904). This completes the generation step. If it is determined in S903 that execution of the teaching data 106 is impossible, this is notified to the instructor 11, and the measurement of the reference path 104 (S901) is performed again.
[0026] (Correction Step) Next, the correction step for correcting the teaching data 106 will be described with reference to Figs. 10 and 11. The necessary calculations will be described in detail later, and the flow of the procedure will be described here. Fig. 10 is a configuration diagram of the work teaching device 1 in the correction step, and Fig. 11 is a flowchart showing the details of the correction step. In the correction step, the instructor 11 leaves the work table 10, and instead the robot 12 moves to a position in front of the work table 10 by an automatic guided vehicle. The robot 12 grasps the micropipette 310 and the test tube 320 to which the marker plates 311 and 321 are attached. The robot 12 measures the marker 12m on the work table 10 with the stereo camera 12c, and calculates the coordinate system 2100(Σ W ) (S1101). The position error of the robot 12 relative to the workbench 10 is thereby grasped. Next, the teaching data executing unit 103 operates the robot 12 using the teaching data 106 recorded in the generating step, and grips and moves the micropipette 310 and the test tube 320 to which the marker plates 311 and 321 are attached. At this time, the pose measuring unit 101 measures and records time series data of the poses of the marker plates 311 and 321 as the movement path 105 (S1102). This time series data of the poses of the marker plates 311 and 321 is an example of the movement path (time series data of the position and posture of the grasped object) of the present invention. The time series data of the poses of the marker plates 311 and 321 is time series data of the position and posture of the grasped object of the present invention.
[0027] When measuring the movement path 105 in S1102, the teaching data 106 is not executed as is, but the movement of the robot 12 is stopped at several points, and time-series data of the poses of the marker plates 311 and 321 at the time of stopping is measured as the movement path 105. Next, the teaching data generating and correcting unit 102 calculates an error (hereinafter referred to as a positioning error) between the reference path 104 and the movement path 105 (S1103). If this positioning error exceeds a predetermined specified value (S1104: Yes), the teaching data generating and correcting unit 102 corrects the teaching data 106 using the positioning error (S1105). The corrected teaching data 106 is recorded in the auxiliary storage device 123. Then, the teaching data executing unit 103 operates the robot 12 using the corrected teaching data 106, and again measures the movement path 105 (S1102) and calculates the positioning error (S1103). If the recalculated positioning error exceeds a predetermined value, or until the number of times the teaching data 106 has been corrected exceeds a predetermined number, steps S1102 to S1105 are repeated. If the positioning error is within a predetermined value, or if the number of times the teaching data 106 has been corrected exceeds a predetermined number, the teaching data 106 or the corrected teaching data 106 is recorded (S1106). This completes the correction step.
[0028] (Steps to be taken) Next, an actual task performed by the robot 12 using the teaching data 106 or the corrected teaching data 106 will be described with reference to FIG. 12. FIG. 12 is a configuration diagram of the task teaching device 1 that causes the robot 12, which has completed the teaching stage, to perform an actual task. After the teaching stage is completed, the marker plates 311 and 321 are removed from the micropipette 310 and the test tube 320. In addition, the cameras 201 to 204 are removed from the workbench 10. The robot 12 moves to a position in front of the workbench 10. The robot 12 measures the marker 12m on the workbench 10 with the stereo camera 12c, and calculates the coordinates of the marker 12m in the workbench coordinate system 2100(Σ W) is recognized. Any error in the position of the robot 12 relative to the workbench 10 is thereby grasped. Next, the teaching data execution unit 103 causes the robot 12 to perform an actual task using the teaching data 106 or the corrected teaching data 106. The robot 12 operates according to the teaching data 106 or the corrected teaching data 106, and grasps and moves the micropipette 310 and the test tube 320.
[0029] (Calculation method of teaching data 106 and corrected teaching data 106) Details of the method for calculating the teaching data 106 and the corrected teaching data 106 will be described with reference to Figures 13 and 14. Note that the reference path 104, the movement path 105, and the teaching data 106 all include data for both arms of the robot 12, but for simplicity, the left arm will be described as an example here.
[0030] First, a method for obtaining the teaching data 106 from the reference path 104 will be described. Fig. 13 is a schematic diagram of the reference path 104 measured in the generation step. The reference path 104 measured in S901 is time-series data of the pose of the marker plate 321 measured at a predetermined sampling period by a motion capture system, and is expressed by the following equation.
number
[0031] In formula 1, the upper left subscript "W" represents the worktable coordinate system 2100(Σ W ) is the reference, and the subscript "LM" at the bottom right indicates that the marker plate coordinate system 3200(Σ LM The argument "i" indicates the order of the time series data. The subscripts in the formulas below have the same meaning. W T LM (i) is a 4×4 homogeneous transformation matrix, W R LM (i) is a 3x3 rotation matrix that represents the pose, W p LM(i) is a 3×1 position vector that represents the position. The orientation may be expressed using Euler angles or quaternions in addition to a rotation matrix. Fig. 13 shows an example in which only the X coordinate 104x of the orientation and position included in the reference path 104 is illustrated, and x(i) is arranged in a predetermined sampling period Δt. In S902, the teaching data 106 is generated from the reference path 104 by the following formula.
number
[0032] In formula 2, LT T LM is the left grip coordinate system 6200(Σ LT ) from the marker plate coordinate system 3200(Σ LM ) pose, which is known from Figure 6. W T LT (i) is the worktable coordinate system 2100(Σ W ) based on the left grip coordinate system 6200(Σ LT ) pose time series data, which is the teaching data 106.
[0033] Next, a method for correcting the teaching data 106 from the movement path 105 in the correction step will be described. Fig. 14 is a schematic diagram of the movement path 105 measured in the correction step. The movement path 105 is time-series data of the pose of the marker plate 321 measured by moving the robot 12 using the teaching data 106 in S1102 and stopping the robot 12 every few points, and is expressed by the following equation with a prime added to the right shoulder of Equation 1.
number
[0034] FIG. 14 is an example showing only the X coordinate 105x of the positions and orientations included in the movement path 105, similarly to FIG. 13, and is measured at every two points, that is, i=1, 4, 7 . . .
[0035] The positioning error in S1103 is the difference between the reference path 104 and the motion path 105, and at i where the motion path 105 is measured, the position error p e (i) and attitude error R e (i) is defined as follows:
number
number
[0036] As can be seen from Equations 4 and 5, the position error p e (i) is the simple difference between position vectors, and the attitude error R e (i) is a rotation matrix of the orientation of the motion path 105 seen from the orientation of the reference path 104. For i where the motion path 105 is not measured, the positioning error is calculated by interpolating the positioning errors obtained by Equations 4 and 5. The interpolation method may be any interpolation method such as linear interpolation or spline interpolation. However, if the orientation error R e Since (i) is a 3×3 rotation matrix, if the matrix format is used for the interpolation calculation, the constraints of the rotation matrix may not be satisfied. Therefore, by converting it into another representation format such as Euler angles or quaternions, performing the interpolation calculation, and then converting it back into a rotation matrix, the position error p e (i) and attitude error R e (i) can be obtained.
[0037] If the positioning error exceeds a specified value, the positioning error is used to correct the teaching data 106. The corrected teaching data 106 is obtained by the following equation so that the motion path 105 coincides with the reference path 104.
number
[0038] It is obtained by Equation 6 W T LT(i) is the teaching data 106 corrected due to the positioning error. The robot is operated using the corrected teaching data 106, and the newly measured motion path 105 is expressed as the following equation by adding a prime to the right shoulder of Equation 3.
number
number
number
[0039] If the positioning error exceeds the specified value again in S1104, the corrected teaching data 106 is corrected in S1105. The calculation of the teaching data 106 at this time is expressed by the following equation using both the first positioning error of the equations 4 and 5 and the second positioning error of the equations 8 and 9.
number
[0040] As described above, by repeating the measurement of the movement path 105 and the correction of the teaching data 106, it is possible to obtain the corrected teaching data 106 in which the movement path 105 approximately coincides with the reference path 104.
[0041] (Effects of Example 1) In the first embodiment, the correction step S702 is executed before the robot 12 performs the actual work. As a result, during the actual work by the robot 12, it is not necessary to provide the marker plates 311 and 321 and the cameras 201 to 204 that capture the images of the marker plates 311 and 321, so that the robot can be made to perform the actual work with a simple configuration.
[0042] In addition, in the first embodiment, by executing the correction step S702 before the robot 12 actually performs the work, the instructor 11 can teach the robot 12, which performs the work at high speed, with high accuracy.
[0043] In the first embodiment, the robot 12 is located in the worktable coordinate system 2100 (Σ W ) and then the worktable coordinate system 2100 (Σ W ) as a reference. As a result, even if the accuracy of the position of the robot 12 relative to the worktable 10 is poor, the robot 12 operates according to the worktable coordinate system 2100 (Σ W ) makes it possible to grasp the error in the position of the robot 12 relative to the worktable 10. As a result, the absolute positioning accuracy of the object to be grasped by the robot 12 is guaranteed.
[0044] In the first embodiment, the four reflective markers 311a to 311d and 321a to 321d of the marker plates 311 and 321 are captured by the cameras 201 to 204, so that the three-dimensional position and orientation of the object to be grasped can be easily measured.
[0045] In the first embodiment, since both the reference path 104 and the motion path 105 are time-series data of the position and posture of the object to be grasped, the position and posture of the object to be grasped held by the instructor 11 and the position and posture of the object to be grasped held by the robot 12 can be accurately determined. match It can be done.
[0046] In the first embodiment, a correction step is executed when the error between the reference path 104 and the motion path 105 is greater than a specified value, so that it is possible to prevent unnecessary correction steps from being executed. Also, in the correction step, the teaching data 106 can be corrected using the error between the reference path 104 and the motion path 105, so that the teaching data 106 can be appropriately corrected according to the error.
[0047] In the first embodiment, it is possible to obtain teaching data 106 in which the error (positioning error) between the reference path 104 when the instructor 11 demonstrates the work in the generation step and the motion path 105 when the robot 12 is operated in the correction step is within a specified value. This makes it possible to cause the robot to perform a highly accurate work that matches the work demonstrated by the instructor 11. Note that the cause of the positioning error includes various factors such as an error caused by the actual link parameters of the robot 12 not strictly matching those in the control device 100, an error caused by deflection deformation due to the weight of the robot 12 or the object to be grasped, a measurement error of the motion capture system, and an error in the mounting positions of the marker plates 311 and 321. However, in the first embodiment, the teaching data 106 is corrected so that the reference path 104 and the motion path 105 when the instructor 11 and the robot 12 grasp and move the object to be grasped in the same measurement environment are matched. This makes it possible to eliminate the influence of any of the errors described above and to operate the robot 12 according to the reference path 104 taught by the instructor 11.
[0048] In the first embodiment, an example is shown in which an optical motion capture system is used to measure the pose of the grasped object, but other non-contact three-dimensional measurement means may be used. For example, a magnetic three-dimensional measurement device, a laser tracker, a system that analyzes an image of the instructor 11's movement captured by a camera, etc. are listed. By using a non-contact three-dimensional measurement means, the instructor 11 can teach the work without being restricted by the measurement means.
[0049] In the first embodiment, in S901, the reference route 104 is recorded as time series data measured at a predetermined sampling period. However, the sampling period does not necessarily have to be a predetermined period, and representative points of the trajectory may be recorded. By recording only characteristic points as time series data, the size of the data to be handled can be reduced.
[0050] Furthermore, when generating the teaching data 106 in S902, the time widths of the time-series data of the reference path 104 and the teaching data 106 do not need to match. For example, the reference path 104 may be downsampled before generating the teaching data 106. This can reduce the data size of the teaching data 106, and ultimately reduce the data required to operate the robot 12.
[0051] In the first embodiment, when measuring the movement path 105 in S1102, the robot 12 is operated using the teaching data 106, and the movement of the robot 12 is stopped every few points and measured by the pose measurement unit 101. However, it may be stopped at each point for measurement. In this case, it takes time to measure the movement path 105, but the positioning error at each point can be obtained without interpolation. Furthermore, when stopping at each point, the robot 12 is operated at a low speed, so that it is possible to check whether the robot 12 operates safely. Note that the time interval when stopping the robot 12 at every few points does not have to be constant. For example, the movement amount between each point of the teaching data 106 is obtained in advance, and the robot 12 is stopped every time a predetermined movement amount is exceeded to measure the movement path 105. This prevents the robot 12 from being stopped more than necessary at points where the movement amount is small, and the time required for the correction step can be shortened.
[0052] <Example 2> In the second embodiment, a robot work teaching device that can teach work more easily than in the first embodiment will be described with reference to Figs. 15 and 16. Fig. 15 is a diagram showing a coordinate system set in the left hand 620 of the robot 12 in the second embodiment, and Fig. 16 is a diagram showing the relationship between the coordinate system set in the left hand 620 and the coordinate system set in the marker plate 321. In Figs. 15 and 16, the same components or components having the same functions as those in Figs. 5 and 6 are given the same reference numerals and redundant description is omitted. The left hand 620 of the robot 12 in the second embodiment has two fingers 620d and 620e that open and close, and the test tube 320 is pinched and grasped by the recesses at the tips of these fingers 620d and 620e. A marker plate installation part 620f on which the marker plate 321 can be installed is formed on the side of the finger 620e.
[0053] The procedure of work teaching in the second embodiment, like the first embodiment, has two steps: a generation step in which the instructor 11 demonstrates the work and generates teaching data 106, and a correction step in which the instructor 11 has the robot 12 perform the work and correct the teaching data 106. After that, the instructor 11 has the robot 12 perform the work using the teaching data 106 corrected in the correction step. Below, the differences from the first embodiment will be described in detail. The procedure of the generating step is the same as that of Example 1. Note that the time series data of the poses of the marker plates 311 and 321 generated in the generating step of Example 2 is an example of a reference path (time series data of the position and posture of the gripping part) of the present invention.
[0054] (Correction Step) The procedure of the correction step will be described with reference to the flowchart in Fig. 11. In the correction step, the instructor 11 leaves the work table 10, and instead moves the robot 12 to a predetermined position in front of the work table 10 by an automated guided vehicle. Unlike the first embodiment, the robot 12 does not hold anything, and a marker plate 321 is placed on the finger 620e of the left hand 620 as shown in Fig. 16. The marker plate coordinate system 3200(Σ LM ) and the left grip coordinate system 6200(Σ LTThe width of the spread between fingers 620d and 620e is adjusted so that the positional relationship between the left hand 620 and the test tube 320 is consistent with that when the left hand 620 is gripping the test tube 320. Similarly, a marker plate 311 is placed on the right hand, and the width of the spread between the fingers of the right hand is adjusted. The robot 12 measures the marker 12m on the workbench 10 with the stereo camera 12c, and calculates the position of the marker 12m in the workbench coordinate system 2100 (Σ W ) (S1101). This allows the error in the position of the robot 12 relative to the workbench 10 to be grasped. Next, the teaching data execution unit 103 operates the robot 12 using the teaching data 106 recorded in the generation step, and operates both hands to which the marker plates 311 and 321 are attached. At this time, the pose measurement unit 101 measures and records time series data of the poses of the marker plates 311 and 321 as the movement path 105 (S1102). The time series data of the poses of the marker plates 311 and 321 is an example of a movement path (time series data of the position and posture of the gripping parts) of the present invention.
[0055] When measuring the movement path 105 in S1102, the teaching data 106 is not executed as it is, but the movement of the robot 12 is stopped at several points, and time-series data of the poses of the marker plates 311 and 321 at the time of stopping is measured as the movement path 105. Next, the teaching data generating and correcting unit 102 calculates an error (positioning error) between the reference path 104 recorded in the generating step and the movement path 105 recorded in the correcting step (S1103). If this positioning error exceeds a predetermined specified value (S1104: Yes), the teaching data generating and correcting unit 102 corrects the teaching data 106 using the positioning error (S1105). The corrected teaching data 106 is recorded in the auxiliary storage device 123. Then, the teaching data executing unit 103 operates the robot 12 using the corrected teaching data 106, and again measures the movement path 105 (S1102) and calculates the positioning error (S1103). If the recalculated positioning error exceeds a predetermined value, or until the number of times the teaching data 106 has been corrected exceeds a predetermined number, steps S1102 to S1105 are repeated. If the positioning error is within a predetermined value, or if the number of times the teaching data 106 has been corrected exceeds a predetermined number, the teaching data 106 or the corrected teaching data 106 is recorded (S1106). This completes the correction step.
[0056] (Steps to be taken) Next, an actual work performed by the robot 12 using the teaching data 106 or the corrected teaching data 106 will be described. After the teaching stage is completed, the marker plates 311 and 321 are removed from both hands of the robot 12. Also, the cameras 201 to 204 are removed from the work table 10. The robot 12 is placed in front of the work table 10. The robot 12 measures the marker 12m on the work table 10 with the stereo camera 12c, and calculates the coordinates of the work table coordinate system 2100(Σ W) is recognized. Any error in the position of the robot 12 relative to the workbench 10 is thereby grasped. Next, the teaching data execution unit 103 operates the robot 12 using the teaching data 106 or the corrected teaching data 106, and causes the robot 12 to grasp and move the micropipette 310 and the test tube 320. The manner in which the taught robot 12 grasps the test tube 320 with its left hand during actual work is as shown in FIG. 15.
[0057] (Effects of Example 2) In the second embodiment, since it is not necessary to make the robot 12 grasp the object to be grasped in the correction step, when there are many types of objects to be grasped, the man-hours required for work teaching can be reduced.
[0058] In addition, in the second embodiment, the marker plates 311 and 321 are removed from both hands of the robot 12 when the robot 12 performs an actual work. However, if the marker plates 311 and 321 do not interfere with the actual work, they do not need to be removed. If two sets of marker plates 311 and 321 are prepared, one for work teaching by the instructor 11 and the other for attachment to both hands of the robot 12, the transition from the generation step to the correction step during work teaching can be simplified. Also, the man-hours for work teaching can be reduced. The other effects are the same as those of the first embodiment.
[0059] <Modification> The present disclosure is not limited to the above-described embodiments, and includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present disclosure, and it is not necessary to include all of the configurations described. In addition, a part of an embodiment can be replaced with a configuration of another embodiment. In addition, a configuration of another embodiment can be added to a configuration of an embodiment. In addition, a part of the configuration of each embodiment can be added to, deleted from, or replaced with a part of the configuration of another embodiment. [Explanation of symbols]
[0060] 1...robot work teaching device, 10...work table, 11...teacher, 12...robot, 12c...stereo camera, 12m...marker, 100...control device, 101...pose measurement unit, 102...teaching data generation / correction unit, 103...teaching data execution unit, 104...reference path, 105...motion path, 106...teaching data, 201-204...camera, 210...camera field of view, 310...micropipette, 320...test tube, 311, 321...marker plate, 311a-311d, 321a-321d...reflective marker, 312, 322...attachment, 620...left hand, 2100...work table coordinate system, 3100, 3200...marker plate coordinate system, 6200...left gripping part coordinate system, 7200...left link tip coordinate system
Claims
1. A work teaching method executed by a work teaching device that has a gripping unit that grips a gripped object and teaches a work to a robot that moves the gripped object, comprising: The work teaching method includes: a first step of providing a marker for defining a coordinate system of a working environment and attaching a marker for measuring the object to be grasped; a second step of detecting measurement markers of the object to be grasped when an instructor grasps and moves the object in the working environment by an imaging means installed in the working environment, outputting first time series data of the position and posture of the object to be grasped, and generating first teaching data of the operation of the robot corresponding to the first time series data; a third step of moving the robot to the working environment after the second step; a fourth step of detecting, after the third step and before the robot performs an actual task, a measurement marker of the object to be grasped when the robot is operated using the first teaching data by an imaging means installed in the working environment, and outputting second time-series data of the position and orientation of the object to be grasped or the gripping unit; a fifth step of generating second teaching data of an operation of the robot according to a difference between the first time-series data and the second time-series data; having the first time series data, the second time series data, the first teaching data, and the second teaching data are generated based on the coordinate system; the third step includes detecting a marker defining a coordinate system of the working environment by an imaging means provided in the robot, and causing the robot to recognize the coordinate system of the working environment; A work teaching method comprising:
2. A marker defining a coordinate system of the working environment is provided on the work table.
2. The operation teaching method according to claim 1 .
3. the second step includes outputting the first time-series data from a plurality of images captured by a plurality of imaging means installed in the work environment where the robot performs the work, the plurality of images capturing measurement markers of the object to be grasped and moved by the instructor, The fourth step includes outputting the second time-series data from a plurality of images captured by the plurality of imaging means, the plurality of images capturing measurement markers of the object to be grasped and moved by the robot.
2. The operation teaching method according to claim 1 .
4. A work teaching method executed by a work teaching device that has a gripping unit that grips a gripped object and teaches a work to a robot that moves the gripped object, comprising: The work teaching method includes: A first step of providing a coordinate system marker in a working environment; a second step of outputting first time series data of a position and a posture of the object to be grasped when an instructor grasps and moves the object to be grasped in the working environment, and generating first teaching data of an operation of the robot corresponding to the first time series data; a third step of moving the robot to the working environment after the second step; a fourth step of outputting, after the third step and before the robot performs an actual task, second time-series data of the position and orientation of the object to be grasped or the gripping unit when the robot is operated using the first teaching data; and a fifth step of generating second teaching data of an operation of the robot according to a difference between the first time-series data and the second time-series data; having the first time series data, the second time series data, the first teaching data, and the second teaching data are generated based on the coordinate system; the third step includes causing the robot to measure a marker of the coordinate system and recognize the coordinate system; the second step includes outputting the first time-series data from a plurality of images of the object to be grasped and moved by the instructor, the images being captured by a plurality of imaging means installed in the work environment where the robot performs the work, the fourth step includes outputting the second time-series data from a plurality of images of the object to be grasped and moved by the robot, the plurality of images being captured by the plurality of imaging means; The plurality of imaging means images three or more markers attached to the object to be grasped or three or more markers attached to the grasping unit. A work teaching method comprising:
5. The first time series data and the second time series data are time series data of a position and a posture of the object to be grasped.
2. The operation teaching method according to claim 1 .
6. The generation of the second teaching data is executed when an error between the second time series data and the first time series data is larger than a specified value.
2. The operation teaching method according to claim 1 .
7. In the fifth step, the second teaching data is generated by using an error between the second time series data and the first time series data.
2. The operation teaching method according to claim 1 .
8. A marker for defining a coordinate system in a working environment; a robot capable of moving to the working environment, having a gripping unit that grips an object to be grasped, and moving the object to be grasped that is gripped by the gripping unit; an output means for detecting a measurement marker of the object to be grasped by an image capturing means installed in the working environment when an instructor grasps and moves the object to be grasped in the working environment, and outputting first time-series data of a position and an orientation of the object to be grasped; a generating means for generating first teaching data of a motion of the robot corresponding to the first time-series data, the output means detects, with an imaging means installed in the working environment, a measurement marker of the object to be grasped when the robot is operated using the first teaching data, before the robot performs an actual task, in a state in which the robot has been moved to the working environment, and outputs second time-series data of a position and an orientation of the object to be grasped or the gripping unit; the generating means generates second teaching data of an operation of the robot according to a difference between the first time-series data and the second time-series data before an actual operation is performed by the robot; the first time series data, the second time series data, the first teaching data, and the second teaching data are based on the coordinate system; the robot detects a marker that defines a coordinate system of the working environment by an imaging means provided in the robot, and recognizes the coordinate system of the working environment; A work instruction system comprising:
9. A marker defining a coordinate system of the working environment is provided on the work table.
9. The operation teaching system according to claim 8,
10. The output means for the first time series data and the second time series data outputs the first time series data and the second time series data from a plurality of images of the measurement markers of the grasped object captured by an imaging means installed in the work environment.
9. The operation teaching system according to claim 8,
11. A coordinate system marker provided in the work environment; a robot capable of moving to the working environment, having a gripping unit that grips an object to be grasped, and moving the object to be grasped that is gripped by the gripping unit; an output means for outputting first time series data of a position and an orientation of the object to be grasped when an instructor grasps and moves the object to be grasped in the working environment; a generation unit that generates first teaching data of a motion of the robot corresponding to the first time-series data, the output means outputs, with the robot moved to the working environment and before the robot performs an actual task, second time-series data of the position and orientation of the object to be grasped or the gripping unit when the robot is operated using the first teaching data; the generating means generates second teaching data of an operation of the robot according to a difference between the first time-series data and the second time-series data before an actual operation is performed by the robot; the first time series data, the second time series data, the first teaching data, and the second teaching data are based on the coordinate system; The robot measures a marker of the coordinate system and recognizes the coordinate system; The work teaching system further includes a plurality of imaging means installed in a work environment in which the robot performs work, The output means outputs the first time series data and the second time series data from a plurality of images of the object to be grasped captured by the imaging means, The imaging means images three or more markers attached to the object to be grasped or three or more markers attached to the grasping unit. A work instruction system comprising:
12. The first time series data and the second time series data are time series data of a position and a posture of the object to be grasped.
9. The operation teaching system according to claim 8,
13. The generating means generates the second teaching data when an error between the second time series data and the first time series data is larger than a specified value.
9. The operation teaching system according to claim 8,
14. The generating means generates the second teaching data by using an error between the second time series data and the first time series data.
9. The operation teaching system according to claim 8,
Citation Information
Patent Citations
Robot motion optimization system and method
EP3733355A1
Teaching device for industrial robot
JP1994250730A
Teaching point tracing type device with operation program correction device
JP1994324732A
Control device for robot
JP2011062793A
Information processing device, robot, scenario information creation method and program
JP2015054378A
