ROBOT SYSTEM AND METHOD FOR CONTROLLING ROBOT SYSTEM
The robot system addresses calibration inaccuracies by using encoders and a control device to align the tool tip with a reference coordinate system, ensuring precise calibration and reducing picking failures through automated adjustments.
Patent Information
- Application Number
- JP2021042526
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Filing Date
- 2021-03-16
- Publication Date
- 2024-11-20
- Estimated Expiration
- 2041-03-16
AI Technical Summary
Existing robot calibration methods suffer from inaccuracies due to lens distortion in scanners, which affects the precision of calibration processes.
A robot system equipped with encoders, a sensor, and a control device that uses a recognition unit to align the tool tip with a reference coordinate system, adjusting the robot's posture to compensate for lens distortion, and calculating a tool offset based on encoder values.
This approach enables highly accurate calibration of the robot system, reducing picking failures and improving throughput by automating the calibration process without human intervention.
Smart Images

Figure 0007587448000001 
Figure 0007587448000002 
Figure 0007587448000003
Abstract
Description
[Technical field]
[0001] The present invention relates to a robot system and a control method for a robot system. [Background technology]
[0002] The technology disclosed in the following Patent Document 1 is a technology related to a robot system in which a robot recognizes, judges, and operates by itself. This Patent Document 1 states that "The robot system 1 includes a scanner 2, a robot 3, and a control device 5, and uses a sphere 70 fixed to the end effector 40 to perform calibration from the scanner coordinate system to the robot coordinate system with high accuracy and automatically. ... The control device 5 acquires, from the scanner 2, point cloud data of the sphere 70 when the hand unit 42 is moved to various positions and postures, and estimates the position of the sphere 70. The control device 5 estimates the rotation component RV and translation component SV of the calibration based on the position data of the sphere 70 estimated in each calculation process." [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2017-100202 A Summary of the Invention [Problem to be solved by the invention]
[0004] In the above-mentioned technology, the spherical point cloud data is acquired from the scanner when the hand unit is moved to various positions and orientations. Therefore, the data acquired from the scanner contains distortion of the lens that constitutes the scanner, which is a factor that reduces the accuracy of the calibration.
[0005] SUMMARY OF THE PRESENT EMBODIMENTS Accordingly, an object of the present invention is to provide a robot system capable of performing highly accurate calibration and a method for controlling the robot system. [Means for solving the problem]
[0006] In order to solve the above problems, for example, the configurations described in the claims are adopted. The present application includes multiple means for solving the above-mentioned problems. One example is a robot system having a multi-jointed robot equipped with multiple encoders, a sensor that captures an image of a working range of the robot, and a control device that controls driving of the robot based on information from the sensor, wherein the control device is a robot system including a recognition unit that recognizes coordinates in a sensor coordinate system of a tip of a work tool attached to the robot based on the information from the sensor, a control unit that sets the tip position of the tool relative to the robot as a tool offset and changes the posture of the robot based on the tool offset and the information from the recognition unit so that the tip of the tool coincides with a reference coordinate set in the sensor coordinate system, an encoder reading unit that reads encoder values from an encoder of the robot in each of the changed postures, and a tool offset calculation unit that calculates a calibration value of the tool offset based on the encoder values in each of the postures read by the encoder reading unit. Effect of the Invention
[0007] According to the present invention, it is possible to provide a robot system capable of performing highly accurate calibration and a method for controlling the robot system. [Brief description of the drawings]
[0008] [Figure 1] 1 is a configuration diagram of a robot system according to a first embodiment. [Diagram 2] FIG. 1 is a configuration diagram of a main part of a robot system according to a first embodiment. [Diagram 3] FIG. 1 is a functional block diagram of a robot system according to a first embodiment. [Figure 4] 4 is a diagram showing an example of coordinate data of a tip of a tool stored in a tip coordinate storage unit of the robot system according to the first embodiment. FIG. [Diagram 5]4A to 4C are diagrams illustrating a first example of posture control of the robot according to the first embodiment. [Figure 6] 5A to 5C are diagrams illustrating a second example of posture control of the robot according to the first embodiment. [Figure 7] 5A to 5C are diagrams illustrating an example of control of the tool tip in each posture of the robot according to the first embodiment. [Figure 8] 3 is a diagram showing an example of robot posture data stored in a robot posture storage unit of the robot system according to the first embodiment. FIG. [Figure 9] 4 is a flowchart showing a control method for the robot system according to the first embodiment. [Figure 10] 5A to 5C are diagrams illustrating a procedure for making the X coordinate of the tip position of the tool coincide with the reference coordinate in the control method of the first embodiment. [Figure 11] 5A to 5C are diagrams illustrating a procedure for making the Y coordinate of the tip position of the tool coincide with the reference coordinate in the control method of the first embodiment. [Figure 12] 5A to 5C are diagrams illustrating a procedure for making the Z coordinate of the tip position of the tool coincide with the reference coordinate in the control method of the first embodiment. [Figure 13] FIG. 1 is a configuration diagram (part 1) of a main part of a robot system according to a second embodiment. [Figure 14] FIG. 11 is a second configuration diagram of a main part of a robot system according to a second embodiment. [Figure 15] FIG. 11 is a configuration diagram of a main part of a robot system according to a third embodiment. [Figure 16] FIG. 11 is a diagram for explaining a main part of a robot system according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0009] Hereinafter, embodiments to which the present invention is applied will be described in detail with reference to the drawings. In each embodiment, the same components are given the same reference numerals, and duplicated descriptions of the same components will be omitted.
[0010] First Embodiment -Robot System- FIG. 1 is a configuration diagram of a robot system 1 according to a first embodiment. The robot system 1 shown in this figure is a picking system in which an autonomous robot that recognizes, judges, and operates itself picks and places a target workpiece [W]. Pick and place is a series of operations in which the target workpiece [W] is held (picked), moved, and placed (placed) in a desired position. Such a robot system 1 includes a robot 10, a sensor 20, an input device 30, a control device 40, a supply conveyor 50, and a placement table 60. Next, the configuration of these components will be described in order.
[0011] <Robot 10> The robot 10 holds a workpiece [W] supplied from a supply conveyor 50, and moves the held workpiece [W] to place it on the top of a mounting table 60. As an example, the robot 10 is a multi-joint robot in which three joints 11, 12, and 13 and three arms 14, 15, and 16 are alternately connected. The robot 10 is also equipped with an encoder (not shown), and is a six-axis robot in which the angles of the rotational and swinging directions of the joints 11, 12, and 13 are freely controlled by feedback of measurements (encoder values) measured by the encoder.
[0012] Various tools 100 for holding a workpiece [W] are attached to such a robot 10. In the robot 10, the position where the tool 100 is attached is, for example, the end of the arm 16 on the tip side of the robot 10, which is a part called the wrist, but may be another part.
[0013] FIG. 2 is a configuration diagram of the main parts of the robot system according to the first embodiment, and is a diagram for explaining the configuration of a tool 100 attached to the robot 10. The tool 100 may be any tool capable of holding a workpiece [W], and in this example, the tool 100 is a bellows that holds the workpiece [W] by suction. The tool 100 has a cylindrical outer diameter shape extended from the arm 16 on the tip side of the robot 10, and the bottom surface on the tip side serves as a suction surface 101. The suction surface 101 has a suction opening 102 in the center, and holds the workpiece [W] (see FIG. 1) on the suction surface 101 by suction from the suction opening 102.
[0014] <Sensor 20> Returning to FIG. 1, the sensor 20 is for acquiring information on the position and shape of the workpiece [W] supplied from the supply conveyor 50. Such a sensor 20 is a camera having a lens, for example, that recognizes the surrounding image in two dimensions or three dimensions, and is preferably a two-dimensional camera in terms of the accuracy of the position information. The two-dimensional camera outputs, for example, an RGB image as sensing data. The sensor 20 has the location of the robot 10, the supply conveyor 50, and the placement table 60 as the position measurement range, and is installed, for example, on a ceiling or wall. Also, a configuration in which information on the position measurement range is acquired by multiple sensors 20 may be used. Furthermore, sensors other than cameras may be attached.
[0015] <Input device 30> The input device 30 is a device for inputting instructions for the operation of the robot 10, and is, for example, a touch panel type input terminal, a keyboard, or an operation button. The operations of the robot 10 instructed by the input device 30 are, here, at least a picking operation and a calibration operation of the tool 100. Here, the calibration operation of the tool 100 is an operation for calibrating (correcting errors) the tip position of the tool 100 relative to the attachment position of the tool 100 in the robot 10, i.e., the tool offset. Such an input device 30 may be formed integrally with the robot 10, or may be formed integrally with a control device 40 described next.
[0016] <Control device 40> The control device 40 performs control for executing a picking operation or a calibration operation based on information input from the input device 30 and information acquired by the sensor 20. Such a control device 40 is configured by a calculator. The calculator is hardware used as a so-called computer. The calculator includes a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). Furthermore, the calculator includes a non-volatile storage and a network interface.
[0017] Fig. 3 is a functional block diagram of the robot system 1 according to the first embodiment, and is a block diagram mainly for explaining the functional configuration of the control device 40. The configuration of the control device 40 will be explained below based on Fig. 3 with reference to Figs. 1 and 2 and other necessary drawings. Note that each unit constituting the control device 40 may be formed integrally with the robot 10 in part or in whole, or may be provided in the cloud.
[0018] The control device 40 is connected to the robot 10, the sensor 20, and the input device 30. This connection may be wired or wireless. The control device 40 also has a recognition unit 41, a reference coordinate holding unit 42, a trajectory planning unit 43, a control unit 44, an encoder reading unit 45, a robot posture storage unit 46, and a tool offset calculation unit 47. Each of these functional units executes the following functions by the CPU of the computer constituting the control device 40 reading out a program stored in the ROM.
[0019] [Recognition unit 41] The recognition unit 41 recognizes the position in the sensor coordinate system of each part recognized by the sensor 20 based on the information (sensing data) acquired from the sensor 20.
[0020] When a picking operation instruction is input from the input device 30, the recognition unit 41 recognizes the position of the workpiece [W] (see Figure 1) in the sensor coordinate system based on the information obtained from the sensor 20, and sends the recognized position information to the trajectory planning unit 43.
[0021] Furthermore, when an instruction for a calibration operation is input from the input device 30, the recognition unit 41 recognizes the position of the tip [P] of the tool 100 attached to the robot 10 in the sensor coordinate system based on the information acquired from the sensor 20. In this case, the tip [P] of the tool 100 (see FIG. 2) is an arbitrary set point on the tip side of the tool 100, and in the illustrated example, it is, for example, the center point of the suction surface 101 of the tool 100 and the center point of the suction opening 102.
[0022] When recognizing the tip [P] of the tool 100, the recognition unit 41 first recognizes the periphery of the suction surface 101 centered on the tip [P] as a marker [M] based on sensing data acquired from the sensor 20. The periphery of the suction surface 101 that becomes the marker [M] may be colored so as to be easily recognized by the sensor 20. After recognizing the marker [M], the recognition unit 41 calculates the position of the tip [P] in the sensor coordinate system based on information on the positional relationship between the marker [M] and the tip [P] that is held by, for example, prior input, and recognizes the calculated position as the coordinates of the tip [P] (tip coordinates).
[0023] Furthermore, when an instruction for a calibration operation is input from the input device 30, the recognition unit 41 transmits the tip coordinates recognized the first time to the reference coordinate holding unit 42. Furthermore, the recognition unit 41 transmits the tip coordinates recognized in times other than the first time to the trajectory planning unit 43. Note that the timing of recognition of the position (tip coordinates) of the tip [P] of the tool 100 by the recognition unit 41 during the calibration operation will be described in detail below in the control method of the robot system.
[0024] [Reference coordinate holding unit 42] The reference coordinate holding unit 42 holds the tip coordinates transmitted from the recognition unit 41 as the reference coordinates [Sp] of the tip [P] of the tool 100. FIG. 4 is a diagram showing an example of the reference coordinates [Sp] of the tip [P] of the tool 100 stored by the reference coordinate holding unit 42 of the robot system 1 according to the first embodiment. The reference coordinates [Sp] shown in FIG. 4 are recognized as a deviation from a reference point in the sensor 20, for example. The reference coordinate holding unit 42 stores the reference point in the sensor 20 and the position at which the tip [P] is recognized in the X, Y, and Z directions in the posture data defined in the sensor 20. The timing of storing the reference coordinates [Sp] by the reference coordinate holding unit 42 will be described in detail in the following robot system control method.
[0025] [Trajectory planning unit 43 (robot control unit)] The trajectory planning unit 43 constitutes the robot control unit together with the next control unit 44. This trajectory planning unit 43 plans a trajectory of the robot 10 based on instructions from the input device 30, tip coordinates acquired from the recognition unit 41, and reference coordinates acquired from the reference coordinate holding unit 42. The trajectory planning unit 43 transmits the planned trajectory to the control unit 44.
[0026] When a picking operation instruction is input from the input device 30, the trajectory planning unit 43 plans a trajectory of the robot 10 for picking and placing the workpiece [W] based on the position information of the workpiece [W] obtained from the recognition unit 41.
[0027] Furthermore, when an instruction for a calibration operation is input from the input device 30, the trajectory planning unit 43 plans a trajectory of the robot 10 for matching the tool tip [P] with the reference coordinate [Sp] stored in the reference coordinate holding unit 42. Details of the trajectory planning of the robot 10 during the calibration operation by the trajectory planning unit 43 will be described in detail below in the description of a control method for a robot system.
[0028] [Control unit 44 (robot control unit)] The control unit 44, together with the trajectory planning unit 43, constitutes a robot control unit, and controls the operation of the robot 10 based on the trajectory acquired from the trajectory planning unit 43 and a tool offset stored in advance.
[0029] When a picking operation instruction is input from the input device 30, the control unit 44 controls the operation of the robot 10 so that the tip [P] of the tool 100 picks up and places the workpiece [W].
[0030] Furthermore, when an instruction for a calibration operation is input from the input device 30, the control unit 44 performs control for the calibration operation of the robot 10. In this case, the control unit 44 controls the operation of the robot 10 based on the trajectory plan acquired from the trajectory planning unit 43 so that the tip [P] of the tool 100 is at a predetermined position that can be sensed by the sensor 20. Note that, when the control device 40 has a detection unit that detects a failure of a picking operation and a judgment unit that judges the occurrence of an idle time during which no picking operation is performed, the control unit 44 may automatically start control for the calibration operation of the robot 10 based on the detection by these detection units and the judgment by the judgment unit.
[0031] Furthermore, in this case, the control unit 44 controls the posture of the robot 10 so that the robot 10 assumes different postures in which the tip [P] coincides with the reference coordinates [Sp] stored in the reference coordinate holding unit 42. Figs. 5 and 6 are diagrams showing a first example and a second example of posture control of the robot 10 according to the first embodiment. As shown in these figures, the control unit 44 changes the posture of the robot 10 so that the tip [P] of the tool 100 coincides with the reference coordinates [Sp] in the coordinate system of the sensor 20.
[0032] In this case, the control unit 44 controls the operation of the robot 10 changed to each different posture so that the tip [P] of the tool 100 coincides with the reference coordinate [Sp] stored in the reference coordinate holding unit 42 when the robot 10 is changed to each different posture. FIG. 7 is a diagram showing an example of control of the tip [P] of the tool 100 in each posture of the robot according to the first embodiment. As shown in this figure, the control unit 44 controls the operation of the robot 10 so that the tip [P] of the tool 100 in the robot 10 whose posture has been changed coincides with the reference coordinate [Sp] in the sensor coordinate system. Details of the operation control of the robot 10 by the control unit 44 will be described in detail later in the robot system control method.
[0033] [Encoder reading unit 45] The encoder reader 45 reads the measurement values (encoder values) of the encoders provided on the robot 10. The encoder reader 45 transmits the multiple encoder values that it has read to the control unit 44.
[0034] [Robot posture memory unit 46] The robot posture storage unit 46 stores posture data of the robot 10. The posture data of the robot 10 stored in the robot posture storage unit 46 is an encoder value in each posture in which the coordinates of the tip [P] of the tool 100 are matched with the reference coordinates [Sp] stored in the reference coordinate holding unit 42.
[0035] FIG. 8 is a diagram showing an example of robot posture data stored in the robot posture storage unit of the robot system according to the first embodiment. As shown in FIG. 8, the robot posture storage unit 46 stores the number of posture data required to obtain a tool offset, which will be described later. Here, each posture data is each encoder value in a state where the tip [P] of the tool 100 is matched with the reference coordinate [Sp] in each state where the posture of the robot 10 is changed. As an example, if the robot 10 is a six-axis robot, one (each time) posture data is composed of angle data (encoder values) of six axes (j1 to j6). In this case, the number of posture data required to obtain a tool offset is three or more. Details of the timing of holding posture data by the robot posture storage unit 46 will be described in detail in the control method of the robot system below.
[0036] [Tool offset calculation part 47] The tool offset calculation unit 47 calculates a tool offset based on a plurality of posture data stored in the robot posture storage unit 46, for example, by the method described in Patent Document 1. Here, the tool offset is the position of the tip [P] of the tool 100 relative to the robot 10 when the tool 100 is attached to the robot 10. If an error occurs in the tool offset, even if the drive of the robot 10 is controlled so that the tip [P] of the tool 100 is placed at the same point, a difference occurs in the position where the tip of the tool 100 is placed depending on the posture of the robot 10. The tool offset calculation unit 47 transmits the calculated tool offset to the control unit 44 as a calibration value of the tool offset. The control unit 44 overwrites the originally held tool offset with the received calibration value of the tool offset.
[0037] <Supply Conveyor 50> The supply conveyor 50 transports the workpiece [W], which is the target of picking, to a position where the robot 10 can easily hold the workpiece [W]. The workpiece [W] is not limited to being contained in a box [W1] as shown in the figure, and may be supplied from the supply conveyor 50 as a single piece.
[0038] <Placement table 60> The placement table 60 is a table on which the workpiece [W] held by the robot 10 is placed.
[0039] -Control method of robot system 1- Fig. 9 is a flowchart showing a control method for the robot system according to the first embodiment, and shows the procedure for calibrating the tool 100 executed by the control program stored in the control device 40 described with reference to Figs. 1 to 8. The control method for the robot system will be described below along the flowchart in Fig. 9 with reference to Figs. 1 to 8 and other figures as necessary. Note that this flow is triggered by, for example, inputting a calibration instruction to the input device 30, or by the control device 40 detecting a failure in a picking operation, or by determining the occurrence of an idle time during which no picking operation is being performed.
[0040] <Step S101> First, in step S101, the control unit 44 controls the robot 10 so that the marker [M] (see FIG. 2) of the tool 100 is within the angle of view of the sensor 20. The posture of the robot 10 at this time is arbitrary.
[0041] <Step S102> In S102, the recognition unit 41 recognizes the position of the marker [M] set on the tool 100 from the sensing data acquired from the sensor 20. The position of the marker [M] recognized here is a position in the sensor coordinate system.
[0042] <Step S103> In step S103, the recognition unit 41 recognizes the position of the tip [P] of the tool 100, i.e., the tip coordinates, from the position of the marker [M]. At this time, the recognition unit 41 calculates the coordinates of the tip [P] in the sensor coordinate system as the tip coordinates based on information on the positional relationship between the marker [M] and the tip [P] that is held by, for example, prior input, and recognizes the tip coordinates. The recognized tip coordinates are transmitted to the reference coordinate holding unit 42.
[0043] In addition, here, the dimensions of the marker image of the marker [M] recognized in step S102 are measured. If the tool 100 is cylindrical, the marker image is a circle or an ellipse, so the dimensions of the marker image are at least any one of the length of the long side, the length of the short side, and the diameter of the circle.
[0044] <Step S104> In step S104, the recognition unit 41 determines whether the recognition of the tip coordinates of the tool 100 in the previous step S103 is the first recognition in this calibration flow. If it is determined that it is the first recognition (YES), the recognition unit 41 transmits the tip coordinates of the tool 100 recognized in step S103 and the dimensions of the marker image to the reference coordinate holding unit 42 and proceeds to step S105. On the other hand, if it is determined that it is not the first recognition (NO), the recognition unit 41 transmits the tip coordinates of the tool 100 recognized in step S103 to the trajectory planning unit 43 and proceeds to step S201.
[0045] <Step S105> In step S105, the reference coordinate holding unit 42 stores the tip coordinates and the dimensions of the marker image received from the recognition unit 41 as the reference coordinates [Sp] of the tool tip [P] (see FIG. 4). Then, the process proceeds to step S106.
[0046] <Step S106> In step S106, the robot posture storage unit 46 stores the measurement values of the multiple (six in this case) encoders read by the encoder reader 45 as posture data of the robot 10 (see FIG. 8).
[0047] <Step S107> In step S107, the robot posture storage unit 46 judges whether the number of times the posture data was saved in step S106 has reached a predetermined number. Here, the predetermined number is set to be equal to or greater than the number of posture data required for the subsequent calculation of the tool offset (step S302). In the case where the robot 10 is a six-axis robot, the number of posture data required for the calculation of the tool offset is three or more, and the predetermined number is set to, for example, four. If the robot posture storage unit 46 judges that the predetermined number has been reached (YES), the process proceeds to step S301, and if it judges that the predetermined number has not been reached (NO), the process proceeds to step S108.
[0048] <Step S108> In step S108, the control unit 44 changes the attitude of the robot 10, and controls the attitude of the robot 10 so that in the changed attitude, the tip [P] of the tool 100 is at the reference coordinate [Sp] held in the reference coordinate holding unit 42. The attitude control here is assumed to be control based on the encoder value. Thereafter, the process returns to step S102, and steps S102 and S103 are carried out in order to recognize the tip coordinates of the tool 100, and the process proceeds to step S104, where the tip coordinates are transmitted to the trajectory planning unit 43, and the process proceeds to step S201.
[0049] <Step S201> On the other hand, S201 is a step to which the process proceeds after determining in step S104 that the recognition of the tip coordinates of the tool 100 is not the first recognition. In this case, the trajectory planning unit 43 is in a state of having received the tip coordinates of the tool 100 in the robot 10 whose posture has been changed from the recognition unit 41. In this step S201, the trajectory planning unit 43 acquires the reference coordinates [Sp] of the tip [P] held in the reference coordinate holding unit 42, and proceeds to step S202.
[0050] <Step S202> In step S202, the trajectory planning unit 43 plans a trajectory for aligning the tip [P] of the tool 100 in the robot 10, whose posture has been changed in step S108, with the reference coordinate [Sp]. The control unit 44 also controls the robot 10 based on the trajectory plan to align the tip [P] of the tool 100 with the reference coordinate [Sp].
[0051] Here, the trajectory planning unit 43 plans a trajectory such that the difference between the reference coordinate [Sp] acquired from the reference coordinate holding unit 42 and the tip coordinate received from the recognition unit 41 becomes zero. The tip coordinate received from the recognition unit 41 is the tip coordinate of the tool 100 in the robot 10 whose posture has been changed in step S108. The trajectory planning unit 43 transmits the planned trajectory to the control unit 44. The control unit 44 controls the robot 10 based on the trajectory plan transmitted from the trajectory planning unit 43, and causes the tip [P] of the tool 100 to coincide with the reference coordinate [Sp] (see FIG. 7). At this time, the control unit 44 may be configured to control the robot 10 based on the trajectory plan that is successively corrected. In this case, the trajectory planning unit 43 can improve the accuracy of the trajectory plan by periodically acquiring the position of the tip [P] of the tool 100 on the sensor coordinate system from the recognition unit 41 and successively correcting the trajectory.
[0052] Here, Figs. 10 to 12 are diagrams showing a procedure for matching the X-coordinate, Y-coordinate, and Z-coordinate of the tip position of the tool 100 with the reference coordinate [Sp] in the control method of the first embodiment. These diagrams are images 90 captured by the sensor 20 (see Figs. 5 and 6) using a two-dimensional camera, and show a state in which an image (marker image 90m) of the marker [M] (see Fig. 2) centered on the tip [P] of the tool 100 can be recognized. Here, the marker [M] is the periphery of the suction surface 101 centered on the tip [P] of the cylindrical tool 100 as shown in Fig. 2. Therefore, as shown in Figs. 10 to 12, the marker image 90m is recognized as an image of a circle or an ellipse. Hereinafter, a specific example of a procedure for matching the tip [P] of the tool 100 with the reference coordinate [Sp] in steps S201 and S202 will be described based on these Figs. 10 to 12.
[0053] First, as shown in Fig. 10, the trajectory planning unit 43 plans a trajectory so that the difference in X coordinate between the reference coordinate [Sp] and the tip coordinate of the tool 100 calculated from the currently recognized marker image 90m of the tool 100 becomes zero. The control unit 44 controls the drive of the robot 10 according to this plan. At this time, the robot 10 may be controlled according to a random trajectory so that the robot 10 stops moving the moment the difference in X coordinate becomes zero, or feedback control may be performed while dynamically monitoring the difference in the X direction so that the difference in X coordinate becomes zero, or other methods may be used.
[0054] Next, using a procedure similar to that described above, as shown in Figure 11, the difference in Y coordinate in the sensor coordinate system between the reference coordinate [Sp] and the tip coordinate of the tool 100 calculated from the marker image 90m of the tool 100 currently recognized is set to zero.
[0055] 12, the trajectory planning unit 43 plans a trajectory so that the length [L90] of the long side of the marker image 90m of the tool 100 currently recognized is the same as the length of the long side of the marker image of the reference coordinate [Sp] recognized initially. The dimensions of the marker image of the reference coordinate [Sp] recognized initially are measured in step S102, and are stored in accordance with the reference coordinate holding unit 42 in step S105.
[0056] According to this plan, the control unit 44 controls the driving of the robot 10 to set the difference in the Z coordinate in the sensor coordinate system to zero. Note that the above-mentioned adjustment of the Z coordinate may be performed so that the lengths of the short sides of the marker images are made equal.
[0057] The above-mentioned control uses a two-dimensional camera capable of highly accurate sensing as the sensor 20, and therefore it is possible to control the position of the tip [P] of the tool 100 with high precision. Furthermore, since an ellipse or a circle, for which there is a relatively high-precision recognition method, is recognized as the target and the tip [P] position is obtained, it is possible to perform alignment with higher precision.
[0058] The above-described determination of the coincidence of coordinates where the difference between the coordinates is 0 may be performed by the trajectory planning unit 43. Also, a coincidence determination unit may be implemented in the control device 40, and the determination unit may determine whether the position of the tip [P] recognized by the recognition unit 41 overlaps with the reference coordinate [Sp] of the tip [P] acquired from the reference coordinate holding unit 42. Also, it may be performed by other methods.
[0059] After determining that the tip [P] of the tool 100 coincides with the reference coordinate [Sp], the control unit 44 proceeds to step S106 described above. As a result, the robot posture storage unit 46 stores the encoder value read by the encoder reading unit 45 at this point in time as posture data of the robot 10 (step S106, see FIG. 8).
[0060] <Step S301> On the other hand, step S301 is a step to which the process proceeds after determining in step S107 that the number of times the posture data has been stored has reached a predetermined number (YES). In this step S301, the tool offset calculation unit 47 calculates a tool offset based on a plurality of posture data stored in the robot posture storage unit 46. At this time, the tool offset calculation unit 47 calculates each tip coordinate of the tool 100 based on the plurality of posture data stored in the robot posture storage unit 46, and calculates a tool offset that minimizes the difference (error) from the reference coordinate [Sp]. The tool offset calculation unit 47 transmits the calculated tool offset to the control unit 44 as a calibration value of the tool offset.
[0061] <Step S302> In step S302, the control unit 44 overwrites the initial value of the tool offset that is originally held with the received calibrated value of the tool offset, updates the tool offset, and ends the calibration.
[0062] In the above description, the control unit 44 has been described as holding the tool offset in advance, but the tool offset may be held in advance by the trajectory planning unit 43. In this case, the trajectory planning unit 43 plans a trajectory of the robot 10 based on an instruction from the input device 30, the tip coordinates acquired from the recognition unit 41, the reference coordinates acquired from the reference coordinate holding unit 42, and the held tool offset.
[0063] In this case, in the previous step S301, the tool offset calculation unit 47 transmits the calibrated value of the tool offset to the trajectory planning unit 43. In addition, in this step S302, the trajectory planning unit 43 overwrites the initial value of the tool offset originally held by the trajectory planning unit 43 with the received calibrated value of the tool offset, updates the tool offset, and ends the calibration.
[0064] -Effects of the first embodiment- The first embodiment described above is configured to control the robot 10 in each posture so that the tip [P] of the tool 100 coincides with the reference coordinate [Sp] in the sensor coordinate system, and calculate the tool offset based on the posture data of the robot 10 in each posture. Therefore, in controlling the robot 10 in each posture, the tip [P] of the tool 100 can be coincided with the reference coordinate [Sp] in the sensor coordinate system without being affected by the distortion of the lens constituting the sensor 20. This makes it possible to control the robot in each posture without considering the distortion of the lens constituting the sensor 20, and as a result, it is possible to calculate a highly accurate tool offset based on posture data that does not include the influence of the distortion, and improve the accuracy of the calibration.
[0065] In addition, it is possible to perform calibration automatically without human intervention. Therefore, it is possible to configure the system to detect picking failures or the occurrence of picking idle time and automatically start calibration operations. This makes it possible to reduce the picking failure rate and improve throughput.
[0066] Second Embodiment -Robot System- 13 and 14 are configuration diagrams (part 1) and (part 2) of the main parts of a robot system 2 according to the second embodiment. The robot system 2 shown in these figures differs from the robot system 1 of the first embodiment described with reference to Figs. 1 to 12 in the configuration of a tool 200 attached to the robot 10. The sensor 20 is a three-dimensional camera. Furthermore, as will be described in the control method of the robot system below, the procedure for recognizing the tip of the tool 200 in the recognition unit 41 of the control device 40 shown in Fig. 3 is different. The other configurations are similar to those of the robot system 1 of the first embodiment.
[0067] 13 and 14 is a gripper having multiple fingers 201, and holds a workpiece (not shown) by the multiple fingers 201. The number of fingers 201 that the tool 200 has is two in the drawings, but it may be three or more.
[0068] The recognition unit 41 (see FIG. 3) recognizes, for example, the center position of the tips of the multiple fingers 201 as the tip [P] of the tool 200. The recognition unit 41 first recognizes the tips of the fingers 201 constituting the tool 200 as markers [M] based on sensing data acquired from the sensor 20. The tips of the fingers 201 that become the markers [M] may be colored so as to be easily recognized by the sensor 20. After recognizing the marker [M], the recognition unit 41 calculates the position of the tip [P] in the sensor coordinate system based on information on the positional relationship between the marker [M] and the tip [P] that is held by, for example, prior input, and recognizes the calculated position as the coordinates of the tip [P] (tip coordinates).
[0069] -Method for controlling a robot system- The control method for the robot system in the second embodiment is different from the control method for the robot system in the first embodiment described with reference to FIG. 9 only in steps S102 and S202.
[0070] That is, in step S102, the recognition unit 41 (see FIG. 3) recognizes the tip of the finger 201 as the position of the marker [M] (see FIGS. 13 and 14) from the sensing data. In the next step S103, the recognition unit 41 recognizes the position of the tip [P] of the tool 200, i.e., the tip coordinates, from the positions of the multiple markers [M]. At this time, the recognition unit 41 calculates the coordinates of the tip [P] in the sensor coordinate system as the tip coordinates based on information on the positional relationship between the multiple markers [M] and the tip [P] that is held by, for example, prior input, and recognizes the tip coordinates. The recognized tip coordinates are transmitted to the reference coordinate holding unit 42.
[0071] Furthermore, in step S202, when the tip [P] of the tool 200 is made to coincide with the reference coordinates [Sp], the trajectory planning unit 43 may execute control based on sensing data from the three-dimensional camera used as the sensor 20. That is, the trajectory may be planned so that the X-coordinate, Y-coordinate, and Z-coordinate of the tip coordinates recognized by the recognition unit 41 coincide with the X-coordinate, Y-coordinate, and Z-coordinate of the reference coordinates [Sp].
[0072] -Effects of the second embodiment- In the second embodiment, as in the first embodiment, it is possible to improve the accuracy of calibration by calculating a precise tool offset based on the attitude data that does not include the influence of distortion constituting the sensor 20. In addition, it is also possible to configure the calibration operation to start automatically, which can reduce the picking failure rate and improve the throughput.
[0073] Third Embodiment -Robot System- Fig. 15 is a configuration diagram of the main parts of a robot system 3 according to the third embodiment. The robot system 3 shown in this figure differs from the robot system 1 of the first embodiment described with reference to Figs. 1 to 12 in the configuration of a tool 300 attached to the robot 10. Also, it is preferable to use a two-dimensional camera as the sensor 20. Furthermore, as will be described in the control method of the robot system below, the procedure for recognizing the tip of the tool 300 in the recognition unit 41 of the control device 40 shown in Fig. 3 is different. The other configurations are similar to those of the robot system 1 of the first embodiment.
[0074] 15 is a gripper having multiple fingers 301, which hold a workpiece (not shown here). The number of fingers 301 held by the tool 300 is two in the drawing, but may be three or more. In particular, at least one of the fingers 301 has a recess 301a on the surface that holds the workpiece.
[0075] Fig. 16 is a diagram for explaining a main part of the robot system 3 according to the third embodiment, and is a diagram for explaining the configuration of the tool 300. As shown in Fig. 15 and Fig. 16, the recess 301a has a shape capable of reliably holding in a predetermined state a sphere having a predetermined diameter that serves as a marker [M] in a calibration operation to be described later.
[0076] The recognition unit 41 (see FIG. 3) recognizes an arbitrary point set on the surface side of the tool 300 that holds the workpiece as the tip [P] (see FIG. 15) of the tool 300. For example, here, the center of the depression 301a provided on one finger 301 is set as the tip [P]. The recognition unit 41 first recognizes a sphere having a predetermined diameter held in a predetermined state by the tool 300 as a marker [M] based on the sensing data acquired from the sensor 20. After recognizing the marker [M], the recognition unit 41 calculates the position of the tip [P] in the sensor coordinate system based on information on the positional relationship between the marker [M] and the tip [P] held by, for example, a previous input, and recognizes the calculated position as the coordinates of the tip [P] (tip coordinates).
[0077] -Method for controlling a robot system- The control method of the robot system in the third embodiment includes a step of holding a sphere having a predetermined diameter to be the marker [M] by the tool 300 before step S101 in the control method of the robot system in the first embodiment described with reference to FIG. 9. The sphere to be the marker [M] is placed within the reach of the robot 10 and in a location that does not affect normal picking operations. If the error in the tool offset is large, a deviation occurs in the holding position of the marker [M] by the tool 300 when holding the marker [M], but the deviation is absorbed by the recess 301a provided in the tool 300, and the marker [M] can be held in a predetermined state.
[0078] After that, only step S102 is different. That is, in step S102, the recognition unit 41 (see FIG. 3) recognizes the position of the marker [M] consisting of a sphere from the sensing data. In the next step S103, the recognition unit 41 recognizes the position of the tip [P] of the tool 300, that is, the tip coordinates, from the positions of the multiple markers [M]. At this time, the recognition unit 41 calculates the coordinates of the tip [P] in the sensor coordinate system as the tip coordinates based on information on the positional relationship between the multiple markers [M] and the tip [P] that is held by, for example, prior input, and recognizes the tip coordinates. The recognized tip coordinates are transmitted to the reference coordinate holding unit 42.
[0079] However, in steps S101 and S108, the posture of the robot 10 is controlled so that the marker [M] can be confirmed by the sensor 20. In addition, in step S202, when the tip [P] of the tool 200 is made to coincide with the reference coordinate [Sp], control based on sensing data from the two-dimensional camera used as the sensor 20 may be performed in the procedure described in the first embodiment with reference to Figs. 10 to 12.
[0080] -Effects of the third embodiment- In the third embodiment, as in the first embodiment, it is possible to improve the accuracy of calibration by calculating a precise tool offset based on the attitude data that does not include the influence of distortion constituting the sensor 20. It is also possible to configure the calibration operation to start automatically, which can reduce the picking failure rate and improve the throughput.
[0081] The present invention is not limited to the above-described embodiments, and further includes various modified examples. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those having all the configurations described. For example, the work tool attached to the robot 10 is not limited to a bellows or a gripper, and may be other tools. In addition, the marker [M] may be anything that assists in recognizing the tip [P], and is not limited to a circle or a sphere, and may be a mark of other shapes (such as a square or a triangle) or a solid shape of other shapes (such as a cube, a cone, a square pyramid, or a cylinder). In addition, it is possible to replace a part of the configuration of a certain embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of a certain embodiment. In addition, it is possible to add, delete, or replace a part of the configuration of each embodiment with another configuration. [Explanation of symbols]
[0082] 1,2,3…Robot system 10. Robot 20…Sensor 40...Control device 41...Recognition part 42...Reference coordinate holding unit 43...Trajectory planning unit (robot control unit) 44...Control unit (robot control unit) 45…Encoder reading section 46…Robot posture memory unit 47...Tool offset calculation section 100, 200, 300…Tools [P]…Tip [M]…Marker
Claims
1. A robot system including a multi-joint robot equipped with a plurality of encoders, a sensor for capturing an image of a work area of the robot, and a control device for controlling the operation of the robot based on information from the sensor, The control device includes: a recognition unit that recognizes coordinates in a sensor coordinate system of a tip of a work tool attached to the robot based on information from the sensor; a robot control unit that defines a tip position of the work tool relative to the robot as a tool offset, and changes the posture of the robot based on the tool offset and information from the recognition unit so that the tip of the work tool coincides with a reference coordinate set in the sensor coordinate system; an encoder reading unit that reads an encoder value from an encoder of the robot in each of the changed postures; a tool offset calculation unit that calculates a calibration value of the tool offset based on the encoder value in each of the attitudes read by the encoder reading unit, the work tool includes a marker that assists the recognition unit in recognizing a tip of the work tool, The recognition unit recognizes the marker based on information from the sensor, and recognizes the tip based on the recognized marker. Robot system.
2. The work tool is a bellows having a columnar tip surface as an adsorption surface, The recognition unit recognizes a periphery of the suction surface based on information from the sensor, and recognizes a center of the suction surface as a tip of the work tool based on the recognized periphery. The robot system of claim 1 .
3. the work tool being a multi-fingered gripper; The recognition unit recognizes an arbitrary point set on any one of the plurality of fingers as a tip of the work tool. The robot system of claim 1 .
4. The sensor is a two-dimensional camera. The robot system according to any one of claims 1 to 3.
5. The robot control unit performs control to change the posture of the robot when detecting an idle time during which no work is being performed by the work tool. The robot system according to any one of claims 1 to 4.
6. The robot control unit performs control to change the posture of the robot when a failure in the work performed by the work tool is detected. The robot system according to any one of claims 1 to 4.
7. A method for controlling a robot system having a multi-joint robot equipped with a plurality of encoders, a sensor for capturing an image of a working range of the robot, and a control device for controlling driving of the robot based on information from the sensor, comprising: a recognition unit of the control device recognizes coordinates in a sensor coordinate system of a tip of a work tool attached to the robot based on information from the sensor; a robot control unit of the control device defines a tip position of the work tool relative to the robot as a tool offset, and changes the posture of the robot based on the tool offset and information from the recognition unit so that the tip of the work tool coincides with a reference coordinate set in the sensor coordinate system; an encoder reading unit of the control device reads an encoder value from an encoder of the robot for each of the changed postures; a tool offset calculation unit of the control device calculates a calibration value of the tool offset based on the encoder value in each of the attitudes read by the encoder reading unit, The recognition unit of the control device recognizes a marker provided on the work tool to assist in recognition of the tip when recognizing the coordinates of the tip of the work tool in a sensor coordinate system, and recognizes the tip based on the recognized marker. A method for controlling a robot system.
Citation Information
Patent Citations
Photographic film unit for color diffusion transfer
JP1985079352A
Microdrop drop device
JP1995003760U
Universal end effector applied to robot
JP2000024978A
Calibration method of robot
JP2006110705A
Robot and robot system
JP2016120558A
