Method for avoiding interference between robot arm and obstacle, and robot system

Abstract

To provide a technique that can avoid interference between a fingertip and an obstacle.SOLUTION: The present method includes: (a) a step of recognizing a position of an obstacle from an image of the obstacle taken by a camera that captures an operation area of a robot arm; (b) a step of calculating a distance between the obstacle and the robot arm when a fingertip of the robot arm is moved toward a target position; (c) when a first condition including a distance between a specific portion other than the fingertip of the robot arm and the obstacle being smaller than a distance between the fingertip and the obstacle holds, a step of causing the fingertip to perform an action of moving toward the target position and determining command values for a plurality of joints in accordance with a first control law for avoiding interference between the obstacle and the specific portion; and (d) when a second condition including the distance between the fingertip and the obstacle being smaller than the distance between the specific portion of the robot arm and the obstacle holds, a step of determining command values for the plurality of joints in accordance with a second control law for avoiding interference between the obstacle and the fingertip.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present disclosure relates to a method for avoiding interference between a robot arm and an obstacle, and a robot system. [Background technology]

[0002] Consider the case where a target position is given to a robot's hand, and you want to control it so that it reaches that target position. In this case, the degrees of freedom required to execute the task of getting the robot's hand to reach the target position are three degrees of freedom (x, y, z) in three-dimensional space. For example, if a six-axis robot is used, there are six degrees of freedom in the joints, so three degrees of freedom remain even when executing this task. This is called "redundancy" for the task, and it is known that the remaining redundancy enables obstacle avoidance.

[0003] Non-Patent Document 1 discloses a technology that uses multiple cameras to detect the positions of the hand, joints, and obstacles from images, and performs feedback control to reduce the deviation between the hand position and the target position to zero, thereby preventing the joint closest to the obstacle from colliding with the obstacle. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Masahiko Mikawa, Koichi Yoshida, Mizuki Tanno, and Michito Matsumoto, "Vision-Based Manipulator Redundant Control and Image Feature Estimation for Obstacle Avoidance," Journal of the Robotics Society of Japan, Vol. 17, No. 4, pp. 534-539, 1999. Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned conventional technology has a problem in that it is impossible to avoid interference between the hand and an obstacle. [Means for solving the problem]

[0006] According to a first aspect of the present disclosure, there is provided a method for avoiding interference between a robot arm having multiple joints and an obstacle, the method including: (a) recognizing a position of the obstacle from an image of the obstacle captured by a camera capturing an image of an area in which the robot arm operates; (b) calculating a distance between the obstacle and the robot arm when moving a hand of the robot arm toward a target position; (c) determining command values ​​for the multiple joints in accordance with a first control law that causes the hand to move toward the target position and avoids the interference between the obstacle and the specific part when a first condition is satisfied, the first condition including the distance between the obstacle and a specific part of the robot arm other than the hand is smaller than the distance between the hand and the obstacle; and (d) determining command values ​​for the multiple joints in accordance with a second control law that avoids the interference between the obstacle and the hand when a second condition is satisfied, the second condition including the distance between the obstacle and the specific part of the robot arm is smaller than the distance between the obstacle and the specific part of the robot arm.

[0007] According to a second aspect of the present disclosure, there is provided a robot system including a robot having a robot arm with multiple joints, a camera that captures an image of an area in which the robot arm operates, and a control device that controls the robot. The control device is configured to execute the following processes: (a) recognizing the position of the obstacle from an image of the obstacle captured by the camera; (b) calculating the distance between the obstacle and the robot arm when moving the hand of the robot arm toward a target position; (c) determining command values ​​for the plurality of joints in accordance with a first control law that causes the hand to move toward the target position and avoids the interference between the obstacle and the specific part, when a first condition is satisfied, the first condition including that the distance between the obstacle and a specific part of the robot arm other than the hand is smaller than the distance between the hand and the obstacle; and (d) determining command values ​​for the plurality of joints in accordance with a second control law that avoids the interference between the obstacle and the hand, when a second condition is satisfied, the second condition including that the distance between the hand and the obstacle is smaller than the distance between the specific part of the robot arm and the obstacle. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is an explanatory diagram showing the configuration of a robot system according to a first embodiment. [Figure 2] FIG. 2 is a functional block diagram of the information processing device according to the first embodiment. [Figure 3] 4 is a flowchart showing a processing procedure for obstacle avoidance control in the first embodiment. [Figure 4] 10 is a flowchart showing a processing procedure for obstacle avoidance control in the second embodiment. [Figure 5] FIG. 10 is an explanatory diagram showing the configuration of a robot system according to a third embodiment. [Figure 6] FIG. 10 is a functional block diagram of an information processing device according to a third embodiment. [Figure 7] 10 is a flowchart showing a processing procedure for obstacle avoidance control in the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] A. First embodiment: 1 is an explanatory diagram showing an example of a robot system according to the first embodiment. The robot system includes a robot 100, a control device 200 that controls the robot 100, an information processing device 300, a camera 400, and a workbench 500. The information processing device 300 is, for example, a personal computer. The control device 200 and the information processing device 300 correspond to the "control device" in this disclosure.

[0010] The robot 100 includes a base 110 and a robot arm 120. An end effector 150 for holding a workpiece WK is connected to an arm end 130, which is the tip of the robot arm 120. In this example, the end effector 150 is a suction hand that picks up the workpiece WK. A hand marker MKe is attached to the workpiece WK or the end effector 150 to recognize the position of the hand. The "hand" usually means the end effector 150. The "position of the hand" means the TCP (Tool Center Point) as the control point of the robot 100.

[0011] The robot arm 120 is connected in sequence by six joints J1 to J6. All of these joints J1 to J6 are rotary joints. More specifically, three joints J2, J3, and J5 are bending joints, and the other three joints J1, J4, and J6 are torsion joints. Although a six-axis robot is exemplified in this embodiment, a robot having any robot arm mechanism with multiple joints can be used. Furthermore, although the robot 100 in this embodiment is a vertical multi-joint robot, a horizontal multi-joint robot may also be used.

[0012] The camera 400 captures an operating area RWA of the robot arm 120. The operating area RWA is a three-dimensional area that encompasses positions where the robot arm 120 can move. As the camera 400, for example, a stereo camera, an RGBD camera including an RGB camera and a D camera (depth camera), a monocular camera, or the like can be used. Furthermore, multiple cameras 400 may be installed to capture the entire operating area RWA.

[0013] A target position marker MKd indicating the target position DP of the hand is set on the workbench 500. As the target position marker MKd and the hand marker MKe, for example, an AR marker (AprilTag) can be used. Instead of markers, the hand position and the target position DP may be indicated by other types of geometric or graphical features whose positions can be recognized from the image captured by the camera 400.

[0014] In the state shown in FIG. 1, the position of the hand of the robot arm 120 is at the start position SP of the task. The task of moving the robot arm 120 is the operation of moving the hand from the start position SP to the target position DP. An obstacle OB exists in the middle of the trajectory TR of the robot arm 120. This obstacle OB may interfere with the hand or other parts of the robot arm 120. The obstacle OB may be a fixed object fixed to the floor or a wall, or may be a movable object.

[0015] FIG. 1 shows a robot coordinate system Σr and a camera coordinate system Σc. The X and Y axes of the robot coordinate system Σr are horizontal axes, and the Z axis is a vertical axis. For the camera coordinate system Σc, the symbols distinguishing between the three coordinate axes are omitted. The robot coordinate system Σr is an orthogonal coordinate system with a coordinate origin set at a predetermined position of the robot 100. The camera coordinate system Σc is an orthogonal coordinate system with a coordinate origin set at a predetermined position of the camera 400. In this disclosure, the two-dimensional coordinate system on the image plane of the camera 400 is referred to as the "image coordinate system."

[0016] The robot system of the first embodiment is configured as a visual servo system that moves a workpiece WK to a target position DP while recognizing the robot arm 120, the target position DP, and an obstacle OB from an image captured by a camera 400. In the first embodiment, it is assumed that the camera 400 is not accurately calibrated with respect to the robot coordinate system Σr. In a visual servo system, even if the accurate coordinate transformation between the camera coordinate system Σc and the robot coordinate system Σr is unknown, it is possible for the robot 100 to perform a task using an image captured by the camera 400.

[0017] 2 is a block diagram showing the functions of an information processing device 300 according to the first embodiment. The information processing device 300 includes a processor 310, a memory 320, and an interface circuit 330. An input device 340 and a display device 350 are connected to the interface circuit 330, and the control device 200 and a camera 400 are also connected to the interface circuit 330.

[0018] The processor 310 has the function of a visual servo control unit 610. The visual servo control unit 610 recognizes the robot arm 120, the target position DP, and the obstacle OB using the image captured by the camera 400, and executes processing to determine a command value for the robot arm 120 so that the obstacle OB does not interfere with the robot arm 120. The visual servo control unit 610 includes the functions of a control parameter setting unit 611, a position recognition unit 612, an interference determination unit 613, a command value determination unit 614, and a command value transmission unit 615.

[0019] The control parameter setting unit 611 accepts settings of various control parameters for performing visual servo control from the user. Some of the control parameters may be automatically determined by the control parameter setting unit 611. Examples of the control parameters will be described later.

[0020] The position recognition unit 612 recognizes the positions of the hand and joints of the robot arm 120, the target position DP, and the position of the obstacle OB using images captured by the camera 400. A hand marker MKe is used to recognize the hand position, and a target position marker MKd is used to recognize the target position DP. Note that it is preferable that the joints of the robot arm 120 and the obstacle OB are also provided with characteristic parts such as recognition markers.

[0021] The interference determination unit 613 determines whether the robot arm 120 will interfere with an obstacle OB when the hand of the robot arm 120 is moved toward the target position DP. Specifically, the interference determination unit 613 determines whether the distance between the obstacle OB and the robot arm 120 is equal to or less than a preset interference threshold. As the "distance between the obstacle OB and the robot arm 120," for example, the distance between the obstacle OB and the hand and the distance between the obstacle OB and each joint are calculated.

[0022] The command value determiner 614 determines a command value so as to avoid interference between the robot arm 120 and an obstacle OB. In this embodiment, the command value is the rotation angle or the displacement (angle change amount) of the joints J1 to J6. The command value is determined in accordance with the following first control law CR1 or second control law CR2.

[0023] <First control law CR1> The command values ​​for the multiple joints J1 to J6 are determined so that the hand moves toward the target position DP and so that interference between the obstacle OB and a specific part of the robot arm 120 is avoided. In the first control law CR1, it is preferable to further determine a command value so as to avoid a singular posture of the robot arm 120. In this case, the task according to the first control law CR1 includes the following two subtasks. (1) First subtask ST11: Move the hand closer to the target position DP. (2) Second subtask ST12: Avoid interference between the obstacle OB and a specific part of the robot arm 120, and avoid a singular posture. It is preferable that the "particular part of the robot arm 120" is any one of the joints J1 to J6.

[0024] <Second control law CR2> Command values ​​for the multiple joints J1 to J6 are determined so as to avoid interference between the obstacle OB and the hand. In the second control law CR2 as well, it is preferable to determine a command value so as to avoid a singular posture of the robot arm 120. In this case, the task according to the second control law CR2 includes the following two subtasks. (1) First subtask ST21: The hand is moved closer to the target position DP while giving priority to avoiding interference between the obstacle OB and the hand. (2) Second subtask ST22: Avoid singular postures.

[0025] The method of determining the command values ​​according to the first control law CR1 and the second control law CR2 will be described later.

[0026] The command value transmitting unit 615 transmits the determined command value to the control device 200 to operate the robot 100. The command value is transmitted to the control device 200 at every predetermined control cycle.

[0027] The functions of the visual servo control unit 610 are realized by the processor 310 executing a computer program stored in the memory 320. However, some or all of the functions of the visual servo control unit 610 may be realized by a hardware circuit.

[0028] The following method can be used, for example, to determine the command values ​​according to the first control law CR1 and the second control law CR2. Note that the dots "·" placed above the parameters in the following formulas are written as superscripts in the explanation due to limitations on character notation.

[0029] In general, the displacement of the hand position in the image coordinate system is · e and the displacement θ of the joint angle θ of all the joints of the robot arm 120. ·The relationship between the image Jacobian matrix J e can be expressed as follows:

number

[0030] Image Jacobian matrix J e may be predetermined or may be estimated by the visual servo control 610 while moving the robot arm 120.

[0031] The general solution of the above equation (q1) is the angular velocity command value θ · is defined by the following formula:

number

[0032] The first term on the right side of the above equation (q2) corresponds to the first command value component for the first subtasks ST11 and ST21, and the second term on the right side corresponds to the second command value component for the second subtasks ST12 and ST22.

[0033] The command value according to the first control law CR1 can be determined as follows: Since the first subtask ST11 of the first control law CR1 is to "move the hand tip closer to the target position DP," the first command value component for the first subtask ST11 is given by the following equation.

number

[0034] Using this equation (q3), it is possible to calculate the first command value component for the first subtask ST11, which brings the deviation between the hand position and the target position closer to zero. e is set by the control parameter setting unit 611.

[0035] The second subtask ST12 of the first control law CR1 is to "avoid interference between the obstacle OB and a specific part of the robot arm 120, and also avoid a singular posture." This can be achieved by appropriately defining the vector z in the second term on the right-hand side of equation (q2). In this embodiment, the vector z is decomposed into a first vector z1 for obstacle avoidance and a second vector z2 for singular posture avoidance, as shown in the following equation.

number

[0036] The weight α can be changed arbitrarily depending on the situation, and is set by the control parameter setting unit 611. The weight α may be automatically set depending on the distance between the obstacle OB and a specific part of the robot arm 120. For example, when the distance is smaller than a reference value, the weight α may be increased to prioritize obstacle avoidance, and when the distance is larger than the reference value, the weight α may be decreased to prioritize singular posture avoidance. In the following description, the "specific part of the robot arm 120" is defined as any one of the joints J1 to J6.

[0037] The first vector z1 for obstacle avoidance is the image Jacobian matrix J for the joint position closest to the obstacle OB. j and vector P indicating the direction away from obstacle OB. j Using this, it is given by the following equation:

number

[0038] In addition, the vector P j means moving the joint in a direction away from the obstacle OB, and can be expressed as follows:

number

[0039] In the above equation (q6), the reason for dividing by the norm is that the greater the distance between the obstacle OB and the joint, the smaller the vector P jThis is because we want to reduce the magnitude of the gain λ j is set by the control parameter setting unit 611.

[0040] The second vector z2 for avoiding the singular attitude can be given by, for example, the following equation:

number

[0041] The command value according to the second control law CR2 can be determined as follows: Since the first subtask ST21 of the second control law CR2 is to "move the hand toward the target position DP while prioritizing avoidance of interference between the obstacle OB and the hand," the first command value component for the first subtask ST21 is given by the following equation instead of the above equation (q3).

number

[0042] In this way, in the second control law CR2, the hand position X e and target position X d From the difference between these, the component P for hand obstacle avoidance is e The first command value component is calculated using the term obtained by subtracting '.

[0043] The above component P e For example, ' can be defined as follows:

number

[0044] As can be seen from the above equations (q9a) and (q9b), the component P eThe smaller the difference (d - D) between the distance d between the hand and the obstacle OB and its allowable lower limit D, the larger the repulsive force R. According to the above equations (q8), (q9a), and (q9b), the hand and the obstacle OB can be kept at a sufficient distance without approaching below the allowable lower limit D. MAX The maximum repulsive force R can be set empirically or can be calculated theoretically. MAX and the coefficient β are set by the control parameter setting unit 611.

[0045] Since the second subtask ST22 of the second control law CR2 is to "avoid a singular posture," the second command value component for the second subtask ST22 can be calculated by appropriately defining the vector z in the second term on the right-hand side of the above equation (q2). The vector z for the second subtask ST22 is given by the following equation instead of the above equation (q4).

number

[0046] This vector z2 can be calculated using the above-mentioned equations (q7a) to (q7c).

[0047] The various formulas described above are merely examples, and the command value may be determined using formulas other than these.

[0048] 3 is a flowchart showing the procedure for obstacle avoidance control in the first embodiment. In step S11, the position recognition unit 612 uses the camera 400 to capture an image of the operating area RWA of the robot 100. In step S12, the position recognition unit 612 recognizes the hand position, the positions of each joint, and the target position DP from the image captured by the camera 400. In step S13, the position recognition unit 612 recognizes an obstacle OB from the image captured by the camera 400. Note that steps S12 and S13 may be executed simultaneously.

[0049] In step S14, the interference determination unit 613 calculates the distance between the obstacle OB and the robot arm 120. The positions of the hand tip of the robot arm 120 and the positions of one or more specific parts of the robot arm 120 other than the hand tip are selected in advance as targets for calculating the distance to the obstacle OB. In this embodiment, multiple joints J1 to J6 are used as the "specific parts of the robot arm 120." However, since the position of the joint J1 between the base 110 and the robot arm 120 is fixed, it may be excluded from the targets for calculating the distance.

[0050] In step S15, the interference determination unit 613 determines whether the distance between the obstacle OB and the robot arm 120 is equal to or smaller than an interference threshold. The interference threshold is set in advance as a distance value at which interference between the obstacle OB and the robot arm 120 may occur. The interference threshold is preferably determined according to the thickness of the robot arm 120. That is, the interference threshold is preferably set to a larger value as the thickness of the robot arm 120 increases. The thickness of the robot arm 120 is set in advance. If the links of the robot arm 120 are not cylindrical, the thickness of the robot arm 120 may change depending on the angle. Therefore, if the distance between the obstacle OB and the robot arm 120 is calculated in real time from images captured by the camera 400, the obstacle OB can be avoided more accurately.

[0051] The interference threshold may be set to the same value for the hand and the joints J1 to J6 of the robot arm 120, or individual values ​​may be set. In the latter case, some of the interference thresholds for the hand and the joints J1 to J6 are set to different values ​​from the others. It is also preferable that the interference threshold for the hand be set to a different value from the interference threshold for the joints. In this case, it is preferable that the interference threshold for the hand be set to a value that corresponds to the model of the end effector 150.

[0052] If the distance between the obstacle OB and the robot arm 120 is greater than the interference threshold, the process proceeds from step S15 to step S17, where command values ​​for the multiple joints are determined according to the first control law CR1.

[0053] On the other hand, if the distance between the obstacle OB and the robot arm 120 is equal to or less than the interference threshold, the process proceeds from step S15 to step S16, where the interference determination unit 613 determines whether the part where the distance to the obstacle OB is equal to or less than the interference threshold is one of the joints J1 to J6 or the hand tip. Hereinafter, the "part where the distance to the obstacle OB is equal to or less than the interference threshold" will be referred to as an "interference candidate."

[0054] If the interference candidate is any of the joints J1 to J6, the process proceeds to step S17, where the command value determiner 614 determines command values ​​for the joints J1 to J6 in accordance with the first control law CR1.

[0055] On the other hand, if the interference candidate is the hand, the process proceeds to step S18, where the command value determiner 614 determines command values ​​for the plurality of joints J1 to J6 in accordance with the second control law CR2.

[0056] Once the command values ​​are determined in step S17 or step S18, the process proceeds to step S19, where the command value transmitting unit 615 transmits the command values ​​of the multiple joints to the control device 200. As a result, it is possible to operate the robot arm 120 while avoiding interference between the obstacle OB and the robot arm 120.

[0057] In step S20, the visual servo control unit 610 determines whether the tip of the robot arm 120 has reached the target position DP. If the tip of the robot arm 120 has not reached the target position DP, the process returns to step S11 and steps S11 to S19 are executed again. If the tip of the robot arm 120 has reached the target position DP, the process of FIG. 3 is completed.

[0058] As described above, in the first embodiment, when the distance between a specific part other than the hand of the robot arm 120 and an obstacle OB is equal to or less than the interference threshold, the command values ​​of the multiple joints are determined according to the first control law CR1, and when the distance between the hand and the obstacle OB is equal to or less than the interference threshold, the command values ​​of the multiple joints are determined according to the second control law CR2. As a result, it is possible to achieve both obstacle avoidance of the hand and obstacle avoidance of the specific part other than the hand while maintaining target reachability.

[0059] B. Second embodiment: Fig. 4 is a flowchart showing the processing procedure of obstacle avoidance control in the second embodiment. The system configuration and device configuration of the second embodiment are the same as those of the first embodiment shown in Figs. 1 and 2. The processing procedure of the second embodiment is the same as that of the first embodiment shown in Fig. 3, except that step S15 is omitted and step S16 is changed to step S16a. The other steps are the same as those in Fig. 3. The first control law CR1 and the second control law CR2 are also the same as those in the first embodiment.

[0060] In step S16a, the interference determination unit 613 determines whether the part of the robot arm closest to the obstacle OB is one of the joints J1 to J6 or the hand. If the part closest to the obstacle OB is one of the joints J1 to J6, the process proceeds to step S17, where the command value determination unit 614 determines command values ​​for the joints J1 to J6 in accordance with the first control law CR1. On the other hand, if the part closest to the obstacle OB is the hand, the process proceeds to step S18, where the command value determination unit 614 determines command values ​​for the joints J1 to J6 in accordance with the second control law CR2. Note that if the smallest value of the distances between the obstacle OB and the joints is equal to the distance between the obstacle OB and the hand, either the first control law CR1 or the second control law CR2 may be used.

[0061] As in the first embodiment, instead of the joints J1 to J6, other parts of the robot arm 120 may be selected as the "specific part of the robot arm 120" that is the target for avoiding interference with the obstacle OB.

[0062] As described above, in the second embodiment, when the condition that the distance between a specific part of the robot arm 120 other than the hand and an obstacle OB is smaller than the distance between the hand and the obstacle OB is satisfied, the command values ​​of the multiple joints are determined according to the first control law CR1. Also, when the condition that the distance between the hand and the obstacle OB is smaller than the distance between a specific part of the robot arm 120 and the obstacle OB is satisfied, the command values ​​of the multiple joints are determined according to the second control law CR2. As a result, it is possible to achieve both obstacle avoidance of the hand and obstacle avoidance of the specific part other than the hand while maintaining target reachability.

[0063] In the first embodiment described above, as in the second embodiment, (i) when the first control law CR1 is used, a condition is satisfied that the distance between a specific portion of the robot arm 120 other than the hand and the obstacle OB is smaller than the distance between the hand and the obstacle OB, and (ii) when the second control law CR2 is used, a condition is satisfied that the distance between the hand and the obstacle OB is smaller than the distance between the specific portion of the robot arm 120 and the obstacle OB. In consideration of this point, it can be understood that the first embodiment and the second embodiment are common in that (i) when a first condition is satisfied, including that the distance between a specific portion of the robot arm 120 other than the hand and the obstacle OB is smaller than the distance between the hand and the obstacle OB, the command values ​​of the multiple joints are determined in accordance with the first control law CR1, and (ii) when a second condition is satisfied, including that the distance between the hand and the obstacle OB is smaller than the distance between the specific portion of the robot arm 120 and the obstacle OB, the command values ​​of the multiple joints are determined in accordance with the second control law CR2.

[0064] C. Third embodiment: 5 is an explanatory diagram showing an example of a robot system according to the third embodiment. The main differences from the first embodiment shown in FIG. 1 are as follows. (1) The hand marker MKe and the target position marker MKd are omitted. (2) The camera 400 is calibrated, and the coordinate transformation between the camera coordinate system Σc and the robot coordinate system Σr is known. (3) The joint command values ​​are determined according to a robot control program so that the robot 100 moves along a predetermined target trajectory TT. (4) The hand position and joint positions of the robot arm 120 are calculated from the joint command values.

[0065] 6 is a block diagram showing the functions of an information processing device 300 in the third embodiment. The difference from the first embodiment shown in FIG. 2 is that a collision avoidance control unit 620 is provided instead of the visual servo control unit 610. The collision avoidance control unit 620 recognizes an obstacle OB using an image captured by the camera 400, and executes processing to correct a command value for the robot arm 120 so that the obstacle OB does not interfere with the robot arm 120. The collision avoidance control unit 620 includes the functions of a target trajectory calculation unit 621, a position recognition unit 622, an interference determination unit 623, a command value correction unit 624, and a command value transmission unit 625.

[0066] The target trajectory calculation unit 621 calculates a target trajectory TT for moving the hand of the robot arm 120 from the task start position SP to the target position DP, and calculates command values ​​for multiple joints for each constant control period. The target trajectory TT is, for example, a three-dimensional trajectory that connects the start position SP and the target position DP with a straight line or a curve. In the third embodiment, the start position SP and the target position DP are set in the robot control program RP, and joint command values ​​for moving the hand along the target trajectory TT are calculated for each control period in accordance with the robot control program RP.

[0067] The position recognition unit 622 recognizes the position of the obstacle OB using the image captured by the camera 400. The interference determination unit 623 determines whether or not the robot arm 120 will interfere with the obstacle OB on the target trajectory TT of the robot arm 120. Specifically, the interference determination unit 623 determines whether or not the distance between the obstacle OB and the robot arm 120 is equal to or less than a preset interference threshold. As the "distance between the obstacle OB and the robot arm 120," for example, the distance between the obstacle OB and the hand tip and the distance between the obstacle OB and each joint are calculated.

[0068] The command value correction unit 624 corrects the command value so as to avoid interference between the robot arm 120 and the obstacle OB. This correction is performed in accordance with the first control law CR1 or the second control law CR2 described in the first embodiment. The content of the first control law CR1 or the second control law CR2 is almost the same as in the first embodiment, but the method of correcting the command value differs from that in the first embodiment. This point will be described in more detail later.

[0069] The command value transmitting unit 625 transmits the corrected command value to the control device 200 to operate the robot 100. The command value is transmitted to the control device 200 at every predetermined control cycle.

[0070] The functions of the collision avoidance control unit 620 are realized by the processor 310 executing a computer program stored in the memory 320. However, some or all of the functions of the collision avoidance control unit 620 may be realized by a hardware circuit.

[0071] A robot control program RP is stored in the memory 320. The robot control program RP is composed of a plurality of commands that cause the robot 100 to operate.

[0072] 7 is a flowchart showing the procedure for obstacle avoidance control in the third embodiment. In step S110, the target trajectory calculation unit 621 calculates a target trajectory TT for moving the tip of the robot arm 120 from the start position SP to the target position DP, and calculates command values ​​for the multiple joints J1 to J6 for each fixed control period.

[0073] In step S111, the target trajectory calculation unit 621 selects a command value for one control cycle later. This command value is transmitted to the interference determination unit 623, the command value correction unit 624, and the command value transmission unit 625. In step S112, the position recognition unit 622 uses the camera 400 to capture an image of the operating area RWA of the robot 100. In step S113, the position recognition unit 622 recognizes an obstacle OB from the image captured by the camera 400. The position recognition unit 622 further converts the position coordinates of the recognized obstacle OB from the camera coordinate system Σc to the robot coordinate system Σr.

[0074] In step S114, the interference determination unit 623 calculates the distance between the obstacle OB and the robot arm 120 one control cycle later. In step S115, the interference determination unit 623 determines whether the distance between the obstacle OB and the robot arm 120 is equal to or less than the interference threshold. The interference threshold is the same as in the first embodiment. As in the first embodiment, the targets for calculating the distance to the obstacle OB are the hand position of the robot arm 120 and the positions of one or more specific parts of the robot arm 120 other than the hand position. It is preferable to use multiple joints J1 to J6 as the "specific parts of the robot arm 120." In the third embodiment, the hand position and the positions of the multiple joints J1 to J6 can be easily calculated from the rotation angle of each joint and the lengths of the multiple links, so the distance to the obstacle OB can also be easily calculated. However, parts other than joints, such as the ends of links, may also be used as the "specific parts of the robot arm 120." In this embodiment, multiple joints J1 to J6 are used as the "specific parts of the robot arm 120." It should be noted that the joint J1 between the base 110 and the robot arm 120 has a fixed position and may therefore be excluded from the calculation of the distance.

[0075] If the distance between the obstacle OB and the robot arm 120 is greater than the interference threshold, the process proceeds from step S115 to step S119, and the command value preset in step S111 is sent directly from the command value sending unit 625 to the control device 200, causing the robot 100 to perform one control cycle of operation.

[0076] On the other hand, if the distance between the obstacle OB and the robot arm 120 is equal to or less than the interference threshold, the process proceeds from step S115 to step S116, and the interference determination unit 623 determines whether the interference candidate, which is the part where the distance from the obstacle OB is equal to or less than the interference threshold, is one of the joints J1 to J6 or the hand.

[0077] If the interference candidate is any of the joints J1 to J6, the process proceeds to step S117, where the command value corrector 624 corrects the command values ​​of the joints J1 to J6 in accordance with the first control law CR1. The first control law CR1 corrects the command values ​​of the joints J1 to J6 so that the hand moves toward the target position DP according to the target trajectory TT and the joints avoid interference with the obstacle OB. The command values ​​are corrected so that the distances between the obstacle OB and the joints J1 to J6 are all greater than the interference threshold. The command values ​​may be corrected according to the first control law CR1 by, for example, determining multiple combinations of command values ​​for the joints to achieve the position and posture of the hand required by the robot control program RP, and selecting one of these combinations that does not cause interference between the obstacle OB and the joints.

[0078] On the other hand, if the interference candidate is the hand, the process proceeds to step S118, where the command value modifying unit 624 modifies the command values ​​of the joints J1 to J6 in accordance with the second control law CR2. In the second control law CR2, the command values ​​of the joints J1 to J6 are modified so as to avoid interference between the obstacle OB and the hand. The command values ​​are modified so that the distance between the obstacle OB and the hand is greater than the interference threshold. The command value modification in the second control law CR2 may be, for example, a process of modifying the command value of one joint closest to the hand among one or more joints whose command value can avoid interference between the obstacle OB and the hand. Specifically, if interference between the obstacle OB and the hand can be avoided by modifying the command value of any of the joints J1 to J5, only the command value of the joint J5 closest to the hand may be modified. This facilitates modification of the command value to avoid interference between the obstacle OB and the hand in accordance with the second control law CR2. Furthermore, since the command values ​​(rotation angles and their displacements) of the other joints are maintained at preset values, the movement of the robot arm 120 can be made smooth before and after avoiding interference between the obstacle OB and the hand.

[0079] The corrected command value for "the one joint closest to the hand" in the second control law CR2 can be calculated as, for example, one of the following two corrected command values ​​CV1 and CV2. CV1 = CV + Δ …(q11a) CV2 = CV - Δ …(q11b) Here, CV is the command value before correction, and Δ is the amount of correction. A predetermined constant value can be used as the correction amount Δ. Of these two corrected command values ​​CV1 and CV2, it is preferable to adopt the one that increases the distance between the obstacle OB and the hand as the corrected command value. In this way, interference between the obstacle OB and the hand can be more reliably avoided. Note that the correction amount Δ may be searched for so that the distance between the obstacle OB and the hand is greater than the interference threshold.

[0080] If the command value is corrected in step S117 or step S118, the corrected command value is transmitted to the control device 200 by the command value transmitting unit 625 in step S119. As a result, it is possible to operate the robot arm 120 while avoiding interference between the obstacle OB and the robot arm 120.

[0081] In step S120, the collision avoidance control unit 620 determines whether the end effector of the robot arm 120 has reached the target position DP. This determination can be made by checking whether all the command values ​​calculated in step S110 have been used. If the target position DP has not been reached, the process returns to step S111 and steps S111 to S119 are executed again. If the target position DP has been reached, the process of FIG. 7 is completed.

[0082] As described above, in the third embodiment, similar to the first embodiment, when the distance between a specific portion of the robot arm 120 other than the hand and the obstacle OB is equal to or less than the interference threshold, the command values ​​of the multiple joints are determined according to the first control law CR1. When the distance between the hand and the obstacle OB is equal to or less than the interference threshold, the command values ​​of the multiple joints are determined according to the second control law CR2. As a result, it is possible to achieve both obstacle avoidance of the hand and obstacle avoidance of the specific portion of the robot arm 120 other than the hand while maintaining target reachability. Note that, similar to the first and second embodiments, the third embodiment also performs the following control: when a first condition is met, including (i) the distance between a specific portion of the robot arm 120 other than the hand and the obstacle OB is smaller than the distance between the hand and the obstacle OB, the command values ​​of the multiple joints are determined according to the first control law CR1; and when a second condition is met, including (ii) the distance between the hand and the obstacle OB is smaller than the distance between the specific portion of the robot arm 120 and the obstacle OB, the command values ​​of the multiple joints are determined according to the second control law CR2.

[0083] Other forms: The present disclosure is not limited to the above-described embodiments and can be realized in various forms without departing from the spirit thereof. For example, the present disclosure can also be realized in the following aspects. The technical features in the above embodiments corresponding to the technical features in each aspect described below can be appropriately replaced or combined to solve some or all of the problems of the present disclosure or to achieve some or all of the effects of the present disclosure. Furthermore, if a technical feature is not described as essential in this specification, it can be appropriately deleted.

[0084] (1) According to a first aspect of the present disclosure, there is provided a method for avoiding interference between a robot arm having multiple joints and an obstacle, the method including: (a) recognizing a position of the obstacle from an image of the obstacle captured by a camera capturing an image of an area in which the robot arm operates; (b) calculating a distance between the obstacle and the robot arm when moving a hand of the robot arm toward a target position; (c) determining command values ​​for the multiple joints in accordance with a first control law that causes the hand to move toward the target position and avoids the interference between the obstacle and the specific part when a first condition is satisfied, the first condition including the distance between the obstacle and a specific part of the robot arm other than the hand is smaller than the distance between the hand and the obstacle; and (d) determining command values ​​for the multiple joints in accordance with a second control law that avoids the interference between the obstacle and the hand when a second condition is satisfied, the second condition including the distance between the obstacle and the specific part of the robot arm is smaller than the distance between the obstacle and the specific part of the robot arm. According to this method, it is possible to achieve both obstacle avoidance of the hand and obstacle avoidance of a specific part other than the hand while maintaining the ability to reach the target.

[0085] (2) In the above method, the specific part may be any one of the plurality of joints. This method allows for both hand and joint obstacle avoidance.

[0086] (3) In the above method, the first condition may include the distance between the specific part of the robot arm and the obstacle being equal to or less than an interference threshold, and the second condition may include the distance between the end effector and the obstacle being equal to or less than the interference threshold. This method allows the selection of a control law using an interference threshold.

[0087] (4) In the above method, the step (d) may include: (d1) determining a first command value component for moving the end effector closer to the target position while prioritizing avoidance of interference between the obstacle and the end effector; (d2) determining a second command value component for avoiding a singular posture of the robot arm; and (d3) determining the command value by adding the first command value component and the second command value component. According to this method, it is possible to obtain a command value that avoids a singular posture while performing obstacle avoidance of the hand.

[0088] (5) In the above method, the step (d1) may include a step of calculating the first command value component using a term obtained by subtracting a component for avoiding the obstacle from a difference between the position of the hand and the target position. According to this method, it is possible to calculate the first command value component that realizes obstacle avoidance of the hand.

[0089] (6) In the above method, the step (d) may include a process of correcting the command value from a preset value for one joint that is closest to the hand among one or more joints that can avoid interference between the obstacle and the hand by changing the command value. According to this method, it is possible to easily determine a command value for avoiding interference between the obstacle and the end effector in accordance with the second control law.

[0090] (7) According to a second aspect of the present disclosure, there is provided a robot system including a robot having a robot arm with multiple joints, a camera that captures an image of an area in which the robot arm operates, and a control device that controls the robot. The control device is configured to execute the following processes: (a) recognizing the position of the obstacle from an image of the obstacle captured by the camera; (b) calculating the distance between the obstacle and the robot arm when moving the hand of the robot arm toward a target position; (c) determining command values ​​for the plurality of joints in accordance with a first control law that causes the hand to move toward the target position and avoids the interference between the obstacle and the specific part, when a first condition is satisfied, the first condition including that the distance between the obstacle and a specific part of the robot arm other than the hand is smaller than the distance between the hand and the obstacle; and (d) determining command values ​​for the plurality of joints in accordance with a second control law that avoids the interference between the obstacle and the hand, when a second condition is satisfied, the second condition including that the distance between the hand and the obstacle is smaller than the distance between the specific part of the robot arm and the obstacle.

[0091] The present disclosure may be realized in various forms other than those described above, such as a robot system including a robot and a robot control device, a computer program for implementing the functions of the robot control device, or a non-transitory storage medium on which the computer program is recorded. [Explanation of symbols]

[0092] 100...robot, 110...base, 120...robot arm, 130...arm end, 150...end effector, 200...control device, 300...information processing device, 310...processor, 320...memory, 330...interface circuit, 340...input device, 350...display device, 400...camera, 500...work table, 610...visual servo control unit, 611...control parameter setting unit, 612...position recognition unit, 613...interference detection unit, 614...command value determination unit, 615...command value transmission unit, 620...collision avoidance control unit, 621...target trajectory calculation unit, 622...position recognition unit, 623...interference detection unit, 624...command value correction unit, 625...command value transmission unit

Claims

1. A method for avoiding interference between a robot arm having multiple joints and an obstacle, comprising: (a) recognizing a position of an obstacle from an image of the obstacle captured by a camera that captures an operating area of ​​the robot arm; (b) calculating a distance between the obstacle and the robot arm when moving the tip of the robot arm toward a target position; (c) when a first condition is satisfied, the first condition including that the distance between the obstacle and a specific part of the robot arm other than the hand is smaller than the distance between the hand and the obstacle, causing the hand to move toward the target position and determining command values ​​of the plurality of joints in accordance with a first control law that avoids the interference between the obstacle and the specific part; (d) determining command values ​​of the plurality of joints in accordance with a second control law that avoids the interference between the obstacle and the endoscope when a second condition is satisfied, the second condition including that the distance between the endoscope and the obstacle is smaller than the distance between the specific portion of the robot arm and the obstacle; A method comprising:

2. 2. The method of claim 1 , The method, wherein the specific part is any one of the plurality of joints.

3. 2. The method of claim 1 , the first condition includes that the distance between the specific portion of the robot arm and the obstacle is equal to or less than an interference threshold; The method, wherein the second condition includes the distance between the hand and the obstacle being equal to or less than the interference threshold.

4. 2. The method of claim 1 , The step (d) (d1) determining a first command value component for moving the hand closer to the target position while giving priority to avoiding interference between the obstacle and the hand; (d2) determining a second command value component for avoiding a singular posture of the robot arm; (d3) determining the command value by adding the first command value component and the second command value component; A method comprising:

5. 5. The method of claim 4, The method, wherein the step (d1) includes a step of calculating the first command value component using a term obtained by subtracting a component for avoiding the obstacle from a difference between a position of the hand end effector and the target position.

6. 2. The method of claim 1 , The method, wherein step (d) includes a process of modifying the command value from a preset value for one joint closest to the hand among one or more joints that can avoid interference between the obstacle and the hand by changing the command value.

7. 1. A robot system comprising: A robot having a robot arm with multiple joints; A camera that photographs the operating area of ​​the robot arm; A control device for controlling the robot; Equipped with The control device includes: (a) a process of recognizing a position of an obstacle from an image of the obstacle captured by the camera; (b) calculating a distance between the obstacle and the robot arm when moving the tip of the robot arm toward a target position; (c) when a first condition is satisfied, the first condition including that the distance between a specific part of the robot arm other than the hand and the obstacle is smaller than the distance between the hand and the obstacle, a process of causing the hand to move toward the target position and determining command values ​​of the plurality of joints in accordance with a first control law that avoids the interference between the obstacle and the specific part; (d) determining command values ​​of the plurality of joints in accordance with a second control law that avoids the interference between the obstacle and the endoscope when a second condition is satisfied, the second condition including that the distance between the endoscope and the obstacle is smaller than the distance between the specific portion of the robot arm and the obstacle; A robotic system configured to:

Images