Robot, robot system, and control method

Abstract

The robot (10) comprises: a base (11); a rotating part (12) rotating relative to the base (11); a multi-joint arm (20) connected to the rotating part (12) and changing the position and posture of a tool (30) used for operation; and a camera (40) fixed to the rotating part (12), wherein the movable range (31) of the tool (30) of the multi-joint arm (20) overlaps with the field of view 41 of the camera (40).

Description

Technical Field

[0001] The present disclosure relates to a robot, a robot system, and a control method. Background Art

[0002] Patent Document 1 discloses a robot system including a work table for supporting a workpiece, a robot arm for working on the workpiece, and a camera provided above the work table, and specifies the position and posture of the workpiece based on an image acquired from the camera.

[0003] Patent Document 1: Japanese Patent Application Publication No. 2014-180720 Summary of the invention

[0004] The present disclosure provides a robot that effectively achieves both effective use of camera images for robot control and simplification of system configuration.

[0005] A robot according to one aspect of the present disclosure comprises: a base; a rotating part that rotates relative to the base; a multi-joint arm that is connected to the rotating part and changes the position and posture of a tool used for an operation; and a camera that is fixed to the rotating part, and the movable range of the tool based on the multi-joint arm overlaps with the field of view of the camera.

[0006] A robot system according to another aspect of the present disclosure includes: the robot described above; a command generation unit that generates an operation command based on an image captured by a camera of a workpiece; and an arm control unit that controls a multi-joint arm based on the operation command to perform a first operation on the workpiece using a tool.

[0007] Another aspect of the present disclosure involves a control method that includes: acquiring an image of a workpiece arranged in a working area around a base of the robot from a camera provided on a rotating part of the robot, and generating an action instruction based on the image, wherein the robot comprises a base, a rotating part rotating relative to the base, and a multi-joint arm connected to the rotating part and changing the position and posture of a tool used for working; and controlling the multi-joint arm based on the action instruction to perform a first operation on the workpiece through the tool.

[0008] According to the present disclosure, a robot is provided that effectively achieves both effective use of camera images for robot control and simplification of system configuration. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] Figure 1 This is a schematic diagram showing the general structure of the robot system.

[0010] Figure 2 yes Figure 1 Front view of the robot system.

[0011] Figure 3 This is a block diagram illustrating the hardware configuration of the controller.

[0012] Figure 4 This is a flowchart illustrating a workpiece conveyance control procedure.

[0013] Figure 5 This is a flowchart illustrating a workpiece conveyance control procedure.

[0014] Figure 6 This is a flowchart showing a modified example of the workpiece transport control procedure. DETAILED DESCRIPTION

[0015] Hereinafter, the embodiments will be described in detail with reference to the drawings. In the description, the same elements or elements having the same functions are denoted by the same reference numerals, and repeated descriptions are omitted.

[0016] [Robotic system]

[0017] Figure 1 The robot system 1 shown is a system that performs a predetermined operation on an object to be operated (hereinafter referred to as a "workpiece") on a production line of a product. As a specific example of the operation, there can be cited a pick-up and place operation of picking up a workpiece and transporting it to a predetermined transport target position and posture and placing it, but it is not limited to this. As other examples of the operation, there can be cited an operation of welding a workpiece to other workpieces, an operation of fastening a workpiece to other workpieces by bolts, etc., an operation of performing processing such as grinding on a workpiece, etc. Figure 1 As shown, the robot system 1 includes a robot 10 and a controller 100 .

[0018] The robot 10 is a so-called vertically articulated industrial robot, and includes a base 11 , a rotating portion 12 , a multi-jointed arm 20 , actuators 71 , 72 , 73 , 74 , 75 , 76 , 77 , a tool 30 , and a camera 40 .

[0019] The base 11 is fixed to the robot support 3 located near the work area 5. The base 11 may also be fixed to an AGV (Automated Guided Vehicle). The rotating portion 12 is provided on the base 11. The rotating portion 12 rotates relative to the base 11. For example, the rotating portion 12 can rotate around a vertical axis 51.

[0020] The multi-joint arm 20 is connected to the rotary unit 12 to change the position and posture of the working tool 30. The multi-joint arm 20 has more than 6 independent degrees of freedom. For example, the multi-joint arm 20 is a 6-axis serial link type arm, and has a first arm 21, a second arm 22, a third arm 23 and a tool holding unit 24.

[0021] The first arm 21 is connected to the rotating part 12, and swings around an axis 52 passing through the connecting part of the first arm 21 and the rotating part 12. The axis 52 (first axis) intersects (for example, is orthogonal to) the central axis of rotation of the rotating part 12 (the above-mentioned axis 51). The intersection here also includes a torsional relationship such as a so-called three-dimensional intersection. The same applies to the following. The first arm 21 has an arm base 25 and a rotating arm 26. The arm base 25 is connected to the rotating part 12, and is along an axis 53 that intersects (for example, is orthogonal to) the axis 52. The rotating arm 26 is connected to the top end of the arm base 25, and rotates along and around the axis 53.

[0022] The second arm 22 is connected to the top end of the first arm 21, rotates around an axis 53 (second axis) along the first arm 21, and swings around an axis 54 (third axis) intersecting (e.g., orthogonal to) the axis 53. For example, the second arm 22 is connected to the top end of the revolving arm 26, and swings around an axis 54 passing through the connection portion between the second arm 22 and the revolving arm 26. In addition, the second arm 22 rotates around the axis 53 together with the revolving arm 26. The second arm 22 has an arm base 27 and a revolving arm 28. The arm base 27 is connected to the top end of the revolving arm 26, and rotates along an axis 55 intersecting (e.g., orthogonal to) the axis 54. The revolving arm 28 is connected to the top end of the arm base 27, and rotates along and around the axis 55.

[0023] The third arm 23 is connected to the top end of the second arm 22, rotates around the axis 55 (fourth axis) along the second arm 22, and swings around the axis 56 (fifth axis) intersecting (e.g., orthogonal) the axis 55. For example, the third arm 23 is connected to the top end of the rotating arm 28, and swings around the axis 56 passing through the connection portion between the third arm 23 and the rotating arm 28. In addition, the third arm 23 rotates around the axis 55 together with the rotating arm 28. The third arm 23 is along the axis 57 intersecting (e.g., orthogonal) the axis 56.

[0024] The tool holding portion 24 is provided at the tip end portion of the third arm 23 , holds the tool 30 , and rotates around an axis 57 (sixth axis) along the third arm 23 .

[0025] In this way, the robot 10 has: a joint 61 connecting the base 11 and the rotating part 12; a joint 62 connecting the rotating part 12 and the arm base 25; a joint 63 connecting the arm base 25 and the rotating arm 26; a joint 64 connecting the rotating arm 26 and the arm base 27; a joint 65 connecting the arm base 27 and the rotating arm 28; a joint 66 connecting the rotating arm 28 and the third arm 23; and a joint 67 connecting the third arm 23 and the tool holding part 24.

[0026] The actuators 71, 72, 73, 74, 75, 76, 77 include, for example, electric motors and reducers, and drive the joints 61, 62, 63, 64, 65, 66, 67, respectively. For example, the actuator 71 is built into the base 11 (for example, housed in the housing 81 of the base 11), and causes the rotating portion 12 to rotate around the axis 51. The actuator 72 is built into the rotating portion 12 (for example, housed in the housing 82 of the rotating portion 12), and causes the arm base 25 to swing around the axis 52. The actuator 73 is built into the arm base 25 (for example, housed in the housing 83 of the arm base 25), and causes the rotating arm 26 to rotate around the axis 53. The actuator 74 is built into the rotating arm 26 (for example, housed in the housing 84 of the rotating arm 26), and causes the arm base 27 to swing around the axis 54. The actuator 75 is built into the arm base 27 (for example, housed in the housing 85 of the arm base 27), and causes the swivel arm 28 to swivel around the axis 55. The actuator 76 is built into the swivel arm 28 (for example, housed in the housing 86 of the swivel arm 28), and causes the third arm 23 to swing around the axis 56. The actuator 77 is built into the third arm 23 (for example, housed in the housing 87 of the third arm 23), and causes the tool holding portion 24 to swivel around the axis 57.

[0027] The tool 30 acts on the workpiece W when the robot 10 performs work on the workpiece W. Specific examples of the tool 30 include a suction nozzle that holds the workpiece W by suction, a hand that holds the workpiece W by grasping the workpiece W, and the like. Other examples of the tool 30 include a welding gun, a screw tightening tool (e.g., an electric screwdriver), and a grinding tool (e.g., a grinder).

[0028] The camera 40 is fixed to the rotating part and photographs the working area 5. The camera 40 has a photographing element such as a CCD (Charged-Coupled Devices) or a CMOS (Complementary Metal-Oxide-Semiconductor) sensor, and an optical system that images the image of the field of view 41 on the photographing element. The camera 40 can be a camera that obtains a two-dimensional image (e.g., a color image, a black and white image) that represents the brightness, hue, etc. of the photographing object for each pixel, or a ToF (Time-of-Flight) camera that obtains a distance image that represents the distance to the photographing object for each pixel. The camera 40 can also be a camera that obtains both a two-dimensional image and a distance image.

[0029] The camera 40 is fixed to the rotating unit 12 so that the relationship between the coordinate system based on the camera 40 (hereinafter referred to as the “camera coordinate system”) and the coordinate system based on the rotating unit 12 (hereinafter referred to as the “rotating coordinate system”) does not change. Figure 1The rotation coordinate system illustrated in the example has an X-axis, a Y-axis, and a Z-axis. The Z-axis is a coordinate axis that faces vertically upward. The Y-axis is a coordinate axis along the axis 52. The X-axis is a coordinate axis that is perpendicular to the Y-axis and the Z-axis. The origin of the rotation coordinate system is located at the intersection of the axis 52 and the reference plane 43 (see Figure 2 ) Hereinafter, the positive direction of the X-axis is set to the front of the robot 10. Figure 1 The camera coordinate system illustrated in FIG. 4 has a CX axis, a CY axis, and a CZ axis. The CX axis is a coordinate axis along the optical axis 42 at the center of the field of view 41, and the direction away from the camera 40 is set as the positive direction. The CZ axis is a coordinate axis with the upper side of the image formed on the imaging element as the positive direction. The CY axis is a coordinate axis perpendicular to the CX axis and the CZ axis.

[0030] The camera 40 may also be built into the rotating part 12 (for example, stored in the housing 82). For example, the camera 40 is arranged in front of the actuator 72 in the housing 82. A window 82a for shooting is formed in the front part of the housing 82 (see FIG. Figure 2 ), the camera 40 captures an image of the front oblique lower portion of the rotating portion 12 through the window 82a.

[0031] The camera 40 on the rotary unit 12 is arranged so that the movable range 31 of the tool 30 based on the multi-jointed arm 20 overlaps with the field of view 41 of the camera 40. For example, the camera 40 is arranged so that the movable range 31 and the field of view 41 overlap in the work area 5.

[0032] For example, the camera 40 is arranged with the optical axis 42 along a reference plane 43 (refer to Figure 2 ) is fixed to the rotating part 12. The camera 40 can also be fixed to the rotating part 12 in a manner that the optical axis 42 is oriented obliquely downward in the reference plane 43. As an example, the camera 40 is fixed to the rotating part 12 in a manner that the CX axis (optical axis 42) is oriented obliquely downward. Specifically, the CX axis is perpendicular to the Y axis, and the positive direction of the CX axis is oriented in the positive direction of the X axis and the negative direction of the Z axis. The angle formed by the CX axis and the X axis around the Y axis is, for example, 30 to 60 degrees.

[0033] Figure 2 This is the front view of the robot 10 as viewed from the positive direction of the X-axis. Figure 2 As shown, the movable range A1 of the first arm 21 may be away from the reference plane 43 of the camera 40 in the direction along the Y axis (axis line 52). For example, the movable range A1 is offset relative to the reference plane 43 in the negative direction of the Y axis.

[0034] Assuming that the camera 40 is located within the movable range of the first arm 21 when the movable range A1 in the direction along the axis 52 is consistent with the reference plane 43, in order to fully ensure the movable angle of the first arm 21, the axis 52 needs to be arranged at a position sufficiently above the camera 40. Therefore, the multi-joint arm 20 is enlarged. On the other hand, if the configuration height of the camera 40 is lowered in order to suppress the enlargement of the multi-joint arm 20, the shooting range of the camera 40 becomes narrower. In this way, it is difficult to take into account both the space saving of the multi-joint arm 20 and the widening of the shooting range of the camera 40. In contrast, by configuring the movable range A1 in the direction along the axis 52 to be away from the reference plane 43, the configuration height of the camera 40 can be increased without affecting the movable range of the first arm 21, thereby achieving a widening of the shooting range of the camera 40. Figure 1 In the figure, the camera 40 is located below the axis 52 , but the camera 40 may be arranged above the axis 52 .

[0035] In addition, the movable range A2 of the second arm 22 may be away from the first arm 21 in the direction along the axis 54. Furthermore, the movable range A3 of the third arm 23 may be away from the second arm 22 in the direction along the axis 56. Hereinafter, the direction in which the movable range A2 of the second arm 22 is away from the first arm 21 along the axis 54 is referred to as the "deviation direction of the second arm 22". The direction in which the movable range A3 of the third arm 23 is away from the second arm 22 along the axis 56 is referred to as the "deviation direction of the third arm 23".

[0036] Figure 2 This indicates a state where both the offset direction of the second arm 22 and the offset direction of the third arm 23 are oriented in the positive direction of the Y axis. In this state, the movable range A2 of the second arm 22 may overlap the reference plane 43. In addition, the movable range A3 of the third arm 23 may be away from the reference plane 43.

[0037] return Figure 1 The robot system 1 may further include a support portion 2 that supports the workpiece W in the working area 5 around the base 11 . The upper surface 2 a of the support portion 2 may be located below the rotating portion 12 or below the base 11 .

[0038] The controller 100 controls the robot 10. For example, the controller 100 is configured to: acquire an image of the workpiece W arranged in the working area 5 from the camera 40, and generate an action command based on the image; and control the multi-joint arm 20 based on the action command to perform a first operation on the workpiece W through the tool 30. For example, the controller 100 has an image processing unit 112, an instruction generation unit 113, an arm control unit 114, and a rotation control unit 111 as functional components (hereinafter referred to as "functional blocks").

[0039] The image processing unit 112 calculates the position and posture of the workpiece W in the three-dimensional space based on the captured image obtained by capturing the workpiece W (for example, the workpiece W disposed on the support unit 2) by the camera 40. For example, the image processing unit 112 calculates the position and posture of the workpiece W in the camera coordinate system based on the shape and size of the workpiece W in the captured image and the known three-dimensional shape and size of the workpiece W, and performs coordinate conversion on the calculation result, thereby calculating the position and posture of the workpiece W in the rotation coordinate system.

[0040] The robot 10 may also include two cameras 40 for stereoscopic observation. In this case, the image processing unit 112 may calculate the position and posture of the workpiece W in the camera coordinate system based on the images captured by the two cameras 40 .

[0041] The command generation unit 113 generates an action command based on a captured image of the workpiece W (e.g., the workpiece W disposed on the support unit 2) captured by the camera 40. The action command includes, for example, a plurality of commands in a time series. Each command includes at least a target position and posture of the tool 30. For example, the command generation unit 113 generates an action command based on the position and posture of the workpiece W calculated by the image processing unit 112.

[0042] As an example, when generating an action command for picking up a workpiece W, the command generation unit 113 calculates the target position and target posture (hereinafter referred to as "holding position and posture") of the tool 30 for holding the workpiece W based on the position and posture of the workpiece W calculated by the image processing unit 112, and calculates an action command to shift the tool 30 from the current position and posture to the holding position and posture. Furthermore, the command generation unit 113 calculates an action command to shift the tool 30 holding the workpiece W at the holding position and posture to a predetermined lifting position and posture.

[0043] The arm control unit 114 controls the multi-joint arm 20 based on the motion command so that the tool 30 performs the first operation on the workpiece W. For example, the arm control unit 114 calculates the motion angles of the joints 62, 63, 64, 65, 66, and 67 through inverse kinematics calculation so that the tool 30 moves according to the motion command, and controls the actuators 72, 73, 74, 75, 76, and 77 so that the joints 62, 63, 64, 65, 66, and 67 move according to the calculated motion angles.

[0044] The arm control unit 114 moves the tool 30 in conjunction with the displacement of the tool 30 by the multi-joint arm 20. For example, when the first operation is to pick up the workpiece W, the arm control unit 114 causes the tool 30 to hold the workpiece W while the tool 30 is arranged to maintain the position and posture.

[0045] The swing control unit 111 swings the swing unit 12 so that the camera 40 faces the workpiece W before the command generation unit 113 generates an operation command, and then stops the swing unit 12 until the multi-joint arm 20 completes the first operation.

[0046] The controller 100 may further include a work monitoring unit 115. The work monitoring unit 115 detects an irregular state in the first work based on an image captured by the camera 40 of the workpiece W while the multi-joint arm 20 is performing the first work. When the first work is a work of picking up the workpiece W, specific examples of the irregular state include displacement of the workpiece W in the process of moving the tool 30 to a position for holding the workpiece W, poor holding of the workpiece W by the tool 30, and the like.

[0047] When the operation monitoring unit 115 detects an irregular state, the command generation unit 113 may also correct the motion command based on the captured image of the workpiece W captured by the camera 40. For example, when the operation monitoring unit 115 detects the displacement of the workpiece W in the process of moving the tool 30 to the position for holding the workpiece W, the command generation unit 113 corrects the motion command based on the position and posture of the workpiece W after the displacement calculated by the image processing unit 112. When the operation monitoring unit 115 detects that the tool 30 does not hold the workpiece W properly, the arm control unit 114 releases the tool 30 from holding the workpiece W, and then the command generation unit 113 corrects the motion command based on the position and posture of the workpiece W calculated by the image processing unit 112. The arm control unit 114 causes the multi-joint arm 20 to continue the first operation based on the corrected motion command.

[0048] The instruction generation unit 113 may further generate a second action instruction based on a captured image of other workpieces W (e.g., other workpieces W on the support unit 2) captured by the camera 40 during the multi-joint arm 20 performing the first operation. For example, based on a captured image of the other workpieces W (the second workpiece) captured by the camera 40 during the multi-joint arm 20 performing the first operation, the image processing unit 112 calculates the position and posture of the second workpiece. The instruction generation unit 113 calculates the second action instruction based on the calculation result of the position and posture of the second workpiece by the image processing unit 112. After the multi-joint arm 20 performs the first operation, the arm control unit 114 controls the multi-joint arm 20 based on the second action instruction so as to perform the second operation on the second workpiece through the tool 30.

[0049] When the first operation is an operation of picking up the workpiece W, the rotation control unit 111 rotates the rotation unit 12 when the multi-joint arm 20 has picked up the workpiece W through the first operation. The arm control unit 114 may also control the multi-joint arm 20 during at least a part of the period when the rotation control unit 111 rotates the rotation unit 12 so as to arrange the workpiece W being picked up within the field of view of the camera 40.

[0050] In this case, the controller 100 may further include a holding state monitoring unit 116 . The holding state monitoring unit 116 checks the holding state of the workpiece W by the tool 30 based on an image captured by the camera 40 while the rotating unit 12 is rotating by the rotation control unit 111 .

[0051] In addition, each functional block described above is a component of the controller 100 , and therefore, the processing executed by each functional block is equivalent to the processing executed by the controller 100 .

[0052] Figure 3 is a block diagram illustrating the hardware configuration of the controller 100. Figure 3 As shown, the controller 100 has a circuit 190. The circuit 190 includes one or more processors 191, a memory 192, a storage 93, an image processing circuit 194, and a drive circuit 195. The storage 93 is, for example, a non-volatile semiconductor memory, etc., and has a computer-readable storage medium. The storage 93 stores a program that enables the controller 100 to perform the following processing: acquiring an image of a workpiece W configured in the working area 5 from the camera 40, generating an action instruction based on the image; and controlling the multi-joint arm 20 based on the action instruction to perform a first operation on the workpiece W through the tool 30.

[0053] The memory 192 temporarily stores the program downloaded from the storage medium of the memory 93 and the calculation results of the processor 191. The processor 191 cooperates with the memory 192 to execute the above program, thereby constituting each functional block of the controller 100. The image processing circuit 194 performs image processing according to the request from the processor 191. As a specific example of image processing, the recognition of the shape and size of the workpiece W in the captured image obtained from the camera 40 can be cited. The drive circuit 195 outputs drive power to the actuators 71, 72, 73, 74, 75, 76, and 77 according to the instruction from the processor 191.

[0054] In addition, the circuit 190 is not necessarily limited to configuring each function by a program. For example, the circuit 190 may configure at least a part of the functions by a dedicated logic circuit or an ASIC (Application Specific Integrated Circuit) integrating the dedicated logic circuit.

[0055] [Control sequence of the robot]

[0056] Next, as an example of a control method, a control sequence of the robot 10 executed by the controller 100 is illustrated. The sequence includes: acquiring an image of the workpiece W arranged in the working area 5 around the base 11 of the robot 10 from the camera 40 provided on the rotating part 12, generating an action command based on the image; and controlling the multi-joint arm 20 based on the action command to perform a first operation on the workpiece W by using the tool 30. The robot 10 includes a base 11, a rotating part 12 rotating relative to the base 11, and a multi-joint arm 20 connected to the rotating part 12 and changing the position and posture of the tool 30 used for operation.

[0057] The following is an example of a sequence for causing the robot 10 to perform a pick-up and placement operation on a plurality of workpieces W on a support portion 2. The sequence includes: further generating a second motion instruction based on a captured image of another workpiece W (a second workpiece) captured by the camera 40 while the multi-joint arm 20 performs a first operation on the workpiece W; and controlling the multi-joint arm 20 based on the second motion instruction to perform a second operation on the second workpiece through the tool 30 after the multi-joint arm 20 performs the first operation.

[0058] like Figure 4 As shown, first, the controller 100 executes steps S01, S02, S03, S04, S05, S06, S07, S08, S09, and S11. In step S01, the rotation control unit 111 rotates the rotation unit 12 so that the camera 40 faces the plurality of workpieces W on the support unit 2, and stops the rotation unit 12 in a state where the camera 40 faces the plurality of workpieces W. Thereafter, the rotation control unit 111 maintains the rotation unit 12 in a stopped state until the multi-joint arm 20 completes the picking and placing operation of the plurality of workpieces W.

[0059] In step S02, the image processing unit 112 acquires the captured images of the plurality of workpieces W from the camera 40. In step S03, the image processing unit 112 calculates the position and posture of any one of the workpieces W (hereinafter referred to as the "target workpiece") in the camera coordinate system based on the captured images acquired from the camera 40. In step S04, the image processing unit 112 performs coordinate conversion on the position and posture in the camera coordinate system, and calculates the position and posture of the target workpiece in the rotation coordinate system.

[0060] In step S05, the command generation unit 113 calculates an action command for holding the target workpiece (hereinafter referred to as a "holding command") based on the captured image of the target workpiece. For example, the command generation unit 113 calculates the target position and posture (the above-mentioned holding position and posture) of the tool 30 for holding the target workpiece based on the position and posture of the target workpiece calculated by the image processing unit 112. In step S06, the arm control unit 114 controls the multi-joint arm 20 based on the holding command so as to configure the tool 30 at the holding position and posture.

[0061] In step S07, the image processing unit 112 acquires a captured image of the target workpiece from the camera 40. In step S08, the image processing unit 112 calculates the position and posture of the target workpiece in the camera coordinate system based on the captured image acquired from the camera 40. In step S09, the image processing unit 112 performs coordinate conversion on the position and posture in the camera coordinate system, and calculates the position and posture of the target workpiece in the rotation coordinate system.

[0062] In step S11, the operation monitoring unit 115 checks whether the position and posture of the target workpiece are within a normal range. For example, the operation monitoring unit 115 checks whether the difference between the position and posture of the target workpiece calculated by the image processing unit 112 in step S04 and the position and posture of the target workpiece calculated by the image processing unit 112 in step S09 is within an allowable range.

[0063] When it is determined in step S11 that the position and posture of the target workpiece are not within the normal range, the controller 100 returns the process to step S03. Thus, the command generation unit 113 corrects the holding command based on the captured image of the target workpiece. The arm control unit 114 controls the multi-joint arm 20 based on the corrected holding command so as to configure the tool 30 at the corrected holding position and posture.

[0064] When it is determined in step S11 that the position and posture of the target workpiece are within the normal range, the controller 100 executes steps S12 , S13 , S14 , S15 , S16 , S17 , and S18 .

[0065] In step S13, the command generation unit 113 calculates an action command (hereinafter referred to as a "lifting command") so that the tool 30 holding the target workpiece is shifted to a predetermined lifting position and posture. In step S14, the arm control unit 114 controls the multi-joint arm 20 based on the lifting command so that the tool 30 is arranged in the lifting position and posture.

[0066] In step S15, the image processing unit 112 acquires the captured images of the plurality of workpieces W from the camera 40. In step S16, the image processing unit 112 calculates the position and posture of the target workpiece in the camera coordinate system based on the captured images acquired from the camera 40. In step S17, the image processing unit 112 performs coordinate conversion on the position and posture in the camera coordinate system, and calculates the position and posture of the target workpiece in the rotation coordinate system.

[0067] In step S18, the work monitoring unit 115 checks whether the position and posture of the target workpiece are within a normal range. For example, the work monitoring unit 115 checks whether the relative position and posture of the target workpiece with respect to the tool 30 are within a normal range.

[0068] When it is determined in step S18 that the position and posture of the target workpiece are not within the normal range, the controller 100 executes step S19. In step S19, the arm control unit 114 releases the tool 30 from holding the workpiece W. The arm control unit 114 may also release the tool 30 from holding the workpiece W after controlling the multi-joint arm 20 to return the tool 30 from the raised position and posture to the held position and posture. Thereafter, the controller 100 returns the processing to step S03. Thus, the instruction generation unit 113 corrects the holding instruction based on the captured image of the target workpiece. The arm control unit 114 controls the multi-joint arm 20 based on the corrected holding instruction so as to keep the tool 30 in the corrected holding position and posture.

[0069] When it is determined in step S18 that the position and posture of the target workpiece are within the normal range, Figure 5 As shown, the controller 100 executes steps S21, S22, and S23. In step S21, the instruction generation unit 113 calculates an action instruction (hereinafter referred to as a "transport instruction") so that the tool 30 is shifted to a target position and posture for configuring the target workpiece at the transport target position and posture (hereinafter referred to as a "release target position and posture"). In step S22, the arm control unit 114 controls the multi-joint arm 20 based on the transport instruction so that the tool 30 starts to shift toward the release target position and posture. In step S23, the instruction generation unit 113 confirms whether there is a workpiece W that has not been transported among the multiple workpieces W.

[0070] When it is determined in step S23 that there are still untransported workpieces W, the controller 100 executes steps S24, S25, S26, S27, S28, and S29. In step S24, the image processing unit 112 obtains a captured image of the untransported workpiece W from the camera 40 among the plurality of workpieces W. In step S25, the image processing unit 112 selects the next target workpiece (the second workpiece) from the untransported workpiece W based on the captured image obtained from the camera 40, and calculates the position and posture of the next target workpiece in the camera coordinate system. In step S26, the image processing unit 112 performs coordinate conversion on the position and posture in the camera coordinate system, and calculates the position and posture of the next target workpiece in the rotation coordinate system.

[0071] In step S27, the command generation unit 113 calculates the holding command (second action command) of the next target workpiece based on the captured image of the next target workpiece. For example, the command generation unit 113 calculates the target position and posture (the above-mentioned holding position and posture) of the tool 30 for holding the next target workpiece based on the position and posture of the next target workpiece calculated by the image processing unit 112.

[0072] In step S28, the arm control unit 114 waits for the tool 30 to reach the release target position and posture. In step S29, the arm control unit 114 releases the tool 30 from holding the target workpiece. After that, the controller 100 returns the process to step S06. Thus, the pick-up and placement operation for the next target workpiece is started based on the holding instruction (second action instruction) calculated in step S27.

[0073] When it is determined in step S23 that there are no uncarried workpieces W left, the controller 100 executes steps S31 and S32. In step S31, the arm control unit 114 waits for the tool 30 to reach the release target position and posture. In step S32, the arm control unit 114 releases the tool 30 from holding the target workpiece. The above-mentioned picking and placing operation of multiple workpieces W is completed.

[0074] (Variation of control sequence)

[0075] The following is an example of a sequence of causing the robot 10 to perform a picking operation on the workpiece W on the support part 2, then causing the rotating part 12 to rotate, and then causing the robot 10 to perform an operation of transporting the workpiece W to a predetermined transport target position and posture and placing it. The sequence includes: rotating the rotating part 12 in a state where the multi-joint arm 20 has picked up the workpiece W by the first operation; controlling the multi-joint arm 20 so that the workpiece W being picked up is arranged within the field of view of the camera 40 during at least a portion of the period during which the rotating part 12 is rotated by the rotation control part 111; and checking the holding state of the workpiece W by the tool 30 based on the captured image obtained by the camera 40 capturing the workpiece W being picked up during the period during which the rotation control part 111 rotates the rotating part 12. In this sequence, the sequence of causing the robot 10 to perform the picking operation on the workpiece W is the same as the above-mentioned steps S01 to S19, so the description is omitted.

[0076] When it is determined in step S18 that the position and posture of the target workpiece are within the normal range, Figure 6As shown, the controller 100 executes steps S41, S42, S43, S44, S45, S46, and S47. In step S41, the rotation control unit 111 starts to rotate the rotation unit 12 so that the camera 40 is directed toward the target position of the target workpiece. In step S42, the arm control unit 114 controls the multi-joint arm 20 so that the target workpiece being picked up is arranged within the field of view 41 of the camera 40.

[0077] In step S43, the image processing unit 112 acquires the captured images of the plurality of workpieces W from the camera 40. In step S44, the image processing unit 112 calculates the position and posture of the target workpiece in the camera coordinate system based on the captured images acquired from the camera 40. In step S45, the image processing unit 112 performs coordinate conversion on the position and posture in the camera coordinate system, and calculates the position and posture of the target workpiece in the rotation coordinate system.

[0078] In step S46, the holding state monitoring unit 116 checks the holding state of the workpiece W by the tool 30 based on the position and posture of the target workpiece calculated in step S45. For example, the holding state monitoring unit 116 calculates the relative position and posture of the workpiece W with respect to the tool 30. In addition, if the workpiece W at the time of completion of the picking is located within the field of view 41 of the camera 40, and a captured image for checking the holding state of the workpiece W by the tool 30 can be obtained, step S42 can be omitted.

[0079] In step S47, the rotation control unit 111 checks whether the rotation unit 12 rotates until the camera 40 faces the target position of the target workpiece. If it is determined in step S47 that the rotation unit 12 has not rotated until the camera 40 faces the target position of the target workpiece, the controller 100 returns the process to step S43. Thereafter, the tool 30 repeatedly checks the holding state of the workpiece W until the camera 40 faces the target position of the workpiece.

[0080] When it is determined in step S47 that the swivel unit 12 swivels until the camera 40 faces the target position of the target workpiece, the controller 100 executes steps S48, S51, S52, S53, S54, S55, S56, and S57. In step S48, the swivel control unit 111 stops the swivel unit 12. Thereafter, the swivel control unit 111 maintains the swivel unit 12 in a stopped state until the multi-joint arm 20 completes the operation of transporting the target workpiece to the transport target position and posture and placing it.

[0081] In step S51, the arm control unit 114 controls the multi-joint arm 20 so that the target workpiece being picked up is retracted from the field of view 41 of the camera 40. In step S52, the image processing unit 112 obtains a captured image of the transport target position from the camera 40. In addition, even if the workpiece W is within the field of view 41 of the camera 40, step S51 can be omitted if the captured image of the transport target position can be obtained. In step S53, the image processing unit 112 calculates the transport target position and posture in the camera coordinate system based on the captured image obtained from the camera 40. In step S54, the image processing unit 112 performs coordinate conversion on the transport target position and posture in the camera coordinate system, and calculates the transport target position and posture in the rotation coordinate system.

[0082] In step S55, the above-mentioned transport instruction is calculated so that the tool 30 is shifted to the above-mentioned release target position and posture. In step S56, the arm control unit 114 controls the multi-joint arm 20 based on the transport instruction so that the tool 30 is shifted to the release target position and posture. In step S57, the arm control unit 114 releases the tool 30 from holding the workpiece W. Above, the picking and placing operation of the target workpiece is completed.

[0083] [Effects of this embodiment]

[0084] As described above, the robot 10 includes a base 11, a rotating part 12 that rotates relative to the base 11, a multi-joint arm 20 that is connected to the rotating part 12 and changes the position and posture of a tool 30 for operation, and a camera 40 fixed to the rotating part 12, wherein the movable range 31 of the tool 30 based on the multi-joint arm 20 overlaps with the field of view 41 of the camera 40.

[0085] In the robot 10, the camera 40 is provided at the rotating part 12. Thus, the camera 40 is integrated with the robot 10, so that the system configuration can be simplified. In addition, the movable range of the tool 30 based on the multi-joint arm 20 overlaps with the field of view of the camera 40. Thus, images including both the tool 30 and the work object of the tool 30 (the above-mentioned workpiece W) can be acquired from the camera 40. Therefore, it is easy to effectively use the image of the camera 40 for the control of the robot 10. Therefore, the robot 10 effectively takes into account both the effective use of the camera image for robot control and the simplification of the system configuration.

[0086] The multi-joint arm 20 may have six or more degrees of freedom. In this case, a shake-free image can be acquired from the camera 40 , and the position and posture of the tool 30 can be freely adjusted by the multi-joint arm 20 .

[0087] The multi-joint arm 20 may include: a first arm 21 connected to the rotating portion 12, swinging around an axis 52 intersecting the rotation center axis of the rotating portion 12; a second arm 22 connected to the top end of the first arm 21, rotating around an axis 53 along the first arm 21, and swinging around an axis 54 intersecting the axis 53; a third arm 23 connected to the top end of the second arm 22, rotating around an axis 55 along the second arm 22, and swinging around an axis 56 intersecting the axis 55; and a tool holding portion 24 provided at the top end of the third arm 23, holding the tool 30, and rotating around an axis 57 along the third arm 23. In this case, six degrees of freedom can be given to the multi-joint arm 20 by a simple structure.

[0088] Alternatively, the optical axis 42 of the first arm 21 and the center of the field of view 41 may be along a reference plane 43 intersecting the axis 52, and the movable range A1 of the first arm 21 may be away from the reference plane 43 in a direction along the axis 52. In this case, by making the base end of the first arm 21 deviate from the center of the field of view 41, it is difficult to form a blind spot caused by the multi-jointed arm 20.

[0089] The robot system 1 includes: a robot 10; a command generation unit 113 that generates an action command based on an image captured by a camera 40 capturing a workpiece W; and an arm control unit 114 that controls the multi-joint arm 20 based on the action command so that the tool 30 performs a first operation on the workpiece W. In this case, the image of the camera 40 can be effectively used for controlling the robot 10.

[0090] The robot system 1 may further include a work monitoring unit 115 that detects an irregularity in the first work based on an image captured by the camera 40 of the workpiece W while the multi-joint arm 20 is performing the first work. In this case, the image of the camera 40 can be used more effectively for controlling the robot 10.

[0091] Alternatively, when the operation monitoring unit 115 detects an irregular state, the command generating unit 113 corrects the operation command based on the image captured by the camera 40 capturing the workpiece W, and the arm control unit 114 causes the multi-joint arm 20 to continue the first operation based on the corrected operation command. In this case, the image of the camera 40 can be more effectively used for controlling the robot 10.

[0092] Alternatively, the instruction generation unit 113 further generates a second motion instruction based on an image captured by the camera 40 of another workpiece W (second workpiece) while the multi-joint arm 20 is performing the first operation, and the arm control unit 114 controls the multi-joint arm 20 based on the second motion instruction to perform the second operation on the second workpiece through the tool 30. In this case, an image of the second workpiece is acquired during the execution of the first operation, thereby shortening the operation time for the first and second workpieces.

[0093] Alternatively, the robot system 1 further includes a rotation control unit 111, which rotates the rotation unit 12 so that the camera 40 faces the workpiece W before the command generation unit 113 generates an action command, and then stops the rotation unit 12 until the multi-joint arm 20 completes the first operation. In this case, an image without shaking during the first operation can be obtained from the camera 40. Therefore, the image of the camera 40 can be used more effectively for controlling the robot 10.

[0094] Alternatively, the first operation may be an operation of picking up the workpiece W, the rotation control unit 111 rotates the rotation unit 12 in a state where the multi-joint arm 20 has picked up the workpiece W through the first operation, and the arm control unit 114 controls the multi-joint arm 20 so as to arrange the workpiece W being picked up within the field of view 41 of the camera 40 during at least a portion of the period during which the rotation control unit 111 rotates the rotation unit 12. In this case, the rotation of the rotation unit 12 does not affect the relative positional relationship between the workpiece W being picked up and the camera 40, and the image of the camera 40 can be more effectively used for controlling the robot 10.

[0095] The robot system 1 may further include a holding state monitoring unit 116 that checks the holding state of the tool 30 on the workpiece W based on an image captured by the camera 40 of the workpiece W being picked up while the rotation control unit 111 rotates the rotation unit 12. In this case, the image acquired from the camera 40 during the rotation can be used more effectively.

[0096] Alternatively, the robot system 1 may further include an image processing unit 112 that calculates the position and posture of the workpiece W in the three-dimensional space based on an image captured by the camera 40 capturing the workpiece W, and the command generation unit 113 generates an action command based on the position and posture of the workpiece W calculated by the image processing unit 112. In this case, the image from the camera 40 can be used more effectively for controlling the robot 10.

[0097] Alternatively, the rotating part 12 may be disposed above the base 11, and the camera 40 may be disposed on the rotating part 12 with the optical axis 42 of the center of the field of view 41 directed obliquely downward. In this case, the camera 40 observes the workpiece W in a three-dimensional manner from obliquely upward, so that an image including both the tool 30 and the workpiece W can be easily obtained.

[0098] The robot system 1 may further include a support unit 2 that supports the workpiece W around the base 11, wherein the upper surface 2a of the support unit 2 is located below the rotating unit 12. In this case, an image including both the tool 30 and the workpiece W can be acquired more easily.

[0099] As mentioned above, although embodiment was described, this invention is not necessarily limited to the said embodiment, Various deformation|transformation is possible within the range which does not deviate from the summary.

[0100] Explanation of symbols:

[0101] 1: robot system; 2: support part; 2a: upper surface; 5: working area; 10: robot; 11: base; 12: rotating part; 20: multi-joint arm; 21: first arm; 22: second arm; 23: third arm; 24: tool holding part; 30: tool; 31: movable range of tool 30; 40: camera; 41: field of view; 42: optical axis; 43: reference plane; 51: axis (rotation center axis of rotating part 12); 5 2: axis (first axis); 53: axis (second axis); 54: axis (third axis); 55: axis (fourth axis); 56: axis (fifth axis); 57: axis (sixth axis); 111: rotation control unit; 112: image processing unit; 113: instruction generating unit; 114: arm control unit; 115: operation monitoring unit; 116: holding status monitoring unit; A1: movable range of the first arm 21; W: workpiece.

Claims

1. A robot having: base; A rotating portion, rotating relative to the base; A multi-jointed arm connected to the rotating part and configured to change the position and posture of a tool used for operation; and A camera is fixed to the rotating part. The multi-joint arm comprises a first arm, the first arm is connected to the rotating part and swings around a first axis, and the first axis intersects with the rotation center axis of the rotating part. Based on the overlap between the movable range of the tool of the multi-jointed arm and the field of view of the camera, The position of the first arm and the position of the camera are different from each other in a direction along the first axis, so that the camera is located outside the movable range of the first arm around the first axis, The optical axis of the center of the field of view is along a reference plane intersecting the first axis, and the movable range of the first arm is away from the reference plane in a direction along the first axis. The camera is arranged above the first axis.

2. The robot according to claim 1, wherein: The multi-joint arm has more than 6 degrees of freedom.

3. The robot according to claim 2, wherein: The multi-jointed arm has: A second arm connected to the top end of the first arm, rotating around a second axis along the first arm and swinging around a third axis, the third axis intersecting the second axis; a third arm connected to the top end of the second arm, rotating around a fourth axis along the second arm and swinging around a fifth axis, the fifth axis intersecting the fourth axis; as well as The tool holding portion is provided at the top end of the third arm, holds the tool, and rotates around a sixth axis along the third arm.

4. A robot system comprising: The robot according to any one of claims 1 to 3; a command generating unit that generates an action command based on an image captured by the camera of the workpiece; and The arm control unit controls the multi-joint arm based on the motion command so as to perform a first operation on the workpiece by using the tool.

5. The robot system according to claim 4, further comprising: The work monitoring unit detects an irregular state in the first work based on an image captured by the camera of the workpiece while the multi-joint arm is performing the first work.

6. The robot system according to claim 5, wherein: When the operation monitoring unit detects an irregular state, the command generating unit corrects the operation command based on an image captured by the camera of the workpiece. The arm control unit causes the multi-joint arm to continue the first task based on the corrected motion command.

7. The robot system according to claim 5 or 6, wherein: The command generation unit further generates a second motion command based on a captured image of a second workpiece captured by the camera while the multi-joint arm is performing the first operation. The arm control unit controls the multi-joint arm based on the second motion command so as to perform a second operation on the second workpiece using the tool.

8. The robot system according to any one of claims 4 to 6, further comprising: The rotation control unit rotates the rotation unit so that the camera faces the workpiece before the command generation unit generates the motion command, and then stops the rotation unit until the multi-joint arm completes the first operation.

9. The robot system according to claim 8, wherein: The first operation is an operation of picking up the workpiece, The rotation control unit rotates the rotation unit in a state where the multi-joint arm picks up the workpiece by the first operation, The arm control unit controls the multi-joint arm so that the workpiece being picked up is arranged within the field of view of the camera during at least a portion of a period in which the rotation control unit rotates the rotation unit.

10. The robot system according to claim 9, further comprising: The holding state monitoring unit checks the holding state of the workpiece by the tool based on an image captured by the camera while the rotating unit is rotating the rotating unit.

11. The robot system according to any one of claims 4 to 6, further comprising: an image processing unit that calculates the position and posture of the workpiece in a three-dimensional space based on an image captured by the camera. The command generation unit generates the operation command based on the position and posture of the workpiece calculated by the image processing unit.

12. The robot system according to any one of claims 4 to 6, wherein: The rotating part is arranged above the base. The camera is installed on the rotating part in such a manner that the optical axis of the center of the field of view faces obliquely downward.

13. The robot system according to claim 12, further comprising: a support portion that supports the workpiece around the base portion, The upper surface of the support portion is located below the rotating portion.

Images