robot systems

Abstract

A robot system according to one aspect of the present disclosure comprises: a model storing unit that stores a workpiece model; a workpiece detecting unit that detects the position and orientation of a workpiece by comparing features of a surface shape measured by a three-dimensional sensor to features of the workpiece model; a workpiece model positioning unit that positions the workpiece model in a virtual space; and a path setting unit that, in the virtual space, sets a removal path so as to move and withdraw the workpiece model that is an object to be removed without interfering with another workpiece model.

Description

Technical Field

[0001] The present invention relates to a robot system.

Background Art

[0002] Robot systems that acquire surface shape information of a subject such as a distance image and point cloud data and identify the position and orientation of a workpiece by matching processing, and perform workpiece extraction by a robot are widely used. In some cases, it may be necessary to extract workpieces one by one in order from the one placed on the uppermost side among a plurality of randomly overlapping workpieces.

[0003] When a robot extracts a workpiece, it is necessary to determine the posture of the robot so that the hand of the robot does not interfere with other workpieces, a container that houses the workpiece, etc. There has also been proposed to perform overlap determination etc. between a recognition target object and a related object by an information processing apparatus having a reception unit that receives a distance image of a subject, a recognition unit that recognizes a recognition target object (workpiece) in the distance image, a conversion unit that converts information on a predetermined surface of a related object (hand) related to the recognition target object into information on the distance image, and an output unit that outputs an evaluation result based on the converted information (see Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the apparatus described in Patent Document 1, the presence or absence of interference is confirmed using distance images representing the surface shapes of the workpiece and the hand. However, in cases where, for example, the tip of a workpiece is inserted into the opening of another workpiece, it is not possible to accurately determine whether or not there is interference between workpieces based solely on the front-to-back relationship of the distance images. For this reason, even if the shape and arrangement of the workpieces are complex, there is a need for a technology that can more reliably prevent interference between the target workpiece and the hand holding it, as well as other workpieces and other obstacles, when removing a workpiece. [Means for solving the problem]

[0006] A robot system according to one aspect of the present disclosure includes a robot, a three-dimensional sensor for measuring the surface shape of a target area where a workpiece may exist, and a control device for generating an extraction path for the robot to extract at least one of the workpieces based on the surface shape measured by the three-dimensional sensor, wherein the control device includes a model storage unit for storing a workpiece model that models the three-dimensional shape of the workpiece, a workpiece detection unit for detecting the position and orientation of the workpiece by comparing the surface shape characteristics measured by the three-dimensional sensor with the characteristics of the workpiece model, a workpiece model placement unit for arranging the workpiece model in a virtual space at the position and orientation of the workpiece detected by the workpiece detection unit, and a path setting unit for setting the extraction path in the virtual space by moving one of the workpiece models so as not to interfere with other workpiece models. [Effects of the Invention]

[0007] According to this disclosure, when removing a workpiece, interference between the target workpiece and other workpieces or other obstacles in the hand holding it can be prevented. [Brief explanation of the drawing]

[0008] [Figure 1] This is a schematic diagram showing the configuration of a robot system according to the first embodiment of this disclosure. [Figure 2] This is a schematic diagram showing the same workpiece removal process as in the conventional method. [Figure 3]Figure 1 is a schematic diagram illustrating the removal of a workpiece by the robot system. [Modes for carrying out the invention]

[0009] Embodiments of this disclosure will be described below with reference to the drawings. Figure 1 is a schematic diagram showing the configuration of a robot system 1 according to the first embodiment of this disclosure. The robot system 1 picks up at least one of the randomly arranged workpieces W (hereinafter, if it is necessary to distinguish between workpieces, a number will be added to the end of the reference numeral) one at a time. In other words, the robot system 1 picks up one workpiece W from among a plurality of workpieces W.

[0010] The robot system 1 comprises a robot 10, a hand 20 attached to the tip of the robot 10 and capable of holding a workpiece W, a three-dimensional sensor 30 for measuring the surface shape of a target area where the workpiece W may be located, and a control device 40 for generating an operation program for the robot 10 based on the surface shape measured by the three-dimensional sensor 30. In the illustrated example, the multiple workpieces W1, W2, and W3 are identical short tubular parts with a flange at one end.

[0011] Robot 10 determines the position and orientation of hand 20, that is, the coordinates of the reference point of hand 20 and the orientation of hand 20. Robot 10 can be a vertical articulated robot as illustrated in Figure 1, but is not limited to this, and may be a Cartesian coordinate robot, a SCARA robot, a parallel link robot, etc.

[0012] The hand 20 can be any device capable of holding the workpiece W. In the illustrated example, the hand 20 has a pair of finger-like members 21 that grip the workpiece W from the outside or engage with the workpiece W by being inserted inside the workpiece W and expanding outwards. However, it is not limited to this, and the hand 20 may have other holding mechanisms, such as a vacuum pad that sucks on the workpiece W.

[0013] The 3D sensor 30 measures the distance from the 3D sensor 30 to the surface of an object within its field of view (in the illustrated example, three workpieces W1, W2, and W3, and a tray-shaped container C on which workpieces W1, W2, and W3 are placed) in the direction of the center axis of the field of view, at each position in the planar direction perpendicular to the center axis of the field of view. In other words, the 3D sensor 30 acquires surface shape information that can be used to create a 3D image of the object being measured, such as a distance image and point cloud data.

[0014] The 3D sensor 30 can be configured to include two 2D cameras 31 and 32 that capture a 2D image of the object to be measured, and a projector 33 that projects an image containing a grid of reference points onto the object to be measured. Such a 3D sensor 30 captures the object to be measured, onto which the grid of reference points is projected, using the two 2D cameras 31 and 32, and makes it possible to calculate the distance from the 3D sensor 30 to each grid based on the positional shift of the grid caused by the parallax of the images captured by the two 2D cameras 31 and 32. Alternatively, the 3D sensor 30 may be another device capable of performing 3D measurement, such as a 3D laser scanner.

[0015] The control device 40 includes a model storage unit 41, a work detection unit 42, a work model placement unit 43, a target selection unit 44, a hand model placement unit 45, a route setting unit 46, a program generation unit 47, and a program execution unit 48. The control device 40 can be implemented by one or more computer devices that have, for example, memory, a CPU, an input / output interface, etc., and execute an appropriate program. Each component of the control device 40 is classified by its function and does not necessarily have to be clearly distinguishable in terms of physical structure and program structure.

[0016] The model storage unit 41 pre-stores a work model (e.g., CAD data) that models the three-dimensional shape of the workpiece W before performing the operation to remove the workpiece W. Preferably, the model storage unit 41 also pre-stores a hand model that models the three-dimensional shape of the hand 20 and a robot model that models the three-dimensional shape of at least the tip of the robot 10. The model storage unit 41 may store multiple work models with different shapes, and may further store obstacle models that model objects other than the workpiece W that may be present, such as a container C on which the workpiece W is placed.

[0017] The workpiece detection unit 42 converts surface shape information, which represents the surface shape measured by the 3D sensor 30, into 3D point data that can be handled in the same coordinate space as the workpiece model. The workpiece detection unit 42 then detects the position and orientation of each workpiece W1, W2, and W3 by comparing the features of this 3D point data with the features of the workpiece model. Such detection of workpieces W1, W2, and W3 can be performed by a well-known matching process. It is preferable that the workpiece detection unit 42 also detects the position and orientation of obstacles by the matching process. For example, if a container C has a shape that could interfere with the workpiece W or the hand 20 when removing the workpiece W, it is preferable that the workpiece detection unit 42 also detects the position and orientation of the container C.

[0018] The work model placement unit 43 places work models in the virtual space at the positions and orientations of the multiple workpieces W1, W2, and W3 detected by the work detection unit 42. When the target for removal has been selected in advance, it is preferable for the work model placement unit 43 to register only the work models of other workpieces W as obstacles, but it is also possible to temporarily register the work models corresponding to all workpieces W1, W2, and W3 detected by the work detection unit 42 as obstacles. Furthermore, the work model placement unit 43 may also place obstacle models such as containers C on which the multiple workpieces W1, W2, and W3 are placed in the virtual space.

[0019] The target selection unit 44 selects any one of the workpieces W1, W2, and W3 detected by the workpiece detection unit 42 as the extraction target. The target selection unit 44 may select the one located at the uppermost position as the extraction target based on the position and orientation data of the workpieces W1, W2, and W3 detected by the workpiece detection unit 42. However, it is preferable to check the workpiece models arranged by the workpiece model arrangement unit 43 in the virtual space and select the extraction target. As an example, the target selection unit 44 may select, as the extraction target, a workpiece model that has no other workpiece model abutting on the upper side (the side of the three-dimensional sensor 30).

[0020] When the workpiece model arrangement unit 43 registers the workpiece models of all the workpieces W1, W2, and W3 as obstacles, the target selection unit 44 excludes the selected workpiece model of the extraction target from the obstacles. In this way, by temporarily registering all the detected workpieces W1, W2, and W3 as obstacles and then excluding the selected extraction target from the obstacles, the extraction target can be appropriately selected and accurate information can be provided to the path setting unit 46. In addition, when the extraction of the selected workpiece (for example, W2) of the extraction target is completed, the target selection unit 44 configured in this way can select the next extraction target from the remaining workpieces (workpieces W1, W3) registered as obstacles, so there is no need to acquire the surface shape data by the three-dimensional sensor 30 and perform the matching process by the workpiece detection unit 42 again. At this time, the target selection unit 44 appropriately updates the obstacle information by excluding the workpiece model of the new extraction target from the obstacles.

[0021] The hand model placement unit 45 places the hand model in the virtual space at the position and orientation that holds the work model to be taken out. Thereby, interference between the workpieces W1, W2, W3 and the hand 20 can be confirmed. The hand model placement unit 45 may generate a composite model by combining the work model to be taken out and the placed hand model. By generating the composite model, it is only necessary to perform a simulation in which only a single composite model is moved, so the computational load can be suppressed. Further, the hand model placement unit 45 may place the robot model together with the hand model. By performing a simulation that also includes the robot model, interference between the workpieces W1, W2, W3, the hand 20, and the robot 10 can also be confirmed.

[0022] The path setting unit 46 sets a take-out path in the virtual space so that the work model to be taken out does not interfere with other work models, that is, work models registered as obstacles and other obstacle models, and moves and evacuates. The path setting unit 46 preferably evacuates the work model to be taken out without changing the relative positional relationship between the work model and the hand model, that is, moves the work model to be taken out integrally with the hand model, for example, as the above-described composite model. The path setting unit 46 is more preferably configured to set a take-out path so that the work model, the hand model, and the robot model do not interfere with each other. Further, the path setting unit 46 may define the take-out path by a plurality of straight lines or curves joined at one or more intermediate points.

[0023] As an example, when the work W2 to be taken out is lifted in the vertical direction as shown in FIG. 2, the short pipe portion of the work W2 to be taken out interferes with the flange portion of the adjacent work W3. Therefore, as illustrated in FIG. 3, the path setting unit 46 sets a take-out path so that the work W2 to be taken out is lifted in a direction in which it is inclined in order to avoid interference between the work W2 to be taken out and the work W1 and the work W3.

[0024] The program generation unit 47 generates an operation program for the robot 10 that moves the hand 20 along the retrieval path set by the path setting unit 46. Such operation program generation can be performed using well-known techniques.

[0025] The program execution unit 48 operates the robot 10 according to the operation program generated by the program generation unit 47. Specifically, the program execution unit 48 converts the instructions of the operation program into the position or speed of each drive axis of the robot 10 required for that program, generates command values ​​for each drive axis of the robot 10, and inputs these command values ​​to the servo amplifiers that drive each drive axis of the robot 10.

[0026] Robot system 1 places a work model of the workpiece to be retrieved W2, work models of other workpieces W1 and W3 that act as obstacles, and a hand model of the hand 20 that holds the workpiece W2 in a virtual space, and sets up a simulated retrieval path so that the work model of the workpiece W2 and the hand model do not interfere with the work models of the other workpieces W1 and W3. Therefore, robot system 1 only needs to check for interference between the obstacle models and the hand model, which have a smaller data volume compared to the surface shape data acquired by the 3D sensor 30, thus reducing the computational load. In addition, robot system 1 uses data from which noise has been removed from the surface shape data acquired by the 3D sensor 30, so it can set up a more appropriate retrieval path. Furthermore, robot system 1 can consider the shapes of the back sides of the hidden workpieces W1, W2, and W3 that cannot be confirmed by the surface shape data acquired by the 3D sensor 30, so it can set up an even more appropriate retrieval path, and it may be possible to retrieve workpieces even when workpieces with complex shapes are arranged to interlock and there are no workpieces that are fully exposed.

[0027] Although embodiments of this disclosure have been described above, the present invention is not limited to the embodiments described above. Furthermore, the effects described in the embodiments described above are merely a list of preferred effects arising from the present invention, and the effects of the present invention are not limited to those described in the embodiments described above. For example, the robot system according to the present invention may not use a hand model and may only check for interference between the workpiece model to be picked up and other workpiece models. [Explanation of Symbols]

[0028] 1. Robot System 10 Robots 20 hands 31,32 2D cameras 33 Projectors 30 3D sensors 40 Control device 41 Model Memory Unit 42 Workpiece detection unit 43 Work Model Placement Section 44. Target Selection Department 45 Hand model placement section 46 Route setting section 47 Program Generation Unit 48 Program Execution Unit C container W1, W2, W3 Work

Claims

1. A robot having a hand at its tip for gripping a workpiece, A three-dimensional sensor that measures the surface shape of the target area where the aforementioned workpiece may exist, A control device that generates an extraction path for the robot to extract at least one of the workpieces based on the surface shape measured by the three-dimensional sensor, Equipped with, The control device is A model storage unit that stores a work model and a hand model that model the three-dimensional shapes of the work and the hand, A workpiece detection unit detects the position and orientation of the workpiece by comparing the surface shape characteristics measured by the three-dimensional sensor with the characteristics of the workpiece model. A work model placement unit places the work model in a virtual space at the position and orientation of the work detected by the work detection unit, A hand model placement unit that places the hand model in the virtual space in a position and orientation that holds one of the work models, In the virtual space, a path setting unit sets the extraction path in such a way that the relative positional relationship between the first work model and the hand model is maintained and the first work model and the hand model do not interfere with other work models. A robotic system having

2. The robot system according to claim 1, wherein the control device further comprises a target selection unit that selects any of the workpieces detected by the workpiece detection unit as a target for removal.

3. The robot system according to claim 1 or 2, wherein the control device selects the work model arranged by the work model arrangement unit as the target for removal.

4. The work model placement unit registers the other work models as obstacles, A robotic system according to any one of claims 1 to 3, which removes the aforementioned work model from the obstacle.

5. The robot system according to any one of claims 1 to 4, wherein the hand model placement unit generates a composite model by combining the one work model and the hand model.

6. The model storage unit further stores a robot model which is a model of the three-dimensional shape of at least the tip of the robot. The hand model placement unit places the robot model together with the hand model. The robot system according to claim 4 or 5, wherein the path setting unit sets the extraction path so that the work model, the hand model, and the robot model do not interfere with each other.

7. The robot system according to any one of claims 1 to 6, wherein the control device further comprises a program generation unit that generates an operation program for moving the robot along the extraction path set by the path setting unit.

8. The robot system according to claim 7, wherein the control device further comprises a program execution unit that operates the robot in accordance with the operation program generated by the program generation unit.

9. A control device for setting the extraction path of a robot, A model storage unit that stores workpiece models and hand models that model the three-dimensional shapes of the workpiece and the hand attached to the robot, A workpiece detection unit detects the position and orientation of the workpiece by comparing the surface shape characteristics measured by a 3D sensor with the characteristics of the workpiece model, A work model placement unit places the work model in a virtual space at the position and orientation of the work detected by the work detection unit, A hand model placement unit that places the hand model in the virtual space in a position and orientation that holds one of the work models, A path setting unit sets the extraction path so as to maintain the relative positional relationship between the first work model and the hand model, and so as not to interfere with other work models. A control device having

10. A robot having a hand at its tip for gripping a workpiece, A three-dimensional sensor that measures the surface shape of the target area where the aforementioned workpiece may exist, A control device that generates an extraction path for the robot to extract at least one of the workpieces based on the surface shape measured by the three-dimensional sensor, A control method for a robot system comprising: The three-dimensional shapes of the workpiece and the hand are modeled and stored as workpiece models and hand models. By comparing the surface shape characteristics measured by the three-dimensional sensor with the characteristics of the workpiece model, the position and orientation of the workpiece are detected. In the virtual space, the workpiece model is placed at the detected position and orientation of the workpiece. In the virtual space, the hand model is positioned and oriented to hold one of the work models. In the virtual space, the extraction path is set up so as to maintain the relative positional relationship between the first work model and the hand model, and so as not to interfere with other work models. A method for controlling a robotic system.

11. A method for generating a robot extraction path, The workpiece model and hand model, which are 3D models of the workpiece and the hand attached to the robot, are stored. By comparing the surface shape characteristics measured by the 3D sensor with the characteristics of the workpiece model, the position and orientation of the workpiece are detected. In the virtual space, the workpiece model is placed at the detected position and orientation of the workpiece. In the virtual space, the hand model is positioned and oriented to hold one of the work models. A method for setting the extraction path while maintaining the relative positional relationship between the first work model and the hand model, so as not to interfere with other work models.

Images