ROBOT SYSTEM

Abstract

Robot system (1) comprising: a first imaging unit (13) that images workpieces (2) conveyed by a conveying unit; a robot (11) comprising a second imaging unit (14) that images the workpieces (2) conveyed by the conveying unit; a position determination unit (301) which determines positions of the workpieces (2) in accordance with the workpieces (2) recorded by the first imaging unit (13); a three-dimensional information detecting unit (302) that detects three-dimensional information about the workpieces whose positions are detected by the position detecting unit, in accordance with the workpieces (2) captured by the second imaging unit (14); a determination unit (303) which determines, on the basis of the three-dimensional information on the workpieces (2) determined by the three-dimensional information determination unit, whether another workpiece (2) is concealed by an exposed workpiece (2) among the workpieces (2); a receiving unit (112) which receives the exposed workpiece (2); a detection unit (304) which, in a case where it is determined that the other workpiece (2) is hidden by the exposed workpiece (2), detects the other workpiece by causing the second imaging unit (14) to image the other workpiece after exposing the other workpiece (2); and a transmitting unit (305) that transmits a result of detection by the detecting unit (304) to the outside of the detecting unit (304) for use in a picking operation by a robot (11) in a next process.

Description

BACKGROUND OF THE INVENTIONField of the InventionThe present invention relates to a robot system.Related ArtVisual tracking in a robot system refers to an application in which the system uses a camera to find a large number of objects which pass through a conveyor system as a conveying unit, and a robot takes out the objects in accordance with the pass through the conveyor system. This makes it possible to remove the objects passing through the conveyor jointly by a plurality of robots.As a device using visual tracking, there is known a capturing device that recognizes the position and posture of a workpiece in the initial stage capturing operation using a first stereo camera, repeatedly operates a second stereo camera while moving the hand of the robot by tracking the moving position of the workpiece, calculates the displacement amount of the position and the posture of the workpiece, followed by correction of the displacement amount, thereby controlling the target position for the final capturing operation and the hand posture (see, e.g., JP 2004-001 122 A).DE 10 2018 000 730 A1 describes a workpiece pickup device including a sensor that measures a plurality of workpieces randomly stacked in a three-dimensional space, a robot that holds the workpieces, a hand mounted on the robot, the hand being capable of holding the workpieces, a holding position posture calculation unit that calculates holding position posture data of a position and a posture for holding the workpieces by the robot based on an output of the sensor, a loading state improvement operation generation unit that generates data on an improving loading state improvement operation of an improving loading state of the workpieces by the robot based on an output of the sensor, a robot control unit, which controls the robot and the hand, a candidate operation output unit including the at least one holding position posture calculation unit and the at least one state-of-charge improvement operation generation unit and outputs at least one candidate operation, and an operation determination unit which determines an operation from the at least one candidate operation output by the candidate operation output unit and controls the robot via the robot control unit based on the determined operation.DE 10 2011 108 169 B4 describes a workpiece extraction apparatus comprising a camera for capturing a workpiece loading area comprising a plurality of workpieces arranged in a stack, a workpiece detection area for detecting a workpiece based on the camera image captured by the camera, a workpiece selection area for selecting a workpiece to be unloaded based on the detection result of the workpiece detection area, a robot for unloading the workpiece selected by the workpiece selection area, a loading state determination section for determining whether a positional state of the workpieces in the workpiece loading area has changed due to an operation of the robot, and an area determination section for determining a workpiece detection area in which the workpiece detection section detects a workpiece. If the posture state acquisition section acquires a change in the posture state of the workpieces, the range determination section determines the workpiece acquisition range as a circumferential range around a change position of the posture state, i.e., in a portion of the workpiece loading range.EP 3 346 446 B1 describes an image processing system for receiving overlapping workpieces which are transported without changing the position of the individual workpieces. An image processing unit has a field of view including the conveyor and captures an object in the image processing field. The imaging unit generates a luminance image representing the object and a distance image including height information indicating the height of at least one point on the object from the conveyor by imaging the object. A measurement unit measures the position of each workpiece in the luminance image. A detection unit determines an overlapping order of the workpieces on the basis of the height information in the range image corresponding to the position of each workpiece in the luminance image. A determination unit determines the workpiece pick order so that a workpiece that is the first place in the overlapping order can be picked up with higher priority than other workpieces.EP 3 290 165 A2 describes that, based on received first area information indicating a first area which is an area determined for a captured image of a plurality of target objects and second area information indicating a second area different from the first area, a robot control device causes a robot to grasp the target object for which the second area which does not overlap with the first area of another one of the target objects is determined and for which the first area which does not overlap with the first area determined for the other target object is determined.The document EP 2 636 493 A2 describes that the position and orientation of a target object is determined on the basis of first measurement data which are obtained by a first sensor (first calculation). The position and orientation of the target object is determined based on second measurement data obtained from a movable second sensor set in a position and orientation different from that of the first sensor (second calculation). When the position and orientation obtained by the first calculation and the position and orientation obtained by the second calculation coincide with each other, the position and orientation obtained by the first or second calculation or a position and orientation calculated from the position and orientation obtained by the first calculation and the position and orientation obtained by the second calculation is output as the position and orientation of the target object.Document EP 1 589 483 B1 describes a system for receiving objects, one after the other, of a plurality of objects. The system includes a capturing section that captures, as an image, an object to be captured from among a plurality of objects arranged so as to be at least partially superimposed on each other, a storage section that stores appearance information of a predetermined portion of a reference object having an outer appearance identical to an outer appearance of the object to be captured, a determination section that determines, in the image of the object to be captured captured captured captured by the capturing section, whether an inspected portion of the object to be captured corresponding to the predetermined portion of the reference object is hidden by another object based on the appearance information of the reference object stored in the storage section, a control section that determines a capturing motion for the object to be captured based on a determination result of the determination section and outputs a control signal for the capturing motion, and a capturing mechanism, executing the picking motion for the object to be picked up in accordance with the control signal outputted from the control section.It is therefore an object of the claimed invention to provide an efficient robot system with which errors in the picking up of a workpiece in the robot system can be reduced.SUMMARY OF THE INVENTIONThis object is achieved by the subject matter of the independent claim. Further embodiments form the subject matter of the dependent claims.However, the pickup device disclosed in JP 2004-001 122 A is not a device in a case where workpieces are stacked.When the workpieces passing through the conveyor are stacked using visual tracking, the workpieces hidden under a workpiece disposed above are not detected because they cannot be measured even when the image is taken, and thereby are omitted. As a solution to this problem, a method of causing a workpiece to circle through an arcuate conveyor, a method of providing a return mechanism, or a method of performing multiple imaging and measurement by a single fixed camera are conceivable. However, the installation of such a system is costly.It is desirable to provide a robot system that can efficiently pick up conveyed workpieces stacked at low cost.According to an aspect of the present disclosure, a robot system includes: a first imaging unit that images workpieces conveyed by a conveying unit; a robot including a second imaging unit that images the workpieces conveyed by the conveying unit; a position acquisition unit that acquires positions of the workpieces according to the workpieces picked up by the first imaging unit; a three-dimensional information acquisition unit that acquires three-dimensional information on the workpieces according to the workpieces picked up by the second imaging unit, the positions of which are acquired by the position acquisition unit; a determination unit that determines whether another workpiece is hidden by an exposed workpiece among the workpieces according to the three-dimensional information acquired by the three-dimensional information acquisition unit; and a pickup unit that picks up the exposed workpiece; a detection unit that detects the other workpiece by causing the second imaging unit to image the other workpiece after exposing the other workpiece in a case where it is determined that the other workpiece is hidden by the exposed workpiece; and a transmission unit that transmits a result of the detection by the detection unit to the outside of the detection unit for use in a robot capturing operation in a next process.According to one aspect, it is possible to provide a robot system that can efficiently pick up conveyed workpieces stacked at low cost.BRIEF DESCRIPTION OF THE DRAWINGSFIG. 1 is a diagram of the overall configuration of an embodiment; FIG. 2 is a functional block diagram of a control unit according to an embodiment; FIG. 3 is a flowchart showing the operation flow in an embodiment; FIG. 4 is a diagram showing the positional relationship of each component during operation according to an embodiment; FIG. 5 is a diagram showing the positional relationship of each component during operation according to an embodiment; FIG. 6 is a diagram showing the positional relationship of each component during operation according to an embodiment; FIG. 7 is a diagram showing the positional relationship of each component during operation according to an embodiment; and FIG. 8 is a diagram showing the positional relationship of each component during operation according to an embodiment.DETAILED DESCRIPTION OF THE INVENTIONNext, referring to Figs. 1 to 8, description will be made of the embodiment of the present invention.FIG. 1 is a diagram showing the overall configuration of a robot system 1 according to an embodiment of the present invention. As shown in FIG. 1, the robot system 1 according to the embodiment includes a robot 11, a conveyor 12 (hereinafter also referred to as a "conveyance unit"), a fixed camera 13 (hereinafter also referred to as a "first imaging unit"), a 3D image sensor 14 (hereinafter also referred to as a "second imaging unit"), and a control unit 30.Moreover, workpieces 2 including, for example, a workpiece 2A and a workpiece 2B are conveyed on the conveyor 12, the workpiece 2A being stacked on the workpiece 2B.The robot 11 has a plurality of drive axes for positioning the front portion. The robot 11 is typically a robot of the vertically articulated type such as that shown. However, the robot 11 may be, for example, a robot having rectangular coordinates, a scalar robot, a robot having parallel connections, or the like.Specifically, in the present embodiment, the robot 11 is a robot for picking up the workpiece 2 passing through from the upstream side on the conveyor 12. The robot 11 includes an arm 111 and a hand 112 (hereinafter also referred to as a "pick unit"). The hand 112 is provided at the front portion of the arm 111. The robot 11 moves the hand 112 by driving the arm 111 close to the workpiece 2 as a target, and takes the workpiece 2 with the hand 112.The conveyor 12 is a device for conveying the workpiece 2 from the upstream side of the robot 11 to the downstream side of the robot 11.The fixed camera 13 is a device for imaging the workpiece 2 passing further on the upstream side than the robot 11 on the conveyor 12, and it is preferable that the relative position of the fixed camera 13 with respect to the conveyor 12 be fixed. However, the present invention is not limited thereto. Moreover, the fixed camera 13 is specifically a camera for imaging a two-dimensional image of the workpiece 2, and is generally less expensive than the 3D image sensor 14 described later.In particular, in the present embodiment, the fixed camera 13 is used to determine the rough position of the workpiece 2 on the conveyor 12. Moreover, it is also possible for the fixed camera 13 to have typical functions.The 3D image sensor 14 is a sensor capable of capturing a three-dimensional image of an object. Specifically, the 3D image sensor 14 in the present embodiment is attached to the arm 111 of the robot 11. With such a configuration, the 3D image sensor 14 can acquire the three-dimensional information on the workpiece 2 before and after the workpiece 2 is picked up by the hand 112 of the robot 11 by an approach to the workpiece 2. However, the present invention is not limited thereto, and the 3D image sensor 14 may be installed at any position as long as the position allows the three-dimensional image of the workpiece 2 to be captured by the 3D image sensor.The control unit 30 is constituted by a microprocessor (not shown) including a CPU and a memory. In the control unit 30, the CPU of the microprocessor corresponding to the robot system 1 performs various kinds of control in accordance with predetermined programs read out from the memory. A description will be given of a part of the functions of the control unit 30.FIG. 2 is a functional block of the control unit 30. the control unit 30 includes a position determination unit 301, a three-dimensional information determination unit 302, a determination unit 303, an acquisition unit 304, and a transmission unit 305.The position acquisition unit 301 acquires a rough position of the workpiece 2 on the conveyor 12 from a captured image of the workpiece 2 captured by the fixed camera 13.The three-dimensional information acquisition unit 302 acquires, from the captured image of the workpiece 2 captured by the 3D image sensor 14, the three-dimensional information on the workpiece 2 whose position is acquired by the position acquisition unit 301. Here, the three-dimensional information includes, among other things, position information, posture information, and height information on the workpiece 2. in particular, in the present embodiment, the three-dimensional information acquisition unit 302 acquires, when the hand 112 approaches the workpiece 2 whose position is acquired by the position acquisition unit 301, the three-dimensional information on the workpiece 2 by driving the arm 111 using the 3D image sensor 14 installed on the arm 111.For example, the determination unit 303 determines whether the workpiece 2B is hidden by the workpiece 2A exposed when viewed from the fixed camera 13 on the workpiece 2 based on the three-dimensional information on the workpiece 2 acquired by the three-dimensional information acquisition unit 302.In particular, in the present embodiment, in a case where the height in the height information included in the three-dimensional information on the workpiece 2 acquired by the three-dimensional information acquisition unit 302 is higher than the thickness of each of the workpieces 2A and 2B, it is possible to determine that the workpiece 2B is hidden by the exposed workpiece 2A. Note that the position where the workpiece 2B is hidden is not limited to the position under the exposed workpiece 2A, and moreover, the workpiece 2B does not need to be completely hidden.Alternatively, the determination unit 303 may compare the ratio of the area occupied by the workpiece 2B in the image of the workpiece 2 with a predetermined threshold in the three-dimensional image captured by the 3D image sensor 14, and determine whether the workpiece 2B is hidden by the exposed workpiece 2A based on the result of the comparison.In addition, for example, JP 2010-184 308 A, JP 2014-046 371 A, and JP 2018-134 698 A have already filed applications related to the technology for detecting and detecting the relative position and posture of a plurality of workpieces when stacked workpieces. By incorporating these technologies into the present invention, it is possible to determine whether the workpiece 2B is hidden by the exposed workpiece 2A.For example, in the present invention, a technology may be employed in which any two workpieces among a plurality of workpieces and an overlapping surface on which the two workpieces overlap each other are estimated from the position and posture of each workpiece and the dimension information on the workpieces, and in a case where a certain shape of a workpiece among the two workpieces is included in this overlapping surface, the gradient information on the individual pixels of the image of the overlapping surface is calculated and a workpiece among the two workpieces is located on the other workpiece based on the gradient information.Alternatively, in the present invention, a technology may be adopted in which the surface position of a plurality of articles arranged in bulk in a three-dimensional space is measured, the position information on a plurality of three-dimensional points is acquired, a set of joints formed by connecting the three-dimensional points close to each other among the plurality of acquired three-dimensional points is calculated, and the position and posture of an article is determined from the position information on the three-dimensional points belonging to the set of joints.Alternatively, in the present invention, a technology may be adopted in which three-dimensional points each having height position information are acquired as three-dimensional information on the workpiece in the container, a plurality of sets of the three-dimensional points in which the adjacent points satisfy a predetermined condition are created among groups of the three-dimensional points, a set that is within at least one of the reference value to a predetermined size, the reference value to a predetermined area, and the reference value to a predetermined length is set as a set of exclusions among the plurality of sets of three-dimensional points, By excluding the points included in the set of exclusions from the three-dimensional point group for acquiring a workpiece or the plurality of three-dimensional point sets, a three-dimensional point group for acquiring a workpiece is acquired, and the acquisition of a workpiece for extraction by the robot is performed using the three-dimensional point for acquiring the workpiece.In a case where the determination unit 303 determines that the workpiece 2B is hidden by the exposed workpiece 2A, the detection unit 304 detects the workpiece 2B with the 3D image sensor 14 after the exposed workpiece 2A is picked up by the hand 112, thereby exposing the workpiece 2B.In particular, in the present embodiment, it is possible to acquire the workpiece 2B by causing the 3D image sensor 14 to image the workpiece 2B during the "retracting operation" after the exposure of the exposed workpiece 2A by the hand 112, that is, during the operation of conveying the workpiece 2A to another position by the hand 112 after gripping the workpiece 2A.The transmission unit 305 transmits a result of the detection by the detection unit 304 to the outside of the detection unit 304 for use in the operation of being taken by the robot 11.In the present embodiment, the transmission of the result of the detection by the detection unit 304 to the control unit of the robot 11A is performed for use in the process of capturing by the robot 11A in the next process by the transmission unit 305.FIG. 3 is a flowchart showing the operation of the robot system 1 according to the present embodiment. Moreover, FIGS. 4 to 8 are views showing positional relationships and states of the respective components of the robot system 1 at each step of the flowchart shown in FIG. 3. Next, referring to FIGS. 3 to 8, the operation of the robot system 1 will be described.In step S 1, as shown in FIG. 4, the position acquisition unit 301 acquires the rough position of the workpiece 2 on the conveyor 12 from a captured image of the workpiece 2 ( 2A, 2B) passing through the conveyor 12 captured by the fixed camera 13.In step S 2, as shown in FIG. 5, the three-dimensional information obtaining unit 302 obtains, from the captured image of the workpiece 2 captured by the 3D image sensor 14, the three-dimensional information on the workpiece 2 whose position is obtained by the position obtaining unit 301.In step S 3, as shown in FIG. 6, the hand 112 receives the exposed workpiece 2A.In step S 4, in a case where the hidden workpiece 2B is present when the workpiece 2A (S 4: YES), the processing proceeds to step S 5. In a case where there is no hidden workpiece 2B when the workpiece 2A is picked up (S 4: NO), the processing is reset to step S 2.In step S 5, as shown in FIG. 7, during the retraction operation after picking up the workpiece 2A, the detection unit 304 detects the workpiece 2B that has been hidden.In step S 6, as shown in FIG. 8, the transmission unit 305 transmits the result of the detection of the hidden workpiece 2B by the next process detection unit 304 to the control unit of the robot 11A. With such a configuration, in the next process, the robot 11A receives the workpiece 2B that has been hidden. Subsequently, the processing is reset to step S 2.The robot system 1 according to the present embodiment includes: the fixed camera 13 that images the workpieces 2 conveyed on the conveyor 12; the robot 11 including the 3D image sensor 14 that images the workpieces 2 conveyed by the conveyor 12; the position determination unit 301 that determines the positions of the workpieces 2 according to the workpieces 2 captured by the fixed camera 13; the three-dimensional information determination unit 302 that determines three-dimensional information on the workpieces 2 whose positions are determined by the position determination unit 301 according to the workpieces 2 captured by the 3D image sensor 14; the determination unit 303 that determines whether another workpiece 2B is hidden from the exposed workpiece 2A among the workpieces 2 from the three-dimensional information acquired by the three-dimensional information acquisition unit 302; the hand 112 that picks up the exposed workpiece 2A; a detection unit 304 that detects the other workpiece 2B by causing the 3D image sensor 14 to image the other workpiece 2B after exposing the workpiece 2B in a case where it is determined that the other workpiece 2B is hidden from the exposed workpiece 2A; and the transmission unit 305 that transmits a result of the detection by the detection unit 304 to the outside of the detection unit 304 for use in an operation of picking up the robot 11.With such a configuration, it is possible to efficiently receive a workpiece 2 stacked on the conveyor 12. Moreover, since it is unnecessary to prepare another mechanism such as a sheet conveying apparatus, it is possible to reduce the cost.In the robot system 1 according to the present embodiment, the transmission unit 305 transmits the result of detection for use in the operation of capturing by the robot 11A in the next process to the control unit of the robot 11A.By configuring this way, it is possible to reduce errors in the picking up of the workpiece 2 in the robot system 1.Moreover, in the robot system 1 according to the present embodiment, the fixed camera 13 may acquire a two-dimensional image of the workpiece 2, and the relative position of the fixed camera 13 with respect to the conveyor 12 may be fixed.By virtue of such a configuration, it is possible to ascertain a rough position of the workpiece 2 on the conveying installation 12 using a relatively inexpensive fixed camera 13.Moreover, in the robot system 1 according to the present embodiment, the 3D image sensor 14 may be attached to the arm 111 of the robot 11.With such a configuration, it is possible to detect, by the 3D image sensor 14, the workpiece 2B that has been hidden during the retraction operation.Moreover, in the robot system 1 according to the present embodiment, the above-described three-dimensional information may include the height information, the position information, and the posture information about the workpiece 2.With such a configuration, by comparing the height of the workpiece 2 with the thickness of the workpiece 2, it is possible to determine whether there is stacking of the workpieces 2.EXPLANATION OF THE REFERENCE NUMERALS1 Robot system 2, 2 a, 2 b Werkstück 11 Robot 12 Conveyor 13 Fixed camera (first imaging unit) 14 3D Image sensor (second imaging unit) 30 Control unit 111 Arm 112 Hand (capturing unit) 301 Position obtaining unit (position obtaining unit) 302 Three-dimensional information obtaining unit (three-dimensional information obtaining unit) 303 Determining unit (determining unit) 304 Detecting unit (detecting unit) 305 Transmitting unit (transmitting unit)

Claims

A robot system (1) comprising: a first imaging unit (13) that images workpieces (2) conveyed by a conveying unit; a robot (11) that includes a second imaging unit (14) that images the workpieces (2) conveyed by the conveying unit; a position obtaining unit (301) that obtains positions of the workpieces (2) according to the workpieces (2) taken by the first imaging unit (13); a three-dimensional information obtaining unit (302) that obtains three-dimensional information on the workpieces whose positions are obtained by the position obtaining unit according to the workpieces (2) taken by the second imaging unit (14); a determination unit (303) that determines whether another workpiece (2) is hidden by an exposed workpiece (2) among the workpieces (2) based on the three-dimensional information on the workpieces (2) acquired by the three-dimensional information acquisition unit (2); a pickup unit (112) that picks up the exposed workpiece (2); a detection unit (304) that, in a case where it is determined that the other workpiece (2) is hidden by the exposed workpiece (2), detects the other workpiece by causing the second imaging unit (14) to image the other workpiece after the exposure of the other workpiece (2); and a transmission unit (305) that transmits a result of the detection by the detection unit (304) to the outside of the detection unit (304) for use in a pickup operation by a robot (11) in a next process.The robot system (1) according to claim 1, wherein the first imaging unit (13) acquires a two-dimensional image of the workpieces and fixes a relative position with respect to the conveying unit.The robot system (1) according to claim 1 or 2, wherein the second imaging unit (14) is attached to an arm of the robot (11).The robot system (1) according to any one of claims 1 to 3, wherein the three-dimensional information includes height information, position information, and posture information about the workpiece (2).

Images