Robotic system

By using the first imaging unit to obtain the workpiece position and the second imaging unit to obtain three-dimensional information, combined with the determination and detection unit, the robot system can effectively select stacked workpieces, solve the problem that stacked workpieces cannot be effectively processed in the prior art, and reduce the problem of reducing the problem of stacked workpieces Cost and efficiency are improved.

CN112060073BActive Publication Date: 2024-11-12FANUC LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202010517737.6
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2019-06-11
Filing Date
2020-06-09
Publication Date
2024-11-12
Estimated Expiration
2040-06-09

AI Technical Summary

Technical Problem

Existing robotic systems cannot effectively detect and remove the workpieces hidden on them when processing stacked workpieces, and the existing solutions are costly and difficult to achieve low-cost and efficient picking.

Method used

The first camera unit is used to obtain the position of the workpiece, the second camera unit (3D vision sensor) obtains three-dimensional information, and the judgment unit determines whether the exposed workpiece hides other workpieces, and after the exposed workpiece is taken out through the robot arm, the 3D vision sensor detects the hidden workpiece, and the transmission unit Pass the test results to guide the removal action.

Benefits of technology

The stacked workpieces are removed at low cost and efficiently, reducing system leakage and avoiding additional equipment costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112060073B_ABST
    Figure CN112060073B_ABST
Patent Text Reader

Abstract

The present invention provides a robot system that can efficiently take out the stacked workpieces being transported at a low cost. The robot system comprises: a position acquisition unit that acquires the position of the workpiece based on the workpiece photographed by the first camera unit; a three-dimensional information acquisition unit that acquires the three-dimensional information of the workpiece whose position is acquired based on the workpiece photographed by the second camera unit; a determination unit that determines whether the exposed workpiece hides other workpieces among the workpieces based on the three-dimensional information of the workpiece; a taking-out unit that is used to take out the exposed workpiece; a detection unit that, when it is determined that the exposed workpiece hides other workpieces, after the other workpieces are exposed, causes the second camera unit to photograph the other workpieces to detect the other workpieces; and a transmission unit that transmits the detection result of the detection unit to the outside of the detection unit for use in the robot's taking-out action.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a robot system. Background Art

[0002] Visual tracking in a robot system refers to the following application: a camera is used to find a large number of objects transported on a conveyor as a transport unit, and the robot takes them out in coordination with the conveyor. In this way, multiple robots can share the task of taking out the objects transported on the conveyor.

[0003] As a device using visual tracking, there is a known picking device that identifies the position and posture of a workpiece in an initial detection operation using a first stereo camera, causes a robot hand to move to track the moving position of the workpiece, and causes a second stereo camera to repeat the operation to find the offset of the position and posture of the workpiece, which is then corrected to control the target position and hand posture of the final picking action (see, for example, patent document 1).

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2004-1122 Summary of the invention

[0007] Problem that the invention aims to solve

[0008] However, the picking device disclosed in Patent Document 1 is not a device that can cope with a situation where workpieces are piled up.

[0009] When using visual tracking, when the workpieces transported by the conveyor are stacked together, even if the workpiece hidden under the workpiece located above is photographed, it cannot be measured, so it is not detected and missed. As a solution, a method of using an arc conveyor to rotate the workpiece, preparing a return mechanism, and using a fixed camera to perform multiple photography and measurement is considered, but it costs money to set up the system.

[0010] It is desirable to provide a robot system that can efficiently remove stacked workpieces being transported at low cost.

[0011] Solutions for solving problems

[0012] One embodiment of the present disclosure is a robot system, comprising: a first camera unit that photographs a workpiece transported by a transport unit; a robot that comprises a second camera unit that photographs the workpiece transported by the transport unit; a position acquisition unit that acquires the position of the workpiece based on the workpiece photographed by the first camera unit; a three-dimensional information acquisition unit that acquires three-dimensional information of the workpiece whose position is acquired by the position acquisition unit based on the workpiece photographed by the second camera unit; a determination unit that determines whether an exposed workpiece hides other workpieces among the workpieces based on the three-dimensional information of the workpiece acquired by the three-dimensional information acquisition unit; a removal unit that is used to remove the exposed workpiece; a detection unit that, when it is determined that the exposed workpiece hides the other workpiece, causes the second camera unit to photograph the other workpiece to detect the other workpiece after the other workpiece is exposed; and a transmission unit that transmits the detection result of the detection unit to the outside of the detection unit for use in the removal action of the robot.

[0013] Effects of the Invention

[0014] According to one embodiment, it is possible to provide a robot system capable of efficiently taking out stacked workpieces being conveyed at low cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is an overall structural diagram of an implementation method.

[0016] Figure 2 This is a functional block diagram of a control unit in one embodiment.

[0017] Figure 3 is a flowchart showing the operation of one embodiment.

[0018] Figure 4 The diagram shows the positional relationship between components during operation of one embodiment.

[0019] Figure 5 The diagram shows the positional relationship between components during operation of one embodiment.

[0020] Figure 6 The diagram shows the positional relationship between components during operation of one embodiment.

[0021] Figure 7 The diagram shows the positional relationship between components during operation of one embodiment.

[0022] Figure 8The diagram shows the positional relationship between components during operation of one embodiment.

[0023] Description of Reference Numerals

[0024] 1: Robot system; 2, 2A, 2B: Workpiece; 11: Robot; 12: Conveyor; 13: Fixed camera (first camera unit); 14: 3D vision sensor (second camera unit); 30: Control unit; 111: Arm; 112: Hand (removal unit); 301: Position acquisition unit (position acquisition unit); 302: Three-dimensional information acquisition unit (three-dimensional information acquisition unit); 303: Judgment unit (judgment unit); 304: Detection unit (detection unit); 305: Transfer unit (transfer unit). DETAILED DESCRIPTION

[0025] Below, refer to Figures 1 to 8 Embodiments of the present invention will be described.

[0026] [Structure of 1 embodiment]

[0027] Figure 1 1 is a diagram showing the overall configuration of a robot system 1 according to an embodiment of the present invention.

[0028] like Figure 1 As shown, a robot system 1 based on one embodiment includes a robot 11, a conveyor (hereinafter also referred to as a "transporting unit") 12, a fixed camera (hereinafter also referred to as a "first camera unit") 13, a 3D vision sensor (hereinafter also referred to as a "second camera unit") 14 and a control unit 30.

[0029] In addition, it is assumed that, on the conveyor 12 , for example, the workpiece 2 including the workpiece 2A and the workpiece 2B is conveyed in a state where the workpiece 2A is stacked on the workpiece 2B.

[0030] The robot 11 has a plurality of drive axes for positioning the front end portion. The robot 11 is typically a vertical multi-joint robot as shown in the figure, but may also be an orthogonal coordinate robot, a SCARA (Selective Compliance Articulated Robot Arm), a parallel link robot, etc.

[0031] In particular, in the present embodiment, the robot 11 is a robot for picking up the workpiece 2 conveyed from upstream on the conveyor 12. The robot 11 includes an arm 111 and a hand (hereinafter also referred to as a "picking unit") 112. The hand 112 is provided at the front end of the arm 111. The robot 11 drives the arm 111 to bring the hand 112 close to the workpiece 2 as a target object, and picks up the workpiece 2 with the hand 112.

[0032] The conveyor 12 is a device that conveys the workpiece 2 from a position upstream of the robot 11 to a position downstream of the robot 11. The conveyor 12 may be, for example, a belt conveyor.

[0033] The fixed camera 13 is a device for photographing the workpiece 2 conveyed on the conveyor 12 at a position upstream of the robot 11. Preferably, the relative position of the fixed camera 13 to the conveyor 12 is fixed, but the present invention is not limited thereto. In addition, the fixed camera 13 is a camera for photographing a two-dimensional image of the workpiece 2, and is generally cheaper than the 3D vision sensor 14 described later, but the present invention is not limited thereto.

[0034] In particular, in the present embodiment, the fixed camera 13 is used to obtain the approximate position of the workpiece 2 on the conveyor 12 , and in addition to this, it can also have the functions that the fixed camera 13 generally has.

[0035] The 3D vision sensor 14 is a sensor capable of capturing a three-dimensional image of an object. In particular, in the present embodiment, the 3D vision sensor 14 is fixed to the arm 111 of the robot 11. Thus, before and after the robot 11 picks up the workpiece 2 by the hand 112, the 3D vision sensor 14 can obtain three-dimensional information of the workpiece 2 due to its proximity to the workpiece 2. However, this is not limited to this, and the 3D vision sensor 14 can be set at any position as long as it is in a position capable of capturing a three-dimensional image of the workpiece 2.

[0036] The control unit 30 is composed of a microprocessor (not shown) including a CPU and a memory. In the control unit 30, the CPU of the microprocessor executes various controls related to the robot system 1 according to a predetermined program read from the memory. Next, some functions of the control unit 30 are described.

[0037] Figure 2 30 is a functional block diagram of the control unit 30. The control unit 30 includes a position acquisition unit 301, a three-dimensional information acquisition unit 302, a determination unit 303, a detection unit 304, and a transmission unit 305.

[0038] The position acquisition unit 301 acquires the approximate position of the workpiece 2 on the conveyor 12 based on the image of the workpiece 2 captured by the fixed camera 13 .

[0039] The three-dimensional information acquisition unit 302 acquires the three-dimensional information of the workpiece 2 whose position is acquired by the position acquisition unit 301, based on the camera image of the workpiece 2 captured by the 3D vision sensor 14. Here, the three-dimensional information particularly includes the position information, posture information, and height information of the workpiece 2. In particular, in the present embodiment, when the hand 112 is moved closer to the workpiece 2 whose position is acquired by the position acquisition unit 301 by driving the arm 111, the three-dimensional information acquisition unit 302 acquires the three-dimensional information of the workpiece 2 by the 3D vision sensor 14 provided on the arm 111.

[0040] The determination unit 303 determines whether another workpiece 2B is hidden in the workpiece 2 , for example, in the workpiece 2A exposed from the fixed camera 13 , based on the three-dimensional information of the workpiece 2 acquired by the three-dimensional information acquisition unit 302 .

[0041] In particular, in this embodiment, when the height information included in the three-dimensional information of the workpiece 2 acquired by the three-dimensional information acquisition unit 302 is greater than the thickness of each of the workpiece 2A and the workpiece 2B, the determination unit 303 can determine that the exposed workpiece 2A hides the other workpiece 2B. In addition, the position where the other workpiece 2B is hidden is not limited to under the exposed workpiece 2A, and the other workpiece 2B may be partially hidden.

[0042] Alternatively, the determination unit 303 may compare the ratio of the area occupied by other workpieces 2B in the image of the workpiece 2 in the three-dimensional image captured by the 3D vision sensor 14 with a specified threshold value, and determine whether the exposed workpiece 2A hides other workpieces 2B based on the comparison result.

[0043] In addition, the applicant has so far applied for technologies for recognizing and detecting the relative positions or postures of multiple workpieces when they are stacked, such as in Japanese Patent Application No. 2009-28968 (Japanese Patent Publication No. 2010-184308), Japanese Patent Application No. 2012-188485 (Japanese Patent Publication No. 2014-046371), and Japanese Patent Application No. 2017-29853 (Japanese Patent Publication No. 2018-134698). By applying these technologies to the present invention, it is possible to determine whether the exposed workpiece 2A hides other workpieces 2B.

[0044] For example, the following technology can be applied to the present invention: select any two workpieces from a plurality of workpieces, estimate the overlapping area where the two workpieces overlap based on the position and posture of each workpiece and the size information of the workpiece, and when the overlapping area contains a specific shape of either of the two workpieces, calculate gradient information for each pixel of the image of the overlapping part, and determine which of the two workpieces is located on the upper side based on the gradient information.

[0045] Alternatively, the following technology can also be applied to the present invention: measure the surface positions of multiple objects randomly stacked in three-dimensional space, obtain the position information of multiple three-dimensional points, find a connection set that connects adjacent three-dimensional points from the acquired multiple three-dimensional points, and determine the position and posture of the object based on the position information of the three-dimensional points belonging to the connection set.

[0046] Alternatively, the following technology can also be applied to the present invention: as three-dimensional information of the workpiece in the container, three-dimensional points with height position information of each point are obtained, and multiple three-dimensional point sets that satisfy specified conditions between adjacent points are created in the three-dimensional point group; a three-dimensional point set in the multiple three-dimensional point sets that meets at least one of a specified size benchmark, a specified area benchmark, and a specified length benchmark is determined as an exclusion set; a three-dimensional point group for workpiece detection is obtained by excluding the points included in the exclusion set from the three-dimensional point group or multiple three-dimensional point sets; and workpiece detection for robot removal is performed based on the three-dimensional points for workpiece detection.

[0047] When the determination unit 303 determines that the exposed workpiece 2A hides the other workpieces 2B, the detection unit 304 detects the other workpieces 2B using the 3D vision sensor 14 after the exposed workpiece 2A is taken out by the hand 112 to expose the other workpieces 2B.

[0048] In particular, in this embodiment, during the "retreat action" after the hand 112 removes the exposed workpiece 2A, that is, the action until the hand 112 grasps the workpiece 2A and transports it to another place, the 3D vision sensor 14 can photograph other workpieces 2B to detect other workpieces 2B.

[0049] The transmitting unit 305 transmits the detection result of the detecting unit 304 to the outside of the detecting unit 304 to be used for the picking operation of the robot 11 .

[0050] In particular, in the present embodiment, the transmission unit 305 can transmit the detection result of the detection unit 304 to the control unit of the robot 11A to be used for the picking operation of the robot 11A in the next step.

[0051] [Operation of Embodiment 2]

[0052] Figure 3 : is a flowchart showing the operation of the robot system 1 according to the present embodiment. Figures 4 to 8 It is shown Figure 3 FIG. 1 is a diagram showing the positional relationship and state of each component in the robot system 1 in each step of the flowchart shown in FIG. Figure 3 to Figure 8 The operation of the robot system 1 will be described.

[0053] In step S1, if Figure 4 As shown, the position acquisition unit 301 acquires the approximate position of the workpiece 2 on the conveyor 12 based on the image of the workpiece 2 ( 2A, 2B) conveyed by the conveyor 12 captured by the fixed camera 13 .

[0054] In step S2, if Figure 5 As shown, the three-dimensional information acquisition unit 302 acquires the three-dimensional information of the workpiece 2 whose position is acquired by the position acquisition unit 301 , based on the captured image of the workpiece 2 captured by the 3D vision sensor 14 .

[0055] In step S3, if Figure 6 As shown, the exposed workpiece 2A is removed by hand 112.

[0056] In step S4, if the workpiece 2A taken out has a hidden workpiece 2B (S4: Yes), the process moves to step S5. If the workpiece 2A taken out has no hidden workpiece 2B (S4: No), the process moves to step S2.

[0057] In step S5, Figure 7 As shown, during the retracting operation after the workpiece 2A is taken out, the detection unit 304 detects the workpiece 2B that was hidden.

[0058] In step S6, if Figure 8 As shown, the transmitting unit 305 transmits the detection result of the once hidden workpiece 2B detected by the detecting unit 304 to the control unit of the robot 11A in the next process. As a result, the robot 11A in the next process takes out the once hidden workpiece 2B.

[0059] After that, the process transfers to step S2.

[0060] [3 Effects of the Implementation Method]

[0061] The robot system 1 involved in this embodiment includes: a fixed camera 13, which photographs the workpiece 2 transported by the conveyor 12; a robot 11, which includes a 3D vision sensor 14, which photographs the workpiece 2 transported by the conveyor 12; a position acquisition unit 301, which acquires the position of the workpiece 2 based on the workpiece 2 photographed by the fixed camera 13; a three-dimensional information acquisition unit 302, which acquires the three-dimensional information of the workpiece 2 whose position is acquired by the position acquisition unit 301 based on the workpiece 2 photographed by the 3D vision sensor 14; and a determination unit 303, which acquires the three-dimensional information of the workpiece 2 whose position is acquired by the position acquisition unit 301 based on the workpiece 2 photographed by the 3D vision sensor 14; The information acquisition unit 302 acquires the three-dimensional information of the workpiece 2 to determine whether the exposed workpiece 2A in the workpiece 2 hides other workpieces 2B; the hand 112 is used to take out the exposed workpiece 2A; the detection unit 304, when it is determined that the exposed workpiece 2A hides other workpieces 2B, after the other workpieces 2B are exposed, the detection unit 304 enables the 3D vision sensor 14 to photograph other workpieces 2B to detect the other workpieces 2B; and the transmission unit 305 transmits the detection result of the detection unit 304 to the outside of the detection unit 304 for use in the removal action of the robot 11.

[0062] Thereby, the workpieces 2 stacked on the conveyor 12 can be efficiently taken out. In addition, since it is not necessary to prepare other mechanisms such as an arc conveyor, the cost can be reduced.

[0063] In the robot system 1 according to the present embodiment, the transmission unit 305 may transmit the detection result to the control unit of the robot 11A to be used in the picking operation of the robot 11A in the next step.

[0064] This can reduce the number of missed workpieces 2 in the robot system 1 .

[0065] In the robot system 1 according to the present embodiment, the fixed camera 13 may acquire a two-dimensional image of the workpiece 2 , and the relative position of the fixed camera 13 to the conveyor 12 may be fixed.

[0066] Thus, the approximate position of the workpiece 2 on the conveyor 12 can be acquired using the relatively inexpensive fixed camera 13 .

[0067] In the robot system 1 according to the present embodiment, the 3D vision sensor 14 may be attached to the arm 111 of the robot 11 .

[0068] Thereby, the workpiece 2B which was hidden can be detected by the 3D vision sensor 14 during the retracting operation.

[0069] In addition, in the robot system 1 according to the present embodiment, the three-dimensional information may include height information, position information, and posture information of the workpiece 2 .

[0070] Thus, by comparing the height of the workpiece 2 with the thickness of the workpiece 2 , it is possible to determine whether or not there is no accumulation in the workpiece 2 .

Claims

1. A robot system comprising: a first imaging unit for capturing a two-dimensional image of the workpiece transported by the transport unit; a robot including a second imaging unit configured to capture a three-dimensional image of the workpiece transported by the transport unit; a position acquisition unit, which acquires the position of the workpiece based on the workpiece photographed by the first camera unit; a three-dimensional information acquisition unit that acquires three-dimensional information of the workpiece whose position is acquired by the position acquisition unit based on the workpiece photographed by the second imaging unit; a determination unit for determining whether an exposed workpiece hides other workpieces among the workpieces based on the three-dimensional information of the workpiece acquired by the three-dimensional information acquisition unit; A taking-out unit, which is used to take out the exposed workpiece; a detection unit, which, when it is determined that the exposed workpiece hides the other workpiece, causes the second camera unit to photograph the other workpiece to detect the other workpiece after the other workpiece is exposed; as well as A transfer unit transfers the detection result of the detection unit to the outside of the detection unit so as to be used for a taking-out action of a robot in a next process.

2. The robot system according to claim 1, It is characterized in that The relative position of the first imaging unit with respect to the transport unit is fixed.

3. The robot system according to claim 1 or 2, It is characterized in that The second camera unit is mounted on the arm of the robot.

4. The robot system according to claim 1 or 2, It is characterized in that The three-dimensional information includes height information, position information, and posture information of the workpiece.