Picking system, picking robot, and picking method

Abstract

To provide a picking system, a picking robot and a picking method capable of reducing a load of arithmetic processing required for work detection in performing picking to a tray in which a plurality of work-pieces and an internal material are mounted.SOLUTION: According to this disclosure, a picking system 400 detects respective positions and attitudes of a plurality of work-pieces 20 on the basis of information a photographic image obtained by photographing a tray 10 in which the plurality of work-pieces 20 are each separated and placed by using an internal material 30. The picking system includes a calculation area detection part 410 for discriminating the plurality of work-pieces 20 and the internal material 30 by performing image processing to the photographic image, and detecting an area including the work-pieces 20 as a calculation area, and a gripping point calculation part 420 for calculating respective gripping points of the plurality of work-pieces 20 in the calculation area on the basis of three-dimensional point group data.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a picking system, a picking robot, and a picking method.

Background Art

[0002] In recent years, techniques have been developed for detecting and picking each work based on a captured image obtained by capturing an image of a tray on which a plurality of works are placed. For example, Patent Document 1 discloses a picking technique that detects each work based on a depth map from a captured image of a tray on which a plurality of works are placed, and picks each detected work.

Prior Art Literature

Patent Literature

[0003]

Patent Document 1

Summary of the Invention

Problem to be Solved by the Invention

[0004] The inventors have found the following problems relating to picking systems, picking robots, and picking methods. When a plurality of works are placed on a tray, there is a risk that the works collide with each other during transportation or the like. Therefore, the interior of the tray is divided by inner materials, and each work is placed in each divided space to suppress collision between the works in some cases. When the interior of the tray is divided by inner materials, the inner materials are detected together with each work in the depth map. Therefore, it is necessary to identify only each work among the detected works and inner materials by machine learning or the like, which complicates the processing.

[0005] This disclosure addresses these challenges and aims to provide a picking system, a picking robot, and a picking method that can reduce the burden of computational processing required for workpiece detection when picking from a tray on which multiple workpieces and internal materials are placed. [Means for solving the problem]

[0006] One embodiment for achieving the above objective is: A picking system that detects the position and orientation of each of the multiple workpieces based on information from images taken of a tray in which multiple workpieces are placed, each separated by an internal material, A calculation region detection unit performs image processing on the captured image to distinguish between the multiple workpieces and the internal material, and detects the region containing the workpieces as a calculation region. The system includes a gripping point calculation unit that calculates the gripping points of each of the multiple workpieces based on 3D point cloud data within the calculation area. [Effects of the Invention]

[0007] According to this disclosure, a picking system, a picking robot, and a picking method can be provided that can reduce the burden of computational processing required for workpiece detection when picking from a tray on which multiple workpieces and internal materials are placed. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of a tray picked by the picking method according to Embodiment 1. [Figure 2] This diagram shows the configuration of the picking robot according to Embodiment 1. [Figure 3] This is a flowchart showing the flow of the picking method according to Embodiment 1. [Figure 4] This is a diagram showing an example of a captured image. [Figure 5]This figure shows an example of a captured image processed in the picking method according to Embodiment 1. [Figure 6] This figure shows an example of a captured image processed in the picking method according to Embodiment 2. [Modes for carrying out the invention]

[0009] Embodiments of the present disclosure will be described in detail below with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and redundant explanations will be omitted where necessary for clarity.

[0010] It should be noted that the right-handed xyz Cartesian coordinate system shown in Figure 1 and other drawings is merely a convenient representation for explaining the positional relationships of the components. Typically, the positive z-axis is vertically upward, and the xy-plane is horizontal; this is consistent across all drawings.

[0011] <Embodiment 1> First, with reference to Figure 1, the configuration of the tray 10 picked by the picking method according to Embodiment 1 will be described. Figure 1 is a diagram showing an example of a tray picked by the picking method according to Embodiment 1. The tray 10 is a component on which a plurality of workpieces 20 can be placed, and the inside of the tray 10 is divided by an inner material 30. The inner material 30 is arranged to form a plurality of spaces divided at equal intervals in the x and y directions, for example, as shown in Figure 1. Each of the plurality of workpieces 20 is placed in a space divided by the inner material 30.

[0012] Next, with reference to Figure 2, the configuration of the picking robot 100 according to Embodiment 1 will be described. As shown in Figure 2, the picking robot 100 comprises a hand 200, a camera 300, and a picking system 400. The hand 200 is a component that grips and moves the workpiece 20 placed on the tray 10. The picking robot 100 includes a hand control unit (not shown). The operation of the hand 200 is controlled by the hand control unit. A camera 300 is mounted on the upper part of the hand 200. The camera 300 is a camera that takes pictures of the inside of the tray 10. The camera 300 can take pictures and acquire 3D point cloud data. The camera 300 is connected to the picking system 400 in a communication manner.

[0013] The picking system 400 includes a calculation area detection unit 410 and a gripping point calculation unit 420. The picking system 400 acquires captured images and 3D point cloud data taken by the camera 300. The calculation area detection unit 410 is configured to perform image processing on the captured images acquired from the camera 300 to distinguish between the workpiece 20 and the inner material 30. The calculation area detection unit 410 is configured to perform image processing, for example, by machine learning. The calculation area detection unit 410 distinguishes between the workpiece 20 and the inner material 30 by performing image processing on the captured images acquired from the camera 300 and detects the area containing the workpiece 20 within the captured image as a calculation area. The gripping point calculation unit 420 calculates the gripping point of the workpiece 20 within the calculation area detected by the calculation area detection unit 410 based on the 3D point cloud data acquired from the camera 300. A control unit (not shown) causes the hand 200 to grip the workpiece 20 based on the gripping point calculated by the gripping point calculation unit 420.

[0014] Next, the picking method according to Embodiment 1 will be further described with reference to Figures 3 to 5. Figure 3 is a flowchart showing the flow of the picking method according to Embodiment 1. Figure 4 is a diagram showing an example of a captured image. Figure 5 is a diagram showing an example of a captured image processed in the picking method according to Embodiment 1.

[0015] When picking each workpiece 20 from a tray 10 on which a plurality of workpieces 20 are placed, first, the picking system 400 acquires the captured image captured by the camera 300 and three-dimensional point cloud data (step S101). FIG. 4 is an example of the captured image acquired in step S101. As shown in FIG. 4, the camera 300 captures, for example, an image of the tray 10 viewed from above. Next, the calculation area detection unit 410 distinguishes between the workpiece 20 and the inner material 30 by performing image processing on the captured image acquired in step S101 (step S102). In the present embodiment, the calculation area detection unit 410 is set to detect the workpiece 20 by image processing. FIG. 5 is a diagram showing an example of a workpiece peripheral area detected by the calculation area detection unit 410 in the present embodiment. The dotted line portion shown in FIG. 5 is the workpiece peripheral area. Next, the calculation area detection unit 410 determines a calculation area (step S103). In the present embodiment, the calculation area detection unit 410 determines the area around each workpiece 20 detected in step S102, that is, the workpiece peripheral area, as the calculation area. Next, the grasping point calculation unit 420 calculates a grasping point in the calculation area determined in step S103 (step S104). Specifically, the grasping point calculation unit 420 calculates a grasping point for the calculation area determined in step S103 using the three-dimensional point cloud data acquired in step S101.

[0016] As described above, in the picking method according to the present embodiment, the grasping point is calculated after determining the area where the workpiece 20 exists by performing image processing. Therefore, compared with a case where grasping point calculation is performed on the entire captured image, the load required for the grasping point calculation process is smaller. In addition, there is no need to perform a step of distinguishing between the workpiece 20 and the inner material 30 after calculating the grasping point. In the picking method according to the present embodiment, since the workpiece 20 and the inner material 30 are distinguished by image processing, a grasping point can be calculated for the workpiece 20 even when the inner material 30 has a height approximately equal to that of the workpiece 20 or the inner material 30 is higher than the workpiece 20.

[0017] <Embodiment 2> In Embodiment 1, the workpiece 20 and the inner material 30 are discriminated by detecting the workpiece 20 in the captured image. In contrast, Embodiment 2 is an embodiment in which the workpiece 20 and the inner material 30 are discriminated by detecting the inner material 30 in the captured image. Since the configuration of the picking system according to Embodiment 2 is the same as that of Embodiment 1, the illustration and description thereof are omitted.

[0018] In this embodiment, the calculation area detection unit 410 is configured to detect the inner material 30 by image processing. FIG. 6 is a diagram showing an example of the area around the inner material 30 detected by the calculation area detection unit 410, that is, the inner material area in this embodiment. The dotted line portion shown in FIG. 6 is the inner material area. In this embodiment, the calculation area detection unit 410 determines an area excluding the periphery of the detected inner material 30 as the calculation area. The picking method according to this embodiment achieves the same effects as Embodiment 1.

[0019] <Embodiment 3> In Embodiments 1 and 2, the workpiece 20 and the inner material 30 are discriminated by detecting the workpiece 20 or the inner material 30 in the captured image. In contrast, Embodiment 3 is an embodiment in which the workpiece 20 and the inner material 30 are discriminated by detecting both the workpiece 20 and the inner material 30 in the captured image. Since the configuration of the picking system according to Embodiment 3 is the same as that of Embodiment 1, the illustration and description thereof are omitted.

[0020] In this embodiment, the calculation area detection unit 410 is configured to detect the workpiece 20 and the inner material 30 by image processing. In this embodiment, the calculation area detection unit 410 determines an area excluding the periphery of the detected workpiece 20 or the periphery of the detected inner material 30 as the calculation area. The picking method according to this embodiment achieves the same effects as Embodiment 1.

[0021] Furthermore, in the picking method according to this embodiment, even if one of the workpiece 20 and the inner material 30 cannot be detected, the workpiece 20 and the inner material 30 can be distinguished by image processing. Therefore, even if the specifications of the workpiece 20 are frequently changed, such as in high-mix low-volume production, the workpiece 20 and the inner material 30 can be distinguished by image processing.

[0022] This disclosure is not limited to the embodiments described above, and can be modified as appropriate without departing from the spirit of the invention. For example, although the above embodiments describe a case in which the picking system 400 is mounted on the picking robot 100, the picking system 400 may be a device independent of the picking robot 100. [Explanation of symbols]

[0023] 10 trays 20 Work 30 Inner material 100 picking robots 200 hands 300 Cameras 400 Picking System 410 Calculation area detection unit 420 Grip point calculation unit

Claims

1. A picking system that detects the position and orientation of each of the multiple workpieces based on information from images taken of a tray in which multiple workpieces are placed, each separated by an internal material, A calculation region detection unit performs image processing on the captured image to distinguish between the multiple workpieces and the internal material, and detects the region containing the workpieces as a calculation region. The system includes a gripping point calculation unit that calculates the gripping points of each of the multiple workpieces using three-dimensional point cloud data acquired within the calculation area. Picking system.

2. A picking robot comprising the picking system described in claim 1.

3. A picking method for picking each of the multiple workpieces from a tray in which the multiple workpieces and the internal material are mixed, using the picking system described in claim 1.

Images