Picking system, picking robot, and picking method

Abstract

To provide a picking system, a picking robot and a picking method capable of reducing a load of arithmetic processing required for work detection in performing picking to a tray in which a plurality of work-pieces and an internal material are mounted.SOLUTION: According to this disclosure, a picking system 400 detects respective positions and attitudes of a plurality of work-pieces 20 on the basis of information a photographic image obtained by photographing a tray 10 in which the plurality of work-pieces 20 are each separated and placed by using an internal material 30. The picking system includes a calculation area detection part 410 for discriminating the plurality of work-pieces 20 and the internal material 30 by performing image processing to the photographic image, and detecting an area including the work-pieces 20 as a calculation area, and a gripping point calculation part 420 for calculating respective gripping points of the plurality of work-pieces 20 in the calculation area on the basis of three-dimensional point group data.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present disclosure relates to a picking system, a picking robot, and a picking method. [Background technology]

[0002] In recent years, a technique has been developed for detecting and picking each work based on a captured image of a tray on which a plurality of workpieces are placed. For example, Patent Document 1 discloses a technology in which each work is detected based on a depth map in a captured image of a tray on which multiple workpieces are placed, and each detected workpiece is picked up. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2021-151699 A Summary of the Invention [Problem to be solved by the invention]

[0004] The inventors have found the following problems regarding the picking system, the picking robot, and the picking method. When multiple workpieces are placed on a tray, there is a risk that the workpieces may collide with each other during transportation, etc. Therefore, the tray may be divided by internal materials, and each workpiece may be placed in each divided space to prevent collisions between the works. When the tray is divided by internal materials, the internal materials are detected in the depth map along with each workpiece. Therefore, it is necessary to identify only each workpiece among the detected workpieces and internal materials by machine learning, etc., which complicates the processing.

[0005] The present disclosure has been made in consideration of such problems, and aims to provide a picking system, picking robot, and picking method that can reduce the burden of the calculation processing required for work detection when picking from a tray on which multiple work pieces and internal materials are placed. [Means for solving the problem]

[0006] One aspect of the present invention for achieving the above object is to A picking system that detects the position and posture of each of a plurality of workpieces based on information of a captured image of a tray on which each of the plurality of workpieces is separated and placed using an inner material, A calculation area detection unit that performs image processing on the captured image to distinguish the plurality of workpieces and the internal material and detects an area including the workpieces as a calculation area; The apparatus further includes a gripping point calculation unit that calculates a gripping point of each of the plurality of workpieces within the calculation area based on three-dimensional point cloud data. Effect of the Invention

[0007] According to the present disclosure, it is possible to provide a picking system, a picking robot, and a picking method that can reduce the burden of the computational processing required for work detection when picking from a tray on which multiple work pieces and internal materials are placed. [Brief description of the drawings]

[0008] [Figure 1] FIG. 2 is a diagram showing an example of a tray to be picked by the picking method according to the first embodiment. [Diagram 2] FIG. 1 is a diagram illustrating a configuration of a picking robot according to a first embodiment. [Diagram 3] 4 is a flowchart showing the flow of a picking method according to the first embodiment. [Figure 4] FIG. 2 is a diagram showing an example of a captured image. [Diagram 5]5A to 5C are diagrams showing an example of a captured image that has been subjected to image processing in the picking method according to the first embodiment. [Figure 6] FIG. 11 is a diagram showing an example of a captured image that has been subjected to image processing in the picking method according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In each drawing, the same or corresponding elements are denoted by the same reference numerals, and duplicated descriptions will be omitted as necessary for clarity of explanation.

[0010] Naturally, the right-handed xyz Cartesian coordinate system shown in FIG. 1 and other drawings is for the convenience of explaining the positional relationship of the components. Usually, the positive direction of the z axis is vertically upward, and the xy plane is a horizontal plane, which is common to all drawings.

[0011] <Embodiment 1> First, the configuration of a tray 10 to be picked by the picking method according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a diagram showing an example of a tray to be picked by the picking method according to the first embodiment. The tray 10 is a member on which a plurality of workpieces 20 can be placed, and the inside of the tray 10 is divided by an internal material 30. For example, as shown in Fig. 1, the internal material 30 is arranged to form a plurality of spaces that are divided at equal intervals in the x direction and the y direction. Each of the plurality of workpieces 20 is placed in a space divided by the internal material 30.

[0012] Next, the configuration of the picking robot 100 according to the first embodiment will be described with reference to FIG. 2. As shown in FIG. 2, the picking robot 100 includes a hand 200, a camera 300, and a picking system 400. The hand 200 is a member that holds and moves the workpiece 20 placed on the tray 10. The picking robot 100 includes a hand control unit (not shown). The operation of the hand 200 is controlled by the hand control unit. The camera 300 is attached to the upper part of the hand 200. The camera 300 is an imaging device that captures images of the inside of the tray 10. The camera 300 can capture images and acquire three-dimensional point cloud data. The camera 300 is connected to the picking system 400 so as to be able to communicate with the picking system 400.

[0013] The picking system 400 includes a calculation area detection unit 410 and a gripping point calculation unit 420. The picking system 400 acquires the captured image and three-dimensional point cloud data captured by the camera 300. The calculation area detection unit 410 is set to perform image processing for distinguishing the workpiece 20 and the internal material 30 from the captured image acquired from the camera 300. The calculation area detection unit 410 is set to execute image processing by, for example, machine learning. The calculation area detection unit 410 distinguishes the workpiece 20 and the internal material 30 by performing image processing on the captured image acquired from the camera 300, and detects an area including the workpiece 20 in the captured image as a calculation area. The gripping point calculation unit 420 calculates the gripping point of the workpiece 20 based on the three-dimensional point cloud data acquired from the camera 300 within the calculation area detected by the calculation area detection unit 410. A control unit (not shown) causes the hand 200 to grip the workpiece 20 based on the gripping point calculated by the gripping point calculation unit 420.

[0014] Next, the picking method according to the first embodiment will be further described with reference to Figs. 3 to 5. Fig. 3 is a flowchart showing the flow of the picking method according to the first embodiment. Fig. 4 is a diagram showing an example of a captured image. Fig. 5 is a diagram showing an example of a captured image that has been subjected to image processing in the picking method according to the first embodiment.

[0015] When picking each workpiece 20 from a tray 10 on which a plurality of workpieces 20 are placed, the picking system 400 first acquires the captured image and three-dimensional point cloud data captured by the camera 300 (step S101). FIG. 4 is an example of the captured image acquired in step S101. As shown in FIG. 4, the camera 300 captures an image of the tray 10 viewed from above, for example. Next, the calculation area detection unit 410 performs image processing on the captured image acquired in step S101 to distinguish the workpiece 20 and the internal material 30 (step S102). In this embodiment, the calculation area detection unit 410 is set to detect the workpiece 20 by image processing. FIG. 5 is a diagram showing an example of the workpiece peripheral area detected by the calculation area detection unit 410 in this embodiment. The dotted line portion shown in FIG. 5 is the workpiece peripheral area. Next, the calculation area detection unit 410 determines the calculation area (step S103). In this embodiment, the calculation area detection unit 410 determines the area around each workpiece 20 detected in step S102, i.e., the workpiece peripheral area, as the calculation area. Next, the gripping point calculation unit 420 calculates a gripping point in the calculation area determined in step S103 (step S104). Specifically, the gripping point calculation unit 420 calculates a gripping point for the calculation area determined in step S103 using the three-dimensional point cloud data acquired in step S101.

[0016] In this way, in the picking method according to the present embodiment, the region in which the workpiece 20 exists is determined by performing image processing, and then the gripping point is calculated, so the load required for the gripping point calculation process is smaller than when the gripping point is calculated for the entire captured image. In addition, there is no need to perform a process of distinguishing between the workpiece 20 and the internal material 30 after the gripping point calculation. In the picking method according to the present embodiment, the workpiece 20 and the internal material 30 are distinguished by image processing, so even if the internal material 30 is about the same height as the workpiece 20 or is higher than the workpiece 20, the gripping point for the workpiece 20 can be calculated.

[0017] <Embodiment 2> In the first embodiment, the work 20 is detected in a captured image to distinguish between the work 20 and the internal material 30. In the second embodiment, the work 20 is detected in a captured image to distinguish between the work 20 and the internal material 30. The configuration of the picking system according to the second embodiment is the same as that of the first embodiment, and therefore illustrations and explanations thereof are omitted.

[0018] In this embodiment, the calculation area detection unit 410 is set to detect the internal material 30 by image processing. Fig. 6 is a diagram showing an example of the area around the internal material 30 detected by the calculation area detection unit 410 in this embodiment, i.e., the internal material area. The dotted line portion shown in Fig. 6 is the internal material area. In this embodiment, the calculation area detection unit 410 determines the area excluding the periphery of the detected internal material 30 as the calculation area. The picking method according to this embodiment has the same effects as those of the first embodiment.

[0019] <Embodiment 3> In the first and second embodiments, the work 20 and the internal material 30 are distinguished from each other by detecting the work 20 or the internal material 30 in the captured image. On the other hand, in the third embodiment, the work 20 and the internal material 30 are distinguished from each other by detecting the work 20 and the internal material 30 in the captured image. The configuration of the picking system according to the third embodiment is the same as that of the first embodiment, and therefore illustrations and explanations thereof are omitted.

[0020] In this embodiment, the calculation area detection unit 410 is set to detect the workpiece 20 and the internal material 30 by image processing. In this embodiment, the calculation area detection unit 410 determines the area excluding the area around the detected workpiece 20 or the area around the detected internal material 30 as the calculation area. The picking method according to this embodiment has the same effects as those of the first embodiment.

[0021] Furthermore, in the picking method according to the present embodiment, even if one of the workpiece 20 and the internal material 30 cannot be detected, the workpiece 20 and the internal material 30 can be distinguished from each other by image processing. Therefore, even if the specifications of the workpiece 20 are frequently changed, for example, in the case of high-mix low-volume production, the workpiece 20 and the internal material 30 can be distinguished from each other by image processing.

[0022] The present disclosure is not limited to the above-described embodiment, and can be modified as appropriate without departing from the spirit of the present disclosure. For example, in the above-described embodiment, the picking system 400 is mounted on the picking robot 100, but the picking system 400 may be a device independent of the picking robot 100. [Explanation of symbols]

[0023] 10 Tray 20 Work 30 Inner material 100 Picking Robot 200 hands 300 Camera 400 Picking System 410 Calculation area detection unit 420 Grip point calculation unit

Claims

1. A picking system that detects the position and posture of each of a plurality of workpieces based on information of a captured image of a tray on which each of the plurality of workpieces is separated and placed using an inner material, A calculation area detection unit that performs image processing on the captured image to distinguish the plurality of workpieces and the internal material and detects an area including the workpieces as a calculation area; A gripping point calculation unit that calculates gripping points of the plurality of workpieces within the calculation area based on three-dimensional point cloud data. Picking system.

2. A picking robot comprising the picking system according to claim 1.

3. A picking method comprising: picking each of the plurality of workpieces from the tray in which the plurality of workpieces and the internal materials are mixed, using the picking system according to claim 1 .

Images