Picking robot system and picking robot control method

Abstract

To provide a picking robot system in which picking from a container can be easily performed according to simple control, even when one container stores a large number of workpieces.SOLUTION: In a configuration of picking workpieces in a container 50, from a top face of the container using a robot 30, the container 50 comprises partitioning plates 51 with a mesh shape that divide the interior part of the container into multiple compartments. Since mesh-shaped partitioning plates 51 are being used, the interior part of the container 50 can be correctly recognized with a camera 20 or the like, and the robot 30 can accurately pick a workpiece disposed in each compartment of the container 50.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a picking robot system and a picking robot control method. [Background technology]

[0002] In recent years, the transportation work at the time of shipment from logistics warehouses has been automated using robots. For example, efforts are underway to automate the task of removing workpieces (items) to be shipped from storage (work) containers, picking them up, and placing the picked workpieces in shipping containers by using picking robots.

[0003] Typically, multiple workpieces are placed in a container, and a picking robot picks the required number of workpieces from the container and places them in a shipping container. If the workpieces in the container are stacked in an orderly fashion, the robot can pick them up easily. However, the workpieces inside the container are not always stacked in an orderly fashion, which may result in poor workability for the picking robot.

[0004] Patent Document 1 describes a technology for a work container for storing workpieces, in which a number of workpiece storage chambers are provided by dividing the container vertically and horizontally with partitions, and each workpiece storage chamber stores a workpiece individually. By storing the workpieces individually, the technology described in Patent Document 1 can prevent the workpieces from colliding with each other and being damaged, and even if multiple types of workpieces are placed in one container, each workpiece is arranged in a separate section. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 5-301143 Summary of the Invention [Problem to be solved by the invention]

[0006] According to the technology described in Patent Document 1, the workpieces are individually accommodated in the work accommodation chamber, and the arrangement of the workpieces is not disturbed in the container, and each workpiece can be picked up appropriately by the robot. However, if small partitions are installed to contain each workpiece individually, even if a camera is used to photograph the inside of the container when the robot picks up the workpiece to check its condition, the partitions create a blind spot and it is not possible to properly check any surfaces other than the top surface of the workpiece.

[0007] To solve this problem, for example, it is possible to arrange many cameras for recognition and recognize the workpiece from images taken from multiple angles by the many cameras. In other words, taking pictures from multiple angles with many cameras reduces blind spots and allows the workpiece to be correctly recognized. However, using multiple cameras for recognition causes the problem that the system configuration becomes expensive accordingly.

[0008] Furthermore, if there is a partition, the tip of the robot's arm may come into contact with the partition, making it impossible to grasp the workpiece. Therefore, if the container is configured to be divided into small sections with partitions as described in Patent Document 1, it is necessary to take measures such as devising a way for the tip of the arm to enter each section inside the container, or to grasp the workpiece by suction, which requires a specialized and expensive robot.

[0009] In view of the above, an object of the present invention is to provide a picking robot system and a picking robot control method that can easily pick workpieces from a container with simple control, even when a single container contains a large number of workpieces. [Means for solving the problem]

[0010] In order to solve the above problems, for example, the configurations described in the claims are adopted. The present application includes multiple means for solving the above problems, and one example is a picking robot system in which a robot picks workpieces inside a container from the top surface of the container, the container having mesh-shaped partitions that divide the interior into multiple compartments, and the robot picks up workpieces placed in each compartment of the container. Effect of the Invention

[0011] According to the present invention, a camera that recognizes the inside of a container can capture images of workpieces behind mesh-shaped partitions, making it possible to recognize workpieces inside the container without creating blind spots, enabling the robot to pick them appropriately. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a configuration diagram showing an example of a picking robot system according to a first embodiment of the present invention. [Diagram 2] 1 is a block diagram showing an example of a control configuration of a picking robot system according to a first embodiment of the present invention. FIG. [Diagram 3] FIG. 2 is a top view showing an example of a container of the picking robot system according to the first embodiment of the present invention. [Figure 4] 1 is a cross-sectional view showing an example of a container of a picking robot system according to a first embodiment of the present invention. [Diagram 5] 1 is a diagram showing an example of the relationship between a mesh-shaped partition plate arranged in a container of a picking robot system according to a first embodiment of the present invention and a workpiece. FIG. [Figure 6] 1 is a diagram showing an example of the arrangement of mesh-shaped partition plates in a picking robot system according to a first embodiment of the present invention. FIG. [Figure 7] 1 is a cross-sectional view showing an example in which a mesh-shaped partition plate of a picking robot system according to a first embodiment of the present invention is partially removed. FIG. [Figure 8] FIG. 2 is a sequence diagram showing an example of a picking operation by the picking robot system according to the first embodiment of the present invention. [Figure 9] FIG. 11 is a block diagram showing an example of a control configuration of a picking robot system according to a second embodiment of the present invention. [Figure 10] FIG. 11 is a diagram showing an example of a state in which the inside of a container is recognized by a picking robot system according to a second embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing an example (after interpolation) of the state of recognition of the inside of a container by the picking robot system according to the second embodiment of the present invention. [Figure 12] FIG. 11 is a top view showing a modified example of a container applied to the picking robot system according to each embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] <First embodiment> A picking robot system and a picking robot control method according to a first embodiment of the present invention will be described below with reference to FIGS.

[0014] [Picking system configuration] FIG. 1 is a diagram showing an example of the overall configuration of a picking system according to a first embodiment. The picking robot system includes a control device 10, a camera 20, a robot 30, and a conveyor 40. The conveyor 40 transports a container 50 in which a workpiece (item) 60 is placed. However, in the state shown in Fig. 1, the workpiece 60 is placed inside the container 50, and the workpiece 60 is not shown. The container 50 has an open top and has a mesh-shaped partition plate 51 disposed inside. The configuration of the mesh-shaped partition plate 51 will be described in detail later.

[0015] The arm of the robot 30 picks up the workpiece 60 from the top surface of the container 50 and places it in a shipping container (not shown). The camera 20 photographs from above the inside of the container 50 within the range in which the robot 30 will perform work, and transmits the photographed image to the control device 10. The control device 10 recognizes the position and shape of the workpiece 60 placed in the container 50 from the image captured by the camera 20, and based on the recognition results, performs the picking operation of the workpiece 60 at the tip of the arm of the robot 30. The control device 10, the camera 20, and the robot 30 are connected via a LAN (Local Area Network) 19.

[0016] [Control device configuration] FIG. 2 shows an example of the configuration of the control device 10. The control device 10 includes an overall control unit 11, a robot control unit 12, a photographing data storage unit 13, a recognition unit 14, and a conveyor control unit 41. The overall control unit 11 controls the operation of the entire picking robot system. The robot control unit 12 issues arm operation instructions to the robot 30 and controls the picking operation by the robot 30. The robot 30 transmits arm posture information to the robot control unit 12.

[0017] The camera 20 supplies the shooting data (moving image data or still image data) to the shooting data storage unit 13. The shooting data storage unit 13 stores the shooting data and supplies the stored shooting data to the robot control unit 12 and the recognition unit 14. The camera 20 takes pictures in response to a shooting instruction from the robot control unit 12, and the camera 20 sends the shooting results to the robot 30. As the camera 20, a camera that takes moving images or still images is used, but various shooting units can be used, such as a camera with a function for measuring the distance to the subject or an infrared camera. The recognition unit 14 performs image analysis processing on the captured data to recognize the workpiece 60 inside the container 50, and transmits the recognition result to the robot control unit 12. The recognition unit 14 recognizes the workpiece 60 based on a recognition instruction from the robot control unit 12.

[0018] The overall control unit 11 sends a conveyor operation command to the conveyor control unit 41, and the conveyor control unit 41 returns conveyor information indicating the operation status of the conveyor 40 based on the conveyor operation command to the overall control unit 11. By the operation of the conveyor 40 controlled by this conveyor control unit 41, the container 50 is transported to the range where the robot 30 performs the picking operation.

[0019] Each of the control units 11, 12, 41 and the recognition unit 14 shown in Fig. 2 is configured by a computer. For example, as shown in Fig. 2, the robot control unit 12 includes a CPU (Central Processing Unit) 12a and a memory 12b, and the CPU 12a executes a program prepared in the memory 12b to execute a processing function for controlling the robot 30. Similarly, the recognition unit 14 also executes a program to recognize a workpiece from photographed data.

[0020] [Container configuration] Next, the configuration of the container 50 according to this embodiment will be described. FIG. 3 is a top view of the container 50. As shown in FIG. As shown in Fig. 3, mesh-shaped partition plates 51 are arranged in three locations in the container 50, and the inside of the container 50 is divided into four sections 50a, 50b, 50c, and 50d. In the example of Fig. 3, the three partition plates 51 are arranged at approximately equal intervals in the width direction of the container 50 (the left-right direction in Fig. 3), and the four sections 50a, 50b, 50c, and 50d are set to approximately the same size. Also, in the example of Fig. 3, one or more workpieces 60 are arranged in each of the sections 50a, 50b, 50c, and 50d. As shown in FIG. 3, the mesh-shaped partition plate 51 is disposed in three locations, but this is merely an example. Depending on the size of the container 50 and the size of the work 60, one or more partition plates may be disposed.

[0021] FIG. 4 is a cross-sectional view of the container 50 shown in FIG. Here, three workpieces 60 are stacked in section 50a, two workpieces 60 are stacked in section 50b, three workpieces 60 are stacked in section 50c, and only one workpiece 60 is placed in section 50d. 4 has a mesh-like shape, when the inside of the container 50 is photographed from above by the camera 20, the recognition unit 14 can correctly recognize the state of the workpieces 60 arranged in each of the sections 50a to 50d from the photographed data of the camera 20. For example, the workpieces 60 in the rightmost section 50d are included in the photographed image of the camera 20 through the mesh-like partition 51, so the recognition unit 14 can correctly recognize the arrangement state of the workpieces 60 in the section 50d.

[0022] If the partition plate 51 is not a mesh-shaped plate, the partition plate will obstruct the image and most of the workpiece 60 in the section 50d will not be included in the image, so the recognition unit 14 will not be able to correctly recognize the position of the workpiece 60 in the section 50d. Even if the partition plate is made of a transparent acrylic plate or the like, a virtual image will be captured due to refraction of light at the partition plate, so the recognition unit 14 will not be able to recognize the correct position of the workpiece 60 from the image captured by the camera 20.

[0023] FIG. 5 shows the mesh shape of the partition plate 51 in detail. The upper side of Fig. 5 shows an enlarged view of a part of the mesh of the partition plate 51. The partition plate 51 is configured in a mesh shape (grid shape) by combining vertical bars 51x and horizontal bars 51y. The vertical bars 51x and horizontal bars 51y are made of metal, resin, or the like. In this case, a flexible resin material or the like may be used to prevent damage to the workpiece 60 that comes into contact with it. Incidentally, it is preferable that the interval w between the vertical bars 51x is equal to half (D / 2) of the minimum width D of the workpiece 60 or equal to or less than D / 2. Similarly, the interval between the horizontal bars 51y is preferably equal to or smaller than half the minimum height of the workpiece 60.

[0024] [Example of removable partitions] The partition plate 51 installed in the container 50 may be fixed to the container 50, or may be configured to be removable. FIG. 6 shows an example in which the partition plate 51 is configured to be removable. That is, as shown in FIG. 6, a mesh-shaped partition plate 51 is held by guides 53 arranged on a container wall surface 52. With this configuration, the mesh-shaped partition plate 51 can be freely removed from the container 50. This makes it possible to freely change the size and number of compartments in the container 50.

[0025] FIG. 7 shows an example in which the mesh-shaped partition plate 51 is removable and can be divided in the height direction. In the example of FIG. 7, the partition plate 51 is divided into a lower half having a mesh-like shape, 51a, and an upper half having a mesh-like shape, 51b. With this configuration, for example, the robot 30 can pick up and remove the upper half partition plate 51b, leaving only the lower half partition plate 51a in the container 50.

[0026] As a result, for example, when the robot 30 is unable to pick up the workpiece 60 in the container 50 because the partition plate is in the way, the upper half of the partition plate 51b can be removed to enable the robot 30 to pick up the workpiece 60. Note that, although an example in which the mesh-shaped partition plate is divided into two, upper and lower portions, is shown in Fig. 7, it may be divided into three or more portions.

[0027] [Example of pick process flow] FIG. 8 is a sequence diagram showing an example of the flow of a picking process by the picking robot system. 8 shows an example of the flow of processing in each section based on the transmission of instructions between the robot control section 12 of the control device 10, the arm section of the robot 30, the camera 20, and the recognition section 14. Here, an example is shown in which the robot 30 fails to perform a pick process using the arm section, and removes a part of the partition board 51.

[0028] First, when a container 50 to be picked up is transported, the robot control unit 12 of the control device 10 instructs the camera 20 to take a photograph (step S11). The camera 20, which has received this photographing instruction, performs a photographing process to photograph the container 50 (step S12). The photographed data obtained here is stored in the photographed data storage unit 13. Next, the robot control unit 12 of the control device 10 issues a recognition instruction to the recognition unit 14 (step S13). The recognition unit 14, which has received this image capture instruction, executes a workpiece recognition process by image analysis of the image capture data stored in the image capture data storage unit 13 (step S14).

[0029] In this recognition process, because the partitions 51 that divide the interior of the container 50 have a mesh shape, it is possible to perform recognition that is essentially as good as if there were no partitions 51. In addition, because there are partitions 51, the positions of the workpieces 60 inside the container 50 are determined to a certain extent, and it is possible to prevent the arrangement position of the workpieces 60 from being disturbed. Data resulting from the recognition process in step S14 is transmitted from the recognition unit 14 to the robot control unit 12 of the control device 10 (step S15).

[0030] The robot control unit 12 of the control device 10 that has received the recognition result creates a trajectory plan for the arm of the robot 30 (step S16). If the creation of this trajectory plan is successful (step S17), the robot control unit 12 of the control device 10 issues an arm operation instruction to the robot 30 based on the trajectory plan (step S18). The robot 30, which has received the arm operation command, performs a gripping operation of the workpiece 60 based on the arm operation command with the arm unit (step S19). Then, the robot 30 transmits the arm operation result to the robot control unit 12 of the control device 10 (step S20). Here, it is assumed that the arm operation result indicating that the picking operation of the workpiece 60 was successful has been acquired.

[0031] Thereafter, the robot control unit 12 of the control device 10 instructs the camera 20 to take a photograph (step S21). The camera 20, which has received this photographing instruction, performs a photographing process to photograph the container 50 (step S22). The robot control unit 12 of the control device 10 also issues a recognition instruction to the recognition unit 14 (step S23). The recognition unit 14, which has received this photographing instruction, executes a workpiece recognition process by image analysis of the photographed data stored in the photographed data storage unit 13 (step S24). The result data of the recognition process in the recognition unit 14 is transmitted to the robot control unit 12 of the control device 10 (step S25). By acquiring this recognition result, the robot control unit 12 of the control device 10 again creates a trajectory plan for the arm of the robot 30 (step S26). However, it is assumed here that the robot control unit 12 fails to create the trajectory plan because the partition plate 51 is in the way (step S27).

[0032] If the creation of the trajectory plan fails, the robot control unit 12 of the control device 10 issues an arm operation command to the robot 30 to remove the partition plate 51 that is in the way of the pick (step S28). Then, the robot 30 performs an operation of removing the partition plate 51 with the arm unit based on the arm operation command (step S29). The operation of removing the partition plate 51 here may be, for example, the removal of the upper half of the partition plate 51b as shown in FIG. 7. Then, the robot 30 transmits the arm operation result (partition board removal result) to the robot control unit 12 of the control device 10 (step S30).

[0033] The robot control unit 12 of the control device 10, which has received the arm operation result for removing the partition plate, creates a trajectory plan for the arm of the robot 30 that reflects the fact that the partition plate has been removed (step S31). In this case, the trajectory plan is successful because there is no partition plate 51 (step S32), and the robot control unit 12 of the control device 10 issues an arm operation instruction to the robot 30 based on the trajectory plan (step S33). The robot 30, which has received the arm operation command, performs a gripping operation of the workpiece 60 based on the arm operation command with the arm unit (step S34). In this case, the gripping operation is successful because there is no partition plate 51. Then, the robot 30 transmits the arm operation result of the successful gripping operation to the robot control unit 12 of the control device 10 (step S35).

[0034] As described above, according to the picking robot system of this embodiment, since the partition plate 51 in the shape of a mesh is present inside the container 50, the position of the workpieces 60 inside the container 50 is determined to a certain extent, and it is possible to prevent the arrangement position of the workpieces 60 from being disturbed. Furthermore, since the partition plate 51 has a mesh shape, when recognizing the shape, position, number, etc. of the workpieces 60 inside the container 50 from an image captured by the camera 20, the recognition unit 14 can perform the recognition without the partition plate 51 getting in the way. Therefore, there is no need to install multiple cameras 20, and the system configuration can be simplified.

[0035] Furthermore, if the partition plate 51 gets in the way when picking the workpiece 60, the robot 30 can remove the partition plate 51 in question, thereby making it possible to appropriately pick the workpiece 60. In this case, as shown in Fig. 7, by removing only a portion of the partition plate 51b in the upper half, the remaining partition plate 51a can be used to separate the workpiece 60, thereby improving the efficiency of the picking operation.

[0036] <Second embodiment> Next, a picking robot system and a picking robot control method according to a second embodiment of the present invention will be described with reference to Figures 9 to 11. In Figures 9 to 11 which explain the second embodiment, the same reference numerals are used for parts corresponding to Figures 1 to 8 which are described in the first embodiment, and duplicated explanations will be omitted.

[0037] [Control device configuration] FIG. 9 shows an example of the configuration of the control device 10. The present embodiment differs from the control device 10 shown in FIG. 2 in that an image processing unit 15 that performs image processing on photographed data (image data) photographed by the camera 20 is provided.

[0038] That is, the photographed data captured by the camera 20 is supplied to the recognition unit 14 via the photographed data storage unit 13. Here, the recognition unit 14 supplies the supplied photographed data (image data) to the image processing unit 15. The image processing unit 15 converts the supplied image data into point cloud image data. Furthermore, the image processing unit 15 performs a process of removing the point cloud at the location of the mesh-shaped partition plate 51 from the point cloud image data, and generates image data by interpolating the point cloud from which the location of the partition plate 51 has been removed. The location of the partition plate 51 inside the container 50 is registered in advance in the image processing unit 15, and the point cloud at the registered position is removed to remove the mesh of the partition plate 51. Alternatively, the mesh-shaped partition plate 51 may be configured in a specific color different from the color of the workpiece 60, and the point cloud at the location of the color of the partition plate 51 may be removed.

[0039] The image data from which the partition plate 51 has been removed is supplied to the recognition unit 14, and the recognition unit 14 performs processing to recognize the workpiece 60 from the image data from which the partition plate 51 has been removed. The other configurations such as the robot control unit 12 shown in FIG. 9 are similar to those shown in FIG.

[0040] [Process to remove the partition plate] 10 and 11 show an example of processing in which the image processor 15 removes the partition plate 51 from the image data. FIG. 10 shows an image of the image data before the partition plate is removed, and FIG. 11 shows an image of the image data after the partition plate is removed. The photographed data from the camera 20 is actually taken from the top side of the container 50, but for the sake of explanation, FIGS. 10 and 11 show the container 50 in cross section.

[0041] As shown in FIG. 10, point cloud data 61 of the photographed data indicates the upper surface of each workpiece 60 and a portion 51c of the partition plate 51 protruding from the workpiece 60. Here, the image processing unit 15 removes the protruding parts 51c of the partition plate 51 contained in the point cloud data 61 and then interpolates the point cloud data to obtain image data 62 showing only the top surface of each workpiece 60, as shown in Figure 11. The recognition unit 14 performs recognition processing of the workpiece 60 using this image data 62, thereby enabling accurate recognition of the workpiece 60 while eliminating the influence of the partition plate 51.

[0042] <Modification> It should be noted that the embodiment examples described so far have been described in detail in order to explain the present invention in an easily understandable manner, and are not necessarily limited to those having all of the configurations described. For example, in the configuration shown in Fig. 3, the partition plates 51 inside the container 50 are arranged parallel to one direction (depth direction) of the container 50. In contrast to this, for example, as shown in Fig. 12, in addition to the partition plate 51 arranged in the depth direction of the container 50, a partition plate 54 arranged in the width direction of the container 50 may be provided to divide the inside of the container 50 into smaller sections. Even in this case, each of the partition plates 51, 54 may be removable. Furthermore, each of the partition plates 51, 54 may be separated into upper and lower halves that can be removed.

[0043] In addition, in the configuration diagrams of the control device shown in Figures 2 and 9, only the control lines and information lines considered necessary for explanation are shown, and not all the control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all the components are connected to each other. In addition, the control device 10 is an example in which each processing unit is configured on a computer by executing a program, but in this case, the program may be stored on a recording medium such as an external memory, an IC card, an SD card, an optical disk, etc., and transferred to the computer. [Explanation of symbols]

[0044] 10...control device, 11...overall control unit, 12...robot control unit, 12a...CPU, 12b...memory, 13...photographing data storage unit, 14...recognition unit, 15...image processing unit, 19...LAN, 20...camera, 30...robot, 40...conveyor, 41...conveyor control unit, 50...container, 50a-50d...section, 51, 51a, 51b...partition plate, 51c...location, 51x...vertical bar, 51y...horizontal bar, 52...container wall surface, 54...partition plate, 60...work, 61...point cloud data, 62...image data

Claims

1. A picking robot system that picks up a workpiece in a container from an upper surface of the container by a robot, The container includes a mesh-shaped partition plate that divides the inside of the container into a plurality of compartments, The robot picks up the workpieces arranged in each compartment of the container. Picking robot system.

2. The partition can be removed by pulling it out from the container. The picking robot system according to claim 1 .

3. The partition plate is divided in the height direction so that only a part of it can be removed by pulling it out from the container. The picking robot system according to claim 2 .

4. The robot recognizes the position of the workpiece in the container, performs a trajectory plan for picking up the recognized workpiece, and removes the partition plate in the container when it is determined that the partition plate is an obstacle to the recognition or arm operation. The picking robot system according to claim 2 .

5. An imaging unit that images the inside of the container; an image processing unit that converts the data captured by the imaging unit into a point cloud; a recognition unit that recognizes the workpiece in the container from the point cloud photographed data obtained by the image processing unit; A robot control unit that causes the robot to pick up the work recognized by the recognition unit, The image processing unit removes the point cloud that exists at the position of the partition plate, and the recognition unit recognizes the workpiece from the point cloud photographed data from which the partition plate has been removed. The picking robot system according to claim 1 .

6. The mesh of the partition plate is in a lattice pattern with intervals equal to or less than half the length of one side of the work to be placed in the container. The picking robot system according to claim 1 .

7. A picking robot control method for picking up a workpiece in a container from an upper surface of the container by a robot, The container has mesh-shaped partitions arranged therein to divide the interior into a plurality of compartments; A recognition process for recognizing the work in each section of the container divided by the mesh-shaped partition plate; A pick process for picking the work in each section recognized by the recognition process. A picking robot control method.

Images