Robot control device, picking system and transportation operation planning method

Abstract

To enable appropriately transporting an article even when the posture of the article is not constant.SOLUTION: A control device 2 includes: a point group data input section 202 for acquiring three dimensional point group data of an article and its peripheral area; a point group filter section 210 for extracting a partial point group region from the three dimensional point group data; a gripping position recognition section 211 for recognizing a gripping position of the article based on information included in the partial point group region; and a transportation operation planning section 212 for planning transportation operation of the article by a gripping mechanism based on information of the gripping position recognized by the gripping position recognition section 211.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a robot control device, a picking system, and a transport operation planning method. [Background technology]

[0002] In recent years, at logistics and production sites, attempts have been made to automate the sorting of goods, which accounts for the majority of manual work in warehouses, by using arm robots and other devices. The sorting of goods involves removing products (goods) stored in containers, transporting them, and sorting them into shipping boxes. A robot control device that controls the transport operation of the arm robot estimates the gripping position of the goods by the arm of the arm robot based on 3D point cloud data of the goods to be picked and the surrounding area, etc.

[0003] For example, Patent Document 1 discloses a control device including an area acquisition unit that acquires areas where the height of a subject obtained from 3D point cloud data is lower than a predetermined value, an object outside area setting unit that sets an area in a camera image corresponding to the area acquisition unit as an object outside area, and an object position recognition unit that recognizes the position of the object relative to the subject from the camera image. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent Publication No. 2021-74799 Summary of the Invention [Problem to be solved by the invention]

[0005] Generally, there are various types of postures of the objects loaded in a container. Therefore, for example, when the position of an object is identified by model matching using a photographed image of the object and preset information on the shape of the object, it is assumed that the object cannot be properly extracted from the photographed image if a part of the object is hidden by another object. In this case, the gripping position of the object by the arm of the arm robot cannot be specified, so that the object cannot be transported and the object remains in the container. Patent Document 1 does not disclose a control method for properly transporting the object even when the object is not in a constant posture.

[0006] The present invention has been made in consideration of the above circumstances, and has an object to make it possible to properly transport an article even when the article is not in a constant position. [Means for solving the problem]

[0007] A robot control device according to one aspect of the present invention is a robot control device that plans an operation of transporting an item by a robot having a gripping mechanism for gripping the item. The robot control device according to one aspect of the present invention includes a three-dimensional data acquisition unit that acquires three-dimensional data of the item and its surroundings, a filter unit that extracts a subspace region from the three-dimensional data, a gripping position recognition unit that recognizes a gripping position of the item based on information included in the subspace region, and a transport operation planning unit that plans an operation of transporting the item by the gripping mechanism based on information of the gripping position recognized by the gripping position recognition unit. Effect of the Invention

[0008] According to at least one aspect of the present invention, an article can be appropriately gripped even when the article is not in a constant position. Problems, configurations and effects other than those described above will become apparent from the following description of the embodiments. [Brief description of the drawings]

[0009] [Figure 1]FIG. 1 is a schematic diagram of a picking system according to an embodiment of the present invention. [Diagram 2] 1A to 1C are diagrams illustrating examples of gripping positions of an article by a finger gripping portion for each orientation of the article according to an embodiment of the present invention. [Diagram 3] 1A to 1C are diagrams showing an overview of a method for gripping an object by a finger gripping portion according to an embodiment of the present invention. [Figure 4] 2 is a block diagram showing an example of a functional configuration of a control device according to an embodiment of the present invention; FIG. [Diagram 5] FIG. 2 is a hardware configuration diagram of a control device according to an embodiment of the present invention. [Figure 6] 11A to 11C are diagrams illustrating an example of a filtering process performed by a point cloud filtering unit according to an embodiment of the present invention. [Figure 7] FIG. 13 is a cross-sectional side view of a pick source container after transport of an upright article according to an embodiment of the present invention. [Figure 8] 11 is a diagram showing an example of a correspondence between the vertical length of a filter and the vertical length of a finger grip portion according to an embodiment of the present invention. FIG. [Figure 9] 11A and 11B are diagrams showing examples of filters that can prevent the tip of the finger grip portion from colliding with the bottom surface of the pick source container in one embodiment of the present invention. [Figure 10] 10 is a flowchart illustrating an example of a procedure for a transport operation planning process performed by a control device according to an embodiment of the present invention. [Figure 11] 13A and 13B are diagrams illustrating an overview of a process for changing the height position of a filter in accordance with a result of recognition of a gripping point according to a modified example of the present invention. [Figure 12] FIG. 13 is a schematic diagram of a picking system according to a modified example of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, examples of modes for carrying out the present invention (hereinafter referred to as "embodiments") will be described with reference to the accompanying drawings. The present invention is not limited to the embodiments, and various numerical values ​​in the embodiments are merely examples. In addition, in this specification and drawings, the same components or components having substantially the same functions are given the same reference numerals, and duplicated explanations will be omitted.

[0011] In this embodiment, a picking system that grasps an article loaded in a container and transports it to another container such as a shipping container will be described as an example. Note that the robot control device according to this embodiment can be applied to a robot control device that controls the transport operation of an arm robot that operates in a logistics base such as a warehouse, a factory, etc., but the installation location of the arm robot is not limited to these locations.

[0012] <Picking system overview> First, an overview of a picking system 1 will be described with reference to Fig. 1. Fig. 1 is a schematic diagram of a picking system 1 in this embodiment. As shown in Fig. 1, the picking system 1 includes a control device 2, an arm robot 3, and a sensor 4.

[0013] The control device 2 (an example of a robot control device) is configured as a so-called computer having a main body 20, a display 21, and an input device 22. The main body 20 of the control device 2 is connected to the arm robot 3 and the sensor 4 via a communication path 8.

[0014] The main body 20 generates a control signal to control the arm robot 3 based on three-dimensional point cloud data (an example of three-dimensional data) of the item 7-1 and its surroundings acquired by the sensor 4, and controls the conveying operation of the arm robot 3 by transmitting the control signal to the arm robot 3.

[0015] A pick source container 5 loaded with an item 7-1 to be picked, and a pick destination container 6 are provided near the picking system 1. Then, the arm robot 3 grasps the item 7-1 and moves from the pick source container 5 to the pick destination container 6. As a result, the item 7-2 is loaded into the pick destination container 6.

[0016] The arm robot 3 performs a picking operation, that is, an operation of grasping and moving the item 7-1, in accordance with a control signal transmitted from the control device 2. Note that the shapes of the pick source container 5 and the pick destination container 6 may differ depending on the shapes of the items to be loaded therein.

[0017] The arm robot 3 has a finger gripping portion 31 and a pad portion 32 provided at the base portion of the finger gripping portion 31 as a gripping mechanism for the item 7-1. The finger gripping portion 31 is a mechanism that transports the item 7-1 to the destination container 6 by gripping the item 7-1 with two substantially planar fingers 311 and 312. The finger gripping portion 31 grips the item 7-1 by applying a predetermined pressure to the item 7-1 pinched between the fingers 311 and 312.

[0018] The pad unit 32 is a mechanism that lifts the item 7-1 by suctioning the surface of the item 7-1. The item 7-1 that has been sucked by the pad unit 32 is transported to the destination container 6 while being held by the finger gripping unit 31. In the following description, when there is no need to distinguish between the items before and after the movement, the items 7-1 and 7-2 are collectively referred to as the item 7.

[0019] The sensor 4 is configured, for example, by a three-dimensional camera, and captures an image of the source container 5 from above to obtain three-dimensional point cloud data of the item 7-1 contained in the source container 5 and its surroundings as subjects, and transmits the obtained three-dimensional point cloud data to the control device 2. The sensor 4 may be configured by a three-dimensional camera that outputs a depth image.

[0020] <Issues in conventional technology> In a conventional picking system having a configuration similar to that of the picking system 1 shown in FIG. 1, the control device can determine the gripping position of an article by an arm robot, for example, by the following two methods. (1) Grasp position determination based on model matching (2) Determining the gripping position by searching for a specific shape part in the photographed image of the container with the goods loaded inside

[0021] In the above method (1), the control device recognizes the shape, dimensions, position, orientation, etc. of the item in the container by comparing (model matching) object information included in the 3D point cloud data captured by the sensor with pre-registered item information. Then, the control device plans the operation of grasping and transporting the item based on the recognition result.

[0022] However, when this method is adopted, depending on the packing style or posture of the item in the container, model matching may not be able to properly recognize the shape and dimensions of the item. For example, assume that the item is a plastic bottle, and the body of the plastic bottle is set as the gripping position. In this case, if the body is not reflected in the 3D point cloud data due to close contact with adjacent parts, the control device cannot determine the gripping position of the item.

[0023] In the above method (2), for example, when the picking mode is a mode in which an item is transported by adsorbing the item with a pad provided on the arm robot, the control device searches the captured image for a plane suitable for the size of the pad, i.e., a plane to be adsorbed by the pad. Then, the control device plans the item transport operation by the arm robot based on the information of the plane detected by the search.

[0024] Also, for example, when the picking mode is a mode in which an item is transported by gripping the item with a finger gripping part (hand) provided on the arm robot, the control device searches the captured image for a space into which the fingers of the finger gripping part can be inserted. Then, the control device plans an item transport operation by the arm robot based on information about the space detected by the search.

[0025] However, for example, when an article is transported by selectively using both a gripping means using a finger gripping portion and a suction means using a pad, erroneous recognition may occur in detecting the suction or gripping position of the article, because the parameters and algorithms used in the recognition process suitable for transport using a gripping means and the recognition process suitable for transport using a suction means are different.

[0026] Therefore, there has been a demand for the ability to properly transport goods even in cases where the position of the goods within the container is not expected to be constant or where multiple means of transportation are used in combination.

[0027] <Overview of how the finger gripper grips an object> Next, an overview of a method for gripping the article 7 by the finger gripping portion 31 according to this embodiment will be described with reference to FIGS. 2 and 3. FIG.

[0028] Fig. 2 is a diagram showing an example of a gripping position of the item 7 by the finger gripping portion 31 for each posture of the item 7. Fig. 2 shows a state in which the item 7, which is a bottle having a cap portion 71, is contained in the pick source container 5. The item 7s shown on the left side of Fig. 2 is an upright item 7, and the item 7f shown on the right side is an item 7 lying sideways.

[0029] 3 is a diagram showing an outline of a method for gripping the article 7 by the finger gripper 31. When the target for picking by the arm robot 3 is an upright article 7s, the finger gripper 31 of the arm robot 3 grips and transports the cap portion 71, which is indicated by a dashed frame in the figure, thereby enabling stable transport of the article 7s.

[0030] On the other hand, when the item 7f to be picked by the arm robot 3 is a fallen item, the pad portion 32 adheres to the upper part of the body of the item 7f, and the finger gripping portion 31 of the arm robot 3 grasps and transports the side portion of the item 7f, thereby enabling the item 7f to be transported stably.

[0031] The control device 2 according to this embodiment extracts a partial point cloud region within a predetermined height range in the vertical direction from the 3D point cloud data of the article 7 and its surroundings acquired by the sensor 4. Then, the control device 2 determines the grip position of the article 7 based on the point cloud information in the partial point cloud region. The partial point cloud region will be described in detail with reference to FIG. 6 described later.

[0032] <Control device functional configuration> Next, a functional configuration of the control device 2 according to the present embodiment will be described with reference to Fig. 4. Fig. 4 is a block diagram showing an example of the functional configuration of the control device 2. As shown in Fig. 4, the control device 2 has a main body unit 20, a display 21, and an input device 22.

[0033] The main body unit 20 has an input unit 201 , a point cloud filter unit 210 , a gripping position recognition unit 211 , a transport operation planning unit 212 , a control signal creation unit 213 , an output unit 214 and a memory unit 216 .

[0034] The input unit 201 receives input of information, data, etc. used for generating a control signal, etc. The input unit 201 includes a point cloud data input unit 202 and an article information input unit 203.

[0035] The point cloud data input unit 202 (an example of a three-dimensional data acquisition unit) receives an input of three-dimensional point cloud data acquired by the sensor 4. The item information input unit 203 accepts input of item information 281, which is information about the item 7. The input item information 281 is stored in an auxiliary storage device 28 shown in Fig. 5, which will be described later. The item information 281 includes, for example, information about the type, shape, and size of the item 7, and the height of the cap portion 71 from the bottom surface when the item is upright.

[0036] The point cloud filter unit 210 (an example of a filter unit) performs a filter process to extract a partial point cloud region (an example of a partial space region) within a predetermined height range in the vertical direction from the three-dimensional point cloud data received by the point cloud data input unit 202.

[0037] The gripping position recognizing unit 211 recognizes the gripping position of the item 7s (see FIG. 2) from within the partial point cloud region based on the height information of the partial point cloud region and the point cloud information in the partial point cloud region extracted by the point cloud filter unit 210. The gripping position recognizing unit 211 also recognizes the suction position of the item 7f by the pad unit 32 based on the three-dimensional point cloud data obtained by photographing the pick source container 5 after the upright item 7 is transported.

[0038] The transport operation planning unit 212 plans the gripping and transporting operation of the item 7 based on the information of the gripping position or suction position recognized by the gripping position recognition unit 211. The control signal generating unit 213 generates a control signal for controlling the picking operation of the arm robot 3 based on the transport operation planned by the transport operation planning unit 212.

[0039] The output unit 214 includes a control signal output unit 215. The control signal output unit 215 outputs the control signal generated by the control signal generation unit 213 to the arm robot 3.

[0040] The input device 22 inputs operation contents according to user operations to the input unit 201. The input unit 201 is also connected to a storage unit 216, and accepts input of information stored in the storage unit 216.

[0041] The output device 21 is composed of, for example, a network card for outputting a control signal to the arm robot 3. The output device 21 is also composed of, for example, an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) display, and displays an input screen (not shown) for various information, etc. The input device 22 is configured with a pointing device such as a keyboard or a mouse, generates an operation signal according to the content of an operation by a user, and inputs the operation signal to the input unit 201. The display 21 and the input device 22 may be integrated into a touch panel.

[0042] <Control device hardware configuration> Next, the hardware configuration of the control device 2 will be described with reference to Fig. 5. Fig. 5 is a hardware configuration diagram of the control device 2 in this embodiment. In this embodiment, the control device 2 is realized by a computer. In Fig. 5, the control device 2 has a display 21, an input device 22, a CPU 23, a RAM 24, a ROM 25, a communication device 26, a media reader 27, and an auxiliary storage device 28. The display 21 and the input device 22 have already been described with reference to Fig. 4, so their description will be omitted.

[0043] The CPU 23 (Central Processing Unit: an example of a control unit) is a unit that executes various calculations. The CPU 23 executes the robot control process according to this embodiment by reading the robot control program 280 from the auxiliary storage device 28, expanding it in the RAM 24, and executing it. Note that a GPU (Graphics Processing Unit) may be used instead of the CPU 23, or the CPU 23 and the GPU may be used in combination. The robot control program 280 stored in the auxiliary storage device 28 is, for example, an application program that can be executed on an OS (Operating System) program. The robot control process by the CPU 23 will be described in detail later with reference to FIG.

[0044] The robot control program 280 may be installed in the auxiliary storage device 28 from, for example, a portable storage medium (not shown) stored in the media reader 27. The CPU 23 may execute processing in accordance with a program other than the robot control program 280. In this case, this program is stored in the auxiliary storage device 28.

[0045] The RAM 24 (Random Access Memory) is a memory that temporarily stores programs executed by the CPU 23, such as the robot control program 280, and data necessary for executing the programs. The ROM 25 (Read Only Memory) is a memory that stores the programs, OS, etc. required for starting the control device 2. The robot control program 280 may be stored in the ROM 25.

[0046] The communication device 26 is connected to the arm robot 3, the sensor 4, and the like via a communication path 8 (see FIG. 1). The communication path 8 is configured by a network such as a LAN (Local Area Network) or the Internet. The communication device 26 functions as an interface connecting to the display 21 and the input device 22 in addition to functioning as the input unit 201 and the output unit 214 in FIG. 2.

[0047] The media reader 27 is configured as a device that reads information from a portable storage medium such as a flash memory or a CD-ROM. The auxiliary storage device 28 is composed of, for example, a hard disk drive (HDD) or a solid state drive (SSD), and stores data and programs for executing various processes, a robot control program 280, and item information 281. The auxiliary storage device 28 may be composed of a flash memory or the like. The RAM 24, the ROM 25, and the auxiliary storage device 28 shown in FIG. 5 correspond to the storage unit 216 in FIG. 4. The auxiliary storage device 28, the portable storage medium, and the ROM 25 are used as an example of a computer-readable non-transitory recording medium that stores a program executed by a computer.

[0048] <Outline of filter processing and grip position recognition processing> Next, an overview of the filtering process by the point cloud filter unit 210 and the grip position recognition process by the grip position recognition unit 211 will be described with reference to FIGS.

[0049] Fig. 6 is a diagram showing an example of filter processing by the point cloud filter unit 210. Fig. 6A is a top view of the source container 5 containing the item 7, and Fig. 6B is a cross-sectional view of the source container 5 containing the item 7, seen from the side. As shown in Figs. 6A and 6B, the source container 5 contains a mixture of upright items 7s and fallen items 7f.

[0050] 6A and 6B is a filter for extracting a partial point cloud area Ar including a point cloud constituting the cap portion 71 of the item 7s. The filter Ft extracts a point cloud existing in a range Hw of the height Hs of the cap portion 71 ±X mm from the bottom surface of the source container 5. Information on the height Hs of the cap portion 71 can be acquired from the item information 281 (see FIG. 5).

[0051] The gripping position recognition unit 211 extracts a point cloud portion corresponding to the cap portion 71 based on information of the partial point cloud area Ar extracted by the point cloud filter unit 210, and recognizes a position (point) that can be gripped by the finger gripping unit 31 from the extracted point cloud portion. Then, based on the information of the gripping position recognized by the gripping position recognition unit 211, the conveyance operation plan unit 212 (see FIG. 4) plans the gripping and conveyance operation of the article 7, and the finger gripping unit 31 operates according to a control signal based on the plan, whereby the cap portion 71 of the article 7 is gripped by the finger gripping unit 31 and conveyed from the pick source container 5 to the pick destination container 6 (see FIG. 1).

[0052] Fig. 7 is a cross-sectional view of the source container 5 after the upright item 7s has been transported from the side. When the point cloud portion corresponding to the cap portion 71 is no longer detected from the partial point cloud area Ar shown in Fig. 6, the CPU 23 (see Fig. 5) presumes that the transport of the upright item 7s has been completed. In other words, it is deemed that only the fallen item 7f remains in the source container 5.

[0053] Then, the gripping position recognition unit 211 recognizes the suction point of the item 7f by the pad portion 32 (see Figure 3) and the gripping point by the finger gripping portion 31 based on the three-dimensional point cloud data obtained by photographing the pick source container 5 in which only the fallen item 7f remains, which is input into the point cloud data input unit 202.

[0054] <Example of determining the vertical length of a filter> The value of the vertical length Hw of the filter Ft used by the point cloud filter unit 210 to extract the partial point cloud region can be set based on the vertical length of the finger gripping portion 31, for example.

[0055] Fig. 8 is a diagram showing an example of the correspondence between the vertical length Hw of the filter Ft and the vertical length of the finger grip portion 31. As shown in Fig. 8, the vertical length Hw of the filter Ft can be set based on the dimensions of the finger grip portion 31 when the finger grip portion 31 grips the cap portion 71 of the article 7s in an upright state. The dimension of the finger grip portion 31 is the length L between the base portion 31r and the tip portion 31t of the finger grip portion 31.

[0056] In addition, when the pad portion 32 adsorbs the side portion of the article 7f in an overturned state, the gripping position of the finger grip portion 31 can be set based on information such as the height of the cap portion 71 of the article 7f from the bottom surface.

[0057] Furthermore, for example, when the height of the cap portion 71 of the article 7s is low, if the vertical position of the filter Ft is determined according to the height of the cap portion 71, it is expected that the position of the filter Ft will be too close to the bottom surface of the source container 5. In this case, if a gripping operation is planned based on information of the partial point cloud region extracted using the filter, there is a possibility that the tip portion 31t of the finger gripping portion 31 will collide with the bottom surface of the source container 5.

[0058] FIG. 9 is a diagram showing an example of a filter Ft that can prevent the tip 31t of the finger grip portion 31 from colliding with the bottom surface of the source container 5. In FIG. In the example shown in Figure 9, the control device 2 sets the vertical position of the length Lh, which is half the length L of the finger gripping portion 31 when the tip 31t of the finger gripping portion 31 is positioned near the bottom surface of the pick source container 5, to the lower limit value L1 of the vertical length of the filter Ft.

[0059] By setting the filter Ft in this manner, it is possible to prevent the tip 31t of the finger grip portion 31 from colliding with the bottom surface of the pick source container 5.

[0060] <Transportation operation planning processing> Next, the transport operation planning process by the control device 2 will be described with reference to Fig. 10. Fig. 10 is a flowchart showing an example of the procedure of the transport operation planning process by the control device 2.

[0061] First, the point cloud data input unit 202 (see FIG. 4) acquires three-dimensional point cloud data from the sensor 4 (see FIG. 1) (step S1). Next, the point cloud filter unit 210 performs a filtering process (step S2). That is, the point cloud filter unit 210 extracts a point cloud (partial point cloud) existing within a range of the height Hs±X mm (see "Hw" in FIG. 8) of the cap portion 71 of the article 7 from the three-dimensional point cloud data acquired by the point cloud data input unit 202 in step S1.

[0062] Next, the gripping position recognition unit 211 performs a process of recognizing a point (grip point) on the item 7 that can be gripped by the finger gripping unit 31 (see FIG. 2) of the arm robot 3 based on the information of the partial point cloud extracted by the point cloud filter unit 210 in step S2 (step S3). The gripping position recognition unit 211 can recognize the gripping point based on, for example, information on the diameter of the cap portion 71 described in the item information 281 (see FIG. 5), a recognition result that the object to be gripped, which is an object of a size that can be gripped by the finger gripping unit 31, is a substantially rectangular parallelepiped having a height within a predetermined height range from the bottom surface of the pick source container 5, etc.

[0063] Next, the grip position recognition unit 211 judges whether or not there is a recognition result (gripping point) in the process of step S3 (step S4). If it is judged that there is a recognition result in step S4 (YES judgment in step S4), the conveyance operation planner 212 (see FIG. 4) determines the grip point recognized by the grip position recognition unit 211 as the grip position by the finger gripping unit 31, and plans a conveyance operation of the article 7 based on the grip position (step S5). After the process of step S5, the CPU 23 of the control device 2 returns the process to step S1.

[0064] The above-described processes from step S1 to step S5 are processes defined in the master for upright products. The master for upright products is a master in which processes for planning the transport operation of an article 7 in an upright state are defined, and is stored in the ROM 25, the auxiliary storage device 28 (see FIG. 5), or the like. In this embodiment, in addition to the master for upright products, there is a fallen product master, which is a master in which processes for planning the transport operation of an article 7 in a fallen (tilted) state are defined. The control device 2 first performs processes based on the master for upright products, and then performs processes based on the master for fallen products.

[0065] If it is determined in step S4 that there is no recognition result (if step S4 is a NO determination), the CPU 23 of the control device 2 (see FIG. 5) switches the master from the upright product master to the fallen product master (step S6). Next, the point cloud data input unit 202 (see FIG. 4) acquires three-dimensional point cloud data from the sensor 4 (step S7). The three-dimensional point cloud data acquired by the point cloud data input unit 202 in step S7 is three-dimensional point cloud data of the pick source container 5 in which only the fallen product 7f remains after the upright product 7s has been transported.

[0066] The gripping position recognizing unit 211 performs a process of recognizing points (suction points) on the article 7 that can be picked up by the pad unit 32 (see FIG. 2) of the arm robot 3 and gripping points by the finger gripping unit 31 from the 3D point cloud data acquired in step S7 (step S8). Next, the gripping position recognizing unit 211 determines whether or not there is a recognition result (suction point and gripping point) in the process of step S9 (step S9).

[0067] If it is determined in step S9 that there is a recognition result (if step S9 is a YES judgment), the transport operation planning unit 212 determines the suction point and gripping point recognized by the gripping position recognition unit 211 as the suction position by the pad unit 32 and the gripping position by the finger gripping unit 31, respectively, and plans a transport operation for the item 7 based on these positions (step S10). After processing in step S10, the CPU 23 of the control device 2 returns the process to step S7. If step S9 is a NO judgment, that is, if it is determined that there is no recognition result, the CPU 23 returns the process to step S1.

[0068] According to the above-described embodiment, even when the posture of the item 7 is not constant, i.e., when the item 7 is in an upright state or a fallen state, the control device 2 can detect the gripping position of the item 7s based on the information contained in the partial point cloud area within a specified height range.

[0069] In the above-described embodiment, when the partial point cloud corresponding to the cap portion 711 of the article 7s is no longer present in the partial point cloud area, the CPU 23 causes the grip position recognition unit 211 to recognize the suction position of the article 7f from the three-dimensional point cloud data without performing filtering by the point cloud filter unit 210. This allows the article 7f that has fallen in the pick source container 5 to be transported appropriately.

[0070] Therefore, according to this embodiment, it is not necessary to perform model matching or the like for the entire part 7, which reduces the processing load on the CPU 23. Furthermore, according to this embodiment, it is not necessary to prepare in advance product information or the like for various packaging styles or attitudes of the item 7, which prevents an increase in the volume of data stored in the storage unit 216 (see FIG. 4).

[0071] In the above embodiment, the control device 2 switches the master to the fallen product master after the grasping position recognition by the upright product master is completed, but the present invention is not limited to this. For example, when multiple items 7 with different heights are loaded into the pick source container 5, the upright product master may perform a process of changing (gradually lowering) the height position of the filter according to the grasping point recognition result.

[0072] Fig. 11 is a diagram showing an overview of the process of changing the height position of the filter depending on the recognition result of the gripping point. Fig. 11 shows a state in which a plurality of items 7s3 to 7s6 of different heights are stored in the source container 5. The items 7s1 and 7s2 shown by dashed lines in Fig. 11 are items that have already been transported to the destination container 6 (see Fig. 1) by the finger gripping unit 31 of the arm robot 3.

[0073] 11, the point cloud filter unit 210 of the control device 2 uses a filter Ft capable of detecting the cap portion 71 of the item 7s3 to extract a partial point cloud area including the point cloud that constitutes the cap portion 71. Then, after the item 7s3 is transported to the destination container 6, based on the information of the extracted partial point cloud area, the point cloud filter unit 210 extracts a partial point cloud area including the point cloud that constitutes the cap portion 71 of the item 7s4 by lowering the height position of the filter Ft.

[0074] In this way, by gradually lowering the height position of the filter Ft according to the height of the item 7s, the control device 2 can appropriately determine the gripping position of each item 7s even when multiple items 7s of different heights are loaded into the pick source container 5.

[0075] In the above embodiment, the arm robot 3 and the control device 2 are connected in a one-to-one relationship, but the present invention is not limited to this. The control device 2 may be connected to a plurality of arm robots 3.

[0076] Fig. 12 is a schematic diagram of a picking system 1A according to a modified example. As shown in Fig. 12, the picking system 1A according to the modified example includes a control device 2A, arm robots 3-1 and 3-2, sensors 4-1 and 4-2, a database 9, a terminal device 10, and a tablet terminal 11. These devices constituting the picking system 1A are connected to each other via a communication path 8 so as to be able to communicate with each other.

[0077] The control device 2A shown in Fig. 12 differs from the control device 2 shown in Fig. 1 in that the control device 2A does not include the display 21, the input device 22, and the media reader 27 (see Fig. 1), and that the product information 281 is not stored in the auxiliary storage device 28. The rest of the configuration is the same as the control device 2 shown in Fig. 1, so a description thereof will be omitted.

[0078] As in the picking system 1A according to the modified example, the display 21, the input device 22, the information managed in the auxiliary storage device 28, and the like, which are included in the control device 2, may be provided in another device connected via a communication path 8. The picking system 1A according to the modified example can also obtain the same effects as those obtained by the above-mentioned embodiment.

[0079] Furthermore, the above-described embodiments provide detailed and specific descriptions of the configurations of the devices and systems in order to clearly explain the present invention, and are not necessarily limited to those having all of the configurations described.

[0080] In addition, the control lines or information lines shown by solid lines in Figures 4, 5, and 12 are those considered necessary for explanation, and do not necessarily show all control lines or information lines in the product. In reality, it can be considered that almost all components are connected to each other.

[0081] In addition, in this specification, the processing steps describing chronological processing include not only processing that is performed chronologically in the order described, but also processing that is not necessarily performed chronologically but is performed in parallel or individually (for example, parallel processing or processing by objects).

[0082] Furthermore, each component of the picking system according to the embodiment of the present invention described above may be implemented in any hardware as long as each hardware can transmit and receive information to and from each other via a network. Furthermore, the processing performed by a certain processing unit may be realized by a single piece of hardware, or may be realized by distributed processing by multiple pieces of hardware. [Explanation of symbols]

[0083] 1, 1A... picking system, 2, 2A... control device, 3, 3-1, 3-2... arm robot, 4, 4-1, 4-2... sensor, 5... pick source container, 6... pick destination container, 7... item, 31... finger gripping section, 32... pad section, 71... cap section, 202... point cloud data input section, 203... item information input section, 210... point cloud filter section, 211... gripping position recognition section, 212... transport operation planning section, 213... control signal generation section, 215... control signal output section, 216... storage section

Claims

1. A robot control device that plans an object transport operation by a robot having a gripping mechanism that grips an object, A three-dimensional data acquisition unit that acquires three-dimensional data of the item and its surroundings; A filter unit that extracts a subspace region from the three-dimensional data; a gripping position recognition unit that recognizes a gripping position of the article based on information included in the subspace region; a conveying operation planning unit that plans a conveying operation of the gripping mechanism based on information of the gripping position recognized by the gripping position recognition unit. Robot control device.

2. The height range of the partial space region extracted by the filter unit is set based on information on the height from the bottom surface of the gripping position of the article by the gripping mechanism, or on information on the height from the bottom surface of the gripping position and the dimensions of the gripping mechanism. The robot control device according to claim 1 .

3. The gripping mechanism includes a finger gripping portion that grips the object with a plurality of fingers, and a pad portion that adheres to the object. The robot control device according to claim 2 .

4. The gripping position recognized by the gripping position recognition unit based on the information included in the subspace region extracted by the filter unit is the gripping position of the article by the finger gripping unit. The robot control device according to claim 3.

5. a control unit that, when the three-dimensional data corresponding to the article is no longer present within the partial space region extracted by the filter unit, causes the filter unit not to extract the partial space region, and causes the grip position recognition unit to perform a recognition process of the suction position of the article by the pad unit. The robot control device according to claim 4.

6. The object has a substantially cubic shape, and the finger grip portion grips an upper portion of the object when the object is upright. The robot control device according to claim 5.

7. The pad portion is attached to the body of the article when the article is in a fallen state. The robot control device according to claim 6.

8. The fingers of the finger gripping portion are configured to be substantially flat, and the finger gripping portion grips the object by pressure applied to the object by the fingers. The robot control device according to claim 7.

9. The grip position recognition unit recognizes an object in the subspace region that is large enough to be gripped by the finger grip unit as an article to be gripped. The robot control device according to claim 8.

10. When information corresponding to the article is no longer present in the subspace region extracted by the filter unit, the height position of the filter used by the filter unit to extract the subspace region is changed to a lower position. The robot control device according to claim 4.

11. A picking system including a robot having a gripping mechanism that grips an item, and a robot control device that plans a transport operation of the item by the robot, The robot control device includes: A three-dimensional data acquisition unit that acquires three-dimensional data of the item and its surroundings; A filter unit that extracts a subspace region from the three-dimensional data; a gripping position recognition unit that recognizes a gripping position of the article based on information included in the subspace region; a conveying operation planning unit that plans a conveying operation of the gripping mechanism based on information of the gripping position recognized by the gripping position recognition unit. Picking system.

12. A transportation motion planning method by a robot control device for planning an object transport motion of a robot having a gripping mechanism for gripping an object, comprising: acquiring three-dimensional data of the item and its surroundings; extracting a subspace region from the three-dimensional data; A step of recognizing a gripping position of the article based on information included in the subspace region; and a step of planning a transport operation of the object by the gripping mechanism based on information of the recognized gripping position. A transportation operation planning method.

Images