Pick-up system

Abstract

Even if shape model comparison is difficult to perform, a gripping position which is easy to satisfy a conveying condition is estimated. A pickup system (1) is provided with: a pickup robot (3) that grips and moves an article (7); and a control device (2) that controls the pickup robot (3), the control device (2) having: an image input unit that receives image data of the article (7) acquired by the sensor (4); an element reliability evaluation unit that evaluates, on the basis of the image data, a plurality of element reliability of an index indicating the possibility of gripping for each physical element of the article (7); a gripping position estimation unit that estimates the gripping position of the pickup robot (3) with respect to the article (7) on the basis of the plurality of element reliability degrees; a conveyance information input unit that receives conveyance information indicating conveyance conditions relating to movement and gripping of the article (7) gripped at the gripping position; and an identification parameter estimation unit that estimates, on the basis of the conveyance information, an identification parameter to be used in the estimation of the gripping position by the gripping position estimation unit.

Description

Technical Field

[0001] The present invention relates to a technique for performing a picking operation of picking up an object (article) using a picking robot, and more particularly to a technique for estimating a gripping position of the picking robot on the picking up object. Background Art

[0002] At logistics sites, production sites, etc., objects are moved and transported as a process of logistics and production. For example, in recent years, at logistics sites such as logistics warehouses, picking robots are used to pick up specific objects from multiple objects in storage. During the picking operation, it is necessary to estimate the gripping position of the picking robot.

[0003] Patent document 1 is proposed as a technology for estimating a gripping position. In Patent document 1, the subject is "determining the gripping position of a gripping device on an object with high precision." Therefore, Patent document 1 discloses "a gripping system (1), which comprises: an acquisition unit (111) that acquires an image including an object as a subject; an estimation unit (112) that uses an estimation model (221) that uses the image as input to estimate multiple gripping candidate positions of the object; and a determination unit (113) that determines the gripping position at which the gripping device (30) grips the object with reference to the multiple gripping candidate positions."

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2022-21147 Summary of the invention

[0007] Problems to be solved by the invention

[0008] In the picking operation, it is necessary not only to consider whether the gripping is successful or not, but also to consider ensuring the accuracy, safety and efficiency of the picking operation. For example, it is expected to prevent defects in the picked object, improve the accuracy of stacking and achieve fast transportation. In order to achieve these, the handling condition that represents the gripping condition of the picking robot becomes important. And the realization of the picking operation that meets the handling condition depends on the gripping position of the picking robot. Therefore, it is required to estimate the gripping position that is easy to meet the handling condition.

[0009] Here, when shape model comparison is used, multiple gripping positions can be pre-set on the shape model and switched according to the transport conditions. However, in a situation where shape model comparison is difficult, it is impossible to pre-teach the gripping positions in this way. Therefore, although it is known to add post-processing corresponding to the transport conditions, the man-hours for manual adjustment of the relevant parameters will become huge.

[0010] Therefore, the present invention aims to estimate a gripping position that easily satisfies the conveying conditions even when shape model comparison is difficult or impossible.

[0011] Means for solving problems

[0012] In order to solve the above-mentioned problems, in the present invention, based on the transport information indicating the transport conditions, the identification parameters used for estimating the gripping position are estimated. More specifically, in a picking system for performing a picking operation on an object to be picked up, there is provided: a picking robot that grips the object and moves the object; and a control device that controls the picking robot, the control device comprising: an image input unit that receives image data of the object acquired by a sensor; an element reliability evaluation unit that evaluates a plurality of element reliability indicators indicating the gripping possibility of each physical element of the object based on the image data; a gripping position estimation unit that estimates the gripping position of the picking robot relative to the object based on the plurality of element reliability indicators; a transport information input unit that receives transport information indicating the transport conditions related to the movement and gripping of the object gripped at the gripping position; and an identification parameter estimation unit that estimates the identification parameters used in the gripping position estimation of the gripping position estimation unit based on the transport information, the picking robot operates using the gripping position estimated by the control device.

[0013] In addition, the present invention also includes devices constituting the picking system and subsystems that are a combination of them. In addition, the present invention also includes a picking method and an auxiliary method thereof using these devices and subsystems. In addition, a program for making the control device constituting the picking system function as a computer and a storage medium storing the program are also included in the present invention.

[0014] Effects of the Invention

[0015] According to the present invention, even when it is difficult or impossible to compare the shape model, it is possible to estimate a gripping position that easily satisfies the conveying conditions in the picking operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is an external view of a pickup system according to one embodiment of the present invention.

[0017] Figure 2 This is a functional block diagram of a control device according to one embodiment of the present invention.

[0018] Figure 3 This is a hardware configuration diagram of a control device according to one embodiment of the present invention.

[0019] Figure 4This is a diagram showing transport information used in one embodiment of the present invention.

[0020] Figure 5 This is a diagram schematically showing reliability data used in one embodiment of the present invention.

[0021] Figure 6 This is a diagram showing history information used in one embodiment of the present invention.

[0022] Figure 7 This is a diagram showing hardware information used in one embodiment of the present invention.

[0023] Figure 8 This is a diagram showing article characteristic information used in one embodiment of the present invention.

[0024] Figure 9 This is a flowchart showing the processing contents of one embodiment of the present invention.

[0025] Figure 10 This is a diagram showing a display screen when inputting weights in one embodiment of the present invention.

[0026] Figure 11 This is a diagram showing a screen of a display that displays a gripping position and an article reliability in one embodiment of the present invention.

[0027] Figure 12 This is a table showing a method of considering reliability according to one embodiment of the present invention.

[0028] Figure 13 It is a diagram showing a modified example of one embodiment of the present invention. DETAILED DESCRIPTION

[0029] Hereinafter, an embodiment of the present invention will be described using the drawings. In this embodiment, a picking system that grasps articles accumulated in a container and moves them to another container is described as an example.

[0030] In addition, the present embodiment can be applied to logistics bases such as warehouses, factories, etc., but the location is not limited.

[0031] first, Figure 1 FIG. 1 is an external view of the picking system 1 of this embodiment. Figure 1 In the embodiment, the picking system 1 includes a control device 2, a picking robot 3, and a sensor 4. In addition, a picking source container 5 and a picking destination container 6 containing an article 7-1 to be picked up are provided near the picking system 1. Then, the picking robot 3 moves the article 7-1 from the picking source container 5 to the picking destination container 6.

[0032] As a result, the article 7-2 is loaded into the pickup destination container 6. Here, the picking robot 3 performs the picking operation, that is, grasps and moves the article 7-1, based on the image data of the article 7-1 captured by the sensor 4 according to the control signal generated by the control device 2. Therefore, the control device 2 is connected to the picking robot 3 and the sensor 4 via the communication path 8. The following describes each device. In addition, in the following, when the article is not distinguished before and after the movement, it is expressed as the article 7, and when it is distinguished, it is expressed as the article 7-1 and the article 7-2 before and after the movement, respectively.

[0033] First, the control device 2 can be realized by a so-called computer having a main body 20, a display 21, and an input device 22. The main body 20 has a function for controlling the picking robot 3. Hereinafter, the functions of the control device 2 will be described mainly with the main body 20 as the focus.

[0034] Figure 2 2 is a functional block diagram of the control device 2 of this embodiment. Figure 2 In the embodiment, the control device 2 includes a main body 20, a display 21, and an input device 22. In addition, the main body 20 as a main part of the present embodiment includes an input unit 201, an element reliability evaluation unit 207, a recognition parameter estimation unit 208, a gripping position estimation unit 209, an article surface extraction unit 210, a reliability calculation unit 211, a control signal generation unit 212, an output unit 213, and a storage unit 216.

[0035] In addition, the input unit 201 receives information and data for the generation of control signals, etc. For this purpose, the input unit 201 includes an image data input unit 202, an item characteristic information input unit 203, a transport information input unit 204, a hardware information input unit 205, and a change instruction input unit 206. Here, the image data input unit 202 receives input of image data captured by the sensor 4. The image data includes an image of the item 7. The image data is in the form of, for example, a grayscale image, an RGB image, a depth image, a three-dimensional point group, or the like.

[0036] In addition, the article characteristic information input unit 203 receives the article characteristic information 292 related to the characteristics of the article 7. In addition, the transport information input unit 204 receives the transport information 287 indicating the transport conditions. In addition, the hardware information input unit 205 receives the hardware information 291 indicating the characteristics of the picking robot 3. The change instruction input unit 206 receives the change instruction for correcting and changing the transport information 287. This is the end of the description of the input unit 201.

[0037] Next, the element reliability evaluation unit 207 evaluates the gripping possibility of each physical element in the article 7, in other words, evaluates the element reliability regarding the ease of gripping. That is, the element reliability evaluation unit 207 calculates the element reliability that is an index representing the gripping possibility of each physical element. In addition, the identification parameter estimation unit 208 estimates the identification parameter used in estimating the gripping position. In addition, the gripping position estimation unit 209 estimates the gripping position of the article 7-1 by the picking robot 3 based on a plurality of element reliabilities.

[0038] In addition, the object surface extraction unit 210 extracts the surface portion of the object 7-1 from the image data received by the image data input unit 202. That is, the object surface extraction unit 210 has an object recognition function, and the control device 2 can also function as an object recognition device. The surface portion extracted in this way can be used to estimate the grasping position.

[0039] In addition, the reliability calculation unit 211 calculates the reliability related to the possibility of grasping the object 7-1 itself based on the element reliability. At this time, it is preferred that the reliability calculation unit 211 uses a weight parameter representing the importance of each element reliability as an identification parameter. And, it is preferred that the reliability calculation unit 211 integrates multiple element reliabilities to calculate the reliability related to the possibility of grasping the object 7-1 itself. Here, the integration includes calculating the sum. In addition, the control signal generation unit 212 generates a control signal for controlling the picking operation of the picking robot 3 based on the gripping position estimated by the gripping position estimation unit 209. In addition, it is preferred that the estimated gripping position is included in the control signal.

[0040] Next, the output unit 213 includes a gripping position output unit 214 and a display data output unit 215. First, the gripping position output unit 214 outputs at least the estimated gripping position to the picking robot 3. In addition, the gripping position output unit 214 may be configured to output the gripping position by outputting a generated control signal to the picking robot 3. In addition, the display data output unit 215 outputs various information displayed by the display 21 to the display 21.

[0041] The display content will be described later. In addition, the storage unit 216 stores information and data used in the processing of the above-mentioned units. The information and data will be described later.

[0042] In addition, the input device 22 accepts the user's operation, and the input unit 201 accepts the operation content. In addition, it is preferred that the input unit 201 is also connected to the storage unit 216 and can receive information and data. In addition, the display 21 displays various information output by the display data output unit 215. Therefore, the display 21 can be implemented by a CRT display, an LCD (Liquid Crystal Display), an organic EL (Electro-Luminescence) display, etc. In addition, the input device 22 can be implemented by a pointing device such as a touch panel, a keyboard, and a mouse. Therefore, the display 21 and the input device 22 can be constructed in a shared manner like a touch panel. End here Figure 2 Next, an installation example of the control device 2 is described.

[0043] Figure 3 FIG. 2 is a hardware configuration diagram of the control device 2 of this embodiment. In this embodiment, the control device 2 is implemented by a computer. Figure 3 In the embodiment, the control device 2 includes a display 21, an input device 22, a CPU 23, a RAM 24, a communication device 26, a ROM 25, a medium reading device 27, and an auxiliary storage device 28. The display 21 and the input device 22 have been described, and the other structures will be described below.

[0044] First, the CPU 23 (Central Processing Unit) can be realized by a so-called processor and is a unit that performs various operations. The CPU 23 performs various processes by executing the picking robot control program 280 loaded from the auxiliary storage device 28 to the RAM 24. In addition, these various processes are equivalent to Figure 2 The details of the processing in the element reliability evaluation unit 207 to the control signal generation unit 212 will be described later using a flowchart.

[0045] Here, the picking robot control program 280 is, for example, an application program that can be executed on an OS (Operating System) program. The picking robot control program 280 is composed of an element reliability evaluation module 281, a recognition parameter estimation module 282, a gripping position estimation module 283, an object surface extraction module 284, a reliability calculation module 285, and a control signal generation module 286. The processing using these modules is equivalent to Figure 2 The processing in the element reliability evaluation unit 207 to the control signal generation unit 212. In addition, in the present embodiment, the picking robot control program 280 is composed of a plurality of modules for each function, but it may be realized by a plurality of independent programs for each function.

[0046] In addition, the picking robot control program 280 can be installed from a removable storage medium to the auxiliary storage device 28 via the medium reading device 27, for example. That is, the picking robot control program 280 can be stored in a storage medium. In addition, the CPU 23 can also execute processing according to a program other than the picking robot control program 280. In this case, it is preferable to store the program in the auxiliary storage device 28.

[0047] In addition, RAM24 (Random Access Memory) is a memory for storing programs such as the picking robot control program 280 executed by the CPU 23, data required for the execution of the program, etc. ROM25 (Read Only Memory) is a memory for storing programs and OS required to start the control device 2. In addition, the communication device 26 is connected to the picking robot 3 and the sensor 4 via the communication path 8. In addition, the communication path 8 can be implemented through a network such as a LAN (Local Area Network) and the Internet. In addition, the communication device 26 is not only used as Figure 2 In addition to the functions of the input unit 201 and the output unit 213, the display 21 and the input device 22 also function as an interface.

[0048] The medium reading device 27 may be realized by a device that reads information from a removable storage medium such as a flash memory or a CD-ROM. The auxiliary storage device 28 may be realized by a hard disk drive (HDD) or the like, and stores data and programs for executing various processes.

[0049] The auxiliary storage device 28 may also be implemented by using an SSD (Solid State Drive) using a flash memory or the like. Figure 1 The storage unit 216 is a storage unit. The auxiliary storage device 28 stores the above-mentioned picking robot control program 280 and the following various information and data. That is, the auxiliary storage device 28 stores the transport information 287, the reliability data 288, the recognition parameter 289, the history information 290, the hardware information 291 and the item characteristic information 292. The following describes each of these information and data.

[0050] Figure 4Indicates the transport information 287 used in this embodiment. The transport information 287 indicates the transport conditions related to the picking operation for the article 7, that is, the movement and grasping. In this embodiment, the transport conditions are defined by each item of the time, collision, speed and accuracy of the picking operation. Here, time indicates the time related to the picking operation. In addition, in this embodiment, as time, the time of "pick-up-and-place" and "until the placing action" are used. "Pick-up-and-place" indicates the time from the time when the article 7-1 is grasped in the picking source container 5 until it is placed in the picking destination container 6.

[0051] “Until placing operation” indicates the time from activation of the picking robot 3 to the start of movement.

[0052] In addition, collision refers to an item related to collision in the picking operation. Moreover, in this embodiment, as collision, "collision probability" and "margin when determining collision" are used. "Collision probability" indicates the probability of collision between the article 7 and the picking robot 3 in the picking operation. In addition, "margin when determining collision" indicates the reference distance of the object to be the collision object when the collision is determined.

[0053] In addition, the speed indicates the speed of the picking robot 3 and the object 7 moving with it during the picking operation. In this embodiment, "maximum speed" and "maximum acceleration" are used as speeds. "Maximum speed" indicates the maximum speed of the picking robot 3 and the object 7 during the picking operation. In addition, "maximum acceleration" indicates the maximum acceleration of the picking robot 3 and the object 7 during the picking operation.

[0054] In addition, the accuracy refers to the accuracy in the picking operation. In this embodiment, "deviation" is used as the accuracy. The deviation includes the position of the picking robot 3 in the operation, for example, the deviation between the target and the actual position of the gripping position or the arrangement position of the article 7-2.

[0055] In addition, each of these items is only an example, and a part of them may be used, or other items may be used. Moreover, in the present embodiment, as a transport condition, a reference for the degree of consideration of each item is recorded. That is, for each item, a numerical value serving as its reference is recorded as "large", "medium" or "small". For example, for "pick-up-and-place", in the case of T1>, it indicates a "large" degree of consideration, in the case of "T1~T2", it indicates a "medium" degree of consideration, and in the case of T2<, it indicates a "small" degree of consideration. In addition, the degree of consideration of each item and its reference are optional items and may be omitted.

[0056] Next, the reliability data 288 used in this embodiment is described. The reliability data 288 is data indicating an index of gripping possibility, and indicates the value of the index at each position of the image data. In addition, the position here can use the pixel of the image data. The reliability data 288 can use a two-dimensional heat map image indicating the plane degree of each pixel for the image data of the processing object, for example. Here, Figure 5 The reliability data 288 used in this embodiment is schematically shown. As described above, the reliability data 288 is data such as a heat map image, but Figure 5 In order to understand the present embodiment, the value of the index at each position of the image data of each object is schematically represented as a scalar value. Figure 5 As shown, the reliability data 288 of this embodiment uses the flatness, distance from the geometric center, height, normal, surface overlap, surface shape, and surface normal as physical elements (elements), and the value of the index at each position can be determined. However, these are examples, and a part of them or other items can be used. In addition, the "geometric center" in the "distance from the geometric center" is defined for each position of the image data that becomes the object of reliability calculation, and the object surface to which it belongs. The object surface here is calculated by the object surface extraction unit 210.

[0057] The identification parameter 289 is a parameter for estimating the gripping position. The calculation of the identification parameter 289 will be described later using a flowchart.

[0058] then, Figure 6 The historical information 290 used in this embodiment is shown. The historical information 290 indicates whether the above-mentioned transport condition was satisfied in the past picking operation, that is, the grasping and moving of the object 7. In the historical information 290 of this embodiment, it is shown to what extent the transport condition was satisfied in the past picking operation. In addition, in this embodiment, the time when the picking operation was performed is also recorded.

[0059] then, Figure 7 The hardware information 291 used in this embodiment is shown. The hardware information 291 is information indicating the characteristics of the picking robot 3. In this embodiment, the performance in the picking operation is used as the characteristics of the hardware information 291. It is particularly preferable to use the size of the hand and the gripping force as the performance in addition to the above-mentioned operation time, speed, and accuracy corresponding to the transport conditions.

[0060] then, Figure 8 Item characteristic information 292 used in this embodiment. Item characteristic information 292 is information related to the characteristics of item 7. Figure 8As shown, there are items of flexibility, strength, weight, and shape of the article as the article characteristic information 292. However, these are examples, and some of them or other items may be used.

[0061] This concludes the description of the information and data used in this embodiment. Figure 1 Next, the structure of the picking system 1 in this embodiment is described. Figure 1 In the embodiment, the picking robot 3 grasps the article 7-1 of the picking source container 5 according to the control signal of the control device 2, and moves to the picking destination container 6. Therefore, the picking robot 3 has a hand that grasps the article 7-1. In addition, the grasping method of the hand is not limited. As a grasping method, for example, the article 7-1 can be sucked, and it can also include grasping or clamping the article 7-1. Then, the arm of the picking robot 3 moves, and the article 7-1 grasped by the hand moves to the picking destination container 6.

[0062] In addition, sensor 4 captures Figure 1 As a result, the object 7-1 is also photographed by the sensor 4. Therefore, the sensor 4 can be implemented by a device such as a camera that can obtain image data that can confirm the shape of the object 7-1.

[0063] In addition, Figure 1 In the example, the object 7-1 to be picked up is loaded in the pickup source container 5. In addition, the object 7-2 moved from the pickup source container 5 is loaded in the pickup destination container 6. In addition, the location where the objects 7-1 and 7-2 are placed is not limited to containers. For example, it can also be a platform of a transport vehicle, a shelf, etc.

[0064] This concludes the description of the structure, information, and data of this embodiment. Next, the processing content in this embodiment will be described. Figure 9 2 is a flowchart showing the processing contents of this embodiment. Figure 2 Each section shown is described as a processing main body.

[0065] First, in step S1, the gripping position estimation unit 209 of the control device 2 determines whether the transport condition of the transport information 287 has been changed. To this end, the gripping position estimation unit 209 can, for example, detect that the transport information 287 of the storage unit 216 has been changed, or can detect that a change instruction has been received by the change instruction input unit 206. As a result, if the transport condition has been changed (yes), the process transfers to step S2. On the other hand, if the transport condition has not been changed (no), the process transfers to step S4. In addition, the processing body in step S1 can also be executed by other structures such as the identification parameter estimation unit 208, or can also be executed by the input unit 201.

[0066] Furthermore, in step S2, the transport information input unit 204 acquires the transport information 287 including the changed transport conditions.

[0067] Furthermore, in step S3, the identification parameter estimation unit 208 estimates the identification parameter based on the acquired transport information 287. For example, the identification parameter estimation unit 208 can calculate the identification parameter using a predetermined calculation formula using the degree of the transport condition of the transport information 287 as a variable.

[0068] In step S4, the image data input unit 202 obtains image data from the sensor 4. That is, image data captured by the sensor 4 and including the object 7-1 is obtained. Alternatively, the image data may be stored in the storage unit 216, and the stored image data may be taken in by the element reliability evaluation unit 207 that executes step S5 described later.

[0069] In addition, in step S5, the element reliability evaluation unit 207 evaluates (calculates) the reliability of multiple elements of the article 7-1 based on the acquired image data. The element reliability is an indicator of the possibility of grasping each physical element at each position (pixel) of the article 7-1. For this purpose, as a preprocessing, the article surface extraction unit 210 performs image processing such as image recognition on the image data to identify the article 7-1 and extract the physical elements of the article 7-1 such as the surface. Then, the element reliability evaluation unit 207 estimates the numerical value related to the physical element at each position (pixel) of the article 7-1. For example, the numerical value of the flatness is estimated.

[0070] Then, the element reliability evaluation unit 207 evaluates the element reliability, which is an index (value) corresponding to the determined numerical value. As a result, the element reliability evaluation unit 207 stores the element reliability as an index (value) of the reliability data 288 in the storage unit 216 .

[0071] In step S6, the reliability calculation unit 211 and the recognition parameter estimation unit 208 calculate the article reliability indicating the grasping possibility of the article 7-1 itself using the plurality of element reliabilities evaluated in step S5. For example, the article reliability is calculated as follows.

[0072] (1) The identification parameter estimation unit 208 receives the transport information 287, calculates the identification parameter based on the transport information, and outputs the identification parameter. The identification parameter estimation unit 208 can be constructed by machine learning as described later, and can calculate a more accurate identification parameter.

[0073] (2) Then, the reliability calculation unit 211 uses the identification parameter to integrate the element reliability and calculate the article reliability. For example, the calculation is Σ(element reliability×identification parameter (weight parameter))=article reliability.

[0074] Here, the method of determining the identification parameter used in (1) is described. In this embodiment, the identification parameter is determined based on the user's designation. To this end, the identification parameter estimation unit 208 displays an input screen for the degree of consideration of the transport condition on the display 21 via the display data output unit 215 . Figure 10 The screen 2100 of the display 21 when inputting the consideration degree in this embodiment is shown. In the screen 2100, a slider for inputting the consideration degree is displayed for each specification of the picking operation related to the conveying conditions ("no collision", "transfer time", "stacking accuracy").

[0075] For the slider, the user uses the input device 22 to specify the degree of consideration that also indicates its importance. That is, the further to the right the slider is, the more importance it has. Therefore, the recognition parameter estimation unit 208 determines the numerical value corresponding to the position of the slider. In addition, the recognition parameter estimation unit 208 can also change the numerical values ​​of other specifications according to the movement of the slider. For example, when emphasizing (moving to the right) "no collision", the numerical values ​​of "conveying time" and "stacking accuracy" are reduced by a corresponding amount. As a result, a balance of the weights of each specification can be achieved. In addition, if Figure 10 As shown, it is preferred to display the slider of the moving object differently from other sliders.

[0076] Then, the identification parameter estimation unit 208 calculates the weight parameter, that is, the identification parameter, based on the weight corresponding to the slider. Then, in the process (2), the element reliability of the physical element is multiplied by the calculated identification parameter value, and the sum is calculated to calculate the item reliability of step S6.

[0077] Here, the identification parameter estimation unit 208 can be constructed as a parameter estimator. For this purpose, the identification parameter estimation unit 208 can be constructed by machine learning, that is, it is possible to achieve improved accuracy. First, the value of the identification parameter and the degree of satisfaction of the handling condition in the moving operation at the gripping point estimated thereby are recorded. In addition, a group of integrations of a sufficient amount for learning is prepared to be used as learning data for the identification parameter estimation unit 208. And, for example, through learning of a neural network, the relationship between the identification parameter and the handling information 287 is reflected in the identification parameter estimation unit. According to the above, a cycle of estimating the gripping position, collecting learning data, executing the picking operation, and model learning using the learning data can be performed, thereby constructing the identification parameter estimation unit 208.

[0078] Furthermore, the identification parameter estimation unit 208 can also be constructed manually. If the index of gripping possibility is a heuristic index, it is easy to know the influence of the index on the transport condition, and the identification parameter estimation unit 208 can be constructed manually. The content is described below.

[0079] For example, the higher the flatness of the gripping position, the greater the suction force of the gripped article, and the faster the conveyance can be. In this case, if you want to make the speed limit strict, increase the recognition parameter of the flatness.

[0080] In addition, the closer the gripping position is to the center of the object surface, the less deformation of the gripped object is, and the easier it is to place (transport). In addition, the object 7-1 is difficult to slip out of the hand, and can be transported quickly.

[0081] If the stacking accuracy constraint is to be made strict, the recognition parameter of the center of the article 7-1 is increased. If the time limit is to be made strict, the recognition parameter of the center of the article 7-1 is increased.

[0082] In addition, the higher the height of the gripping position is, the less likely the article 7-1 is to be located below other articles, and the less likely it is to collide with them. Therefore, if you want to make the collision restriction stricter, you can increase the recognition parameter of the height.

[0083] In addition, the closer the normal direction of the gripping position is to the vertical, the lower the possibility that the object 7-1 or the hand will collide with the pickup source container 5 or the pickup destination container 6. Therefore, when it is desired to make the collision restriction stricter, the recognition parameter of the normal direction is increased.

[0084] In addition, in step S7, the gripping position estimating unit 209 estimates the gripping position of the picking robot 3 for the item 7-1 based on a plurality of element reliabilities, for example, using the item reliability calculated based on the plurality of element reliabilities. At this time, the gripping position estimating unit 209 estimates the gripping position based on the item reliability of each position of the item 7-1. For example, the position with the highest item reliability is estimated as the gripping position. In addition, the position of the present embodiment is not limited to a point (coordinate) on the item 7-1, and preferably has an area of ​​a certain degree. In particular, an area corresponding to the size of the hand of the picking robot 3 is preferred. For example, the area includes the suction area of ​​the hand.

[0085] Furthermore, it is preferable that the gripping position estimating unit 209 causes the display 21 to display the gripping position and the article reliability via the display data output unit 215 . Figure 11 FIG. 2 is a diagram showing a screen 2100 of the display 21 that displays the gripping position and the reliability of the article according to the present embodiment. Figure 11 In the example, the image data A to D of each of the plurality of items 7-1 are displayed on the screen 2100. Moreover, in each of the image data A to D, the value of the item reliability is represented as a gradient. That is, the darker the color, the higher the item reliability. Figure 11 In the example, the grasping position estimated in this step is clearly indicated in the image data A. Furthermore, the reliability of the article at the position specified by the input device 22 may be displayed. Figure 11This example is shown in image data B.

[0086] In step S8, the control signal generating unit 212 generates a control signal including the gripping position estimated in step S7. Figure 11 Screen 2100 shown generates a control signal when a confirmation instruction is received.

[0087] Then, in step S9, the gripping position output unit 214 having a control signal output function outputs the generated control signal to the picking robot 3. As a result, in step S10, the picking robot 3 performs a picking operation on the object 7-1 according to the control signal. That is, the picking robot 3 moves its own hand to the estimated gripping position and grips the object.

[0088] According to the above, the gripping position corresponding to the transporting conditions can be estimated, and a more accurate and efficient picking operation can be performed. To this end, in this embodiment, reliability (element reliability, item reliability) is used to estimate the gripping position. Therefore, the consideration method of this reliability is simply summarized. Figure 12 This is a table showing the reliability consideration method of this embodiment. Figure 12 In the embodiment, the characteristics of each physical element are shown. The characteristics represent the characteristics related to the gripping possibility of the corresponding physical element. By using such a reliability concept, in this embodiment, a more appropriate gripping position can be estimated according to the situation and needs. In addition, in this embodiment, Figure 12 The information shown is stored in the storage unit 216, and the grasping position is estimated using this information.

[0089] Next, a modification of this embodiment is described. Figure 1 , one picking robot 3 is described, but the number of picking robots 3 may be plural. In addition, the display 21 and the input device 22 may be configured differently from the control device 2. Furthermore, the database 9 may be provided separately. Figure 13 Modifications of the above are shown. Figure 13 A variation of the present embodiment is shown. In this variation, the control device 2 is implemented by a so-called server, and controls a plurality of picking robots 3-1 and 3-2. Therefore, the control device 2 is connected to the picking robots 3-1 and 3-2 via a communication path 8. In addition, sensors 4-1 and 4-2 for photographing the picking objects of the picking robots 3-1 and 3-2 are also connected to the communication path 8. Furthermore, in this variation, a terminal device 10 and a tablet terminal 11 are provided instead of the display 21 and the input device 22. The terminal device 10 and the tablet terminal 11 can be implemented by a computer such as a so-called PC, a tablet device, a smartphone, etc. operated by a user. Furthermore, the display in these devicesFigure 10 or Figure 11 Screen 2100 is shown.

[0090] In addition, Figure 13 In the embodiment, the database 9 stores the history information 290, the hardware information 291 and the article characteristic information 292. However, the information sent out from the control device 2 is not limited to this. In addition, the database 9 may be omitted and the control device 2 may store such information.

[0091] In addition, in the present embodiment and the modified example, the transport information 287 may be changed according to a change instruction to the change instruction input unit 206. In this case, the change instruction input unit receives a change instruction to the input device 22 and a change instruction to the terminal device 10 or the tablet terminal 11 via the communication device 26, and makes a change.

[0092] In addition, as a change of the transport information 287, information from the sensor 4 that grasps the status of the picking source, the placement destination (pickup destination), etc., such as a visual sensor, can be used. In addition, the transport information 287 can also be changed using information from a system that manages the entire operation site of the picking robot 3. In addition, as an example of the operation site, a warehouse can be exemplified, and as its system, a WCS (Warehouse Management System) can be used.

[0093] This is the end of the description of this modification example, but the present invention is not limited to this modification example and embodiment.

[0094] For example, the control device 2 may be realized by hardware such as a dedicated circuit.

[0095] Description of Reference Numerals

[0096] 1…Picking system, 2…Control device, 20…Main unit, 21…Display, 22…Input device, 201…Input unit, 202…Image data input unit, 203…Article characteristic information input unit, 204…Transportation information input unit, 205…Hardware information input unit, 206…Change instruction input unit, 207…Element reliability evaluation unit, 208…Recognition parameter estimation unit, 209…Gripping position estimation unit, 210…Article surface extraction unit, 211…Reliability calculation unit, 212…Control signal generation unit, 213…Output unit, 214…Gripping position output unit, 215…Display data output unit, 216…Storage unit, 3…Picking robot, 4…Sensor, 5…Picking source container, 6…Pickup destination container, 7…Article, 8…Communication path, 9…Database.

Claims

1. A picking system, which performs a picking operation on an object to be picked up. It is characterized in that The picking system has: a picking robot that grasps the item and moves the item; and a control device, which controls the picking robot, The control device comprises: An image input unit that receives image data of the object acquired by a sensor; an element reliability evaluation unit that evaluates a plurality of element reliabilities representing indices of gripping possibility of each physical element of the article based on the image data; a gripping position estimating unit for estimating a gripping position of the object by the picking robot based on the plurality of element reliabilities; a transport information input unit that receives transport information indicating transport conditions related to movement and gripping of the article gripped at the gripping position; as well as an identification parameter estimating unit that estimates, based on the transport information, an identification parameter used in estimating the gripping position by the gripping position estimating unit, The picking robot operates using the gripping position estimated by the control device.

2. The picking system according to claim 1, It is characterized in that The transport information indicates, as the transport condition, at least one of a collision of the moving article with another object, a time required for the movement, a moving speed or acceleration during the movement, and a position accuracy of the moved article.

3. The picking system according to claim 1, It is characterized in that In the control device, the identification parameter is a weight parameter indicating the importance of each of the plurality of element reliabilities.

4. The picking system according to claim 3, It is characterized in that The control device further includes a reliability calculation unit that integrates the plurality of element reliabilities using the recognition parameter to calculate an item reliability indicating the possibility of grasping the item itself. The gripping position estimating unit estimates the gripping position using the article reliability.

5. The picking system according to claim 1, It is characterized in that The control device further includes an article characteristic information input unit, which receives article characteristic information related to the characteristics of the article. The identification parameter estimating unit estimates a value of the identification parameter based on the transport information and the article characteristic information.

6. The picking system according to claim 1, It is characterized in that The control device further includes a hardware information input unit for receiving hardware information related to hardware constituting the picking robot. The identification parameter estimation unit estimates a value of the identification parameter based on the transport information and the hardware information.

7. The picking system according to claim 1, It is characterized in that The picking system also has: a database storing historical information indicating whether the transport condition was satisfied in the grasping and moving of the object performed in the past; and A transport information changing unit changes the transport information based on the history information.

8. The picking system according to claim 7, It is characterized in that The picking system further includes a change instruction input unit, the change instruction input unit receiving a change instruction of the transport information. The transport information changing unit changes the transport information according to the change instruction.

Images