Handling system, transport system, control device, control program, and handling method

Abstract

To provide a handling system, a conveyance system, a control device, a control program and a handling method which can shorten operation time of a handling device.SOLUTION: A handling system in an embodiment has a movable arm, a holding part, a sensor, and a control part. The holding part is mounted on the movable arm, and can select one or more holding methods among a plurality of holding methods and can hold an object. The sensor can detect the plurality of objects. The control part controls the movable arm and the holding part. The control part calculates a score based on the selected holding method for each of the objects and the holding methods on the basis of the information acquired from the sensor. The control part selects the object to be held next and the holding method on the basis of the score. The control part calculates the position where the selected object is held and the attitude of the movable arm.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] An embodiment of the present invention relates to a handling system, a transport system, a control device, a control program, and a handling method. [Background technology]

[0002] Conventionally, handling devices in which an end effector holds an object are known. To automate the transfer operation at a logistics site, it is required to be able to hold objects of a wide variety of shapes, sizes, and weights. When holding these objects using a handling device, many calculations are required to determine a holding strategy, such as the holding position, holding method, and posture of the robot arm. When the loading state of the object becomes complex, the time required for the calculations required to determine the holding strategy increases.

[0003] Furthermore, when performing work by selectively using a plurality of holding tools, if the holding method is switched too many times, the time required for the entire work may be increased by the time required for the tool exchange operation. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Patent No. 5130509 [Patent Document 2] JP 2019-188516 A Summary of the Invention [Problem to be solved by the invention]

[0005] The problem to be solved by the present invention is to provide a handling system, a transport system, a control device, a control program, and a handling method that can shorten the operation time of a handling device. [Means for solving the problem]

[0006] The handling system of the embodiment has a movable arm, a holding unit, a sensor, and a control unit. The holding unit is attached to the movable arm and is capable of holding an object by selecting one or more of a plurality of holding methods. The sensor is capable of detecting a plurality of the objects. The control unit controls the movable arm and the holding unit. The control unit calculates a score for each object and each holding method based on information acquired from the sensor, with the selected holding method as a reference. The control unit selects the object and holding method to be held next based on the score. The control unit calculates a position for holding the selected object and an attitude of the movable arm.

[0007] A handling system according to another embodiment includes a movable arm, a holding unit, a sensor, and a control unit. The holding unit is attached to the movable arm and is capable of selecting one or more of a plurality of holding mechanisms to hold a holding object. The sensor is capable of detecting the holding object. The control unit controls the movable arm and the holding unit. The control unit calculates a score based on one of the holding objects and a holding mechanism selected from one of the holding mechanisms based on information acquired from the sensor. The control unit selects the holding object and a holding mechanism that will hold the holding object after the selected holding mechanism based on the score. The control unit calculates a position at which the holding mechanism that will hold the holding object after the selected holding mechanism will hold the holding object, and an attitude of the movable arm. [Brief description of the drawings]

[0008] [Figure 1] 1 is a perspective view showing a transport system including a handling system according to a first embodiment; [Diagram 2] 1 is a block diagram showing a system configuration of a transport system including a handling system according to a first embodiment. [Diagram 3] 4 is a control flowchart of the control device according to the first embodiment. [Figure 4]13 is a diagram showing a temporary mask area in image data of an object when the object O is held by "clamping." [Diagram 5] FIG. 4 is a diagram showing a mask region in image data of an object. [Figure 6] FIG. 1 is a diagram showing a depth image of an object. [Figure 7] 1 is a diagram for explaining the three-dimensional position and orientation of an object. [Figure 8] FIG. 13 is a diagram showing a mask region to which information regarding three-dimensional position and orientation has been added. [Figure 9] 5 is a control flowchart of a retention strategy planning process performed by the control device according to the first embodiment. [Figure 10] 10 is a control flowchart of a retention strategy planning process performed by a control device according to a second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, a handling system, a transport system, a control device, a control program, and a handling method according to embodiments will be described with reference to the drawings. In the following description, components having the same or similar functions are given the same reference numerals. Duplicate descriptions of those components may be omitted. In addition, "based on XX" in this application means "based on at least XX" and includes cases where it is based on other elements in addition to XX. Furthermore, "based on XX" is not limited to cases where XX is used directly, but also includes cases where it is based on XX that has been subjected to calculation or processing. "XX" is any element (for example, any information).

[0010] (First embodiment) An embodiment will be described with reference to Figures 1 to 9. Figure 1 is a perspective view that shows a schematic diagram of a transport system 1 including a handling device 10 (an example of a "handling system") of the present embodiment.

[0011] The transport system 1 is, for example, a handling system (picking system) for logistics. The transport system 1 moves an object (object to be held, object to be transported) O located at a source V1 to a destination V2. For example, the transport system 1 performs a task of picking out a specified number of various kinds of objects O stored at the source V1 and the like and loading them at the destination V2.

[0012] The source V1 may be, for example, any type of conveyor, any type of pallet, or a container such as a tote or an Ori-con. A "container" broadly means a member (e.g., a box-shaped member) capable of housing an object O. However, the source V1 is not limited to the above example. In the following description, the "source V1" may be referred to as the "source container V1."

[0013] Many kinds of objects O with different sizes and weights are randomly placed at the movement source V1. For example, the object O to be held has an uneven shape on at least a part of the surface of the object O. In this embodiment, the outer shape of the object O varies from small ones such as 5 cm square to large ones such as 30 cm square. In addition, the object O varies from light ones such as several tens of grams to heavy ones such as several kilograms. However, the size and weight of the object O are not limited to the above example.

[0014] The destination V2 is, for example, a container such as a tote or an Ori-con. However, the destination V2 is not limited to the above example. In the following description, the "destination V2" may be referred to as a "destination container V2", and the "source V1" and the "destination V2" may be collectively referred to simply as a "container". Note that the transport system 1 may also transport the object O to a destination V2 other than a container.

[0015] The transport system 1 is not limited to a handling system for logistics. The transport system 1 can be widely applied to industrial robot systems and other systems. In this application, the terms "transport system," "handling system," and "handling device" are not limited to systems and devices whose main purpose is to transport objects, but also include systems and systems that involve transporting (moving) objects as part of product assembly or other purposes.

[0016] 1, the transport system 1 includes a handling device 10, a sensor 11, and a control device 12 (an example of a "control unit"). The control device 12 may be incorporated in the handling device 10.

[0017] The handling device 10 is, for example, a robot device. The handling device 10 holds an object O located in a source container V1 and moves the held object O to a destination container V2 (storage area). The handling device 10 can communicate with a control device 12 by wire or wirelessly. In this embodiment, the handling device 10 has a first handling device 10A and a second handling device 10B.

[0018] The first handling device 10A has, for example, a movable arm 100 and a first holding part 200A provided at the tip of the movable arm 100.

[0019] The movable arm 100 is a moving mechanism that moves the first holding unit 200A to a desired position. For example, the movable arm 100 is a six-axis vertical articulated robot arm. The movable arm 100 can take various positions and postures. Like a human arm or hand, the movable arm 100 can also take a variety of postures for holding an object. The movable arm 100 includes, for example, a plurality of arm members 101 and a plurality of rotating units 102 that rotatably connect the plurality of arm members 101.

[0020] The movable arm 100 may be a three-axis orthogonal robot arm. The movable arm 100 may be a mechanism that moves the first holding unit 200A to a desired position by other configurations. For example, the movable arm 100 may be an aircraft (e.g., a drone) that lifts and moves the first holding unit 200A by a rotor.

[0021] The first holding unit 200A is a holding mechanism (end effector) that holds the object O located in the source container V1. The first holding unit 200A has a clamping hand 202.

[0022] The clamping hand 202 is a gripper-type hand that pinches and grasps the object O between two fingers, and is provided at the tip of the movable arm 100. The configuration of the clamping hand 202 is not limited to this, and may be, for example, a gripper-type hand that pinches and grasps the object O between three fingers.

[0023] The first holding unit 200A may be a hybrid hand that further includes a suction device and an adsorption unit communicating with the suction device and holds the object O by clamping and / or adsorption. In this case, the adsorption unit may be provided on the finger tips of the clamping hand 202. A plurality of adsorption units may be provided on the finger tips of the clamping hand 202.

[0024] The second handling device 10B has, for example, an arm (second arm) 100 and a second holding part 200B provided at the tip of the movable arm 100. The movable arm 100 of the second handling device 10B has a similar configuration to the movable arm 100 of the first handling device 10A.

[0025] The second holding unit 200B is a holding mechanism (end effector) that holds the object O located in the source container V1. For example, the second holding unit 200B has a suction device 203 and a suction unit 205 that communicates with the suction device 203. The second holding unit 200B is a suction-type hand that holds the object O by suction.

[0026] The second holding unit 200B may be a mechanism that holds the object O by other holding methods. For example, the second holding unit 200B may be a holding unit that can hold the object O by using magnetic force. For example, the second holding unit 200B may be a holding unit (e.g., a jamming gripper) that is composed of a flexible membrane filled with powder and a vacuum pump that removes air from within the flexible membrane and can hold the object O by using a jamming phenomenon.

[0027] The suction device 203 is, for example, a vacuum pump. The suction device 203 is in communication with each of the plurality of suction units 205 via a hose or the like. When the suction device 203 is driven, the pressure in each suction unit 205 becomes lower than atmospheric pressure, and the object O is adsorbed and held by the suction unit 205.

[0028] The suction unit 205 is provided at the tip of the second holding unit 200B. For example, a plurality of suction units 205 are provided at the tip of the second holding unit 200B. The suction unit 205 has an outer shape smaller than the smallest object O located in the source container V1. The second handling device 10B suctions and holds the object O using only one or more suction units 205 selected from the plurality of suction units 205.

[0029] In the following description, the "first holding unit 200A" and the "second holding unit 200B" are collectively referred to as the "holding unit 200". That is, the "holding unit 200" includes the "first holding unit 200A" and the "second holding unit 200B". Here, the first holding unit 200A is described as a clamping hand and the second holding unit 200B is described as an adsorption hand, but the configuration of the holding unit 200 is not limited to the above-mentioned configuration having one each of the first holding unit 200A of the clamping hand and the second holding unit 200B of the adsorption hand. In this embodiment, the first holding unit 200A and the second holding unit 200B may both be clamping hands or adsorption hands. In this case, the holding unit 200 may be configured to have a plurality of clamping hands that are different in at least one of the characteristics such as the configuration, structure, shape, size, and arrangement. Specifically, for example, the first holding unit 200A and the second holding unit 200B may be two or more clamping type hands with different claw lengths or opening widths. The holding unit 200 may also be configured to have a plurality of suction type hands that differ in at least any of the characteristics such as configuration, structure, shape, size, and arrangement. Specifically, for example, the first holding unit 200A and the second holding unit 200B may be two or more suction type hands with different suction pad arrangements, suction pad diameters, bellows structures, etc. Even in these cases, the present embodiment can be implemented in the same way to obtain the same effects.

[0030] The sensor 11 is controlled by the control device 12 to detect the state of the objects O and / or the holding unit 200. The sensor 11 has a first sensor 11A, a second sensor 11B, a third sensor 11C, a fourth sensor 11D, and a fifth sensor 11E. The first sensor 11A, the second sensor 11B, the third sensor 11C, the fourth sensor 11D, and the fifth sensor 11E are connected to the control device 12 by wire or wirelessly.

[0031] The first sensor 11A is a camera or various sensors arranged near the source V1 (for example, directly above or diagonally above the source V1). The first sensor 11A acquires, for example, information about the object O located at the source V1 and information about the source V1. The information acquired by the first sensor 11A is, for example, "image data", "distance image data", "shape data", etc. The "distance image data" is image data having distance information in one or more directions (for example, depth information from an arbitrary reference plane set above the source V1). The "shape data" is information indicating the outer shape of the object O, etc. The information detected by the first sensor 11A is output to the control device 12. The first sensor 11A may be provided as a part of the handling device 10.

[0032] The second sensor 11B is a camera or various sensors arranged near the destination container V2 (for example, directly above or diagonally above the destination container V2). The second sensor 11B detects, for example, information about the shape of the destination container V2 (including the shapes of the inner walls and partitions) and information about the object O previously placed in the destination container V2. The information acquired by the second sensor 11B is, for example, "image data," "distance image data," "shape data," etc. The information detected by the second sensor 11B is output to the control device 12. The second sensor 11B may be provided as a part of the handling device 10.

[0033] The third sensor 11C is a variety of sensors provided on the first holding unit 200A or near the first holding unit 200A. The third sensor 11C acquires information on the physical state of the first holding unit 200A, such as the distortion of the first holding unit 200A, the pressure applied to the first holding unit 200A, and the surface state of the first holding unit 200A. The third sensor 11C includes one or more physical sensors, such as a distortion sensor, a pressure sensor, and a near-contact sensor. The third sensor 11C may further acquire physical information of the object O. The information detected by the third sensor 11C is output to the control device 12. The third sensor 11C may be provided as a part of the handling device 10.

[0034] The fourth sensor 11D is a variety of sensors provided on the second holding unit 200B or near the second holding unit 200B. The fourth sensor 11D acquires information on the physical state of the second holding unit 200B, such as the distortion of the second holding unit 200B, the pressure applied to the second holding unit 200B, and the surface state of the second holding unit 200B. The fourth sensor 11D includes one or more physical sensors, such as a distortion sensor, a pressure sensor, and a near-contact sensor. The fourth sensor 11D may further acquire physical information of the object O. The information detected by the fourth sensor 11D is output to the control device 12. The fourth sensor 11D may be provided as a part of the handling device 10.

[0035] The fifth sensor 11E acquires information on the usage status of the holding unit 200. For example, the fifth sensor 11E detects the holding unit 200 that is currently being used or selected for use among the first holding unit 200A and the second holding unit 200B (hereinafter, the holding unit that is currently being used or selected for use may be collectively referred to as the "currently selected holding unit" or simply the "selected holding unit"). Information detected by the fifth sensor 11E is output to the control device 12. Note that the fifth sensor 11E may be provided as a part of the handling device 10. Also, the currently selected holding unit 200 may be determined based on other information, such as the control history of the handling device 10 or information acquired by the third sensor 11C or the fourth sensor 11D, instead of the fifth sensor 11E. In this case, the fifth sensor 11E can be omitted. In addition, if the holding portion 200 is a hybrid hand capable of performing clamping, adsorption, and both clamping and adsorption, the control device 12 determines, based on information from the fifth sensor 11E or other information, whether the holding portion 200 is in a state suitable for clamping the object O, a state suitable for adsorption, or a state suitable for both.

[0036] The control device 12 manages and controls the entire transport system 1. For example, the control device 12 acquires information detected by the first sensor 11A to the fifth sensor 11E, and controls the handling device 10 based on the acquired information. The control device 12 is, for example, a programmable device (computer) equipped with a processor, memory, etc.

[0037] FIG. 2 is a block diagram showing a system configuration of the transport system 1. As shown in FIG. The control device 12 includes an input unit 300 , a recognition processing unit 301 , a memory unit 302 , an operation control unit 303 , a score generating unit 304 , a threshold generating unit 305 , a judgment unit 306 , and a retention strategy determining unit 307 .

[0038] The input unit 300 receives an order list relating to the object O to be held from an operator or the system, information acquired by the first sensor 11A to the fifth sensor 11E, and the like.

[0039] The recognition processing unit 301 processes information acquired by the first sensor 11A to the fifth sensor 11E. For example, the recognition processing unit 301 determines the position, posture, shape, characteristics, and the like of the object O at the origin V1 based on the image data acquired by the first sensor 11A.

[0040] The storage unit 302 records shape data on the object O to be picked, an order list received from an operator or the system, various scores generated by a score generation unit 304, a threshold value generated by a threshold generation unit 305, a determination result by a determination unit 306, a holding strategy determined by a holding strategy determination unit 307, a control history and an operation history of the handling device 10, etc. The shape data stored in the storage unit 302 is defined in the local coordinate system of the object O.

[0041] The operation control unit 303 controls the operation of the movable arm 100, the first holding unit 200A, the second holding unit 200B, etc. of the handling device 10. For example, the operation control unit 303 specifically calculates a position for holding an object O and a posture of the movable arm 100 when holding the object O, which are appropriate for holding a specific object O by a specific holding method. The operation control unit 303 also instructs the handling device 10 to cause the first holding unit 200A or the second holding unit 200B to perform a holding operation of the specific object O based on the calculated position for holding the object O and the posture of the movable arm 100 when holding the object O.

[0042] The score generating unit 304 generates a score for determining the priority of the object O to be held and the holding method. For example, the score generating unit 304 calculates a first score and a second score, which will be described later.

[0043] The threshold generating unit 305 generates a threshold for determining whether or not it is necessary to switch the holding method. For example, the threshold generating unit 305 generates a threshold for a second score, which will be described later.

[0044] The determination unit 306 determines whether or not it is necessary to switch the holding method based on the score generated by the score generation unit 304, the threshold generated by the threshold generation unit 305, and the like.

[0045] The retention strategy determination unit 307 determines a retention strategy including high-priority retention objects and retention methods, the order in which retention objects are retained, the order of retention operations and retention method switching operations, etc., based on the score generated by the score generation unit 304, the threshold value generated by the threshold generation unit 305, the result of judgment by the judgment unit 306, etc.

[0046] All or part of the functions of the control device 12 are realized by one or more processors, such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), executing programs stored in a program memory. However, all or part of these functions may be realized by hardware (e.g., a circuit unit) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a PLD (Programmable Logic Device). Also, all or part of the above functions may be realized by a combination of software and hardware. The storage unit 302 is realized by a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), a ROM (Read-Only Memory), or a RAM (Random Access Memory), etc.

[0047] Next, an operation of the transport system 1 will be described with reference to a control flowchart of the control device 12 shown in FIG.

[0048] When the control device 12 is started, the control device 12 initializes the handling device 10 and the sensor 11, and then starts controlling the handling device 10 (step S0). Next, the control device 12 executes step S1.

[0049] In step S1, the input unit 300 of the control device 12 receives an order list of objects O to be picked from an operator or the system (order list receiving step).

[0050] Next, the control device 12 executes step S2. In step S2, the input unit 300 of the control device 12 receives image data, range image data, shape data, etc. related to the object O in the source container V1 from the first sensor 11A. The recognition processing unit 301 of the control device 12 determines whether the object O on the order list is present in the source container V1 based on the data received by the input unit 300. In addition, the recognition processing unit 301 acquires information related to the shape, position, attitude, etc. of the object O to be picked based on the data received by the input unit 300 (information acquisition process).

[0051] FIG. 4 is a diagram showing a temporary mask region R1 in image data of an object O when the object O is held by "pinching." The recognition processing unit 301 of the control device 12 uses a known image segmentation method to set a rectangular area (circumscribed rectangular area) circumscribing the object O to be picked from the image data as a "temporary mask area R1." Image segmentation may be a method using machine learning.

[0052] FIG. 5 is a diagram showing a mask region R2 in the image data of the object O. As shown in FIG. The recognition processing unit 301 sets an area expanded vertically and horizontally from the temporary mask area R1 as a "mask area R2." The mask area R2 is expanded vertically and horizontally of the circumscribed rectangular area by a margin M. For example, the margin M is 100 mm. By using the expanded mask area R2, the recognition processing unit 301 can determine whether there is a space around the object O into which the clamping hand 202 can enter when the holding method by the holding unit 200 is clamping.

[0053] FIG. 6 is a diagram showing a depth image D of an object O. As shown in FIG. The recognition processing unit 301 uses the distance image data to generate a depth image D that visualizes the depth information of the object O in the mask region R2. The depth image D has a height value based on the origin of the world coordinate system (X-axis, Y-axis, Z-axis). The scale of the depth image D can be changed depending on the storage method used by the storage unit 200. The scale of the depth image D can be set to, for example, 1 mm per pixel.

[0054] Fig. 7 is a diagram for explaining the three-dimensional position and orientation of an object O. In Fig. 7, objects O having different shapes are denoted as "O1" and "O2". The recognition processing unit 301 calculates the three-dimensional position and orientation of the object O1 from the acquired image data of the object O1. The recognition processing unit 301 converts the shape data in the local coordinate system of the object O1 recorded in the storage unit 302 into a world coordinate system (X-axis, Y-axis, Z-axis) using a conversion matrix. The Z-axis direction of the world coordinate system is the depth direction from an arbitrary reference plane set above the source container V1, as shown in FIG. 7. The recognition processing unit 301 calculates the three-dimensional position and orientation of the object O1 by comparing the acquired image data of the object O1 with the shape data converted into the world coordinate system.

[0055] 8 is a diagram showing a mask area R2 to which information regarding the three-dimensional position and posture is added. The recognition processing unit 301 sets the three-dimensional position pose serving as the reference of the object O1 as the center CO of a circumscribing rectangular area (temporary mask area R1) in the mask image R2. In addition, the recognition processing unit 301 may calculate and use the centroid FO of the object O1 separately from the center CO of the circumscribing rectangular area, taking into account the ease of gripping by the gripping hand 202.

[0056] On the other hand, when the object O is held by "adsorption", the temporary mask region R1 is divided into mask planes that can be objects to be adsorbed on the surface of the object O. The recognition processing unit 301 generates a mask image R2 and a depth image D for each mask plane, and defines a direction perpendicular to the mask plane as a normal direction. The recognition processing unit 301 can determine the position and orientation by, for example, extracting a plane region from the point cloud of the three-dimensional sensor, and setting the short axis direction as x, the long axis direction as y, and the normal direction of the plane as z, for example, by principal component analysis. In the case of "clamping", only one holding region may be defined for one object O, but in the case of "adsorption", multiple holding regions may exist for one object O. Therefore, the object to be held in "adsorption" may be one region of the object O. Note that, for example, in the case where there are multiple holding regions for "clamping", the object to be held in "clamping" may also be one region of the object O.

[0057] When holding object O by "adhesion," the recognition processing unit 301 can perform a convolution process to calculate how many of the multiple adsorption units 205 to use, at what angle the adsorption units 205 should be brought into contact with object O, and so on.

[0058] The recognition processing unit 301 may use a database in which the three-dimensional position and orientation of the object O when the object O has been successfully picked up in the past are recorded. The recognition processing unit 301 can also use the database to output a recommended holding method and holding position for the object O.

[0059] The input unit 300 receives information on the physical state of the first holding unit 200A from the third sensor 11C as necessary, and receives information on the physical state of the second holding unit 200B from the fourth sensor 11D. The input unit 300 also receives information on the usage status of the holding unit 200, such as the currently selected holding unit, from the fifth sensor 11E as necessary. The recognition processing unit 301 determines which of the first holding unit 200A and the second holding unit 200B is the currently selected holding unit 200, based on the information from the fifth sensor 11E and / or other information. Note that the information on the physical states of the first holding unit 200A and the second holding unit 200B and the information on the currently selected holding unit 200 may be appropriately acquired at a timing other than step S2.

[0060] Furthermore, the input unit 300 receives physical information of the object O to be held from a database. For example, the input unit 300 can receive information on the external shape, weight, surface material, frictional characteristics, and the like of the object O from the database. The database may be defined for each individual object O, or may be used by specifying basic information primitives (e.g., a rectangular parallelepiped, a cylinder, a sphere, a square pyramid, and the like) and applying information that is close to the shape.

[0061] Next, the control device 12 executes step S3. In step S3, the control device 12 calculates a score for each object O or each region of the object O and each holding method, and determines the object O to be held next and the holding method (holding strategy planning step). Fig. 9 is a control flowchart of the holding strategy planning step by the control device 12, and shows the details of step S3.

[0062] In step S301, the score generating unit 304 of the control device 12 calculates the ease with which the holding unit 200 holds the object O as a "first score" based on the information acquired in step S2. The score generating unit 304 calculates a first score (1) for each object O or each region of the object O (hereinafter also simply referred to as "each object O") and (2) for each holding method. For example, the holding method is "clamping" by the first handling device 10A or "suction" by the second handling device 10B. When the holding method is "clamping", for example, the first score is calculated for each object O, and when the holding method is "suction", for example, the first score is calculated for each region of the object O.

[0063] The first score S calculated in step S301 H (I) is calculated, for example, by the evaluation function shown in Equation 1.

number

[0064] In the formula 1, H is the holding method (such as clamping by the first handling device 10A, suction by the second handling device 10B, etc.) that is the evaluation target of the first score. I is the mask region R2 that indicates the object O or the region of the object O that is the evaluation target of the first score. i is the evaluation item of the evaluation function, and w i is the weight of the evaluation function. That is, the above evaluation function is expressed as a linear combination of each evaluation item. However, the evaluation function is not limited to the above example, and an average value of the evaluation items may be used, or a nonlinear function may be used, and any evaluation function can be used. In addition, the evaluation items are not limited to the items described in Equation 1, and may be, for example, a quantity dependent on the physical characteristics of the object O or the clamping hand 202.

[0065] When the holding method is "clamping" (H=p), the first score S p (I) is calculated, for example, by the evaluation function shown in Equation 2.

number

[0066] In formula 2, w p1 +w p2 +w p3 +w p4 +w p5 +w p6 = 1, and w pi ≧0(i=1,2,3,4,5,6). p1 is the position of object O (e.g., the center position of object O and the height of object O). p2 is the roughness ratio of the object O. f p3 is the flatness of object O. f p4 is the depth difference between object O and its surroundings. f p5 is the result of judging whether the object is thin or not. p6 is an evaluation item used when using machine learning, and the above f p1 ~f p5 For example, f p6 When using w p1 ~w p5 may be set to zero. For each object O, the first score S p (I) is calculated.

[0067] If the retention method is "adsorption" (H=s), the first score S s (I) is calculated, for example, by the evaluation function shown in Equation 3.

number

[0068] In formula 3, w s1 +w s2 +w s3 +w s4 +w s5 +w s6 = 1, and w si ≧0(i=1,2,3,4,5,6). s1 is the position of the region of object O (e.g., the center position of the region of object O and the height of object O). s2is the roughness ratio of the area of ​​the object O. s3 is the area of ​​the region of object O. f s4 is the approach angle with respect to the area of ​​object O (for example, the angle between the normal direction of the area of ​​object O and the vertical direction). s5 is the result of judging whether the object is thin or not. s6 is an evaluation item used when using machine learning, and the above f s1 ~f s5 For example, f s6 When using w s1 ~w s5 can be set to zero. The first score S of "attachment" for various mask regions I representing the region of object O s (I) is calculated.

[0069] In this way, the score generation unit 304 generates a first score S for each holding method (here, clamping by the first handling device 10A and suction by the second handling device 10B) for each object O or region of the object O. H Calculate (I).

[0070] Next, the control device 12 executes step S302. In step S302, the score generation unit 304 of the control device 12 calculates a "second score" based on the information acquired in step S2 and the first score calculated in step S301, with the currently selected holding method as a reference. The score generation unit 304 calculates the second score (1) for each object O or region of the object O, and (2) for each holding method.

[0071] The second score T calculated in step S302 H,H0 (I) is calculated, for example, by the evaluation function shown in Equation 4.

number

[0072] In Equation 4, H is the holding method to be evaluated for the second score. H0 is the currently selected holding method. I is the object O or a mask region R2 indicating the region of the object O to be evaluated for the second score. That is, the evaluation function of Equation 4 is the first score S of the currently selected holding method. H0 (I) and the first score S of the retention method that is the subject of the second score evaluation. H However, the evaluation function for the second score is not limited to the above example, and any function can be used. For example, the first score S of the currently selected retention method H0 H0 (I) and the first score S of the retention method H that is the subject of the evaluation of the second score H It may be any function that depends on (I).

[0073] 1st Score S H Since (I) is an index of the “ease of retention” of the target mask region I by the target retention method H, when using the evaluation function of the above formula 4, the second score T H,H0 (I) is the ratio between the ease of holding by the currently selected holding method H0 and the ease of holding by the target holding method H. In other words, when the second score of formula 4 is greater than 1, the currently selected holding method H0 is easier to hold than the target holding method H. When the second score of formula 4 is less than 1, the target holding method H is easier to hold than the currently selected holding method H0. For example, when the second score of formula 4 is 0.5, it can be said that the ease of holding by the currently selected holding method H0 is 0.5 times that of the target holding method H. In other words, it can be said that the ease of holding by the target holding method H is twice that of the currently selected holding method H0.

[0074] Next, the control device 12 executes step S303. In step S303, the determination unit 306 of the control device 12 determines the second score T H,H0 The necessity of switching the retention method is determined based on (I). H,H0When (I) is used, for example, the judgment unit 306 judges that it is necessary to switch the holding method from the currently selected holding method H0 to the target holding method H if the conditional expression of the following mathematical expression 5 is satisfied, and judges that it is not necessary to switch the holding method from the currently selected holding method H0 to the target holding method H if the conditional expression of the mathematical expression 5 is not satisfied.

number

[0075] In Equation 5, Th is a predetermined threshold generated by the threshold generating unit 305. That is, the determining unit 306 determines the second score T H,H0 If (I) is below the threshold, it is determined that the currently selected retention method H0 needs to be switched to the target retention method H, and a second score T H,H0 If (I) is equal to or greater than the threshold, it is determined that there is no need to switch from the currently selected holding method H0 to the target holding method H.

[0076] 1st Score S H If you select the retention method only in (I), the information of the currently selected retention method H0 is added to the first score S H Since the second score T is not reflected in (I), the holding method with the highest success rate is simply selected. For this reason, in some cases, it may be determined that the holding method needs to be changed every time the object O is held, and in such cases, it may take time to switch the holding method. H,H0 (I) is a score that includes information on the currently selected holding method H0 and is calculated based on the currently selected holding method H0. For example, the second score T H,H0 When using (I), the second score T H,H0 Even if (I) is smaller than 1 (i.e., the ease of holding by the target holding method H is greater than the ease of holding by the currently selected holding method H0), the determination unit 306 determines the second score T H,H0If (I) is not so small (i.e., if it is equal to or greater than a predetermined threshold Th), it is determined that there is no need to switch the holding method. As a result, the control device 12 can control the handling device 10 to hold the object as much as possible using the currently selected holding method H0 when the object is relatively easy to hold using the currently selected holding method H0.

[0077] The determination unit 306 calculates a second score T H,H0 The necessity of switching the retention method may be determined based on the magnitude relationship between the average value of (I) and a threshold Th. H,H0 The necessity of switching the holding method may be determined based on the magnitude relationship between each of (I) and the threshold value Th. For example, in the above example, the determination unit 306 may determine the necessity of switching the holding method based on the second score T H,H0 When at least one of (I) is smaller than the threshold value Th, it can be determined that the holding method needs to be switched.

[0078] Next, when it is determined that switching of the holding method is not necessary (step S304: No), the control device 12 executes step S305. In step S305, the holding strategy determination unit 307 of the control device 12 determines a holding strategy including an order of holding the objects O to be held. When the currently selected holding method H0 is "clamping", the holding strategy determination unit 307 determines the order of holding the objects O based on, for example, the first score S for "clamping". p In addition, when the currently selected holding method H0 is "adhesion", the holding strategy determination unit 307 may, for example, rank the holding order of the object O in order of the first score S for "adhesion". s In other words, the retention strategy determination unit 307 may select the first score S, for example, in descending order of the first score S without switching the currently selected retention method H0. H0 A holding strategy can be determined in which the object O is picked up by the currently selected holding method H0 in descending order of (I).

[0079] Next, the control device 12 executes step S307. In step S307, the control device 12 stores in the storage unit 302 the retention strategy including the retention order determined by the retention strategy determination unit 307 in step S305.

[0080] Next, the control device 12 executes step S308. In step S308, the retention strategy determination unit 307 of the control device 12 determines the next retention action. For example, the retention strategy determination unit 307 determines the earliest retention action in the retention strategy determined in step S305 (in the above example, the first score S H0 (Holding the object O with the largest I by the currently selected holding method H0) is set as the next holding operation. After that, the control device 12 proceeds to step S4.

[0081] On the other hand, if it is determined that switching of the holding method is necessary (step S304: Yes), the control device 12 executes step S307. In step S307, the holding strategy determination unit 307 determines a holding strategy including an order of holding and switching of the holding method. That is, the holding strategy determination unit 307 determines a holding strategy including an order in which holding operations and holding method switching operations are performed on multiple objects O to be held. As an example, the holding strategy determination unit 307 can determine the order of holding and switching of the holding method as follows. (1) The first score S according to the currently selected retention method H0 H0 (I) is the first score S by the retention method H1 after switching H1 For an object O or an area of ​​the object O that is larger than (I), a first score S H0 The holding operation is performed in order of increasing (I). (2) The holding method is switched from the currently selected holding method H0 to the target holding method H1. (3) The first score S according to the currently selected retention method H0 H0 (I) is the first score S by the retention method H1 after switching H1 For an object O or an area of ​​the object O that is smaller than (I), a first score S H1Holding operations can be performed in order of increasing (I).

[0082] This makes it possible to create a holding strategy that can pick up all objects O to be held by switching the holding method once, thereby minimizing the number of times the holding method is switched.

[0083] The method of determining the retention strategy is not limited to the above example. For example, the retention strategy determination unit 307 determines the second score T H1,H0 After performing the currently selected retention method H0 for the retention target whose (I) is equal to or greater than the threshold Th, the retention method is switched and the second score T H,H0 The retention strategy may be determined so that retention targets for which (I) is below the threshold Th are retained using the retention method H1 after switching. The retention strategy determination unit 307 may determine the retention strategy using any other method.

[0084] Next, the control device 12 executes step S307. In step S307, the control device 12 stores in the storage unit 302 the retention strategy including the order of retention and retention method switching determined by the retention strategy determination unit 307 in step S306.

[0085] Next, the control device 12 executes step S308. In step S308, the retention strategy determination unit 307 of the control device 12 determines the next retention action. For example, the retention strategy determination unit 307 sets the earliest retention action in the retention strategy determined in step S306 as the next retention action. For example, in the example shown in (1) to (3) above, the retention strategy determination unit 307 sets the first score S by the currently selected retention method H0 as the next retention action. H0 (I) is the first score S by the retention method H1 after switching H1 (I) is the first score S of an object O or a region of an object O that is larger than H0 The next holding operation can be set to hold the one with the largest (I) by the currently selected holding method H0. Note that the first score S by the currently selected holding method H0 is H0 (I) is the first score S by the retention method H1 after switchingH1 If there is no object O or area of ​​the object O larger than (I), the retention strategy determination unit 307 determines the first score S H1 Holding the object O or the area of ​​the object O for which (I) is maximum by the holding method H1 after switching can be set as the next holding operation. After that, the control device 12 proceeds to step S4.

[0086] In the steps up to this point, the control device 12 preferably determines the next holding operation without specifically calculating the position at which the object O is held or the attitude of the movable arm 100.

[0087] In the above example, the control device 12 has been described as determining the entire order of holding and then selecting the earliest holding action among them as the next action, but the determination of the order of holding may be omitted and only the next action may be determined. For example, the holding strategy determination unit 307 may determine the first score S without determining the order of holding in step S305. H The one with the largest (I) may be selected as the next action.

[0088] In determining the retention strategy in step S305 or step S306, the control device 12 uses the first score S H (I) or second score T H,H0 A corrected score obtained by correcting (I) based on the weight of object O, etc. may be used.

[0089] As mentioned above, the first score S is calculated by "holding" and "adhering". H Just as the evaluation items for calculating (I) and the assumed retention objects (object O or area of ​​object O, etc.) differ, the first score S H The calculation criteria of (I) may be different. Therefore, the control device 12 may perform normalization of the scores as appropriate so that the scores of different retention methods can be compared. The control device 12 may perform the normalization process of the scores at any step, such as step S301 for calculating the first score, step S302 for calculating the second score, and steps S305 and S306 for determining the retention strategy.

[0090] Details of step S3 have been described above. As an example, the second score T in the above formula 4 in a case where the selectable holding method is either "clamping" or "suction", the currently selected holding method H0 is "clamping", and the threshold value Th is set to 0.5 is calculated as follows: H,H0 Examples of calculations according to (I) are shown in Tables 1 to 3 below. Tables 1 to 3 are tables showing examples of score calculations. Tables 1 to 3 show, for holding targets 1 to 5 representing object O or a region of object O, a first score indicating the ease of holding with the holding method "clamping", a first score indicating the ease of holding with the holding method "adsorption", a second score calculated from these first scores, and a judgment result based on a threshold value Th. In the judgment result, "True" indicates a second score T H,H0 (I) is smaller than the threshold Th (i.e., the retention method needs to be changed). “False” means that the second score T H,H0 This means that (I) is greater than the threshold value Th (that is, there is no need to switch the holding method). [Table 1] [Table 2] [Table 3]

[0091] In the example shown in Table 1, the second score T H,H0 The average score of subjects 1 to 5 in (I) is 1.03, which is greater than 1. Therefore, on average, it can be said that the ease of holding with the currently selected holding method H0 (clamping) is greater than the ease of holding with the holding method (suction) of the evaluation target. In addition, the second scores T H,H0 Since (I) and the average value thereof are both greater than the threshold value Th=0.5, the determination unit 306 of the control device 12 determines that it is not necessary to switch the holding method.

[0092] Next, the retention strategy determination unit 307 determines the order of retention based on the first score SH0 The holding strategy is determined so that the currently selected holding method, "clamping", is used to hold in the order "target 3 → target 2 and target 4 → target 1 and target 5". Next, the control device 12 stores the determined holding strategy in the memory unit 302. Next, the holding strategy determination unit 307 determines that the next holding action is "holding target 3 by clamping", which is the earliest holding action in the holding strategy. Note that in the example of Table 1, the first score S of "clamping" for two or more targets is H0 When (I) has the same value, which one is to be prioritized in the retention strategy can be appropriately determined by taking into account any factor such as the position or weight of object O.

[0093] In the example shown in Table 2, the second score T H,H0 The average score of subjects 1 to 5 in (I) is 0.79, which is lower than 1. Therefore, it can be said that the ease of holding with the currently selected holding method H0 (clamping) is lower than the ease of holding with the holding method (suction) of the evaluation target on average. However, the second scores T H,H0 Since (I) and the average value thereof are both greater than the threshold value Th=0.5, the determination unit 306 of the control device 12 determines that it is not necessary to switch the holding method.

[0094] Next, as in the case of Table 1, the retention strategy determination unit 307 determines the order of retention based on the first score S H0 The holding strategy is determined so that the objects are held in the order of "object 5 → object 1 → object 2 → object 3 and object 4" by the currently selected holding method "clamping". Next, the control device 12 stores the determined holding strategy in the memory unit 302. Next, the holding strategy determination unit 307 determines that the next holding operation is "holding object 5 by clamping", which is the earliest holding operation in the holding strategy.

[0095] In the example shown in Table 3, the second score T H,H0The average value of subjects 1 to 5 in (I) is 0.38, which is less than 1, so it can be said that the ease of holding with the currently selected holding method H0 (clamping) is lower than the ease of holding with the holding method (suction) of the evaluation target on average. In addition, the second scores T H,H0 Since (I) and the average value thereof are both below the threshold value Th=0.5, the determination unit 306 of the control device 12 determines that it is necessary to switch the holding method.

[0096] Next, the holding strategy determination unit 307 determines a first score S for each of the targets 1 to 5 based on “holding” and “adsorption.” H (I) is compared. In this example, the first score S of "adsorption" is obtained for all subjects 1 to 5. H (I) is the first score S of the currently selected holding method "clamping" H0 Since the retention strategy determination unit 307 determines the retention strategy to first switch the retention method. Furthermore, the retention strategy determination unit 307 determines the order of retention after the switching of the retention method based on the first score S of the retention method after the switching, “adsorption”. H (I) is set in descending order, and a holding strategy is determined so that holding is performed in the order of "target 1 and target 5 → target 2 → target 3 → target 4." Next, the control device 12 stores the determined holding strategy in the memory unit 302. Next, the holding strategy determination unit 307 determines that the next holding operation is "holding target 1 by adhesion" or "holding target 5 by adhesion," which is the earliest holding operation in the holding strategy. Here, targets 1 and 5 have a first adhesion score S H Since (I) is equal, which one is to be given priority can be appropriately determined by taking into consideration any factor such as the positions and weights of objects 1 and 5.

[0097] 3 again, next, the control device 12 executes step S4. In step S4, the control device 12 specifically calculates the position for holding the target object O determined in step S3 and the posture of the movable arm 100 when holding the target object O (holding position and posture planning step).

[0098] If the method of holding object O selected in step S3 is “clamping”, in step S4, the operation control unit 303 of the control device 12 specifically calculates the position at which object O is to be clamped and the posture of the movable arm 100 when clamping, using a method appropriately selected from publicly known methods.

[0099] If the holding method for object O selected in step S3 is “suction”, in step S4, the operation control unit 303 of the control device 12 specifically calculates the position at which object O will be picked up and the posture of the movable arm 100 when picking up the object O, using a method appropriately selected from publicly known methods.

[0100] In this embodiment, the selectable holding methods are either "clamping" or "adsorption". However, if, for example, "clamping and adsorption (hybrid)" is available, in step S4, the operation control unit 303 of the control device 12 can specifically calculate the position for clamping and adsorbing the object O and the posture of the movable arm 100 when clamping and adsorbing, by a method appropriately selected from known methods.

[0101] After calculating the position to hold the object O and the 100 postures when holding it, the operation control unit 303 calculates whether the calculated holding operation is possible as an actual robot operation, and if it is determined that the actual operation is impossible, it again calculates the position to hold the object O and the 100 postures when holding it. Note that if it is determined that the actual operation is impossible, the operation may be repeated from the planning of the holding strategy back to step S3.

[0102] Calculating the position where object O is held and the orientation of the movable arm 100 requires a very large amount of calculations. The control device 12 needs to calculate the position where object O is held and the orientation of the movable arm 100 only for the selected object O. Therefore, the transport system 1 can significantly reduce the amount of calculations required, compared to other transport systems that calculate the position where object O is held and the orientation of the movable arm 100 in order to select object O to be held.

[0103] Next, the control device 12 executes step S5. In step S5, the determination unit 306 of the control device 12 determines whether or not it is necessary to switch the holding method. If the determination unit 306 determines that it is necessary to switch the holding method (step S5: Yes), that is, if the currently selected holding method is different from the holding method in the next holding operation, the control device 12 executes step S6. In step S6, the operation control unit 303 switches the holding method (holding method switching process). For example, when switching from "clamping" to "suction", the operation control unit 303 controls the movable arm 100 of the handling device 10 so as to orient the second holding unit 200B toward the source container V1 instead of the first holding unit 200A facing the source container V1. On the other hand, if the determination unit 306 determines that it is not necessary to switch the holding method (step S5: No), that is, if the currently selected holding method is the same as the holding method in the next holding operation, the control device 12 proceeds to step S7 without executing step S6.

[0104] Next, the control device 12 executes step S7. In step S7, the control device 12 controls the holder 200 and the movable arm 100 based on the position for holding the object O and the posture of the movable arm 100 calculated in step S4 (motion control step). The selected object O is transported by the handling device 10 from the source container V1 to the destination container V2.

[0105] Next, the control device 12 executes step S8. In step S8, the determination unit 306 of the control device 12 determines whether the order written in the order list has been completed. For example, the determination unit 306 determines whether the object O remains in the source container V1. If the determination unit 306 determines that the object O remains in the source container V1 (step S8: No), the control device 12 executes step S2 again. That is, the control device 12 again acquires information on the state of the source container V1 after picking up the object O, the state of each holding unit 200A, 200B, and the like, and creates a next holding strategy based on this. Note that the control device 12 may determine the next holding operation based on a holding strategy including the holding order that has already been created, without performing the holding strategy planning process. Also, the control device 12 may modify or update the already created holding strategy based on the acquired information, instead of performing the same holding strategy planning process. Furthermore, after performing one or more holding operations, the control device 12 may cause the score generating unit 304 to change the parameter value at the time of generating the score, or may cause the threshold generating unit 305 to change the threshold value Th.

[0106] On the other hand, if the determination unit 306 determines that the order written in the order list has been completed (step S8: Yes), the control device 12 executes step S9 and ends the control. For example, if the determination unit 306 determines that no object O remains in the source container V1, the control device 12 executes step S9 and ends the control.

[0107] According to the configuration of the first embodiment as described above, it is possible to efficiently determine a holding strategy for holding object O while suppressing the number of times of switching the holding method. The control device 12 of the transport system 1 selects a holding strategy without specifically calculating the position where object O is held or the posture of the movable arm 100. The holding strategy means, for example, selecting the type of handling device 10 to be used, the object O to be held, and the holding method. The control device 12 of the transport system 1 only needs to calculate the position where object O is held and the posture of the movable arm 100 for the selected object O, and the required amount of calculations can be significantly reduced. In addition, if holding is relatively easy even with the currently selected holding method, the holding strategy is created to preferentially use the currently selected holding method without switching the holding method, thereby saving the time required for switching the holding method, and therefore the overall work time can be shortened.

[0108] In the present embodiment, the plurality of holding methods include clamping and suction, which allows the handling device 10 to hold various objects O, such as a thin object O that is difficult to hold by clamping and an object O having an uneven shape that is difficult to hold by suction.

[0109] Furthermore, in this embodiment, the control device 12 selects the object O to be held and the holding method based on the score so as to reduce the number of times the holding method is switched. The control device 12 also determines whether or not the holding method needs to be switched based on the score. This allows the handling device 10 to execute a series of holding operations so as to minimize the number of times the holding method needs to be switched, thereby reducing the time required for the entire work.

[0110] In this embodiment, the control device 12 determines whether or not it is necessary to switch the retention method based on the magnitude relationship between the score and a predetermined threshold value Th. This allows the control device 12 to easily determine the necessity of switching the retention method. Furthermore, the control device 12 can create a retention strategy according to the situation by adjusting the threshold value Th.

[0111] Furthermore, in this embodiment, the control device 12 determines the order in which the object O is held based on the score. This allows the control device 12 to select the next holding operation in consideration of the entire pickup operation specified in the order list.

[0112] In this embodiment, the control device 12 determines the order of switching the holding method for the object, thereby enabling the control device 12 to select the optimal timing for switching the holding method in consideration of the entire pickup work specified in the order list.

[0113] Furthermore, in this embodiment, when calculating the score when the holding method is suction, the control device 12 calculates the score for each region of the object O. This allows the control device 12 to select a region of the same object O that is easiest to hold as the holding target region, thereby improving holding efficiency.

[0114] Furthermore, in this embodiment, the control device 12 calculates the ease of holding for each object O and for each holding method as a first score, and the score based on the selected holding method is a second score calculated based on the first score for each object O and for each holding method. This allows the control device 12 to determine the holding operation based on two perspectives: the ease of holding the object O represented by the first score, and the priority based on the currently selected holding method represented by the second score.

[0115] In this embodiment, the second score for the object O and the holding method is a value obtained by dividing the first score for the object O or the area of ​​the object O and the selected holding method by the first score for the object O or the area of ​​the object O and the holding method. This allows the control device 12 to easily calculate the second score for determining the need to switch the holding method.

[0116] Second embodiment Next, a second embodiment will be described with reference to Fig. 10. The second embodiment differs from the first embodiment in that a plurality of retention strategies are created. Note that the configuration other than that described below is the same as that of the first embodiment. FIG. 10 is a control flowchart of a retention strategy planning process by the control device 12 according to the second embodiment.

[0117] In the second embodiment, the control device 12 executes steps S316 to S318 instead of step S306 in the first embodiment. In the retention strategy planning step (step S3), the control device 12 calculates the first score S H (I) is calculated (step S311), and the second score T H,H0 (I) (step S312), and determines whether or not the holding method needs to be switched (step S313). If the determination unit 306 determines that the holding method does not need to be switched (S314: No), the holding strategy determination unit 307 of the control device 12 determines a holding strategy including the order of holding, as in the first embodiment (step S315), stores the holding strategy in the storage unit 302 (step S319), and determines the next holding operation based on the holding strategy (step S320). That is, the control contents of the control device 12 when the determination unit 306 determines that the holding method does not need to be switched are the same as in the first embodiment.

[0118] On the other hand, if the judgment unit 306 judges that the retention method needs to be switched (S314: Yes), the control device 12 executes step S316. In step S316, the retention strategy decision unit 307 of the control device 12 creates a plurality of retention strategy proposals including the order of retention and switching. For example, the retention strategy decision unit 307 creates a plurality of retention strategy proposals including the order of retention and switching. H Based on (I), a retention strategy proposal with one switch and a retention strategy proposal with two switches and a relative high overall success rate are created.

[0119] Next, the control device 12 executes step S317. In step S317, the score generation unit 304 of the control device 12 calculates the score of each retention strategy proposal created by the retention strategy determination unit 307. For example, the score generation unit 304 calculates the number of times the retention method is switched in the retention strategy proposal, the first score S for each retention action, H The score of each retention strategy proposal is calculated using the average, maximum, minimum, etc. of (I), the expected time required to complete the retention strategy, etc. as evaluation items. The score generation unit 304 can also change the weight of the evaluation items as appropriate in response to an input from the operator.

[0120] Next, the control device 12 executes step S317. In step S317, the retention strategy determination unit 307 of the control device 12 determines an optimal retention strategy based on the score of each retention strategy proposal generated by the score generation unit 304. For example, the retention strategy determination unit 307 selects the retention strategy proposal with the maximum score as the retention strategy.

[0121] Thereafter, the control device 12 stores the holding strategy determined by the holding strategy determination unit 307 in the storage unit 302 (step S319), and determines the next holding operation based on the holding strategy (step S320).

[0122] The control device 12 may create multiple retention strategy proposals based on the first score, omitting the calculation of the second score and / or the determination of the necessity of switching (steps S312 to S314). For example, the retention strategy determination unit 307 can create a retention strategy proposal that does not switch and has the lowest overall success rate, a retention strategy proposal that switches once and has the second lowest overall success rate, and a retention strategy proposal that switches twice and has the highest overall success rate. The score generation unit 304 calculates a score for each retention strategy proposal, and the retention strategy determination unit 307 determines a retention strategy based on the score. This allows the control device 12 to determine a retention strategy for reducing the number of times the retention method is switched, even without determining whether to switch the retention method.

[0123] According to the configuration of this embodiment, the control device 12 creates a plurality of holding strategies including at least one of the order of holding the object O and the order of switching the holding method, and selects one of the plurality of holding strategies. This allows the control device 12 to compare a wide range of holding strategies and select the holding strategy that is the best in terms of the success rate, the number of times the holding method is switched, and the like.

[0124] In this embodiment, the control device 12 selects one of the multiple holding strategies so as to minimize the number of times the holding method is switched, thereby enabling the handling device 10 to execute a series of holding operations so as to minimize the number of times the holding method is switched, thereby reducing the time required for the entire operation.

[0125] (Third embodiment) Next, a third embodiment will be described. The third embodiment is different from the first embodiment in that a holding method is selected based on the frequency of use of each holding method. Note that the configuration other than that described below is the same as that of the first embodiment.

[0126] The control device 12 records the frequency of use of each holding method in the storage unit 302. For example, the control device 12 can record the holding method used in the storage unit 302 every time a holding operation is performed, and calculate the frequency of use of each holding method based on the history of holding method use. Alternatively, the control device 12 can record in the storage unit 302 that the holding method has been switched, and calculate the frequency of use of each holding method based on the history of holding method switching. This allows the control device 12 to confirm how frequently each holding method has been used in the past.

[0127] When the calculated frequency of use of each holding method is biased or when it is determined that the holding method is approaching its limit of use, the control device 12 can reduce the frequency of use of a holding method that is frequently used. For example, the control device 12 may reduce the frequency of use of a holding method that is frequently used when the second score T H,H0 When calculating (I), a second score T is added to reduce the priority of the frequently used retention method.H,H0 Alternatively, the control device 12 may adjust the parameters in (I) by, for example, adjusting the second score T H,H0 When comparing (I) with the threshold value Th, the threshold value generating unit 305 can be caused to adjust the threshold value Th so as to lower the priority of selecting a frequently used holding method. Alternatively, for example, when determining a holding strategy including the order of holding and switching in step S306 of FIG. 9, the control device 12 can adjust the first score S by another holding method so as to avoid using a frequently used holding method as much as possible. H For objects O with a relatively high (I), the control device 12 can be configured to hold the object by the other holding method. Alternatively, when calculating the score of each holding strategy proposal in step S317 of Fig. 10, the control device 12 can adjust a parameter in the score so as to lower the priority of selection of a holding method that is frequently used.

[0128] For example, the control device 12 can calculate the scores of clamping and suction by the following Equation 6.

number

[0129] In formula 6, w1h1+w2h2+w3=1, and w i ≧0(i=1,2,3), h j ≧0(j=1,2). S p (I) is the first score when the object O or the mask area I indicating the area of ​​the object O is held by "pinching". S p (I) is the first score when object O or mask area I indicating the area of ​​object O is held by "adsorption". g(S H (I),S H0 (I) is a function for calculating the second score. itemis a value normalized by multiplying the weight of the object O by a coefficient. h1 and h2 are weights that adjust the degree of use of the current holding method. For example, the weights of h1 and h2 can be stored in a database or the like as table information as shown below. In addition, the control device 12 calculates the first score S p (I) or S s The weights may be adjusted based on the results of calculation (I). [Table 4]

[0130] According to the configuration of the third embodiment as described above, the control device 12 selects the object O to be held and the holding method based on the frequency of use of each holding method. This allows the control device 12 to suppress bias towards a specific holding method by making an adjustment such as lowering the priority of a holding method that is used frequently, thereby extending the life of the holding unit 200.

[0131] Furthermore, in this embodiment, the control device 12 calculates a score based on the frequency of use of each holding method. Alternatively, the control device 12 determines a threshold value Th based on the frequency of use of each holding method, and judges whether or not it is necessary to switch the holding method based on the magnitude relationship between the score and the threshold value Th. This allows the control device 12 to easily make adjustments such as lowering the priority of a holding method that is frequently used by correcting the score or threshold value based on the frequency of use of each holding method.

[0132] (Fourth embodiment) Next, a fourth embodiment will be described. The fourth embodiment is different from the first embodiment in that a holding method is selected based on the detection result of the physical state of each of the holding parts 200A and 200B. Note that the configuration other than that described below is the same as that of the first embodiment.

[0133] The control device 12 receives information on the physical state of the first holding unit 200A and the second holding unit 200B from the third sensor 11C and the fourth sensor 11D. When a particular holding unit 200 is used repeatedly, the measured values ​​of the physical sensors of the third sensor 11C and the fourth sensor 11D may change due to distortion, shape change, change in surface condition, etc. of the holding unit 200. The control device 12 estimates the usage state, such as the wear state, of the holding units 200A and 200B based on such information on the physical state. As for the holding method using the holding units 200A and 200B estimated to be deteriorated or damaged, the control device 12 adjusts the score, threshold, order of holding and switching in the holding strategy, etc., so as to lower the priority or frequency of selection of the holding method, as in the third embodiment.

[0134] According to the configuration of the fourth embodiment as described above, the control device 12 selects the object O to be held and the holding method based on physical information of the holding unit 200. This makes it possible to adjust the priority of the holding method while actually measuring the physical deterioration of the holding unit 200, thereby making it possible to extend the life of the holding unit 200.

[0135] Fifth embodiment Next, a fifth embodiment will be described. The fifth embodiment is different from the first embodiment in that an algorithm using machine learning is used instead of a rule-based algorithm. Note that the configuration other than that described below is the same as that of the first embodiment.

[0136] In this embodiment, the calculation of the first score and the second score in the first embodiment is performed by machine learning. For example, when supervised learning is assumed, the evaluation function defined above can be used as an evaluation value during learning. The learning algorithm is not limited to supervised learning, and can be changed depending on the type of learning, such as unsupervised learning and reinforcement learning.

[0137] The control device 12 can learn to simultaneously evaluate the success rate of the holding action, the number of times the holding method is switched, and the frequency of use of the holding method, and adjust the balance between them by configuring a learning network based on the information acquired from the sensor 11, the information acquired by the first score, the information acquired by the second score, and the information required for the number of times the holding method is switched, by configuring a learning network based on the information acquired from the sensor 11, and can output a score for each holding object and each holding method, and one or more holding strategies including the order of holding and switching, suitable for increasing the success rate of the holding action, suppressing the switching of the holding method, and suppressing bias in the frequency of use of the holding method.

[0138] When using machine learning, in the evaluation function of the first score of Equation 2 and Equation 3, w p1 =w p2 =w p3 =w p4 =w p5 = 0, and w s1 =w s2 =w s3 =w s4 =w s5 =0.

[0139] According to the configuration of the fifth embodiment as described above, the control device 12 can efficiently output more appropriate scores and retention strategies by repeatedly performing machine learning.

[0140] In each of the above embodiments, the control device 12 calculates the second score T H,H0 The order of the holding action and the switching action is determined after determining the necessity of switching by comparing (I) with the threshold value Th, but the order of the holding action and the switching action may be determined so as to reduce the number of switching actions without determining the necessity of switching. For example, the score generating unit 304 of the control device 12 may generate a first score S which indicates the ease of holding without considering the currently selected holding method. H For (I), the first score S according to the currently selected retention method H0 H0Correct the value of (I) only (for example, by multiplying it by n (n>1)), and use the first score S H (I) is the second score T H,H0 (I) is generated, and this second score T H,H0 Based on (I), a holding strategy including the order of holding operations and switching operations may be determined so as to minimize the number of switching operations. In this case, the control device 12 does not need to determine the necessity of switching the holding method. H The second score T including the correction method of (I) H,H0 The method of generating (I) is not limited to the above example, and any method can be used as long as it has some bias to preferentially use the currently selected retention method.

[0141] In each of the above embodiments, the handling device 10 has a first handling device 10A that performs "clamping" and a second handling device 10B that performs "adsorption", but the configuration of the handling device 10 is not limited to this, and may have one or more holding units that perform any holding method as long as it can select one or more of a plurality of holding methods to hold an object. For example, the handling device 10 may further have another handling device in addition to the first handling device 10A and the second handling device 10B. The handling device 10 may have one hybrid hand that can perform both "clamping" and "adsorption" instead of the first handling device 10A and the second handling device 10B. In addition, at least one of the first handling device 10A and the second handling device 10B may be a hybrid hand that can perform both "clamping" and "adsorption". Such a hybrid hand may be configured to switch between a clamping unit and an adsorption unit by a rotating unit that rotates 180°, may be configured as a revolver like an optical microscope, or may have other configurations. The handling apparatus 10 may have two types of handling devices for suction that differ in size, shape, characteristics, etc. Alternatively, the handling apparatus 10 may have a handling device that holds the object O by a holding method other than clamping and suction.

[0142] According to at least one embodiment described above, the operating time of the handling device can be reduced by creating a holding strategy that prioritizes the use of the currently selected holding method based on a score based on the currently selected holding method.

[0143] Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These embodiments can be implemented in various other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These embodiments and their modifications are included in the scope of the invention and its equivalents described in the claims, as well as in the scope and spirit of the invention. [Explanation of symbols]

[0144] 1...transport system, 10...handling device (handling system), 10A...first handling device, 10B...second handling device, 11...sensor, 11A...first sensor, 11B...second sensor, 11C...third sensor, 11D...fourth sensor, 11E...fifth sensor, 12...control device (control unit), 100...arm, 101...arm member, 102...rotating unit, 200...holding unit, 200A...first holding unit, 200B...second holding unit, 202...clamping hand, 203...suction device, 205...suction unit, 300...input unit, 301...recognition processing unit, 302...memory unit, 303...operation control unit, 304...score generation unit, 305...threshold generation unit, 306...judgment unit, 307...holding strategy determination unit.

Claims

1. A movable arm; a holding section attached to the movable arm and capable of holding an object by selecting one or more of a plurality of holding methods; A sensor capable of detecting a plurality of the objects; A control unit that controls the movable arm and the holding unit; Equipped with The control unit is calculating, for each of the objects, a first score indicating the ease of holding of each holding method corresponding to the object based on the information acquired from the sensor, and using the first score, calculating, for each of the objects, a second score based on the selected holding method and the holding method to be evaluated, the second score being calculated with the first score for the selected holding method as a standard by an evaluation function that depends on the first score for the selected holding method and the first score for the holding method to be evaluated; using the second score to select a next retention method; controlling the movable arm to hold one of the objects in the selected holding manner; Handling system.

2. The plurality of holding methods include clamping and suction. Handling system according to claim 1 .

3. the control unit selects the object to be held next and the holding method so as to reduce the number of times the holding method is switched. Handling system according to claim 1 or 2.

4. The control unit determines whether or not it is necessary to switch the holding method based on the second score. A handling system according to any one of claims 1 to 3.

5. the control unit determines whether or not it is necessary to switch the storage method based on a magnitude relationship between the second score and a predetermined threshold value. Handling system according to claim 4.

6. The control unit determines an order in which the objects are held based on the first scores. A handling system according to any one of claims 1 to 5.

7. The control unit further determines an order of switching the holding method of the object.

7. A handling system according to claim 6.

8. the control unit creates a plurality of holding strategies including at least one of an order of holding the object and an order of switching the holding method, and selects one of the plurality of holding strategies.

8. A handling system according to claim 6 or 7.

9. the control unit selects one of the plurality of retention strategies so as to reduce the number of times the retention method is switched.

9. A handling system according to claim 8.

10. The control unit selects the object and the holding method to be held next based on the frequency of use of each holding method. Handling system according to any one of claims 1 to 9.

11. The control unit calculates the second score based on a frequency of use of each holding method. Handling system according to claim 10.

12. the control unit determines a threshold value based on a frequency of use of each storage method, and determines whether or not it is necessary to switch the storage method based on a magnitude relationship between the second score and the threshold value. Handling system according to claim 10.

13. the control unit selects the object to be held next and a holding method based on the first score, the second score, and physical information of the holding unit. Handling system according to any one of claims 1 to 12.

14. When the control unit calculates the first score or the second score in a case where the holding method is suction, the control unit calculates the first score or the second score for each region of the object. Handling system according to any one of claims 1 to 13.

15. the second score for the object and holding method is the first score for the object and the selected holding method divided by the first score for the object and the holding method. Handling system according to claim 14.

16. A movable arm; a holding section attached to the movable arm and capable of holding an object by selecting one or more of a plurality of holding methods; A sensor capable of detecting a plurality of the objects; A control unit that controls the movable arm and the holding unit; Equipped with The control unit is calculating, for each of the objects, a first score indicating the ease of holding of each holding method corresponding to the object based on the information acquired from the sensor, and using the first score, calculating, for each of the objects, a second score based on the selected holding method and the holding method to be evaluated, the second score being calculated with the first score for the selected holding method as a standard by an evaluation function that depends on the first score for the selected holding method and the first score for the holding method to be evaluated; using the second score to select a next retention method; controlling the movable arm to hold one of the objects in the selected holding manner and moving the object from a first position to a second position; Conveying system.

17. A movable arm; a holding section attached to the movable arm and capable of holding an object by selecting one or more of a plurality of holding methods; A sensor capable of detecting a plurality of the objects; A control device for controlling a conveyance system comprising: calculating, for each of the objects, a first score indicating the ease of holding of each holding method corresponding to the object based on the information acquired from the sensor, and using the first score, calculating, for each of the objects, a second score based on the selected holding method and the holding method to be evaluated, the second score being calculated with the first score for the selected holding method as a standard by an evaluation function that depends on the first score for the selected holding method and the first score for the holding method to be evaluated; using the second score to select a next retention method; controlling the movable arm to hold one of the objects in the selected holding manner; Control device.

18. A movable arm; a holding section attached to the movable arm and capable of holding an object by selecting one or more of a plurality of holding methods; A sensor capable of detecting a plurality of the objects; A control unit that controls the movable arm and the holding unit; A control program for controlling a conveyance system comprising: The control unit, a first score indicating the ease of holding of each holding method corresponding to the object is calculated for each object based on the information acquired from the sensor, and a second score based on the selected holding method and the holding method to be evaluated is calculated for each object using the first score, the second score being calculated based on the first score for the selected holding method and an evaluation function that depends on the first score for the selected holding method and the first score for the holding method to be evaluated, with the first score for the selected holding method as a reference; selecting a next retention method using the second score; controlling the movable arm to hold any of the objects in the selected holding manner; Control program.

19. A movable arm; a holding section attached to the movable arm and capable of holding an object by selecting one or more of a plurality of holding methods; A sensor capable of detecting a plurality of the objects; A control unit that controls the movable arm and the holding unit; A handling method for handling an article by a conveyance system comprising: The control unit is calculating, for each of the objects, a first score indicating the ease of holding of each holding method corresponding to the object based on the information acquired from the sensor, and using the first score, calculating, for each of the objects, a second score based on the selected holding method and the holding method to be evaluated, the second score being calculated with the first score for the selected holding method as a standard by an evaluation function that depends on the first score for the selected holding method and the first score for the holding method to be evaluated; using the second score to select a next retention method; controlling the movable arm to hold one of the objects in the selected holding manner; Handling method.

20. A movable arm; a holding section attached to the movable arm and capable of holding an object to be held by selecting one or more of a plurality of holding mechanisms; A sensor capable of detecting the held object; a control unit that controls the movable arm and the holding unit, The control unit is a first score indicating the ease of holding of each holding mechanism corresponding to each holding object is calculated for each holding object based on the information acquired from the sensor, and a second score based on a selected holding mechanism and a holding mechanism to be evaluated is calculated for each holding object using the first score, and the second score is calculated based on the first score for the selected holding mechanism by an evaluation function that depends on the first score for the selected holding mechanism and the first score for the holding mechanism to be evaluated, selecting a next retention mechanism using the second score; controlling the movable arm so that the selected holding mechanism holds any one of the holding objects; Handling system.

Images