Loading and unloading system, conveying system, control device, storage medium and loading and unloading method

Abstract

The object is to provide a loading and unloading system, a conveying system, a control device, a control program and a loading and unloading method that can shorten the operation time of the loading and unloading device. The loading and unloading system of the technical solution has a movable arm, a holding part, a sensor and a control part. The holding part is installed on the movable arm and can select one or more of a plurality of holding methods to hold an object. The sensor can detect a plurality of the objects. The control part controls the movable arm and the holding part. Based on the information obtained from the sensor, the control part calculates a score based on the selected holding method for each of the objects and for each holding method. Based on the score, the control part selects the next object and holding method to be held. The control part calculates the position of the selected object and the posture of the movable arm.

Description

Technical Field

[0001] The invention relates to a loading and unloading system, a conveying system, a control device, a storage medium and a loading and unloading method. Background Art

[0002] Conventional loading and unloading devices that use end effectors to hold objects are known. Automating transfer operations in logistics sites requires the ability to hold objects of various shapes, sizes, and weights. Using loading and unloading devices to hold these objects requires significant computational effort to determine the holding strategy, including the holding position, holding method, and robot arm posture. The more complex the object's loading state, the longer the computational time required to determine the holding strategy.

[0003] Furthermore, when performing work by selectively using a plurality of holding tools, if the number of times the holding method is switched is too large, the time required for the entire work may be increased by the amount of tool replacement operations.

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent No. 5130509

[0007] Patent Document 2: Japanese Patent Application Publication No. 2019-188516 Summary of the Invention

[0008] Problems to be solved by the invention

[0009] An object of the present invention is to provide a loading and unloading system, a conveying system, a control device, a control program and a loading and unloading method that can shorten the operation time of a loading and unloading device.

[0010] Means used to solve problems

[0011] The loading and unloading system of the technical solution includes a movable arm, a holding unit, a sensor, and a control unit. The holding unit is installed on the movable arm and can select one or more holding methods to hold an object. The sensor can detect multiple objects. The control unit controls the movable arm and the holding unit. Based on the information obtained from the sensor, the control unit calculates a score based on the selected holding method for each object and each holding method. Based on the score, the control unit selects the next object and holding method to be held. The control unit calculates the position of the selected object and the posture of the movable arm.

[0012] The loading and unloading system of another technical solution comprises a movable arm, a holding part, a sensor and a control part. The holding part is mounted on the movable arm and is capable of selecting one or more holding mechanisms to hold the object to be held. The sensor is capable of detecting the object to be held. The control part controls the movable arm and the holding part. Based on the information obtained from the sensor, the control part calculates a score based on the selected holding mechanism according to any one of the held objects and any one of the holding mechanisms. Based on the score, the control part selects the object to be held and the holding mechanism that holds the object to be held after the selected holding mechanism. The control part calculates the position of the object to be held by the holding mechanism that holds the object to be held after the selected holding mechanism and the posture of the movable arm. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 It is a perspective view schematically showing a conveying system including the loading and unloading system according to the first embodiment.

[0014] Figure 2 This is a block diagram showing the system configuration of a transportation system including the loading and unloading system according to the first embodiment.

[0015] Figure 3 This is a control flowchart of the control device according to the first embodiment.

[0016] Figure 4 1 is a diagram showing a temporary masked area in the image data of the object when the object O is held by “clamping”.

[0017] Figure 5 is a diagram showing a masked area in image data of an object.

[0018] Figure 6 It is a graph representing the depth image of an object.

[0019] Figure 7 A diagram that illustrates the three-dimensional position and posture of an object.

[0020] Figure 8 This is a diagram showing a mask area to which information on three-dimensional position and orientation is added.

[0021] Figure 9 This is a control flowchart of the maintenance strategy planning process performed by the control device according to the first embodiment.

[0022] Figure 10 This is a control flowchart of a maintenance strategy planning process performed by the control device according to the second embodiment.

[0023] Label Description

[0024] 1…conveying system; 10…loading and unloading device (loading and unloading system); 10A…first loading and unloading device; 10B…second loading and unloading device; 11…sensor; 11A…first sensor; 11B…second sensor; 11C…third sensor; 11D…fourth sensor; 11E…fifth sensor; 12…control device (control unit); 100…arm; 101…arm component; 102…rotating part; 200…holding part; 200A…first holding part; 200B…second holding part; 202…gripping hand; 203…suction device; 205…adsorption part; 300…input part; 301…recognition processing part; 302…storage part; 303…action control part; 304…score generating part; 305…threshold generating part; 306…judgment part; 307…holding strategy determining part. DETAILED DESCRIPTION

[0025] Hereinafter, a loading and unloading system, a conveying system, a control device, a control program, and a loading and unloading method according to an embodiment will be described with reference to the drawings.

[0026] In addition, in the following description, the same reference numerals are given to structures having the same or similar functions. In addition, there are cases where repeated descriptions of these structures are omitted. In addition, "based on XX" recorded in this application means "at least based on XX", and also includes cases based on other elements in addition to XX. In addition, "based on XX" is not limited to the case of directly using XX, but also includes the case of being based on XX after calculation or processing. "XX" is an arbitrary element (such as arbitrary information).

[0027] (First embodiment)

[0028] Reference Figures 1 to 9 , an implementation method is described. Figure 1 It is a perspective view schematically showing a transport system 1 including a loading and unloading device 10 (an example of a “loading and unloading system”) according to the present embodiment.

[0029] Conveyor system 1 is, for example, a loading and unloading system (picking system) for logistics. Conveyor system 1 moves objects (held objects, conveyed objects) O located at a starting point V1 to a destination V2. For example, conveyor system 1 retrieves a specified number of various objects O stored at starting point V1 and loads them onto destination V2.

[0030] The movement starting point V1 can be, for example, various conveyors, various pallets, or containers such as bags or folding boxes. "Container" generally refers to any component (e.g., a box-shaped component) capable of containing an object O. However, the movement starting point V1 is not limited to the above examples. In the following description, the "movement starting point V1" may be referred to as the "removal source container V1."

[0031] At the movement starting point V1, a variety of objects O of varying sizes and weights are randomly placed. For example, the objects O to be held may have a concave-convex shape on at least a portion of their surface. In this embodiment, the outer shapes of the objects O range from small ones, such as 5 cm on a side, to large ones, such as 30 cm on a side. Furthermore, the objects O range from lightweight ones, such as tens of grams, to heavy ones, such as several kilograms. However, the size and weight of the objects O are not limited to the above examples.

[0032] The moving target V2 is, for example, a container such as a bag or a folding box. However, the moving target V2 is not limited to the above examples. In the following description, the "moving target V2" may be referred to as the "transportation target container V2," and the "movement starting point V1" and "moving target V2" may be collectively referred to as simply the "container." Furthermore, the conveying system 1 may also move the object O to a moving target V2 other than a container.

[0033] The conveying system 1 is not limited to a loading and unloading system for logistics. The conveying system 1 can also be widely applied to industrial robot systems and other systems. The "conveying system," "loading and unloading system," and "loading and unloading device" described in this application are not limited to systems and devices whose primary purpose is to transport objects. They also include systems that transport (move) objects as part of product assembly or other purposes.

[0034] Conveying system 1 Figure 1 As shown, the loader 10 includes a loading and unloading device 10 , a sensor 11 , and a control device 12 (an example of a “control unit”). The control device 12 can be assembled to the loading and unloading device 10 .

[0035] The handling device 10 is, for example, a robot. It holds an object O in a removal source container V1 and moves the held object O to a transfer destination container V2 (storage area). The handling device 10 can communicate with a control device 12 via wired or wireless communication. In this embodiment, the handling device 10 includes a first handling device 10A and a second handling device 10B.

[0036] The first loading and unloading device 10A includes, for example, a movable arm 100 and a first holding portion 200A provided at the front end of the movable arm 100 .

[0037] The movable arm 100 is a moving mechanism that moves the first holding portion 200A to a desired position. For example, the movable arm 100 is a six-axis vertical multi-jointed robotic arm. The movable arm 100 can assume a variety of positions and postures. Similar to a human arm or hand, the movable arm 100 can also assume a variety of postures for holding an object. The movable arm 100 includes, for example, a plurality of arm components 101 and a plurality of rotating portions 102 that rotatably connect the plurality of arm components 101.

[0038] The movable arm 100 may also be a three-axis orthogonal robot arm. Alternatively, the movable arm 100 may utilize other mechanisms to move the first holding portion 200A to a desired position. For example, the movable arm 100 may be an aerial vehicle (e.g., a drone) that uses rotating wings to lift and move the first holding portion 200A.

[0039] The first holding unit 200A is a holding mechanism (end effector) that holds the object O located in the removal source container V1 . The first holding unit 200A includes a gripping hand 202 .

[0040] The gripping hand 202 is a clamp-type hand that grips the object O with two fingers and is provided at the front end of the movable arm 100. The structure of the gripping hand 202 is not limited thereto, and for example, a clamp-type hand that grips the object O with three fingers may be used.

[0041] Alternatively, the first holding portion 200A may be a hybrid hand that further includes a suction device and a suction unit connected to the suction device, and holds the object O by gripping and / or suction. In this case, the suction unit may be provided at the tips of the fingers of the gripping hand 202. Multiple suction units may be provided at the tips of the fingers of the gripping hand 202.

[0042] The second handling device 10B includes, for example, an arm (second arm) 100 and a second holding portion 200B provided at the distal end of the movable arm 100. The movable arm 100 of the second handling device 10B has the same structure as the movable arm 100 of the first handling device 10A.

[0043] The second holding unit 200B is a holding mechanism (end effector) that holds the object O located in the extraction source container V1. For example, the second holding unit 200B includes a suction device 203 and a suction unit 205 connected to the suction device 203. The second holding unit 200B is a suction-type hand that holds the object O by suction.

[0044] The second holding portion 200B may also be a mechanism that holds the object O using other holding methods. For example, the second holding portion 200B may be a holding portion that can use magnetic force to hold the object O. For example, the second holding portion 200B may be a holding portion that is composed of a soft film filled with powder and a vacuum pump that extracts air from the soft film, and can hold the object O using a jamming phenomenon (e.g., a jamming clamp).

[0045] The suction device 203 is, for example, a vacuum pump and is connected to each of the plurality of suction units 205 via a hose or the like. By driving the suction device 203 , the pressure in each suction unit 205 becomes lower than atmospheric pressure, and the object O is sucked and held by the suction unit 205 .

[0046] The suction unit 205 is provided at the front end of the second holding unit 200B. For example, multiple suction units 205 may be provided at the front end of the second holding unit 200B. The suction units 205 have an outer shape smaller than the smallest object O in the removal source container V1. The second handling device 10B suction-holds the object O using only one or more suction units 205 selected from the plurality of suction units 205.

[0047] In the following description, the "first holding part 200A" and the "second holding part 200B" are collectively referred to as the "holding part 200". That is, it is assumed that the "holding part 200" includes the "first holding part 200A" and the "second holding part 200B". In addition, although the first holding part 200A is described as a clamping-type hand and the second holding part 200B is described as a suction-type hand, the structure of the holding part 200 is not limited to having one clamping-type hand, the first holding part 200A, and one suction-type hand, the second holding part 200B, respectively. In this embodiment, the case where both the first holding part 200A and the second holding part 200B are clamping-type hands or suction-type hands is also included. In this case, the holding part 200 may also be a structure having a plurality of clamping-type hands that are different in at least any one of the characteristics such as structure, construction, shape, size, and configuration. Specifically, for example, the first holding portion 200A and the second holding portion 200B may also be two or more clamping-type hands having different claw lengths or opening widths. In addition, the holding portion 200 may also be a structure having multiple suction-type hands that are different in at least one of the characteristics such as structure, construction, shape, size, and configuration. Specifically, for example, the first holding portion 200A and the second holding portion 200B may also be two or more suction-type hands that are different in the configuration of the suction pad, the diameter of the suction pad, or the structure of the bellows. In these cases, the present embodiment can also be implemented in the same manner to obtain the same effect.

[0048] The sensor 11 is controlled by the control device 12 to detect the status of multiple objects O and / or the holding unit 200. The sensor 11 includes a first sensor 11A, a second sensor 11B, a third sensor 11C, a fourth sensor 11D, and a fifth sensor 11E. The first sensor 11A, the second sensor 11B, the third sensor 11C, the fourth sensor 11D, and the fifth sensor 11E are connected to the control device 12 via wired or wireless connections.

[0049] The first sensor 11A is a lens or various sensors that are arranged near the movement starting point V1 (for example, directly above or diagonally above the movement starting point V1). The first sensor 11A obtains, for example, information about the object O located at the movement starting point V1 and information about the movement starting point V1. The information obtained by the first sensor 11A is, for example, "image data", "distance image data", "shape data", etc. "Distance image data" is image data having distance information in one or more directions (for example, depth information from an arbitrary reference surface set above the movement starting point V1). "Shape data" is information indicating the outer shape of the object O, etc. The information detected by the first sensor 11A is output to the control device 12. In addition, the first sensor 11A can also be provided as a part of the loading and unloading device 10.

[0050] The second sensor 11B is a lens or other type of sensor positioned near the target container V2 (e.g., directly above or diagonally above the target container V2). The second sensor 11B detects, for example, information regarding the shape of the target container V2 (including the shape of the inner wall and partitions) and information regarding an object O previously placed within the target container V2. Examples of information acquired by the second sensor 11B include image data, distance image data, and shape data. The information detected by the second sensor 11B is output to the control device 12. Alternatively, the second sensor 11B may be provided as part of the loading and unloading device 10.

[0051] The third sensor 11C is one of various sensors provided at or near the first holding portion 200A. The third sensor 11C obtains information regarding the physical state of the first holding portion 200A, such as deformation of the first holding portion 200A, pressure acting on the first holding portion 200A, and the surface condition of the first holding portion 200A. The third sensor 11C may include, for example, one or more physical sensors such as a strain sensor, a pressure sensor, and a proximity sensor. The third sensor 11C may also obtain physical information about the object O. The information detected by the third sensor 11C is output to the control device 12. Alternatively, the third sensor 11C may be provided as part of the loading and unloading device 10.

[0052] The fourth sensor 11D is one of various sensors provided on or near the second holding portion 200B. The fourth sensor 11D obtains information regarding the physical state of the second holding portion 200B, such as deformation of the second holding portion 200B, pressure acting on the second holding portion 200B, and the surface condition of the second holding portion 200B. The fourth sensor 11D may include, for example, one or more physical sensors such as a strain sensor, a pressure sensor, and a proximity sensor. The fourth sensor 11D may also obtain physical information about the object O. The information detected by the fourth sensor 11D is output to the control device 12. Alternatively, the fourth sensor 11D may be provided as part of the loading and unloading device 10.

[0053] The fifth sensor 11E obtains information about the usage status of the holding portion 200. For example, the fifth sensor 11E detects the holding portion 200 that is currently in use or selected for use among the first holding portion 200A and the second holding portion 200B (hereinafter, the holding portions that are currently in use or selected for use are sometimes collectively referred to as "currently selected holding portions" or simply "selected holding portions"). The information detected by the fifth sensor 11E is output to the control device 12. In addition, the fifth sensor 11E may be provided as a part of the loading and unloading device 10. In addition, the currently selected holding portion 200 may be determined not based on the fifth sensor 11E but based on other information such as the control history of the loading and unloading device 10 or the information obtained by the third sensor 11C or the fourth sensor 11D. In this case, the fifth sensor 11E may be omitted. In addition, in the case where the holding portion 200 is a mixed hand capable of performing clamping, adsorption, and both clamping and adsorption, the control device 12 determines whether the holding portion 200 is in a state suitable for clamping the object O, in a state suitable for adsorption, or in a state suitable for performing both based on information from the 5th sensor 11E or other information.

[0054] The control device 12 manages and controls the entire conveying system 1. For example, the control device 12 obtains information detected by the first sensor 11A to the fifth sensor 11E and controls the loading and unloading device 10 based on the obtained information. The control device 12 is, for example, a programmable device (computer) having a processor, memory, etc.

[0055] Figure 2 1 is a block diagram showing the system configuration of the transportation system 1 .

[0056] The control device 12 includes an input unit 300 , a recognition processing unit 301 , a storage unit 302 , an action control unit 303 , a score generation unit 304 , a threshold generation unit 305 , a determination unit 306 , and a retention strategy determination unit 307 .

[0057] The input unit 300 receives a command list regarding the object O to be held, information acquired by the first to fifth sensors 11A to 11E, and the like from an operator or the system.

[0058] The recognition processing unit 301 processes information acquired by the first to fifth sensors 11A to 11E. For example, the recognition processing unit 301 determines the position, posture, shape, characteristics, etc. of the object O at the movement starting point V1 based on image data acquired by the first sensor 11A.

[0059] The storage unit 302 stores the shape data of the object O to be picked, a list of commands received from the operator or the system, various scores generated by the score generation unit 304, thresholds generated by the threshold generation unit 305, determination results by the determination unit 306, the holding strategy determined by the holding strategy determination unit 307, and the control and operation history of the handling device 10. The shape data stored in the storage unit 302 is defined by the local coordinate system of the object O.

[0060] The motion control unit 303 controls the motion of the movable arm 100, the first holding unit 200A, the second holding unit 200B, and the like of the handling device 10. For example, the motion control unit 303 specifically calculates the appropriate position for holding the specific object O and the posture of the movable arm 100 during holding, in order to hold the specific object O using a specific holding method. Furthermore, the motion control unit 303 instructs the handling device 10 to cause the first holding unit 200A or the second holding unit 200B to perform the holding operation for the specific object O based on the calculated position for holding the specific object O and the posture of the movable arm 100 during holding.

[0061] The score generating unit 304 generates a score for determining the priority of the holding target object O and the holding method. For example, the score generating unit 304 calculates a first score and a second score described below.

[0062] The threshold value generating unit 305 generates a threshold value for determining whether or not the holding method needs to be switched. For example, the threshold value generating unit 305 generates a threshold value for a second score described later.

[0063] The determination unit 306 determines whether or not the holding method needs to be switched based on the score generated by the score generation unit 304 and the threshold generated by the threshold generation unit 305 .

[0064] The retention strategy determination unit 307 determines the retention strategy including the retention objects and retention methods with high priority, the order of retaining the retention objects, the retention actions and the order of switching actions of the retention methods based on the score generated by the score generation unit 304, the threshold generated by the threshold generation unit 305, the result of the determination by the determination unit 306, etc.

[0065] All or part of the various functions of the control device 12 are implemented by, for example, one or more processors such as a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit) executing a program stored in a program memory. However, all or part of these functions may also be implemented by hardware (e.g., circuitry) such as an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a PLD (Programmable Logic Device). In addition, all or part of the above functions may also be implemented by a combination of software and hardware. The storage unit 302 is implemented by a flash memory, an EEPROM (Electrically Erasable Programmable Read-Only Memory), a ROM (Read-Only Memory), or a RAM (Random Access Memory).

[0066] Next, the operation of the conveying system 1 will be described. Figure 3 The control flow chart of the control device 12 shown in FIG.

[0067] When the control device 12 is activated, the control device 12 initializes the loading and unloading device 10 and the sensor 11 and then starts controlling the loading and unloading device 10 (step S0 ). Next, the control device 12 executes step S1 .

[0068] In step S1 , the input unit 300 of the control device 12 receives a command list of objects O to be picked from an operator or the system (command list receiving step).

[0069] Next, the control device 12 executes step S2. In step S2, the input unit 300 of the control device 12 receives image data, distance image data, shape data, and other information regarding the object O in the extraction source container V1 from the first sensor 11A. Based on the data received by the input unit 300, the recognition processing unit 301 of the control device 12 determines whether an object O listed in the command list is present in the extraction source container V1. Furthermore, based on the data received by the input unit 300, the recognition processing unit 301 acquires information regarding the shape, position, and posture of the object O to be picked (information acquisition step).

[0070] Figure 4 A diagram showing a temporary masked region R1 in image data of an object O when the object O is held by “clamping”.

[0071] The recognition processing unit 301 of the control device 12 uses a known image segmentation method to set a rectangular area circumscribing the object O to be sorted (circumscribed rectangular area) as a "temporary mask area R1" based on image data. Image segmentation can be performed using a machine learning method.

[0072] Figure 5 3 is a diagram showing a masked region R2 in image data of an object O.

[0073] The recognition processing unit 301 sets the area that is expanded in both the vertical and horizontal directions of the temporary masking area R1 as "masking area R2." Masking area R2 extends the circumscribed rectangular area by a margin M in both the vertical and horizontal directions. For example, margin M is 100 mm. By using this expanded masking area R2, the recognition processing unit 301 can determine whether there is space around the object O where the gripping hand 202 can intrude, when the gripping method of the gripping unit 200 is gripping.

[0074] Figure 6 is a diagram showing a depth image D of an object O.

[0075] The recognition processing unit 301 uses the range image data to generate a depth image D that visualizes the depth information of the object O in the masked area R2. The depth image D has a value representing a height relative to the origin of the world coordinate system (X-axis, Y-axis, and Z-axis). The scale of the depth image D can be changed based on the storage method of the storage unit 200. For example, the scale of the depth image D can be set to 1 mm per pixel.

[0076] Figure 7 is a diagram illustrating the three-dimensional position and posture of object O. Figure 7 In the example, objects O with different shapes are denoted as "O1" and "O2".

[0077] The recognition processing unit 301 calculates the three-dimensional position and posture of the object O1 based on the image data of the object O1 obtained. The recognition processing unit 301 transforms the shape data of the local coordinate system of the object O1 recorded in the storage unit 302 into the world coordinate system (X axis, Y axis, Z axis) using the transformation matrix. The Z axis direction of the world coordinate system is as follows: Figure 7 The recognition processing unit 301 compares the acquired image data of the object O1 with the shape data converted into the world coordinate system to calculate the three-dimensional position and posture of the object O1.

[0078] Figure 8 This diagram shows the masked region R2, to which information regarding the three-dimensional position and posture is added. The recognition processing unit 301 sets the three-dimensional position pose, which serves as a reference for the object O1, as the center CO of the circumscribed rectangular region (the temporary masked region R1) within the masked region R2. Alternatively, the recognition processing unit 301 may calculate and use the center of gravity FO of the object O1 independently of the center CO of the circumscribed rectangular region, taking into account the gripping ease of the gripping hand 202.

[0079] On the other hand, when holding an object O by "adsorption," the pseudo-masked area R1 is segmented according to a masking plane on the surface of the object O that can serve as an adsorption target. The recognition processing unit 301 generates a masking area R2 and a depth image D according to the masking plane, defining the direction perpendicular to the masking plane as the normal direction. The recognition processing unit 301 can, for example, extract a plane region from the point cloud of the three-dimensional sensor and determine its position and posture by, for example, using principal component analysis, setting the short axis direction to x, the long axis direction to y, and the plane normal direction to z. In the case of "clamping," sometimes only one holding region is defined for one object O, but in the case of "adsorption," multiple holding regions can exist for one object O. Therefore, the holding target of "adsorption" can be a single region of the object O. Furthermore, in cases where, for example, multiple holding regions exist for "clamping," the holding target of "clamping" can also be a single region of the object O.

[0080] When the object O is held by “adsorption”, the recognition processing unit 301 can perform convolution processing to calculate how many of the plurality of adsorption units 205 to use and at what angle the adsorption unit 205 is brought into contact with the object O.

[0081] The recognition processing unit 301 can use a database that records the three-dimensional position and posture of the object O in previous cases where the object O was successfully picked up. The recognition processing unit 301 can also output a recommended holding method and holding position for the object O using the database.

[0082] The input unit 300 receives information about the physical state of the first holding unit 200A from the third sensor 11C and information about the physical state of the second holding unit 200B from the fourth sensor 11D, as needed. Furthermore, the input unit 300 receives information about the usage status of the holding unit 200, such as the currently selected holding unit, from the fifth sensor 11E, as needed. The recognition processing unit 301 determines which of the first holding unit 200A and the second holding unit 200B is the currently selected holding unit 200 based on the information from the fifth sensor 11E and / or other information. Information about the physical states of the first holding unit 200A and the second holding unit 200B and information about the currently selected holding unit 200 may also be obtained at other appropriate times other than step S2.

[0083] Furthermore, the input unit 300 receives physical information of the object O to be held from a database. For example, the input unit 300 receives information about the object O's shape, weight, surface material, friction characteristics, and the like from the database. The database may be defined for each individual object O, or may specify basic information primitives (e.g., a cuboid, cylinder, sphere, pyramid, etc.) and utilize information that approximates their shape by matching.

[0084] Next, the control device 12 executes step S3. In step S3, the control device 12 calculates a score for each object O or region of the object O and for each holding method, and determines the next object O to be held and the holding method (holding strategy planning step). Figure 9 This is a control flowchart of the holding strategy planning process performed by the control device 12 , and shows the details of step S3 .

[0085] In step S301, the score generating unit 304 of the control device 12 calculates the ease with which the holding unit 200 holds the object O as a "first score" based on the information obtained in step S2. The score generating unit 304 calculates the first score (1) for each object O or region of the object O (hereinafter, simply referred to as "for each object O") and (2) for each holding method. For example, the holding method is "clamping" by the first loading and unloading device 10A or "adsorption" by the second loading and unloading device 10B. When the holding method is "clamping", the first score is calculated for each object O, for example. When the holding method is "adsorption", the first score is calculated for each region of the object O, for example.

[0086] The first score S calculated in step S301 H (I) For example, it is calculated using the evaluation function shown in Formula 1.

[0087] [Formula 1]

[0088]

[0089] In Formula 1, H is the holding method (clamping by the first handling device 10A, suction by the second handling device 10B, etc.) that is the evaluation target of the first score. I is the masking area R2 that represents the object O or the area of the object O that is the evaluation target of the first score. i is the evaluation item of the evaluation function, w iis the weight of the evaluation function. That is, the above evaluation function is represented by a linear combination of the evaluation items. However, the evaluation function is not limited to the above example; average values or nonlinear functions may be used for the evaluation items; any evaluation function can be used. Furthermore, the evaluation items are not limited to those described in Equation 1; for example, they may be quantities dependent on the physical properties of the object O or the gripping hand 202.

[0090] When the holding method is "clamp" (assuming H = p), the first score S p (I) For example, it is calculated using the evaluation function shown in Formula 2.

[0091] [Formula 2]

[0092]

[0093] In equation 2, w p1 +w p2 +w p3 +w p4 +w p5 +w p6 =1,w pi ≥0 (i=1, 2, 3, 4, 5, 6). f p1 is the position of the object O (for example, the center position of the object O and the height of the object O). p2 is the concavity and convexity of object O. f p3 is the flatness of object O. f p4 It is the difference in depth between the object O and the surroundings of the object O. p5 Is the result of judging whether the object O is thin. p6 This is an evaluation item used when machine learning is used. p1 ~f p5 Therefore, for example, when using f p6 In the case of p1 ~w p5 All of them can be set to zero. For various masked areas I representing each object O, the first score S of "clamping" is calculated. p (I).

[0094] When the retention method is "adsorption" (assuming H = s), the first score S s (I) For example, it is calculated using the evaluation function shown in Formula 3.

[0095] [Formula 3]

[0096]

[0097] In equation 3, w s1 +w s2+w s3 +w s4 +w s5 +w s6 =1,w si ≥0 (i=1, 2, 3, 4, 5, 6). f s1 is the position of the region of the object O (for example, the center position of the region of the object O and the height of the object O). s2 is the concavity and convexity of the region of object O. s3 is the area of the region of object O. f s4 is the approach angle of the region to the object O (for example, the angle between the normal direction of the region of the object O and the vertical direction). s5 Is the result of judging whether the object O is thin. s6 This is an evaluation item used when machine learning is used. s1 ~f s5 Therefore, for example, when using f s6 In the case of w s1 ~w s5 All of them can be set to zero. For various masking areas I representing the area of the object O, the first score S of "adsorption" is calculated. s (I).

[0098] In this way, the score generating unit 304 calculates the first score S for each object O or region of the object O for each holding method (here, clamping by the first handling device 10A and suction by the second handling device 10B). H (I).

[0099] Next, the control device 12 executes step S302. In step S302, the score generating unit 304 of the control device 12 calculates a "second score" based on the currently selected holding method based on the information obtained in step S2 and the first score calculated in step S301. The score generating unit 304 calculates the second score (1) for each object O or region of object O and (2) for each holding method.

[0100] The second score T calculated in step S302 H，H0 (I) is calculated, for example, using the evaluation function shown in Formula 4.

[0101] [Formula 4]

[0102]

[0103] In Formula 4, H is the holding method to be evaluated for the second score. H0 is the currently selected holding method. I is the masked area R2 representing the object O or the area of the object O to be evaluated for the second score. In other words, the evaluation function of Formula 4 is the first score S of the currently selected holding method. H0 (I) The first score S of the maintenance method as the evaluation target of the second score H However, the evaluation function of the second score is not limited to the above example, and any function can be used, for example, the first score S that depends on the currently selected holding method H0 H0 (I) and the first score S of the holding method H as the evaluation target of the second score H (I) is an arbitrary function.

[0104] Since the first score S H (I) is an indicator of the “ease of maintaining” of the object’s masking area I by the object’s maintaining method H. Therefore, when the evaluation function of the above-mentioned formula 4 is used, the second score T H，H0 (I) is the ratio of the ease of maintenance based on the currently selected maintenance method H0 to the ease of maintenance based on the object maintenance method H. That is, when the second score of Formula 4 is greater than 1, the currently selected maintenance method H0 is easier to maintain than the object maintenance method H. When the second score of Formula 4 is less than 1, the object maintenance method H is easier to maintain than the currently selected maintenance method H0. For example, when the second score of Formula 4 is 0.5, it can be considered that the ease of maintenance based on the currently selected maintenance method H0 is 0.5 times the ease of maintenance based on the object maintenance method H. That is, it can be considered that the ease of maintenance based on the object maintenance method H is twice the ease of maintenance based on the currently selected maintenance method H0.

[0105] Next, the control device 12 executes step S303. In step S303, the determination unit 306 of the control device 12 determines the value of the second score T H，H0 (I) Determine the necessity of switching the holding method. H，H0 In the case of (I), for example, the determination unit 306 determines that it is necessary to switch the holding method from the currently selected holding method H0 to the object's holding method H when the conditional expression of the following formula 5 is satisfied, and determines that it is not necessary to switch the holding method from the currently selected holding method H0 to the object's holding method H when the conditional expression of the following formula 5 is not satisfied.

[0106] [Formula 5]

[0107]

[0108] In Formula 5, Th is a predetermined threshold value generated by the threshold value generating unit 305. That is, the determining unit 306 determines the second score T H，H0 (I) If the score is lower than the threshold, it is determined that the currently selected holding method H0 needs to be switched to the target holding method H. H，H0 If (I) is equal to or greater than the threshold, it is determined that there is no need to switch from the currently selected holding method H0 to the target holding method H.

[0109] Based only on the first score S H (I) When selecting a holding method, the information of the currently selected holding method H0 is not reflected in the first score S. H (I), so the holding method with a high holding success rate is simply selected. Therefore, depending on the situation, it may be determined that the holding method needs to be switched every time the object O is held. In such a case, switching the holding method may take time. On the other hand, the second score T H，H0 (I) contains information about the currently selected holding method H0, which is a score calculated based on the currently selected holding method H0. For example, when using the second score T of the above formula 4 H，H0 In the case of (I), even in the second score T H，H0 (I) is less than 1 (that is, when the holding ease based on the holding method H of the object is greater than the holding ease based on the currently selected holding method H0), the determination unit 306 sets the second score T H，H0 (I) Even when the value is not too small (i.e., it is greater than the predetermined threshold value Th), it is determined that there is no need to switch the holding method. Thus, the control device 12 can control the loading and unloading device 10 so that the currently selected holding method H0 is used as much as possible when the currently selected holding method H0 is easy to hold to a certain extent.

[0110] The determination unit 306 may determine the object O or the area of the object O to be held based on the second score T calculated for each masked area I. H，H0 The necessity of switching the holding method can be determined based on the magnitude relationship between the average value of (I) and the threshold Th. Alternatively, the necessity of switching the holding method can be determined based on the second score T for each mask area I. H，H0 (I) The necessity of switching the holding method is determined by the relationship between the size of each and the threshold value Th. For example, in the above example, the determination unit 306 can determine the necessity of switching the holding method based on the second score T for each mask area I. H，H0 When at least one of (I) is smaller than the threshold value Th, it is determined that the holding method needs to be switched.

[0111] Next, if it is determined that the holding method does not need to be switched (step S304: No), the control device 12 executes step S305. In step S305, the holding strategy determination unit 307 of the control device 12 determines a holding strategy including the order in which the holding is to be performed for the plurality of objects O to be held. If the currently selected holding method H0 is "clamping", the holding strategy determination unit 307 may, for example, set the order in which the objects O are held to the first score S for "clamping". p (I) Order from largest to smallest. In addition, when the currently selected holding method H0 is "adsorption", the holding strategy determination unit 307 may, for example, set the order of holding the object O to the first score S for "adsorption". s (I) Order from largest to smallest. That is, the holding strategy decision unit 307 can decide not to switch the currently selected holding method H0, but to use the first score S H0 (I) A holding strategy in which objects O are picked up in descending order by the currently selected holding method H0.

[0112] Next, the control device 12 executes step S307 . In step S307 , the control device 12 stores the retention policy including the retention order determined by the retention policy determination unit 307 in step S305 in the storage unit 302 .

[0113] Next, the control device 12 executes step S308. In step S308, the holding strategy decision unit 307 of the control device 12 decides the next holding action. For example, the holding strategy decision unit 307 determines the initial holding action (in the above example, the first score S is set to 0) in the holding strategy determined in step S305. H0 (I) The largest object O is held by the currently selected holding method H0 and is set as the next holding action. Then, the control device 12 proceeds to step S4.

[0114] On the other hand, if it is determined that a holding method switch is necessary (step S304: Yes), the control device 12 executes step S307. In step S307, the holding strategy determination unit 307 determines a holding strategy that includes the order of holding and switching holding methods. Specifically, the holding strategy determination unit 307 determines a holding strategy that includes the order in which holding operations and switching holding methods are performed on the plurality of objects O to be held. As an example, the holding strategy determination unit 307 may determine the order of holding and switching holding methods as follows.

[0115] (1) For the first score S based on the currently selected holding method H0 H0 (I) Compared to the first score S based on the holding method H1 after switching H1 (I) A large object O or an area of object O, with the first score SH0 (I) Perform the holding action in order from large to small.

[0116] (2) The holding method is switched from the currently selected holding method H0 to the holding method H1 as the switching target.

[0117] (3) For the first score S based on the currently selected holding method H0 H0 (I) Compared to the first score S based on the holding method H1 after switching H1 (I) A small object O or a region of object O can be scored with the first score S H1 (I) Perform the holding action in order from largest to smallest.

[0118] As a result, a holding strategy can be created that can pick up all the objects O to be held by switching the holding method once, and the number of times the holding method is switched can be suppressed to a minimum.

[0119] In addition, the method of determining the holding strategy is not limited to the above example. For example, the holding strategy determination unit 307 may also determine the holding strategy as follows: H1，H0 (I) After the holding object with a value greater than or equal to the threshold value Th is held using the currently selected holding method H0, the holding method is switched and the second score T is calculated. H，H0 (I) The retention targets whose values are lower than the threshold value Th are retained using the switched retention method H1. The retention policy determination unit 307 may determine the retention policy using any other method.

[0120] Next, the control device 12 executes step S307 . In step S307 , the control device 12 stores the retention policy including the order of retention and switching of retention methods determined by the retention policy determination unit 307 in step S306 in the storage unit 302 .

[0121] Next, the control device 12 executes step S308. In step S308, the holding strategy determination unit 307 of the control device 12 determines the next holding action. For example, the holding strategy determination unit 307 sets the initial holding action in the holding strategy determined in step S306 as the next holding action. For example, in the examples represented by (1) to (3) above, the holding strategy determination unit 307 can maintain the first score S based on the currently selected holding method H0 by the currently selected holding method H0. H0 (I) Compared to the first score S based on the holding method H1 after switching H1 (I) The first score S in the large object O or the area of the object O H0 (I) The largest one is set as the next holding action. In addition, if there is no first score S based on the currently selected holding method H0 H0(I) Compared to the first score S based on the holding method H1 after switching H1 (I) In the case of a large object O or a region of the object O, the holding strategy determination unit 307 can hold the first score S based on the switched holding method H1 using the switched holding method H1. H1 (I) The largest object O or the area of the object O is set as the next holding operation. Then, the control device 12 proceeds to step S4.

[0122] In the steps so far, the control device 12 preferably determines the next holding operation without specifically calculating the position for holding the object O and the posture of the movable arm 100 .

[0123] In the above example, the control device 12 selects the first holding action as the next action after determining the order of all holding actions. However, the determination of the order of holding actions may be omitted and only the next action may be determined. For example, in step S305, the holding strategy determination unit 307 may select the first score S instead of the first score S. H (I) The largest one is taken as the next action.

[0124] When the control device 12 determines the holding strategy in step S305 or step S306, it may also use the first score S H (I) or 2nd score T H，H0 (I) Corrected score corrected by the weight of object O, etc.

[0125] In addition, as described above, in the case of "clamping" and "adsorption", the first score S is calculated. H As the evaluation items of (I) and the assumed holding target (object O or area of object O, etc.) are different, the first score S is obtained according to the holding method. H The calculation basis for (I) may vary. Therefore, the control device 12 may appropriately normalize the scores so that even scores from different retention methods can be compared. The control device 12 may perform score normalization processing at any step, such as step S301 for calculating the first score, step S302 for calculating the second score, or steps S305 and S306 for determining the retention strategy.

[0126] The details of step S3 have been described above. Here, as an example, Tables 1 to 3 below show the second score T obtained by the above formula 4 when the selectable holding method is either "clamping" or "adsorption", the currently selected holding method H0 is "clamping", and the threshold Th is set to 0.5. H，H0(I) Calculation example. Tables 1 to 3 are tables showing an example of score calculation. Tables 1 to 3 show, for holding objects 1 to 5 representing an object O or an area of an object O, a first score indicating the ease of holding based on the holding method "clamping", a first score indicating the ease of holding based on the holding method "adsorption", a second score calculated based on these first scores, and a judgment result based on a threshold value Th. In the judgment result, "True" means that the second score T H，H0 (I) is smaller than the threshold value Th (that is, the method switching needs to be maintained), and "False" means the second score T H，H0 (I) is greater than the threshold value Th (ie, switching of the holding method is not necessary).

[0127] [Table 1]

[0128]

[0129] [Table 2]

[0130]

[0131] [Table 3]

[0132]

[0133] In the example shown in Table 1, since the second score T H，H0 The average value of objects 1 to 5 of (I) is 1.03, which is higher than 1. Therefore, it can be considered that the holding ease based on the currently selected holding method H0 (clamping) is higher than the holding ease based on the holding method (adsorption) to be evaluated. In addition, since the second scores T of objects 1 to 5 are H，H0 Since (I) and its average value are both higher than the threshold value Th=0.5, the determination unit 306 of the control device 12 determines that switching of the holding method is unnecessary.

[0134] Next, the retention strategy determination unit 307 sets the retention order to the first score S H0 (I) In descending order, the holding strategy is determined to be to hold the objects 3 → objects 2 and 4 → objects 1 and 5 in the order of "clamping" which is the currently selected holding method. Then, the control device 12 saves the determined holding strategy to the storage unit 302. Then, the holding strategy determination unit 307 determines "holding object 3 by clamping" which is the initial holding action in the holding strategy as the next holding action. In addition, in the example of Table 1, the first score S of "clamping" among two or more objects is H0 When (I) and (I) are the same value, which one is prioritized in the holding strategy can be appropriately determined in consideration of arbitrary factors such as the position and weight of the object O.

[0135] In the example shown in Table 2, since the second score T H，H0 The average value of objects 1 to 5 of (I) is 0.79, which is lower than 1. Therefore, it can be considered that the holding ease based on the currently selected holding method H0 (clamping) is lower than the holding ease based on the holding method (adsorption) to be evaluated. However, since the second scores T of objects 1 to 5 are H，H0 Since (I) and its average value are both higher than the threshold value Th=0.5, the determination unit 306 of the control device 12 determines that switching of the holding method is unnecessary.

[0136] Next, similarly to the case of Table 1, the retention strategy determination unit 307 sets the retention order to the first score S H0 (I) The holding strategy is determined to hold the currently selected holding method "clamping" in the order of "object 5 → object 1 → object 2 → objects 3 and 4" from largest to smallest. The control device 12 then stores the determined holding strategy in the storage unit 302. The holding strategy determination unit 307 then determines "holding object 5 by clamping," which is the initial holding action in the holding strategy, as the next holding action.

[0137] In the example shown in Table 3, since the second score T H，H0 The average value of objects 1 to 5 of (I) is 0.38, which is lower than 1. Therefore, it can be considered that the holding ease based on the currently selected holding method H0 (clamping) is lower than the holding ease based on the holding method (adsorption) to be evaluated. In addition, since the second scores T of objects 1 to 4 are H，H0 Since (I) and its average value are both lower than the threshold value Th=0.5, the determination unit 306 of the control device 12 determines that the holding method needs to be switched.

[0138] Next, the holding strategy determination unit 307 compares the first scores S for each of the objects 1 to 5 under the conditions of “clamping” and “adsorption”. H In this example, among all objects 1 to 5, the first score S of “Adsorption” is H (I) Higher than the first score S of "clamping" which is the currently selected holding method H0 (I), so the holding strategy decision unit 307 decides the holding strategy to switch the holding method first. Then, the holding strategy decision unit 307 sets the holding order after the holding method is switched to the first score S of the "adsorption" of the holding method after the switching. H(I) The holding strategy is determined to hold objects 1 and 5 in the order of "object 2" and "object 3" and "object 4" from the largest to the smallest. The control device 12 then stores the determined holding strategy in the storage unit 302. The holding strategy determination unit 307 then determines "holding object 1 by adsorption" or "holding object 5 by adsorption" as the next holding action, which is the initial holding action in the holding strategy. Here, the first score S of object 1 and object 5 due to adsorption is 1. H (I) are equal, so which one to prioritize can be appropriately determined by considering arbitrary factors such as the positions and weights of objects 1 and 5.

[0139] Refer again Figure 3 Next, the control device 12 executes step S4. In step S4, the control device 12 specifically calculates the position of the holding target object O determined in step S3 and the posture of the movable arm 100 during holding (holding position and posture planning step).

[0140] When the holding method of the object O selected in step S3 is "clamping", in step S4, the motion control unit 303 of the control device 12 specifically calculates the position of the clamped object O and the posture of the movable arm 100 during clamping by a method appropriately selected from known methods.

[0141] When the holding method of the object O selected in step S3 is "adsorption", in step S4, the motion control unit 303 of the control device 12 specifically calculates the position of the adsorbed object O and the posture of the movable arm 100 during adsorption by a method appropriately selected from known methods.

[0142] In addition, in this embodiment, the selectable holding method is either "clamping" or "adsorption", but for example, when "clamping and adsorption (mixed)" can be used, in step S4, the motion control unit 303 of the control device 12 can specifically calculate the position of the clamped and adsorbed object O and the posture of the movable arm 100 during clamping and adsorption by a method appropriately selected from known methods.

[0143] After calculating the position of the object O to be held and the posture of 100° during holding, the motion control unit 303 calculates whether the calculated holding motion can be used as an actual robot motion. If it is determined that the motion is actually impossible, the motion control unit 303 recalculates the position of the object O to be held and the posture of 100° during holding. If it is determined that the motion is actually impossible, the process may return to step S3 and start again from the planning of the holding strategy.

[0144] The amount of computation required to calculate the position of the held object O and the posture of the movable arm 100 is very large. The control device 12 only needs to calculate the position of the held object O and the posture of the movable arm 100 for the selected object O. Therefore, the conveying system 1 can significantly reduce the amount of computation required compared to other conveying systems that calculate the position of the held object O and the posture of the movable arm 100 in order to select the object O to be held.

[0145] Next, the control device 12 executes step S5. In step S5, the determination unit 306 of the control device 12 determines whether a holding method switch is necessary. If the determination unit 306 determines that a holding method switch is necessary (step S5: Yes), that is, if the currently selected holding method is different from the holding method in the next holding operation, the control device 12 executes step S6. In step S6, the motion control unit 303 switches the holding method (holding method switching process). For example, when switching from "gripping" to "adsorption," the motion control unit 303 controls the movable arm 100 of the handling device 10 to direct the second holding unit 200B toward the removal source container V1 instead of the first holding unit 200A toward the removal source container V1. On the other hand, if the determination unit 306 determines that a holding method switch is not necessary (step S5: No), that is, if the currently selected holding method is the same as the holding method in the next holding operation, the control device 12 skips step S6 and proceeds to step S7.

[0146] Next, the control device 12 executes step S7. In step S7, the control device 12 controls the holding unit 200 and the movable arm 100 based on the position of the held object O and the posture of the movable arm 100 calculated in step S4 (operation control step). The selected object O is transported from the removal source container V1 to the transfer destination container V2 by the loading and unloading device 10.

[0147] Next, the control device 12 executes step S8. In step S8, the determination unit 306 of the control device 12 determines whether the command recorded in the command list has been completed. For example, the determination unit 306 determines whether there is an object O remaining in the extraction source container V1. When the determination unit 306 determines that there is an object O remaining in the extraction source container V1 (step S8: No), the control device 12 executes step S2 again. That is, the control device 12 again obtains information about the state of the extraction source container V1 after picking up the object O, the state of each holding unit 200A, 200B, etc., and creates the next holding strategy based on the information. In addition, the control device 12 may not perform the holding strategy planning process, but may determine the next holding action based on the holding strategy that has already been created and includes the order of holding. In addition, the control device 12 may correct / update the already created holding strategy based on the information that has been obtained instead of performing the same holding strategy planning process. Furthermore, the control device 12 may cause the score generating unit 304 to change the value of the parameter used when generating the score, or cause the threshold generating unit 305 to change the threshold Th, after the holding operation is performed one or more times.

[0148] On the other hand, if the determination unit 306 determines that the command listed in the command list has been completed (step S8: Yes), the control device 12 executes step S9 and ends the control. For example, if the determination unit 306 determines that no object O remains after the source container V1 is removed, the control device 12 executes step S9 and ends the control.

[0149] According to the structure of the first embodiment as described above, it is possible to efficiently determine the holding strategy for the object O while suppressing the number of times the holding method is switched. The control device 12 of the conveying system 1 does not specifically calculate the position of the object O to be held and the posture of the movable arm 100, etc., but selects the holding strategy. The holding strategy is, for example, the type of loading and unloading device 10 to be used, the object O to be held, and the holding method. The control device 12 of the conveying system 1 only needs to calculate the position of the object O to be held and the posture of the movable arm 100 for the selected object O, which can significantly reduce the required amount of calculations. In addition, by making a holding strategy that does not switch the holding method but preferentially uses the currently selected holding method when it is also easy to hold to a certain extent by the currently selected holding method, it is possible to save the time required for switching the holding method, so that the overall operation time can be shortened.

[0150] In this embodiment, the plurality of holding methods include clamping and suction. Thus, the handling device 10 can hold various objects O, such as thin objects O that are difficult to hold by clamping or objects O with uneven shapes that are difficult to hold by suction.

[0151] Furthermore, in this embodiment, the control device 12 selects the object O to be held and the holding method based on the score so as to minimize the number of times the holding method needs to be switched. Furthermore, the control device 12 determines the need (or lack thereof) for switching the holding method based on the score. This allows the handling device 10 to perform a series of holding operations while minimizing the number of times the holding method needs to be switched, thereby reducing the time required for the overall operation.

[0152] Furthermore, in this embodiment, the control device 12 determines the need (presence or absence) for switching the retention method based on the magnitude relationship between the score and a predetermined threshold value Th. This allows the control device 12 to easily determine the necessity of switching the retention method. Furthermore, the control device 12 can create a retention strategy tailored to the situation by adjusting the threshold value Th.

[0153] Furthermore, in the present embodiment, the control device 12 determines the order of holding the objects O based on the scores. This allows the control device 12 to select the next holding operation in consideration of the entire picking operation specified in the command list.

[0154] Furthermore, in this embodiment, the control device 12 determines the order of switching the holding method of the object. This allows the control device 12 to select the optimal timing for switching the holding method in consideration of the entire picking operation specified in the command list.

[0155] Furthermore, in this embodiment, when calculating the score for the holding method of suction, the control device 12 calculates the score for each region of the object O. This allows the control device 12 to select the easiest region to hold of the same object O as the holding target region, thereby improving holding efficiency.

[0156] Furthermore, in this embodiment, the control device 12 calculates the ease of holding for each object O and each holding method as a first score, and the score based on the selected holding method is a second score calculated based on the first score for each object O and each holding method. Thus, the control device 12 can determine the holding action based on two perspectives: the ease of holding the object O represented by the first score, and the priority based on the currently selected holding method represented by the second score.

[0157] Furthermore, in this embodiment, the second score for the object O and the holding method is the value obtained by dividing the first score for the object O or the region of the object O and the selected holding method by the first score for the object O or the region of the object O and the holding method. This allows the control device 12 to easily calculate the second score used to determine the necessity of switching the holding method.

[0158] (Second embodiment)

[0159] Next, refer to Figure 10 The second embodiment will be described. The second embodiment differs from the first embodiment in that a plurality of retention strategies are created. The configuration other than that described below is the same as that of the first embodiment.

[0160] Figure 10 This is a control flowchart of a maintenance strategy planning process performed by the control device 12 according to the second embodiment.

[0161] In the second embodiment, the control device 12 executes steps S316 to S318 instead of step S306 of the first embodiment. In the maintenance strategy planning step (step S3), the control device 12 calculates the first score S H (I) (Step S311), calculate the second score T H，H0 (I) (Step S312), the necessity of switching the holding method is determined (Step S313). If the determination unit 306 determines that switching the holding method is not necessary (S314: No), the holding strategy determination unit 307 of the control device 12 determines a holding strategy including the order of holding, similar to the first embodiment (Step S315), stores the holding strategy in the storage unit 302 (Step S319), and determines the next holding action based on the holding strategy (Step S320). In other words, the control content of the control device 12 when the determination unit 306 determines that switching the holding method is not necessary is the same as in the first embodiment.

[0162] On the other hand, if the determination unit 306 determines that the switching of the holding method is necessary (S314: Yes), the control device 12 executes step S316. In step S316, the holding strategy determination unit 307 of the control device 12 creates a plurality of holding strategy plans including the order of holding and switching. For example, the holding strategy determination unit 307 determines the order of holding and switching based on the first score S H (I) Create a retention strategy plan that performs one switching operation with a relatively low overall success rate and a retention strategy plan that performs two switching operations with a relatively high overall success rate.

[0163] Next, the control device 12 executes step S317. In step S317, the score generating unit 304 of the control device 12 calculates the score of each holding strategy plan created by the holding strategy determining unit 307. For example, the score generating unit 304 calculates the number of switching times of the holding method in the holding strategy plan, the first score S for each holding action, and the score of each holding action. H The score of each retention strategy plan is calculated using the average value, maximum value, minimum value, etc. of (I), the expected time required to complete the retention strategy, etc. as evaluation items. The score generating unit 304 can also appropriately change the weight of the evaluation items based on the operator's input.

[0164] Next, the control device 12 executes step S317. In step S317, the retention strategy determination unit 307 of the control device 12 determines the optimal retention strategy based on the scores of the retention strategy plans generated by the score generation unit 304. For example, the retention strategy determination unit 307 selects the retention strategy plan with the highest score as the retention strategy.

[0165] Then, the control device 12 stores the holding strategy determined by the holding strategy determination unit 307 in the storage unit 302 (step S319 ), and determines the next holding operation based on the holding strategy (step S320 ).

[0166] Alternatively, the control device 12 may omit the calculation of the second score and / or the determination of the necessity of switching (steps S312 to S314), and generate multiple retention strategy scenarios based on the first score. For example, the retention strategy determination unit 307 may generate a retention strategy scenario with the lowest overall success rate, which does not require switching; a retention strategy scenario with the second lowest overall success rate, which requires switching once; and a retention strategy scenario with the highest overall success rate, which requires switching twice. The score generation unit 304 calculates the score for each retention strategy scenario, and the retention strategy determination unit 307 determines the retention strategy based on the score. Thus, even without determining whether to switch the retention method, the control device 12 can determine a retention strategy that reduces the number of times the retention method is switched.

[0167] According to the configuration of this embodiment, the control device 12 creates a plurality of holding strategies including at least one of the order in which the object O is held and the order in which the holding method is switched, and selects one of the plurality of holding strategies. This allows the control device 12 to compare and study a wide range of holding strategies and select the optimal holding strategy in terms of success rate and the number of switching times of the holding method.

[0168] Furthermore, in this embodiment, the control device 12 selects one of the plurality of holding strategies in such a manner as to minimize the number of switching holding methods. Thus, the loading and unloading device 10 can perform a series of holding operations in such a manner as to minimize the number of switching holding methods, thereby reducing the time required for the overall operation.

[0169] (Third embodiment)

[0170] Next, the third embodiment will be described. The third embodiment differs from the first embodiment in that a holding method is selected based on the frequency of use of each holding method. The configuration other than that described below is the same as that of the first embodiment.

[0171] The control device 12 records the usage frequency of each holding method in the storage unit 302. For example, the control device 12 may record the holding method used in the storage unit 302 each time a holding operation is performed, and calculate the usage frequency of each holding method based on the holding method usage history. Alternatively, the control device 12 may record the switching of holding methods in the storage unit 302, and calculate the usage frequency of each holding method based on the switching history of the holding method. In this way, the control device 12 can confirm the frequency with which each holding method has been used in the past.

[0172] The control device 12 can reduce the frequency of use of the frequently used holding method when there is a tendency in the calculated frequency of use of each holding method or when it is determined that the holding method is close to the usage limit. Figure 9 In step S302, the second score T is calculated. H，H0 (I) When the second score T can be adjusted H，H0 (I) in order to reduce the priority of selecting the frequently used holding method. Alternatively, the control device 12 can be, for example, Figure 9 In step S303, the second score T is compared. H，H0 (I) and the threshold value Th, the threshold value generating unit 305 adjusts the threshold value Th to reduce the priority of selecting the frequently used holding method. Alternatively, the control device 12 can, for example, Figure 9 In the case where a holding strategy including the order of holding and switching is determined in step S306, the first score S based on other holding methods H (I) A certain high object O is held by the other holding method so that the frequently used holding method is not used as much as possible. Alternatively, the control device 12 can, for example, Figure 10 When the score of each retention strategy is calculated in step S317, the parameters in the score are adjusted to lower the priority of selecting the retention method that is frequently used.

[0173] For example, the control device 12 can calculate the scores of clamping and adsorption using the following formula 6.

[0174] [Formula 6]

[0175] F(I)=g(w1h1S p (I)+w2h2S s (I))+w3f item

[0176] In formula 6, w1h1+w2h2+w3=1, w i ≥0(i=1, 2, 3), h j ≥0(j=1,2). S p(I) is the first score when the masked area I representing the object O or the area of the object O is held by "clamping". p (I) is the first score when the masked area I representing the object O or the area of the object O is held by "adsorption". g(S H (I), S H0 (I)) is the function used to calculate the second score. item The weight of the object O is multiplied by a coefficient to normalize the value. h1 and h2 are weights for adjusting the degree of utilization of the current holding method. For example, the weights of h1 and h2 can be stored in a database or the like as the following table information. In addition, the control device 12 can also be based on the first score S p (I) or S s (I) to adjust the weight.

[0177] [Table 4]

[0178] Maintenance method <![CDATA[h1]]> <![CDATA[h2]]> adsorption 0.9 0.1 Clamping 0.1 0.9

[0179] According to the configuration of the third embodiment described above, the control device 12 selects the object O to be held and the holding method based on the frequency of use of each holding method. Thus, the control device 12 can suppress a preference for a particular holding method by lowering the priority of the frequently used holding method, thereby extending the life of the holding unit 200.

[0180] Furthermore, in this embodiment, the control device 12 calculates a score based on the frequency of use of each holding method. Alternatively, the control device 12 determines a threshold value Th based on the frequency of use of each holding method and determines the need (or lack thereof) for switching holding methods based on the magnitude relationship between the score and the threshold value Th. Thus, the control device 12 can easily adjust the priority of frequently used holding methods by adjusting the score or threshold value based on the frequency of use of each holding method.

[0181] (Fourth embodiment)

[0182] Next, the fourth embodiment will be described. The fourth embodiment differs from the first embodiment in that a holding method is selected based on the detection results of the physical states of the holding units 200A and 200B. The configuration other than that described below is the same as that of the first embodiment.

[0183] The control device 12 receives information about the physical conditions of the first holding portion 200A and the second holding portion 200B from the third sensor 11C and the fourth sensor 11D. If a specific holding portion 200 is used repeatedly, the measurement values of the physical sensors of the third sensor 11C and the fourth sensor 11D may change due to deformation, shape change, surface condition change, etc. of the holding portion 200. Based on this information about the physical conditions, the control device 12 infers the usage conditions such as the wear condition of the holding portions 200A and 200B. As in the third embodiment, the control device 12 adjusts the score, threshold, and the order of maintenance and switching in the maintenance strategy to reduce the priority or frequency of selecting the maintenance method for the holding portions 200A and 200B that are inferred to have deteriorated or damaged.

[0184] According to the configuration of the fourth embodiment described above, the control device 12 selects the object O to be held and the holding method based on the physical information of the holding unit 200. This allows the priority of the holding methods to be adjusted while actually measuring the physical degradation of the holding unit 200, thereby extending the life of the holding unit 200.

[0185] (Fifth embodiment)

[0186] Next, the fifth embodiment will be described. The fifth embodiment differs from the first embodiment in that an algorithm using machine learning is used instead of a rule-based algorithm. The configuration other than that described below is the same as that of the first embodiment.

[0187] In this embodiment, the calculation of the first and second scores in the first embodiment is performed through machine learning. For example, if supervised learning is envisioned, the evaluation function specified above can be used as the evaluation value during learning. The learning algorithm is not limited to supervised learning and can be modified according to the type of learning, such as unsupervised learning and reinforcement learning.

[0188] The control device 12 constructs a learning network based on information obtained from the sensor 11, information obtained from the first score, information obtained from the second score, and information required for the number of times the holding method is switched. This allows learning to balance the success rate of the holding action, the number of times the holding method is switched, and the frequency of use of the holding method, thereby adjusting the balance between them. For example, the control device 12 trained by machine learning can receive as input the information obtained in the information acquisition process (step S1) performed by the sensor 11, and output a score for each holding object and each holding method that is suitable for increasing the success rate of the holding action, suppressing the switching of holding methods, and suppressing the frequency of use of holding methods, as well as one or more holding strategies including the order of holding and switching.

[0189] When machine learning is used, in the evaluation function of the first score in Formula 2 and Formula 3, w is set p1 =w p2 =w p3 =w p4 =w p5 =0,w s1 =w s2 =w s3 =w s4 =w s5 =0.

[0190] According to the configuration of the fifth embodiment as described above, the control device 12 can efficiently output a more appropriate score and retention strategy by repeatedly performing machine learning.

[0191] In each of the above embodiments, the control device 12 compares the second score T H，H0 (I) The necessity of switching is determined by comparing the threshold value Th and the order of the holding action and the switching action is determined. However, the necessity of switching may be not determined and the order of the holding action and the switching action may be determined so as to reduce the number of switching times. For example, the score generating unit 304 of the control device 12 generates a first score S which simply indicates the ease of holding without considering the currently selected holding method. H (I) It is possible to generate a first score S based on the currently selected holding method H0. H0 (I) value (for example, set to n times (n>1)), the first score S based on other holding methods H (I) Maintain the second score T of the original value H，H0 (I), based on the second score T H，H0 (I) A holding strategy including the order of holding actions and switching actions is determined in such a way as to minimize the number of switching times. In this case, the control device 12 does not need to determine the necessity of switching the holding method. H The second score T of the correction method (I) H，H0 The generation method of (I) is not limited to the above example, and any method may be used as long as a certain bias is applied to preferentially use the currently selected holding method.

[0192] In each of the above-described embodiments, the loading and unloading device 10 includes a first loading and unloading device 10A that performs "clamping" and a second loading and unloading device 10B that performs "adsorption." However, the structure of the loading and unloading device 10 is not limited to this. As long as it can select one or more of a plurality of holding methods to hold an object, it may include one or more holding sections that perform any holding method. For example, the loading and unloading device 10 may include other loading and unloading devices in addition to the first loading and unloading device 10A and the second loading and unloading device 10B. The loading and unloading device 10 may also include a hybrid hand that can perform both "clamping" and "adsorption" in place of the first loading and unloading device 10A and the second loading and unloading device 10B. Furthermore, at least one of the first loading and unloading device 10A and the second loading and unloading device 10B may be a hybrid hand that can perform both "clamping" and "adsorption." Such a hybrid hand may, for example, have a structure in which the clamping section and the adsorption section are switched using a rotating section that rotates 180°, a rotary wheel-type structure like an optical microscope, or other structures. The handling device 10 may include two types of suction handling devices that differ in size, shape, characteristics, etc. Alternatively, the handling device 10 may include a handling device that holds the object O using a holding method other than clamping and suction.

[0193] According to at least one embodiment described above, the operation time of the loading and unloading device can be shortened by creating a holding policy so as to preferentially use the currently selected holding method based on the score based on the currently selected holding method.

[0194] While several embodiments of the present invention have been described, these embodiments are provided as examples and are not intended to limit the scope of the invention. These embodiments may be implemented in various other forms and may be omitted, replaced, or modified in various ways without departing from the spirit of the invention. These embodiments and their variations are intended to be included within the scope and spirit of the invention and within the scope of the invention set forth in the claims and their equivalents.

Claims

1. A loading and unloading system comprising: movable arm; A holding portion, mounted on the movable arm, capable of selecting one or more of a plurality of holding methods to hold an object; a sensor capable of detecting a plurality of said objects; and a control unit for controlling the movable arm and the holding unit; The control unit calculates, for each of the objects, a first score indicating ease of holding for each holding method corresponding to the object based on the information acquired from the sensor; The control unit calculates, for each of the objects, a second score based on the selected holding method and the holding method to be evaluated, using an evaluation function based on the first score; The control unit selects the object to be held next and a holding method based on the second score; The control unit calculates and maintains the position of the selected object and the posture of the movable arm.

2. The loading and unloading system according to claim 1, wherein: The plurality of holding methods include clamping and adsorption.

3. The loading and unloading system according to claim 1, wherein: The control unit selects the object to be held next and the holding method so that the number of times the holding method is switched is reduced.

4. The loading and unloading system according to claim 1, wherein: The control unit determines whether or not switching of the holding method is necessary based on the second score.

5. The loading and unloading system according to claim 4, wherein: The control unit determines whether or not switching of the holding method is necessary based on a magnitude relationship between the second score and a predetermined threshold value.

6. The loading and unloading system according to claim 1, wherein: The control unit determines the order in which the objects are to be retained based on the second score.

7. The loading and unloading system according to claim 6, wherein: The control unit further determines an order of switching the holding methods of the objects.

8. The loading and unloading system according to claim 6, wherein: The control unit creates a plurality of holding strategies including at least one of an order of holding the objects and an order of switching the holding methods, and selects one of the plurality of holding strategies.

9. The loading and unloading system according to claim 8, wherein: The control unit selects one of the plurality of holding strategies so as to reduce the number of times the holding method is switched.

10. The loading and unloading system according to claim 1, wherein: The control unit selects the object to be held next and the holding method based on the usage frequency of each holding method.

11. The loading and unloading system according to claim 10, wherein: The control unit calculates the second score based on the usage frequency of each holding method.

12. The loading and unloading system according to claim 10, wherein: The control unit determines a threshold value based on the frequency of use of each holding method, and determines the necessity of switching the holding method based on a magnitude relationship between the second score and the threshold value.

13. The loading and unloading system according to claim 1, wherein: The control unit selects the object to be held next and a holding method based on the physical information of the holding unit.

14. The loading and unloading system according to claim 1, wherein: When calculating the second score when the holding method is adsorption, the control unit calculates the second score for each region of the object.

15. The loading and unloading system according to any one of claims 1 to 14, wherein: The second score for the object and the holding method is a value obtained by dividing the first score for the object and the selected holding method by the first score for the object and the holding method.

16. A delivery system comprising: movable arm; A holding portion, mounted on the movable arm, capable of selecting one or more of a plurality of holding methods to hold an object; a sensor capable of detecting a plurality of said objects; and a control unit for controlling the movable arm and the holding unit; The control unit calculates, for each of the objects, a first score indicating ease of holding for each holding method corresponding to the object based on the information acquired from the sensor; The control unit calculates, for each of the objects, a second score based on the selected holding method and the holding method to be evaluated, using an evaluation function based on the first score; The control unit selects the object to be held next and a holding method based on the second score; The control unit calculates a position at which the selected object and a posture of the movable arm are maintained, and moves the object from a first position to a second position.

17. A control device for controlling a conveying system, the conveying system comprising: movable arm; A holding portion, mounted on the movable arm, capable of selecting one or more of a plurality of holding methods to hold an object; and a sensor capable of detecting a plurality of said objects; The control device calculates, for each of the objects, a first score indicating ease of holding for each holding method corresponding to the object based on the information acquired from the sensor; The control device calculates, for each of the objects, a second score based on the selected holding method and the holding method to be evaluated, using an evaluation function based on the first score; The control device selects the object to be held next and a holding method based on the second score; The control device calculates and maintains the position of the selected object and the posture of the movable arm.

18. A storage medium storing a control program for controlling a conveying system, wherein the conveying system comprises: movable arm; A holding portion, mounted on the movable arm, capable of selecting one or more of a plurality of holding methods to hold an object; a sensor capable of detecting a plurality of said objects; and a control unit for controlling the movable arm and the holding unit; The control program causes the control unit to execute the following steps: calculating, for each of the objects, a first score indicating ease of holding for each holding method corresponding to the object based on the information acquired from the sensor; calculating, for each of the objects, a second score based on the selected holding method and the holding method to be evaluated, using an evaluation function based on the first score; selecting the object to be held next and a holding method based on the second score; The position of the selected object and the posture of the movable arm are calculated and maintained.

19. A loading and unloading method for loading and unloading articles by a conveying system, wherein the conveying system comprises: movable arm; A holding portion, mounted on the movable arm, capable of selecting one or more of a plurality of holding methods to hold an object; a sensor capable of detecting a plurality of said objects; and a control unit for controlling the movable arm and the holding unit; The control unit calculates, for each of the objects, a first score indicating ease of holding for each holding method corresponding to the object based on the information acquired from the sensor; The control unit calculates, for each of the objects, a second score based on the selected holding method and the holding method to be evaluated, using an evaluation function based on the first score; The control unit selects the object to be held next and a holding method based on the second score; The control unit calculates and maintains the position of the selected object and the posture of the movable arm.

20. A loading and unloading system comprising: movable arm; A holding portion, mounted on the movable arm, capable of selecting one or more of a plurality of holding mechanisms to hold an object; a sensor capable of detecting the held object; and a control unit for controlling the movable arm and the holding unit; The control unit calculates, for each of the objects to be held, a first score indicating the ease of holding of each holding mechanism corresponding to the object to be held based on the information acquired from the sensor; The control unit calculates, for each of the held objects, a second score based on the selected holding mechanism and the holding mechanism to be evaluated, using an evaluation function according to the first score; The control unit selects a holding object and a holding mechanism that holds the holding object subsequent to the selected holding mechanism based on the second score; The control unit calculates a position at which a holding mechanism that holds the object to be held subsequent to the selected holding mechanism holds the object to be held and a posture of the movable arm.

Images