Handling system, control device and robot management system

Abstract

To obtain a novel handling system with fewer inconveniences. [Solution] In one embodiment, a robot system includes a robot having a movable part and an end effector that grasps and moves an object, and a sensor that measures the object, and is capable of processing the object in a plurality of operation modes. A data analysis unit analyzes first data of the object obtained by measuring the object with the sensor and calculates second data of the object. A database stores the second data of the object in association with a status of the object data that is determined corresponding to the operation mode of the robot system when the first data of the object is obtained. An operation instruction unit issues an operation instruction to the robot system that processes the object, in the operation mode determined according to the status of the object stored in the database, to process the object in accordance with an operation command determined according to the second data of the object.

Description

[Technical field]

[0001] The embodiment is a handling system , a control device, and Regarding robot management systems. [Background technology]

[0002] 2. Description of the Related Art Conventionally, systems for automating object handling operations are known. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2018-504333 Summary of the Invention [Problem to be solved by the invention]

[0004] It would be beneficial to have a novel arrangement of handling systems that would be less inconvenient, for example by reducing the effort required to handle new items. [Means for solving the problem]

[0005] The handling system of the embodiment includes a robot system, A database; A plan generation unit; Robot Control Unit The robot system includes a robot having a movable part and an end effector attached to the movable part and configured to grasp an object, and configured to grasp and move the object, and a sensor configured to measure the object, and is capable of processing the object. The database stores an item ID for identifying the object and item information indicating characteristics of the object in association with each other. The plan generating unit is before Robot motion information representing the motion and parameters of the robot is calculated. Robot Control Unit According to the generated robot operation information, The robot control unit controls the robot. When the robot is to grasp the object, the plan generation unit calculates a grasping position and a grasping posture representing a position and a posture of the robot at which the object is grasped, based on the position and posture of the object, and generates the robot operation information including a path of the robot from the current position and posture of the robot to the grasping position and the grasping posture. The item information includes an item type indicating a shape of the object and a degree of ease of shape change. When the robot is to grasp a first object, which is one of the objects, the robot control unit sets a target value of a force to be applied to the first object when grasping the first object, based on at least one of the shape of the item type of the first object stored in the database corresponding to the item ID of the first object and the degree of ease of shape change. [Brief description of the drawings]

[0006] [Figure 1]FIG. 1 is a schematic and exemplary configuration diagram of a handling system according to an embodiment. [Diagram 2] FIG. 2 is an exemplary block diagram of a robot integrated management system according to the embodiment. [Diagram 3] FIG. 3 is a schematic and exemplary configuration diagram of a picking robot system according to an embodiment. [Figure 4A] FIG. 4A is a schematic, exemplary configuration diagram of an end effector according to an embodiment. [Figure 4B] FIG. 4B is a schematic, exemplary structural view of an end effector different from that of FIG. 4A in the embodiment. [Figure 4C] FIG. 4C is a schematic, exemplary configuration diagram of an end effector different from those in FIGS. 4A and 4B according to the embodiment. [Figure 4D] FIG. 4D is a schematic and exemplary configuration diagram of an end effector different from those in FIGS. 4A to 4C according to the embodiment. [Figure 4E] FIG. 4E is a diagram illustrating an example of end effector information according to the embodiment. [Diagram 5] FIG. 5 is an exemplary block diagram of a controller of the picking robot system of the embodiment. [Figure 6] FIG. 6 is an exemplary state transition diagram relating to the recording state of data and the registration state of an end effector in the integrated item database of the embodiment. [Figure 7A] FIG. 7A is a diagram illustrating an example of an item information table in the integrated item database of the embodiment. [Figure 7B] FIG. 7B is a diagram illustrating an example of an end effector information table in the integrated item database of the embodiment. [Figure 8] FIG. 8 is an exemplary flowchart showing the procedure from registration of a new item to picking, and the procedure for updating the integrated item database in the handling system of the embodiment. [Figure 9] FIG. 9 is an exemplary flowchart showing a processing procedure in the photography and registration mode of the picking robot system according to the embodiment. [Figure 10A] FIG. 10A is a schematic exemplary perspective view of an end effector of an embodiment. [Figure 10B] FIG. 10B is an exemplary illustration of a suction pattern by the end effector of FIG. 10A according to an embodiment. [Figure 10C] FIG. 10C is an explanatory diagram showing an example of item recognition by the controller of the picking robot system of the embodiment. [Figure 11A] FIG. 11A is an exemplary illustration showing items placed within a container according to an embodiment. [Figure 11B] FIG. 11B is a diagram showing data such as evaluation values ​​obtained for a plurality of items shown in FIG. 11A of the embodiment. [Figure 11C] FIG. 11C is an exemplary flowchart illustrating a procedure of a tentative determination process of an end effector according to the embodiment. [Figure 11D] FIG. 11D is a diagram showing an example of an item information table in the integrated item database of the embodiment, showing an end effector tentatively determined state. [Figure 11E] FIG. 11E is a diagram showing an example of an end effector information table in the integrated item database of the embodiment, and is a diagram showing an end effector tentatively determined state. [Figure 12] FIG. 12 is an exemplary flowchart showing a processing procedure in the shape registration mode of the picking robot system according to the embodiment. [Figure 13A] FIG. 13A is an exemplary flowchart showing a procedure for updating item information in the integrated item database of an embodiment. [Figure 13B] FIG. 13B is a schematic and exemplary perspective view of an item model modeled in the robot integrated management system of the embodiment. [Figure 13C] FIG. 13C is a diagram showing an example of an end effector information table in the integrated item database of the embodiment, showing a state in which an end effector evaluation value has been assigned. [Figure 14A]FIG. 14A is a diagram showing an example of an item information table in an integrated item database according to an embodiment, and is a diagram showing an end effector determined state. [Figure 14B] FIG. 14B is a diagram illustrating an example of an end effector information table in the integrated item database of the embodiment, showing a state in which end effector compatibility and registration status have been assigned. [Figure 15] FIG. 15 is an exemplary flowchart illustrating a procedure of an item type determination process according to the embodiment. [Figure 16] FIG. 16 is an exemplary flowchart illustrating a procedure of a process for determining flag information indicating attributes of an item according to the embodiment. [Figure 17] FIG. 17 is an exemplary flowchart showing a procedure for processing an item that has reached an end effector determination state by the robot integrated management system according to the embodiment. [Figure 18] FIG. 18 is an exemplary flowchart showing a processing procedure in the normal mode of the picking robot system according to the embodiment. [Figure 19] FIG. 19 is an exemplary flowchart illustrating a procedure of a selection process of an end effector according to the embodiment. [Figure 20A] FIG. 20A is a diagram showing an example of a picking order for the handling system of the embodiment. [Figure 20B] FIG. 20B is a diagram showing an example of a primary end effector and a capable end effector for an unprocessed item in a picking order for the handling system of the embodiment. [Figure 21] FIG. 21 is an exemplary flowchart showing a procedure for switching the gripping mode of an item by the picking robot system of the embodiment. [Figure 22] FIG. 22 is an exemplary flowchart showing a procedure for packing items into boxes by the picking robot system of the embodiment. [Figure 23A] FIG. 23A is a schematic, exemplary perspective view of a box-type item in a handling system of an embodiment. [Figure 23B] FIG. 23B is a schematic and exemplary perspective view of a solid type item in a handling system of an embodiment. [Figure 23C] FIG. 23C is a schematic and exemplary perspective view of a deformable type item in a handling system of an embodiment. [Figure 23D] FIG. 23D is a schematic and exemplary plan view of a container in which the items in FIGS. 23A to 23C are packed by the picking robot system of the embodiment. [Figure 24] FIG. 24 is an exemplary flowchart illustrating a procedure for a logistics system improvement proposal process performed by the robot integrated management system according to the embodiment. [Diagram 25] FIG. 25 is an explanatory diagram illustrating an example of an image displayed in the logistics system improvement proposal function by the robot integrated management system of the embodiment. [Figure 26] FIG. 26 is an exemplary block diagram showing a hardware configuration of a computer constituting the robot integrated management system, the master device, the slave device, and the controller of the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Exemplary embodiments of the handling system are disclosed below. The configurations and controls (technical features) of the embodiments described below, as well as the actions and results (effects) brought about by the configurations and controls, are merely examples.

[0008] [Handling system overview] Fig. 1 shows an overview of a handling system 1 according to an embodiment. As shown in Fig. 1, the handling system 1 includes a robot integrated management system 100, an external system 310, a host system 320, a robot 200, and material handling equipment 250. The handling system 1 may also be referred to as an object handling system or an item handling system.

[0009] The external system 310 is, for example, an automated warehouse, an external transport device, etc. The upper system 320 is, for example, a warehouse management system, an inventory management system, a transportation management system, etc.

[0010] A plurality of robots 200 (automated robots), material handling equipment 250 such as a conveyor system and a lifting system, an information reading inspection system such as OCR, a sorting system, etc. (not shown) are connected to the robot integrated management system 100. The robot integrated management system 100 is an example of a robot management system.

[0011] The robot integrated management system 100 manages or controls the above-mentioned connected systems and devices in an integrated manner so as to handle the object 10 in cooperation with the host system 320 when transferring the object 10 from the external system 310 or to the external system 310. Specifically, the robot integrated management system 100 controls the robot 200 and the material handling equipment 250 so as to carry out processes such as transport, picking, packing, unloading, loading, unpacking, inspection, and quality control for the object 10 such as various items, products, luggage, and boxes. An item is an example of the object 10 and is also referred to as an article or object. The robot 200 and the material handling equipment 250 may also be referred to as a robot system.

[0012] The robot integrated management system 100 includes a master device 110, a plurality of slave devices 120, and an integrated management database .

[0013] The master device 110 acts as an interface between an external system 310 and the integrated management database 130. The master device 110 also performs operations such as the transportation of items between different devices such as the robot 200 and the material handling equipment 250, and optimizes the operations.

[0014] The slave devices 120 are connected to the robot 200 and the material handling equipment 250, respectively, and manage or control the connected equipment. When switching between devices connected to the robot integrated management system 100, the master device 110 can minimize changes to the master device 110 and the integrated management database 130 by switching between the slave devices 120 to be controlled.

[0015] [Outline of the Robot Integrated Management System] Fig. 2 shows the configuration of the robot integrated management system 100. As shown in Fig. 2, the robot integrated management system 100 includes a data management unit 100a, a data analysis unit 100b, an operation unit 100c, a service / UI unit 100d, a business management unit 100e, and an SoS_API 100f.

[0016] The data management unit 100a manages the database, in other words, acquires, stores, and updates data from the integrated management database 130.

[0017] The integrated management database 130 managed by the data management unit 100a may be referred to as the integrated management database 130. The integrated management database 130 includes, for example, item data, robot data, end effector data, and operation data.

[0018] The item data is information about the items that are processed by the robot 200 or material handling equipment 250 as a terminal.

[0019] The robot data is information about the robot 200, and the end effector data is information about an effector as a part of the robot 200. The end effector 220 may also be referred to as a hand.

[0020] Operational data is information generated during operation of the system, such as the operational status of the robot 200 and the results of processing items.

[0021] The data analysis unit 100b acquires necessary data from the data management unit 100a and performs data analysis required for the operation unit 100c to execute the operation. The data associated with the item (object 10) obtained by the analysis of the data analysis unit 100b is an example of second data.

[0022] The operation section 100c executes a data update operation and an action command operation.

[0023] In a data update operation, the operation unit 100c updates the data in the database by determining whether the data stored in the database is valid or invalid based on the analysis by the data analysis unit 100b, selecting the data, or generating new data from multiple pieces of data.

[0024] In the operation command operation, the operation unit 100c controls the robot system and the material handling equipment 250 by controlling the operating state of each of the robots 200 and the material handling equipment 250 based on the analysis results of the data analysis unit 100b, controlling the items to be fed into each of the robots 200 and the material handling equipment 250, changing the end effector 220 of each robot 200 according to the items to be fed, and issuing instructions on the operation mode to each robot 200. The service UI unit 100d inputs and outputs data to and from the data management unit 100a via a user interface, provides maintenance information to the outside based on the analysis performed by the data analysis unit 100b, and provides on-site improvement information to the outside. The service UI unit 100d may also have a function of visualizing the operation status based on information from the data management unit 100a and the data analysis unit 100b, and an operation UI function of receiving or transmitting operation data that realizes remote operation. The business management unit 100e utilizes data from the service / UI unit 100d and the data management unit 100a to perform functions such as computer aided engineering (CAE), product lifecycle management (PLM), and enterprise asset management (EAM). The system of systems application programming interface (SoS_API100f) provides an interface for connecting to external IT systems such as a warehouse management system (WMS), a manufacturing execution system (manufacturing execution system), and a transportation management system (MES).

[0025] [Picking robot system overview] 3 is a schematic configuration diagram of the picking robot system 2. The picking robot system 2 includes a controller 201, a robot 200, a sensor 230, a housing 240, and a material handling device 250. The picking robot system 2 grips and transports the object 10 at a gripping position Ph, and releases it at a release position Pr. Such an operation may be referred to as transporting the object 10, or picking up the object 10. The robot 200 may also be referred to as a manipulator. The controller 201 may also be referred to as a hand control device.

[0026] The robot 200 includes an arm 210 and an end effector 220 .

[0027] The arm 210 is, for example, a multi-joint robot having rotating parts 210a to 210f that are rotated by a plurality of (for example, six) servo motors. The arm 210 is not limited to this, and may be a SCARA robot or a linear motion robot. The arm 210 may also be a combination of at least two of a multi-joint robot, a SCARA robot, and a linear motion robot. The arm 210 is an example of a movable part. The robot 200 may also be called a robot arm.

[0028] 3, the end effector 220 may have, for example, a jamming mechanism, a pinching mechanism, or a multi-finger mechanism. The end effector 220 may have at least two of the suction mechanism, jamming mechanism, pinching mechanism, and multi-finger mechanism. The end effector 220 may also be referred to as a hand.

[0029] The sensor 230 may be a torque sensor, a rotation sensor, or a current sensor of the motors provided in the rotating parts 210a to 210f, in addition to the force sensor 231. The controller 201 may obtain the force acting on the end effector 220 from the detection value of the force sensor 231, or may calculate it from the current value or rotation value (detection value of the rotation sensor) of each motor.

[0030] 3, the picking robot system 2 includes a plurality of cameras 232. The cameras 232 are an example of the sensors 230 and may also be referred to as image sensors. The cameras 232 are sensors capable of acquiring image information and distance information (three-dimensional information), such as an RGB image camera, a distance image camera, a laser range finder, or a laser imaging detection and ranging (LiDAR). The cameras 232 are an example of an observation unit.

[0031] Moreover, the data obtained by measuring the object 10 by the sensor 230 is an example of first data.

[0032] The picking robot system 2 includes, as the cameras 232, for example, a camera 232a for gripping, a camera 232b for checking the gripping state or for calibration, a camera 232c for releasing, and a camera 232d for temporary placement.

[0033] The gripping camera 232a captures and detects the object 10 and its surrounding area at the gripping position Ph where the object 10 to be gripped (transported) is present. At the gripping position Ph, the object 10 is accommodated in a gripping container 11a such as a container or a pallet. In this case, the gripping camera 232a captures and detects a part or the whole of the inside of the container 11a. The gripping position Ph may also be referred to as a movement start position or departure position of the object 10. Note that in FIG. 3, the container 11a is placed on a container transport unit 251 as a transport mechanism such as a belt conveyor, but is not limited to this. The container transport unit 251 is an example of a material handling device 250. The container 11a may also be referred to as an article container or an input container.

[0034] The calibration camera 232b photographs and detects the object 10 at a calibration position of the object 10 that is set outside the gripping container 11a and above the gripping position Ph.

[0035] The release camera 232c photographs and detects the release position Pr and the surrounding area at the release position Pr where the object 10 is released (carried in). At the release position Pr, the object 10 is accommodated in a release container 11b such as a container or a pallet. In this case, the release camera 232c photographs and detects a part or the whole of the inside of the container 11b. The release position Pr may also be referred to as the movement end position or arrival position of the object 10. Note that in FIG. 3, the container 11b is placed on a container conveyor 252 as a conveying mechanism such as a belt conveyor, but is not limited to this. The container conveyor 252 is an example of a material handling device 250. The container 11b may also be referred to as a collection container or a carrying-out container.

[0036] The temporary placement camera 232d captures and detects a temporary placement position Pt where the target object 10 is temporarily placed between the grip position Ph and the release position Pr and its surrounding area. At the temporary placement position Pt, the target object 10 is placed on a temporary placement surface 260 such as a table or stage. In this case, the temporary placement camera 232d captures and detects a part or the whole of the temporary placement surface 260. The temporary placement position Pt may also be referred to as a midway position or a waypoint position of the target object 10. Note that the temporary placement position Pt does not have to be a placement surface.

[0037] The picking robot system 2 may also include, as the sensor 230, a weighing scale 233 that measures the weight of the target object 10. The weighing scale 233 includes a weighing scale 233a that measures the weight of the target object 10 at the gripping position Ph, and a weighing scale 233b that measures the weight of the target object 10 at the release position Pr.

[0038] The housing 240 may house various parts and devices of the picking robot system 2, such as a power supply device for driving an electric actuator such as a motor, a cylinder for driving a fluid actuator, a tank, a compressor, a user interface, various safety mechanisms, etc. The housing 240 may also house a controller 201.

[0039] 4A to 4D show examples of end effectors 220 that can be used in the robot 200. FIGS. 4A to 4C show suction-type end effectors 220A to 220C. The end effector 220A in FIG. 4A has a bent portion 220b, but the end effector 220B in FIG. 4B and the end effector 220C in FIG. 4C do not have a bent portion 220b. The end effector 220B in FIG. 4B has one suction pad 220a, whereas the end effector 220A in FIG. 4A and the end effector 220C in FIG. 4C have multiple suction pads 220a. FIG. 4D also shows a grip-type end effector 220D. The end effector 220D has a clamping mechanism with multiple fingers 220c.

[0040] However, the end effector 220 is not limited to the examples in FIGS. 4A to 4D. For example, the end effector 220 may have an extension mechanism. Specifications such as the size, number, and position of the suction pads 220a in the end effector 220 can be changed in various ways. The force sensor 231 may be provided at the tip of the arm 210. The end effector 220 may have a sensor 230 other than the force sensor 231. The end effector 220 may be provided with a torque sensor that measures the torque of a motor as an actuator that moves the bending portion 220b. In addition, the controller 201 may detect or estimate the force acting on the end effector 220 from the current value or encoded signal of the motor.

[0041] 4E shows an example of end effector information. An end effector ID is assigned to each end effector 220. The characteristic information of the end effector 220 includes the type of the end effector 220, joint information, the number of suction pads, suction pad information, the number of fingers, finger information, external shape information, and the presence or absence of a force sensor 231. Data indicating the characteristic information is stored in the integrated management database 130 in association with the end effector ID. The robot 200 is configured to be able to replace and attach a plurality of different end effectors 220 to the arm 210. An end effector changer (not shown) can replace the end effector 220 according to an instruction from the robot integrated management system 100.

[0042] [Picking robot system controller] 5 is a block diagram of the controller 201. The controller 201 includes, for example, an integration unit 201a, an image processing unit 201b, a signal processing unit 201c, a gripping plan generating unit 201d, a release plan generating unit 201e, a motion plan generating unit 201f, a robot control unit 201g, a peripheral device / I / O control unit 201h (I / O: input / output), a learning control unit 201i, an error detection unit 201j, and an internal database 201k. The controller 201 may also be referred to as a control device.

[0043] The integration unit 201a generates, operates, and manages a work plan for the picking robot system 2 based on input information from the robot integrated management system 100, the state of the picking robot system 2, and values ​​detected by the sensor 230.

[0044] The image processing unit 201b processes images and information (detection values) obtained from the camera 232, and generates information required for operation planning, operation control, error detection, learning, and the like.

[0045] The signal processing unit 201c processes information (detection values) obtained from the sensor 230, and generates information required for operation planning, operation control, error detection, and the like.

[0046] The gripping plan generating unit 201d executes a process of calculating a gripping method of the target object 10 by the robot 200, a gripping position Ph, a gripping posture, and a path and via points by which a necessary part can move to the gripping position Ph and the gripping posture without interfering with the surrounding environment. Here, the necessary part is, for example, the whole robot 200 or a part of the arm 210. Meanwhile, in the picking robot system 2 of this embodiment, since force control is possible, some contact may be allowed for parts having a certain degree of flexibility, such as the suction pad 220a and elastic joints.

[0047] The release plan generating unit 201e executes a process of calculating an installation method such as pressing, a gripping method, a release method, a release position Pr and a release posture, and a path and via points that allow a necessary part to move to the release position Pr and the release posture without interfering with the surrounding environment at a place where the target object 10 gripped by the robot 200 is packed or placed in the container 140b. The release plan generating unit 201e is an example of a plan generating unit. The release posture is substantially the same as the grip posture immediately before the release.

[0048] The motion plan generation unit 201f calculates robot motion information such as the motion method, motion speed, motion path, etc., so that the robot 200 moves from the current position to the grasping position Ph, the temporary placement position Pt, the release position Pr, etc. via a possible path (waypoint) in accordance with instructions from the integration unit 201a.

[0049] The robot control unit 201g controls the picking robot system 2 including the robot 200 according to robot operation information generated by the gripping plan generating unit 201d, the releasing plan generating unit 201e, or the operation plan generating unit 201f and various operation switching instructions from the integrating unit 201a. The robot control unit 201g also controls the robot 200 according to various posture information, trajectory plan information, and the like acquired from the integrating unit 201a.

[0050] The peripheral device / I / O control unit 201h executes input / output (I / O) control for controlling various transport devices, controlling peripheral devices 400 such as safety doors, acquiring information from various sensors, controlling lighting, and so on.

[0051] The learning control unit 201i controls learning functions such as robot model learning for improving the operation accuracy such as vibration suppression of the robot 200, grip control parameter learning for improving the grip performance of the target object 10, grip database learning, and error detection learning for improving the performance of implementing a work plan. In this embodiment, optimal parameter values ​​are selected depending on the situation in the force control. These values ​​may be set by a human being based on experience, but by applying the results of machine learning, more efficient processing can be achieved while saving more labor.

[0052] The error detection unit 201j observes the state of the picking robot system 2, the implementation state of the work plan, the drive control state, the gripping and transporting state of the target object 10, and the like, and detects errors. This error detection can be realized, for example, by observing the output of the force sensor 231, the value obtained by converting the output into hand coordinates, and those values ​​passed through a low-pass filter, and judging an error to have occurred if a preset value is exceeded. As a result, it becomes possible to interrupt the work being processed and then move on to a recovery operation, for example.

[0053] The internal database 201k includes, for example, a robot database, a hand database, an item database, a gripping database, and an environment database (none of which are shown).

[0054] The robot database stores, for example, the structure of the picking robot system 2, the dimensions and weight of each part, the moment of inertia, the operating range and speed of each drive part, and the torque performance.

[0055] The end effector database stores information regarding, for example, the capabilities of the end effector 220 and the grasping characteristics of the end effector 220.

[0056] The item database stores, for example, the name of the object 10, its identification number, its category, image information of the entire surface, CAD model information, weight information, and information on characteristics when grasped (soft, fragile, changes shape, etc.).

[0057] The grasping database stores, for example, for the target object 10, score information such as a graspable position, a graspable posture, and ease of grasping, a possible amount of depression when grasping, a judgment threshold for grasping judgment, and a judgment threshold for error detection, for each grasping method of the end effector 220. Here, the grasping method is, for example, a suction method, a parallel two-finger method, a parallel four-finger method, or a multi-joint method.

[0058] The environment database stores, for example, information about the workbench that the picking robot system 2 is compatible with, and surrounding environment information that indicates the operating range of the picking robot system 2 and surrounding obstacles O.

[0059] The learning results in the learning control unit 201i and the information in the internal database 201k are sequentially shared with information in the integrated management database 130 of the robot integrated management system 100 associated with the data management unit 100a of the robot integrated management system 100.

[0060] In this configuration, the operation unit 100c instructs the controller 201 to change the operation mode of the robot 200 or to replace the end effector 220 depending on the presence or absence of data and the amount of data in the integrated management database 130 managed by the data management unit 100a. The controller 201 changes the processing content for the target object 10 in accordance with the instruction. That is, in this embodiment, the operation of the robot 200 changes depending on the accumulation state of data in the integrated management database 130.

[0061] In the handling system 1 of this embodiment, the operation unit 100c issues an instruction to change the operation mode of the robot 200 or to replace the end effector 220 based on the recording state of the data in the integrated management database 130 managed by the data management unit 100a, in other words, the type of recorded data, the presence or absence of data, the status of the data, the value of the data, etc. Then, the robot 200 changes the processing content for the handling target according to the instruction content (switches the robot operation based on the recording state of the data in the database).

[0062] [Integrated item database and state transitions] Fig. 6 shows a state transition diagram relating to the recording state of data in the integrated item database, which is one of the contents of the integrated management database 130, and the settings of the end effector 220. Fig. 7A and Fig. 7B show examples of records in the integrated item database. An item is an example of an object 10 to be processed by the end effector 220. Records in the integrated item database are generated for each item ID.

[0063] Fig. 7A is called an item information table. The item information table has information (data) fields for the status, item type which is the classification of the item, item weight, various handling flags, etc. Also, Fig. 7B is called an end effector information table. The end effector information table has information (data) fields for each gripping surface of the item, such as the size of the gripping surface and the suitability of each end effector 220.

[0064] 6, the integrated item database has basic states "ST1: item initial registration state", "ST2: end effector tentative determination state", "ST3: end effector confirmed state", and "ST4: handling not supported", as well as intermediate states between these basic states "ST5: item registration updating in progress state", "ST6: model updating in progress state", "ST7: end effector registration updating in progress state", and "ST8: new end effector updating in progress state", and the integrated item database transitions between these eight statuses. Note that in this embodiment, the data management unit 100a executes registration and rewriting of records in the integrated item database.

[0065] "ST1: item initial registration state" is a state in which the data management unit 100a issues a new item ID to the item to be processed and creates a new record.

[0066] "ST2: End effector tentative determination state" is a state in which the data management unit 100a has provisionally registered one or more end effectors 220 that may be able to grip the item to be processed. The end effector tentative determination state is an example of the first state.

[0067] "ST3: End effector determined state" is a state in which the picking operation of the item to be processed has been performed one or more times, the data analysis unit 100b has completed modeling such as acquiring shape information of the item, and the data management unit 100a has determined the end effector 220 suitable for gripping the item. The end effector determined state is an example of the second state.

[0068] "ST4: handling not supported" is a state in which the data analysis unit 100b has determined that it is difficult for the robot 200 to pick up the item to be processed, and has decided not to have the robot 200 handle the item.

[0069] "ST5: Item registration updating state" is a state in which the data analysis unit 100b is calculating and evaluating on a computer the possibility that the item being processed can be grasped by each of the end effectors 220 based on photographed information of the item being processed, and the data management unit 100a is in the process of changing the state of the record.

[0070] "ST6: Model updating state" is a state in which the data analysis unit 100b is modeling the item to be processed based on the shape of the item measured during multiple picking operations, and the data management unit 100a is in the process of changing the state of the record.

[0071] "ST7: End effector registration updating state" is a state in which the data management unit 100a is changing the information of the end effector 220 suitable for grasping the item to be processed and the end effector 220 suitable for each grasping surface of the item, based on the data of the modeled item.

[0072] "ST8: New end effector updating state" is a state in which the data management unit 100a is changing information related to the new end effector 220 based on the data of the modeled item and operational data such as actual grasping success or failure.

[0073] [Picking Integrated Management Processing] FIG. 8 is a flowchart showing the procedure from registration of a new item to picking by the handling system 1, and the procedure for updating the integrated item database.

[0074] In step S0-1, the robot integrated management system 100 receives a picking work instruction from an external system 310 such as a WMS through the SoS_API 100f. The operation unit 100c compares the received picking work instruction data with the information in the integrated management database 130 managed by the data management unit 100a. If the item for which the picking work instruction has been issued is an unregistered item, the data management unit 100a issues a new item ID and adds a record to the integrated item database of the integrated management database 130. At this time, the data management unit 100a sets the status field of the integrated item database to "ST1: Item initial registration state."

[0075] In step S0-2, the operation unit 100c instructs the controller 201 of the robot 200 that can receive a work instruction among the multiple robots 200 to execute the photographing and registration mode, and instructs the material handling device 250 to move the container 11a to the container conveying unit 251 of the robot 200. The controller 201 that receives the instruction executes an operation in the photographing and registration mode. In this operation, the robot 200 does not pick up the specified item, but instead photographs the item with the camera 232 and creates a plan such as a gripping plan. Then, the controller 201 returns the photographed data and related information such as the created plan to the robot integrated management system 100. The data management unit 100a updates the record corresponding to the item ID in the integrated item database based on the received data. During this update operation, the state of the integrated item database is "ST5: Item registration update in progress". The photographing and registration mode is an example of an operation mode and a first mode, and the shape registration mode is an example of an operation mode and a second mode.

[0076] In step S0-3, the data analysis unit 100b executes an end effector tentative determination process for tentatively determining an end effector 220 capable of processing the item, using information from the integrated management database 130. In this end effector tentative determination process, the data analysis unit 100b tentatively determines the end effector 220 that is most likely to be able to grip the item stably as a primary end effector based on the evaluation value of the gripping by each end effector 220 corresponding to the gripping plan, and tentatively determines the end effector 220 that is likely to be able to grip the item, although it has a lower evaluation value than the primary end effector, as a capable end effector.

[0077] If the data analysis unit 100b determines in the tentative decision process of S0-3 that there is no end effector 220 that can perform the process (N in S0-4), the operation unit 100c increments the failure count of the photography registration process in the integrated item database, and the procedure proceeds to S0-15.

[0078] In step S0-15, if the failure count of the photography registration process exceeds a specified number of times (N in S0-15), the procedure proceeds to S0-16, where the data management unit 100a changes the status field of the item to "ST4: handling non-compatible state" and completes the process. On the other hand, in step S0-15, if the failure count of the photography registration process does not exceed a specified number of times (Y in S0-15), the status field remains "ST1: item initial registration state" and the procedure returns to S0-2.

[0079] If the data analysis unit 100b provisionally determines an end effector 220 that can be processed in the provisional determination process of S0-3 (Y in S0-4), in step S0-5, the data management unit 100a changes the status field of the item information table to "ST2: End effector provisional determination state."

[0080] In step S0-6, if all the provisionally determined primary end effectors and capable end effectors are not end effectors 220 that can be attached to the robot 200 currently performing processing, the data management unit 100a searches for a robot 200 to which the end effector 220 can be attached, and the operation unit 100c instructs the robot 200 to which the end effector 220 can be attached (in other words, another robot 200) to perform an operation in the shape registration mode. Furthermore, an instruction is issued to the material handling device 250 to move the container 11a to the container transport unit 251 of the corresponding machine. If the provisionally determined end effector 220 is an end effector 220 that can be attached to the robot 200 currently performing processing, the operation unit 100c instructs the robot 200 to perform an operation in the shape registration mode.

[0081] In step S0-7, the robot 200 that has received the instruction executes an operation in the shape registration mode. In this operation, the robot 200 actually picks up an item using the designated provisionally determined end effector. However, at this point, the item data in the item integrated database is incomplete or insufficient, so the operation speed is limited to a predetermined speed or less. In addition, in order to grasp the shape of the item in more detail, the picking robot system 2 (controller 201) executes more detailed measurement processing of the item using the grasping camera 232a, the weight scale 233a, etc. In the picking operation in the shape registration mode, the picking robot system 2 executes a grasping operation of the item to be processed, and if grasping is possible, carries the grasped item to a measurable area of ​​the grasping camera 232b and measures the shape and characteristics of the grasped item. After that, the picking robot system 2 packs the grasped item into the collection (release) container 11b, and measures the weight using the weight scale 233b. Then, the picking process result and related information such as the item measurement result are returned to the robot integrated management system 100. The data management unit 100a uses the received data to update information related to the item ID in the integrated item database of the integrated management database 130. During this update operation, the status of the integrated item database is "ST6: Model updating".

[0082] In step S0-8, the data analysis unit 100b analyzes the data obtained by the operation in the shape registration mode and updates the item information. The item is modeled based on the measured data such as shape and weight, and the item is evaluated for the gripping suitability of the multiple types of end effectors 220 for each gripping surface of the item based on the information. The data management unit 100a then registers the end effector 220 that was actually capable of gripping the item or the end effector 220 that is estimated to be most stable in the item integrated database as the primary end effector for the item. The data management unit 100a also registers the end effector 220 that has a lower evaluation value than the primary end effector but is likely to be able to grip the item as the capable end effector for the item in the item integrated database. Furthermore, the data analysis unit 100b analyzes the shape and characteristic information of the item and determines the type of the item.

[0083] If the operation in the shape registration mode fails in step S0-9, the operation unit 100c increments the failure count of the shape registration process, and the procedure proceeds to S0-17.

[0084] In S0-17, if the failure count of the shape registration process of all end effectors exceeds a specified number of times (N in S0-17), the procedure proceeds to S0-16, and the data management unit 100a changes the status field of the item to "ST4: Handling non-compatible state" and completes the process. Also, if the failure count of the shape registration process of the primary end effector for the item exceeds a specified number of times, the end effector 220 that is most likely to be able to hold stably among the capable end effectors is changed to the primary end effector. Also, if the operation in the shape registration mode is successful, the success count of the shape registration process is incremented. During this update operation, the state of the integrated item database is "ST8: New end effector registration update in progress."

[0085] In step S0-10, when the item is modeled by the analysis by the data analysis unit 100b and the information related to the characteristics of the item data, such as various flags and compatibility, in the item integrated database is sufficient, the procedure proceeds to step S0-11, and the data management unit 100a changes the status field of the integrated item database to "ST3: End effector determined state". On the other hand, in step S0-10, when the item data in the item integrated database is not sufficient, the status field is maintained as "ST2: End effector tentatively determined state". In addition, when all the operations corresponding to the received picking work instruction are completed as a result of the shape registration process, the operation unit 100c issues an instruction to the robot 200 to sweep out the container 11b, and the data management unit 100a notifies the external system 310 of the completion of the picking work through the business management unit 100e and the SoS_API 100f.

[0086] In step S0-12, if the picking work instruction is not complete, the operation unit 100c instructs the picking robot system 2 (controller 201) to execute the picking process for the next item. At that time, according to the information in the updated integrated item database, if the status field is "ST3: End effector determined state", the operation unit 100c instructs execution of an operation in normal mode. If the status field is "ST2: End effector tentatively determined state", the procedure moves from S0-5 to S0-6, and the operation unit 100c instructs execution of an operation in shape registration mode. The normal mode is an example of the operation mode and the third mode.

[0087] In step S0-13, when the robot 200 operates in the normal mode, it does not perform detailed measurements such as measuring the shape of the item, but only estimates the posture of the object to be grasped, and executes the picking process at a higher speed. Then, it notifies the robot integrated management system 100 of information related to the process such as the success or failure of picking and the processing time at any time. The normal mode may also be called the posture estimation mode. The movement speed of the item in the normal mode is set higher than the movement speed of the same item in the shape registration mode.

[0088] In step S0-14, the data management unit 100a accumulates these processing results in a database. The data analysis unit 100b and the operation unit 100c change and update the primary end effector and the capable end effector based on the accumulated information according to the success rate of the processing, etc. Then, the operation unit 100c issues an instruction to replace the container 11b when the container 11b is full, and issues an instruction to empty the container 11b when all the ordered picking work instructions have been completed. Furthermore, the data management unit 100a notifies the external system 310 of the completion of the picking work through the business management unit 100e and the SoS_API 100f.

[0089] In this way, the handling system 1 of the embodiment models the characteristics of the end effector 220 and manages them in the integrated management database 130. Furthermore, the integrated management database 130 is shared among a plurality of robots 200. Therefore, according to this embodiment, there is no need for each robot 200 to perform the process of updating item data, and the robot integrated management system 100 updates the integrated management database 130 with data obtained from a plurality of robots 200 having different end effectors 220, so that the database can be expanded more quickly.

[0090] [Operation in photo registration mode] 9 shows a procedure of processing in the photography and registration mode of the picking robot system 2. In the photography and registration mode, the picking robot system 2 photographs an item (target object 10) and calculates an evaluation value of a planning operation such as a gripping plan for the conditions set in the internal database 201k of the robot 200.

[0091] In S1-1, when an operation instruction in the photography and registration mode is received from the robot integrated management system 100, the controller 201 executes updating of the internal database 201k.

[0092] In S1-2, the picking robot system 2 confirms the arrival of the container 11a by the camera 232b or a sensor such as a photoelectric sensor or a microswitch, and then in S1-3, the camera 232b photographs the inside of the container 11a.

[0093] In S1-4, the image processor 201b determines the presence or absence of an item from the captured image and identifies the grippable surface of the item. Furthermore, the image processor 201b calculates gripping surface information such as the shape and size of the grippable surface of the item, and its position and orientation in three-dimensional space.

[0094] In S1-5, the grip plan generating unit 201d executes a grip plan calculation to calculate multiple pieces of grip position and posture information for the robot 200 to grip the grippable surface of an item from the grip surface information. At this time, the grip plan calculation is performed for all types of end effector models registered in the updated internal database 201k. Here, the grip plan generating unit 201d assigns an evaluation value indicating the stability and reliability of each grip position and posture to the grip position and posture information, and adds information such as the pressing direction and magnitude of the pressing force of the end effector 220 with force control, and the allowable movement range of the force control, based on information on the surrounding environment such as containers and items other than the grip target, grip surface information, and information on the item integrated database.

[0095] In S1-6, the integration unit 201a selects an item to be grasped, a grasp position, and a grasp posture, and the motion plan generation unit 201f generates robot motion information from the current position of the robot 200 to the grasp position and posture. The integration unit 201a assigns path evaluation values, such as a probability of collision and an estimated processing speed, to the robot motion information.

[0096] In S1-7, the integration unit 201a notifies the robot integrated management system 100 of the gripping positions and postures and robot operation information for all types of end effector models registered in the internal database 201k via the external interface, and completes the processing of the photography and registration mode.

[0097] [Grasp Plan] FIG. 10A shows an example of the end effector 220, and FIG. 10B shows a suction pattern by the suction pads 220a. The end effector 220 shown in FIG. 10A has five suction pads 220a along a plane, and each of these suction pads 220a can be switched on and off. Therefore, as shown in FIG. 10B, items can be suctioned by various suction patterns. In FIG. 10B, the circles with dot patterns are suction pads 220a that are on and perform suction, the white circles are suction pads 220a that are off and do not perform suction, and the squares on the periphery indicate the periphery of the end effector 220. When modeling the end effector 220 for performing a gripping plan, it is sufficient to specify the position of each suction pad 220a relative to the center (centroid, center of gravity) of the suction end face of the end effector 220, the diameter of the suction pads 220a, the outer dimensions of the end effector 220, and the like. FIG. 10C shows an example of recognition of an item placed in the container 11a based on image processing by the controller 201. Here, the dashed area is a circumscribing rectangle of the gripping plane that can be gripped, that is, the grippable area A1. The solid area within the dashed line is the plane P1 that is actually measured and recognized, and this is called the mask information of the grippable area A1. In the gripping plan, as shown in FIG. 10B, based on the specified end effector model, patterns of the suction pads 220a such as (A) to (H) in the figure are listed. Then, while changing the posture of the end effector 220, it is checked whether the area of ​​the ON suction pads 220a in each listed pattern is included in the mask information indicating the grippable area A1 of each grippable surface.

[0098] The gripping plan generating unit 201d determines that a state in which the region of the ON suction pad 220a is included in the mask information is a grippable state, and sets the pad pattern and gripping position / posture as gripping position / posture candidates. The gripping plan generating unit 201d can perform these determinations, for example, by setting the mask region as a bitmap represented by 0 and 1, setting the region of the valid suction pad 220a as a bitmap represented by 0 and 1, and performing convolution of both. Here, it can be determined that a portion where the value after the convolution matches the sum of 1s in the bitmap of the ON suction pad 220a is a portion where the region of the ON suction pad 220a is included in the mask information.

[0099] Next, a specific example of an evaluation value indicating the stability and reliability of gripping calculated for each gripping position and posture candidate will be described. For example, since it is considered that the wider the area of ​​the included ON suction pad 220a, the more stable the gripping can be, so the evaluation value is set higher as the area of ​​the area of ​​the ON suction pad 220a is larger. Furthermore, since the gripping force can be calculated by multiplying the area of ​​the area of ​​the ON suction pad 220a by the degree of vacuum generated there, the evaluation value may be set higher as the ratio of the gripping force to the weight of the item to be gripped is larger. Furthermore, since it is considered that the closer the gripping position is to the center of gravity of an object, the more stable the gripping can be, so the evaluation value may be set higher as the gripping position is to the center of gravity (centroid). Furthermore, in consideration of the possibility that the end effector 220 will collide with surrounding objects during gripping, the evaluation value may be set higher as the distance from the outer periphery (outer edge) of the end effector 220 to the surrounding objects is larger. Note that, although the suction-type end effector 220 has been described here,<https: / / www.jstage.jst.go.jp / article / jrsj1983 / 9 / 1 / 9_1_85 / _pdf / -char / ja> As described above, a grasping plan can be created similarly for the multi-fingered end effector 220. In this case, the evaluation value may be set higher as the opening width of the fingers 220c becomes narrower, or the evaluation value may be set higher as the contact area of ​​the fingers 220c becomes larger.

[0100] This type of grasping planning process can create a grasping plan for an end effector 220 that is not actually installed in the handling system 1 or an end effector 220 that is not interchangeable with the end effector 220, as long as a model of the end effector 220 is registered in the integrated management database 130.

[0101] [End effector tentative determination process] FIG. 11A shows a plurality of items of the same kind placed in a container 11a, and FIG. 11B shows a list of evaluation values ​​obtained by the operation of the photographing and registration mode for the plurality of items in FIG. 11A. Here, evaluation values ​​for a plurality of items of the same kind are considered. For this reason, all of the items (target objects 10) stored in the container 11a in FIG. 11A are the same items of the same kind, and have the same item ID. In the list in FIG. 11B, for the four different end effectors 220A to 220D (220) shown in FIG. 4A to FIG. 4D, the maximum evaluation value and the average evaluation value of the evaluation value according to the gripping plan of each end effector 220 are recorded for each measured gripping surface. In addition, since these processes are the results of the measurement, the registration state of the data is labeled with "D" indicating the measurement result.

[0102] An example of the procedure of the tentative determination process is shown in Fig. 11C. In S2-1, the data analysis unit 100b performs grouping or clustering of the measurement results of the detected gripping surface in order to tentatively determine the end effector. This can be performed by matching to predetermined conditions, statistical evaluation, or machine learning such as a deep neural network.

[0103] Here, Fig. 11D is item information of the integrated item database contained in the integrated management database 130, and Fig. 11E is end effector information of the integrated item database. In the examples of Fig. 11D and Fig. 11E, data analysis unit 100b classifies the item (target object 10) as a rectangular parallelepiped with three sides of different lengths, so the item type in the item information in Fig. 11D is a box type. In addition, data analysis unit 100b determines that the size of the item is sufficiently large, so the small item flag in the item information in Fig. 11D is 0, indicating that it is not a small item.

[0104] Next, in S2-2 of Fig. 11C, the data analysis unit 100b classifies the measurement results of the gripping surfaces. Here, the data analysis unit 100b classifies the measurement results of the multiple gripping surfaces into three rectangular parallelepiped surfaces of different sizes and assigns identification numbers to them.

[0105] In S2-3, the data analysis unit 100b evaluates the suitability of the end effector 220, records it in the end effector information, and ends the process. Here, since the process performed in the end effector tentative determination process is a provisional evaluation, a label "P" representing a provisional determination is recorded in the registration state of each data.

[0106] In the compatibility evaluation of S2-3, the end effector 220 predicted to be graspable and the end effector 220 predicted to be unable to grasp can be classified using a predetermined evaluation value criterion. For example, the data analysis unit 100b determines that an end effector with an evaluation value of 0.3 or less is highly likely to be unable to grasp for each gripping surface, and sets the compatibility as "not capable". The data analysis unit 100b determines that an end effector with an evaluation value of more than 0.3 is capable for grasping, and sets the compatibility as "capable". Furthermore, the data analysis unit 100b determines that the end effector with the largest maximum evaluation value for each gripping surface is "primary". For example, if the gripping surface TS2 is classified as the gripping surface MS2, and the end effector 220 with the largest maximum evaluation value for the gripping surface TS2 is end effector c, the data analysis unit 100b determines that the compatibility of end effector c for gripping surface TS2 is "primary". Similarly, when the gripping surface TS2 is classified into gripping surfaces MS2 and MS3, which are measurement data, and the end effector 220 with the largest maximum evaluation value for the gripping surface TS2 is the end effector a, the data analysis unit 100b determines the suitability of the end effector a for the gripping surface TS2 to be "primary". Finally, the data analysis unit 100b determines the overall suitability of the end effector 220 whose gripping surfaces are either "capable" or "primary" as a comprehensive evaluation to be "capable". Here, when the gripping surfaces are "incapable", the data analysis unit 100b determines the overall suitability of the end effector 220 to be "incapable", and the overall suitability of the end effector 220 with the largest maximum evaluation value to be "primary". Here, the determination of "primary", "capable", "incapable", etc. has been described using the maximum evaluation value. The method using the maximum value has the characteristics of simple processing and high speed. On the other hand, it may be possible to achieve a more accurate judgment by using the average value, a combination of maximum, minimum and average values, taking into account the variance and outliers of multiple data, using clustering, or other statistical evaluation, or by using machine learning such as a deep neural network.

[0107] [Shape registration mode operation] 12 shows the procedure of processing in the shape registration mode of the handling system 1. In the operation in the shape registration mode, the picking robot system 2 executes a gripping operation of the target object 10. At this time, the picking robot system 2 (controller 201) executes measurement processing of the item (target object 10) using the gripping camera 232a, the calibration camera 232b, the release camera 232c, the temporary placement camera 232d, and the weight scale 233.

[0108] First, in S3-1, the integration unit 201a receives an item transport instruction (picking instruction) from the external system 310 via the external interface.

[0109] In S3-2, the integration unit 201a detects the arrival of the container 11a at the gripping position Ph by a sensor 230 such as a camera 232a, a photoelectric sensor, a microswitch, or the like.

[0110] In S3-3, the integration unit 201a controls the camera 232a to capture an image of the grip position Ph of the item and its surroundings, for example, the inside of the container 140a in which the item is stored.

[0111] In S3-4, the image processor 201b executes the determination of the presence or absence of an item and the identification of the grippable surface of the item from the image captured by the camera 232a. Furthermore, the image processor 201b calculates gripping surface information such as the shape, size, and position and orientation in three-dimensional space of the grippable surface of the item.

[0112] In S3-5, the grip plan generating unit 201d calculates a plurality of pieces of grip information for the robot 200 to grip the grippable surface of the item based on the grip surface information. The grip information includes grip position information and grip posture information. Here, the grip plan generating unit 201d calculates additional information such as the movement direction of the end effector 220, the magnitude of the pressing force, and the allowable range of movement when performing the force control described later, based on peripheral information of the grip position Ph such as objects (peripheral parts) other than the item in the container 11a, the grip surface information, and information in the item database, and adds this to the grip information.

[0113] In S3-6, the integration unit 201a selects or determines an item to be grasped, its grasp position Ph, and its grasp posture based on the grasp information. Then, the motion plan generating unit 201f generates robot motion information from the current position of the robot 200 to the grasp position Ph and the grasp posture. Here, the integration unit 201a selects a faster motion in an area where there is no risk of interference or collision between the robot 200 and objects other than the item, or where the probability of interference or collision is relatively low. On the other hand, the integration unit 201a selects a slower motion in an area where there is a risk of interference or collision between the end effector 220 or the item and objects other than the item, or where the probability of interference or collision is relatively high, for example, when the end effector 220 approaches the container 11a or enters the container 11a. Furthermore, the motion plan generating unit 201f determines a target value of the force generated by the end effector 220 in this area. In this case, since the weight of the item and the like are uncertain in the shape registration mode, the motion plan generating unit 201f may set the motion speed to the lower limit within the speed variable range.

[0114] In S3-7, the robot control unit 201g controls the robot 200 and the end effector 220 to operate according to the generated robot operation information and grasp the item at the grasping position Ph and grasping posture. In the peripheral area of ​​the grasping position Ph of the item, the pressing operation of the end effector 220 is performed by only force control, and the grasping operation such as suction or pinching is performed in a state in which the suction pad 220a and the finger 220c are in sufficient contact with the item. In the peripheral area of ​​the grasping position Ph, if the end effector 220 hits an obstacle such as an object other than an item, the robot control unit 201g executes a predetermined operation, for example, an operation of the end effector 220 escaping from the obstacle according to a target value of force. Also, even if the surface to be grasped has an unexpected inclination, the robot control unit 201g executes an operation of making the end effector 220 follow the surface to be grasped by a predetermined pressing method or pressing force.

[0115] The robot control unit 201g and the integration unit 201a monitor the operation of the robot 200 and the state of gripping of the item by the end effector 220. If the end effector 220 succeeds in gripping the item in S3-8 (Y in S3-8), the procedure proceeds to S3-10. On the other hand, if the end effector 220 fails to grip the item in S3-8 (N in S3-8), the procedure proceeds to S3-9.

[0116] In S3-9, the integration unit 201a executes retry preparation operations such as a retreat operation of the robot 200 and a registration process for registering information indicating a gripping failure in the item database, and after S3-9, the procedure returns to S3-3.

[0117] In S3-10, the robot control unit 201g controls the robot 200 to move to a position where an image of the end effector 220 gripping an item can be captured. Then, the integration unit 201a controls the camera 232b to capture an image of the end effector 220 gripping the item at that position.

[0118] In S3-11, the image processing unit 201b measures and generates gripped item information, such as the relative position of the item with respect to the end effector 220, the relative orientation of the item with respect to the end effector 220, and the state and shape of the item, from images captured by the gripping camera 232a and the calibration camera 232b. The signal processing unit 201c can also measure the weight of the item by calculating the difference in weight between before and after lifting the item from the measurement results of the weight scale 233a provided at the gripping position Ph.

[0119] In S3-12, the integrating unit 201a executes a gravity compensation operation such as resetting the force sensor 231. If the next release orientation has already been determined, the integrating unit 201a can execute a more accurate gravity compensation operation by setting the orientation of the end effector 220 to the same or close to the next release orientation and executing the gravity compensation operation. At this time, the integrating unit 201a may further execute a weight measurement of the gripped item by the force sensor 231.

[0120] In S3-13, the integration unit 201a controls the camera 232b to capture an image of the inside of the container 11b. The image processing unit 201b then determines the presence or absence of an object other than the item at the release position Pr and its periphery from the image captured by the camera 232b, and generates peripheral information indicating the size and position of the object and the position of the surface of the object.

[0121] In S3-14, the release plan generating unit 201e generates release information including a method for releasing an item being gripped by the robot 200, a release position, a release posture, a via position, a via posture, etc., using the gripped item information, information in the grip database, peripheral information, etc. The release plan generating unit 201e can further add information such as a pressing direction and magnitude of the pressing force of the end effector 220 involving force control, and a permissible movement range for force control, to the release information.

[0122] In S3-15, the integration unit 201a selects a release position and a release method, and the motion plan generation unit 201f generates robot motion information from the current position of the robot 200 to the release position and posture. Here, in an area where there is no risk of interference or collision with objects around the robot 200 or where the probability of interference or collision is relatively low, the integration unit 201a selects the fastest possible motion, taking into consideration gripped item information such as the weight of the gripped item, information in the gripping database, gripping state, and the like. On the other hand, in an area where there is a risk of interference or collision between the end effector 220 or the item and an object other than the item or where the probability of interference or collision is relatively high, such as when the end effector 220 or the gripped item approaches the container 11b or enters the container 11b, the integration unit 201a selects a slower motion. Furthermore, in this area, a target value of the force to be generated is given to the end effector 220 based on the release information, etc.

[0123] In S3-16, the robot control unit 201g controls the robot 200 and the end effector 220 to operate according to the generated robot operation information and release the item at the release position Pr and release posture. In order to arrange the items in the container 11b with higher density, if there is a wall of the container 11b or a sufficiently large load already placed around the arrival position of the items, the end effector 220 performs a pressing operation of the grasped item. In the peripheral area of ​​the release position Pr, if the end effector 220 or the grasped item hits an unexpected obstacle such as an object already placed, the robot control unit 201g executes a predetermined operation, for example, an operation of the end effector 220 escaping from the obstacle according to a target value of force. Even if the surface to be pressed has an unexpected inclination, the robot control unit 201g executes an operation of making the grasped item follow the surface to be pressed such as the wall of the container 11b by a predetermined pressing method or pressing force, and releasing the item in a state of close contact.

[0124] In S3-17, after releasing the item, the robot control unit 201g controls the end effector 220 to escape from the container 11b, and controls the robot 200 to assume a standby posture. At this time, the robot control unit 201g executes gravity compensation, such as resetting the detection value of the force sensor 231 (setting the detection value in this state to 0). Here, if the gripping posture of the next item has already been determined, the robot control unit 201g controls the robot 200 and the end effector 220 so that the end effector 220 assumes a calibration posture that is substantially the same as or close to the gripping posture, and executes gravity compensation in this state.

[0125] In S3-18, if the integration unit 201a receives a picking instruction for the next item from the external system 310 (N in S3-18), the procedure proceeds to S3-2 and starts a series of controls for the next item. In S3-18, if there is no next item (Y in S3-18), the series of controls ends.

[0126] Here, the process of measuring the shape of an item using a sensor 230 for gripping such as the camera 232a has been exemplified, but the shape of the item may be measured using a sensor 230 for temporary placement such as the camera 232d. In this case, the robot 200 moves the item gripped by the end effector 220 to a temporary placement position Pt. By installing devices such as a turntable, a scanner, and a weighing scale 233 at the temporary placement position Pt, it becomes possible to acquire data on the shape and weight of the item at the temporary placement position Pt. When the process of acquiring the item data is completed, the end effector 220 grips the item placed at the temporary placement position Pt, the robot 200 moves to the container 11b, and the end effector 220 releases it inside the container 11b, thereby completing a series of controls.

[0127] [Item information update process: Item categorization] In the operation in the photographing and registration mode, the shape and feature information of the item are extracted from the photographing information of the container 11a, but in such a situation, the information of the entire item may not be obtained due to occlusion caused by the wall of the container 11a or overlapping items. On the other hand, in the operation in the shape registration mode, it is possible to photograph and measure the overall information of the item alone, including the area hidden in the container 11a, when the item is lifted or placed in the temporary placement position Pt. Therefore, in the item information update process, the provisional information generated based on the information obtained in the operation in the photographing and registration mode is updated with the information obtained in the operation in the shape registration mode.

[0128] Fig. 13A shows the procedure of the item information update process. Fig. 13B shows an item model, and Fig. 13C shows a list of evaluation values. First, in S4-1, the data management unit 100a generates an item model as shown in Fig. 13B and a list of evaluation values ​​as shown in Fig. 13C from various data acquired by operation in the shape registration mode. Here, the solid line Cr in Fig. 13B indicates the circumscribed rectangular parallelepiped of the item, and the dot pattern area Mo indicates three-dimensional model data.

[0129] In S4-2, the data management unit 100a updates the list of evaluation values ​​shown in FIG. 11B to the list of evaluation values ​​shown in FIG. 13C in accordance with the modeled information. Here, the registration status of each gripping surface is recorded as "M" indicating that it has been modeled, and the size of the gripping surface is updated in accordance with the modeling results. In addition, weight information is recorded based on the measurement results, and shape information is registered as a box type. Furthermore, the evaluation value of each hand is recalculated and updated based on the modeled information.

[0130] In S4-3, the data management unit 100a updates the integrated item database. FIG. 14A shows item information in the integrated item database. The item type is registered as a BOX type based on the classification result after modeling, and the item weight is recorded as a measured value based on the measurement result and the confirmation result by modeling. The classification method will be described later. The data management unit 100a also updates the small item flag and the heavy item flag based on the modeled information. FIG. 14B shows end effector information in the integrated item database. The registration status of the gripping surface information is recorded as "M", which indicates that it has been updated by the modeled information. The data management unit 100a also updates the suitability of each end effector 220 according to the list of updated evaluation values. Here, the suitability of the gripping surface MDS1 is the result of actually performing gripping and succeeding in gripping, so the registration status is "G", which indicates successful gripping.

[0131] 15 shows an example of a procedure for determining an item type. First, in S5-1, the data analysis unit 100b acquires modeling data. In S5-2, the data analysis unit 100b acquires information about shape change from the modeling data and confirms the shape change. This shape change can be confirmed by measuring the shape change when gripping (when clamping, when suctioning) or transporting (when changing direction, when changing acceleration, when arriving at the end point), measuring the shape change during measurement in the shape registration mode, or by differences or changes in the results of multiple operations in the shape registration mode.

[0132] In S5-3, the data analysis unit 100b performs a determination as to the shape change. If it is determined in S5-3 that the shape change, for example the amount of deformation caused by a predetermined magnitude of external force, is equal to or greater than a threshold (Y in S5-4), the procedure proceeds to S5-4. In step S5-4, the item type is set to deformable, and the process ends.

[0133] On the other hand, if it is determined in S5-3 that the shape change is less than the threshold value (N in S5-4), the procedure proceeds to S5-5. In S5-5, the data analysis unit 100b performs shape confirmation or classification. Classification can be achieved by matching with a predetermined template, etc.

[0134] In S5-6, the data analysis unit 100b judges whether the shape is a substantially rectangular parallelepiped. If it is judged in S5-6 that the shape is a substantially rectangular parallelepiped, for example that each face is within a three-dimensional predetermined range close to a rectangular parallelepiped (Y in S5-6), the procedure proceeds to S5-7. In step S5-7, the data management unit 100a registers the item type as a box, and ends the determination of the item type.

[0135] If it is determined in S5-6 that the shape is not the above-mentioned predetermined rectangular parallelepiped (N in S5-6), the procedure proceeds to S5-8. In S5-8, the data management unit 100a sets the item type to solid, and ends the item type determination process.

[0136] In this specification, a "box" refers to an item whose shape is approximately a rectangular parallelepiped, and a "solid" refers to an item whose shape is difficult to change and is not approximately a rectangular parallelepiped. Also, a "deformable" refers to an item whose shape is not fixed, such as clothing or a soft bag. Here, in the case of a rectangular parallelepiped item such as a "box", a packing algorithm such as the BL method can be applied relatively easily. In addition, in the case of an item whose shape is difficult to change, such as a "solid", a packing plan can be calculated relatively easily using the measurement results and model information registered in a database. In addition, in the case of an item whose shape is not fixed, such as a "deformable", a packing process is required that takes into account that the item itself follows the shape of its surroundings. In this way, in the handling system 1 of this embodiment, by classifying items taking into account the packing after picking, a more efficient handling process can be realized. "Box", "solid", and "deformable" are examples of classifications according to the amount of deformation (ease of deformation) of an item against an external force of a predetermined magnitude and its external shape, respectively. As an example, the data analysis unit 100b here analyzes and classifies the items based on whether their external shape is approximately rectangular prism-like or not, but the shape is not limited to rectangular prism-like and the items may be analyzed and classified based on whether their external shape is another shape such as a thin rectangular plate, cylinder, or column, or the items may be classified by matching with various shapes.

[0137] 16 shows a procedure for determining flag information indicating attributes of an item. In S6-1, the data analysis unit 100b acquires modeled item information from the integrated item database.

[0138] Here, in S6-2, the data analysis unit 100b starts judging whether or not the target item is an item for which a small item flag should be set, based on the acquired information. First, in S6-3, the data analysis unit 100b judges the three side sizes of the circumscribed rectangular parallelepiped of the item. Here, if the size of each of the three sides and the total size are equal to or less than the size specified value (Y in S6-3), the procedure proceeds to S6-4. In S6-4, the data analysis unit 100b judges whether or not the weight of the item is equal to or less than the weight specified value. Here, if the weight of the item is equal to or less than the weight specified value (Y in S6-4), the data analysis unit 100b judges that the target item is a small item, and the data management unit 100a sets the small item flag of the item in the integrated item database to "1" in S6-5.

[0139] If the result of S6-3 is N or the result of S6-4 is N, that is, if the data analysis unit 100b determines that the size of the item is equal to or smaller than the size specification value, Not less than or the item's weight exceeds the weight limit Not less than If so, in S6-6, the data analysis section 100b determines that the target item is not a small item, and the data management section 100a sets the small item flag of the item in the integrated item database to "0."

[0140] Next, in S6-7, the data analysis unit 100b starts judging the fragile item flag. First, in S6-8, the data analysis unit 100b checks whether there is data that can be used to judge fragile items. Examples of data that can be used to judge fragile items include data from an external database such as a fragile item list, photo information of items that can be classified as fragile items, and information input by a user through an external user interface.

[0141] When the data analysis unit 100b determines that there is data that can be used to determine whether the target item is a fragile item (Y in S6-8), in S6-9, the data analysis unit 100b determines whether the target item is a fragile item. This determination can be made by directly using data acquired from an external source, or by using a classifier for fragile item determination using machine learning or the like, when primary data such as photographic information is used.

[0142] If the item is determined to be a fragile item in S6-9 (Y in S6-9), the data management unit 100a sets the fragile item flag of the item in the integrated item database to "1" in S6-10. On the other hand, if the item is determined not to be a fragile item in S6-9 (N in S6-9), the data management unit 100a sets the fragile item flag of the item in the integrated item database to "0" in S6-11.

[0143] Furthermore, if it is determined in S6-8 that there is no data available for determination (N in S6-8), the data management unit 100a sets the perishable item flag of the item in the integrated item database to "NA" in S6-12. Here, NA means that the item has not been determined.

[0144] Next, in S6-13, the data analysis unit 100b starts judging the heavy item flag. First, in S6-14, the data analysis unit 100b judges the three side sizes of the circumscribed rectangular parallelepiped of the item. Here, if the size of each of the three sides and the total size are equal to or larger than the size specified value (Y in S6-14), the procedure proceeds to S6-15. In S6-15, the data analysis unit 100b judges whether the weight of the item is equal to or larger than the weight specified value. If the weight of the item is equal to or larger than the weight specified value (Y in S6-15), in S6-16, the data analysis unit 100b judges that the target item is a heavy item, and the data management unit 100a sets the heavy item flag of the item in the integrated item database to "1", and the process ends.

[0145] If the answer is N in S6-14 or N in S6-15, that is, if the data analysis unit 100b determines that the size of the item is less than the specified size value or the weight of the item is less than the specified weight value, then in S6-17 the data analysis unit 100b determines that the item in question is not heavy, and the data management unit 100a sets the heavy item flag for that item in the integrated item database to "0" and completes the processing.

[0146] In this way, in the robot integrated management system 100, the picking robot system 2 can automatically improve processing efficiency by sequentially updating the information in the database using item information modeled based on information obtained by operation in the photography registration mode and operation in the shape registration mode.

[0147] In the robot integrated management system 100, the data in the database can be supplemented or updated by performing analysis in the data analysis unit 100b and performing database update processing by the operation unit 100c. As a result, the processing efficiency of the handling system 1 can be improved as the processing progresses. In addition, the above description mainly describes an example in which the handling system 1 automatically generates, supplements, and updates various information in the database, but the handling system 1 can also create and update the integrated management database 130 based on information obtained by direct input by the user via the service U100dI unit or information obtained from an external database or the like via the SoS_API100f.

[0148] [Integrated picking management of handling systems after database update] FIG. 17 shows the procedure for the robot integrated management system 100 to process an item that is in the “end effector determined state” in the integrated management database 130.

[0149] In S7-1, the robot integrated management system 100 receives a picking operation instruction from an external system 310 such as a WMS via the SoS_API 100f.

[0150] In S7-2, the operation unit 100c compares the received picking work instruction data with the information in the integrated management database 130 managed by the data management unit 100a. If the ID of the item for which the picking work instruction has been issued is an item that has already been registered in the robot integrated management database, the operation unit 100c checks the information of the registered item.

[0151] If the state of the item data is "end effector determined state" in S7-2, the data analysis unit 100b determines the end effector 220 that is optimal for the item to be picked and the picking robot system 2 that has the optimal end effector 220 in S7-3.

[0152] In S7-4, the operation unit 100c instructs the controller 201 of the picking robot system 2 determined to be optimal to perform picking in normal mode using the end effector 220 determined to be optimal.

[0153] In S7-5, while the robot 200 is performing a picking operation, the data management unit 100a updates the integrated management database 130 in accordance with the picking operation. For example, when picking is performed with a combination of a gripping surface and an end effector 220 with a registered status of "M" and the gripping is successful, the data management unit 100a updates the registered status from "M" to "G." In addition, even if the combination of the gripping surface and the end effector 220 is capable, if gripping fails multiple times, the compatibility is updated to incapable and the registered status is changed to "C" indicating that reconfirmation is required.

[0154] In S7-6, the robot integrated management system 100 executes an end process. Specifically, when all the ordered picking work instructions are completed, the robot integrated management system 100 issues an instruction to sweep out the container 11b to the material handling device 250. In addition, the robot integrated management system 100 notifies the outside of the completion of the picking work via the data management unit 100a, the business management unit 100e, and the SoS_API 100f. If the picking fails during the picking work, the robot integrated management system 100 notifies the external system 310 of the error via the data management unit 100a, the business management unit 100e, and the SoS_API 100f, or notifies the user of the error via the service / UI unit 100d.

[0155] [Operation in normal mode] FIG. 18 shows a processing procedure of the picking robot system 2 in the normal mode.

[0156] First, in S8-1, the integration unit 201a receives an item transport instruction (picking instruction) from the external system 310 via the external interface.

[0157] In S8-2, the integration unit 201a detects the arrival of the container 11a at the gripping position Ph by a sensor 230 such as a camera 232a, a photoelectric sensor, a microswitch, or the like.

[0158] In S8-3, the integration unit 201a controls the camera 232a to capture an image of the grip position Ph of the item and its surroundings, for example, the inside of the container 140a in which the item is stored.

[0159] In S8-4, the image processor 201b determines the presence or absence of an item and identifies the grippable surface of the item from the image captured by the camera 232a. Furthermore, the image processor 201b calculates gripping surface information such as the shape, size, and position and orientation in three-dimensional space of the grippable surface of the item.

[0160] In S8-5, the grip plan generating unit 201d calculates a plurality of pieces of grip information for the robot 200 to grip the grippable surface of the item based on the grip surface information. The grip information includes grip position information and grip posture information. Here, the grip plan generating unit 201d calculates additional information such as the movement direction of the end effector 220, the magnitude of the pressing force, and the allowable range of movement when performing the force control described later, based on peripheral information of the grip position Ph such as objects (peripheral parts) other than the item in the container 11a, the grip surface information, and information in the item database, and adds this to the grip information.

[0161] In S8-6, the integration unit 201a selects or determines an item to be grasped, its grasp position Ph, and its grasp posture based on the grasp information. Then, the motion plan generation unit 201f generates robot motion information from the current position of the robot 200 to the grasp position Ph and the grasp posture. Here, the integration unit 201a selects a faster motion in an area where there is no risk of interference or collision between the robot 200 and objects other than the item, or where the probability of interference or collision is relatively low. On the other hand, the integration unit 201a selects a slower motion in an area where there is a risk of interference or collision between the end effector 220 or the item and objects other than the item, or where the probability of interference or collision is relatively high, for example, when the end effector 220 approaches the container 11a or enters the container 11a. Furthermore, the motion plan generation unit 201f determines a target value of the force generated by the end effector 220 in this area.

[0162] In S8-7, the robot control unit 201g controls the robot 200 and the end effector 220 to operate according to the generated robot operation information and grasp the item at the grasping position Ph and grasping posture. In the peripheral area of ​​the grasping position Ph of the item, the pressing operation of the end effector 220 is performed by only force control, and the grasping operation such as suction or pinching is performed in a state in which the suction pad 220a and the finger 220c are in sufficient contact with the item. In the peripheral area of ​​the grasping position Ph, if the end effector 220 hits an obstacle such as an object other than the item, the robot control unit 201g executes a predetermined operation, for example, an operation of the end effector 220 escaping from the obstacle according to a target value of force. Also, even if the surface to be grasped has an unexpected inclination, the robot control unit 201g executes an operation of making the end effector 220 follow the surface to be grasped by using a predetermined pressing method or pressing force.

[0163] The robot control unit 201g and the integration unit 201a monitor the operation of the robot 200 and the state of gripping of the item by the end effector 220. If the end effector 220 succeeds in gripping the item in S8-8 (Y in S1-8), the procedure proceeds to S8-10. On the other hand, if the end effector 220 fails to grip the item in S8-8 (N in S1-8), the procedure proceeds to S8-9.

[0164] In S8-9, the integration unit 201a executes retry preparation operations such as a retreat operation of the robot 200 and a registration process for registering information indicating a gripping failure in the item database, and after S8-9, the procedure returns to S8-3.

[0165] In S8-10, if the posture of the item to be grasped has not been determined in S8-3, the robot control unit 201g controls the robot 200 to move to a position where an image of the state of the item being grasped by the end effector 220 can be captured. Then, the integration unit 201a controls the camera 232b to capture an image of the end effector 220 grasping the item at that position.

[0166] In S8-11, the integration unit 201a acquires the relative position and relative orientation between the end effector 220 and the item based on images captured by the grasping camera 232a and the calibration camera 232b, and the shape information of the item registered in the integrated management database 130.

[0167] In S8-12, the integrating unit 201a executes a gravity compensation operation such as resetting the force sensor 231. If the next release orientation has already been determined, the integrating unit 201a can execute a more accurate gravity compensation operation by setting the orientation of the end effector 220 to the same or close to the next release orientation and executing the gravity compensation operation. At this time, the integrating unit 201a may further execute a weight measurement of the gripped item by the force sensor 231.

[0168] In S8-13, the integration unit 201a controls the camera 232b to capture an image of the inside of the container 11b. The image processing unit 201b then determines the presence or absence of an object other than the item at the release position Pr and its periphery from the image captured by the camera 232b, and generates peripheral information indicating the size and position of the object and the position of the surface of the object.

[0169] In S8-14, the release plan generating unit 201e generates release information including a method for releasing an item being gripped by the robot 200, a release position, a release posture, a via position, a via posture, etc., using the gripped item information, information in the grip database, peripheral information, etc. The release plan generating unit 201e can further add information such as a pressing direction and magnitude of the pressing force of the end effector 220 involving force control, and a permissible movement range for force control, to the release information.

[0170] In S8-15, the integration unit 201a selects a release position and a release method, and the motion plan generation unit 201f generates robot motion information from the current position of the robot 200 to the release position and posture. Here, in an area where there is no risk of interference or collision with objects around the robot 200 or where the probability of interference or collision is relatively low, the integration unit 201a selects the fastest possible motion, taking into consideration gripped item information such as the weight of the gripped item, information in the gripping database, gripping state, and the like. On the other hand, in an area where there is a risk of interference or collision between the end effector 220 or the item and objects other than the item or where the probability of interference or collision is relatively high, such as when the end effector 220 or the gripped item approaches the container 11b or enters the container 11b, the integration unit 201a selects a slower motion. Furthermore, in this area, a target value of the force to be generated is given to the end effector 220 based on the release information, etc.

[0171] In S8-16, the robot control unit 201g controls the robot 200 and the end effector 220 to operate according to the generated robot operation information and release the item at the release position Pr and release posture. In order to arrange the items in the container 11b with higher density, if there is a wall of the container 11b or a sufficiently large load already placed around the arrival position of the items, the end effector 220 performs a pressing operation of the grasped item. In the peripheral area of ​​the release position Pr, if the end effector 220 or the grasped item hits an unexpected obstacle such as an object already placed, the robot control unit 201g performs a predetermined operation, for example, an operation of the end effector 220 escaping from the obstacle according to a target value of force. Even if the surface to be pressed has an unexpected inclination, the robot control unit 201g performs an operation of making the grasped item follow the surface to be pressed such as the wall of the container 11b by a predetermined pressing method or pressing force, and releasing the item in a state of close contact.

[0172] In S8-17, after releasing the item, the robot control unit 201g controls the end effector 220 to escape from the container 11b, and controls the robot 200 to assume a standby posture. At this time, the robot control unit 201g executes gravity compensation, such as resetting the detection value of the force sensor 231 (setting the detection value in this state to 0). Here, if the gripping posture of the next item has already been determined, the robot control unit 201g controls the robot 200 and the end effector 220 so that the end effector 220 assumes a calibration posture that is substantially the same as or close to the gripping posture, and executes gravity compensation in this state.

[0173] In S8-18, if the integration unit 201a receives a picking instruction for the next item from the external system 310 (N in S8-18), the procedure proceeds to S8-2 and starts a series of controls for the next item. In S8-18, if there is no next item (Y in S8-18), the series of controls ends.

[0174] [Online use of databases in an integrated system: end effector selection] 19 shows a procedure for determining an optimal end effector 220 by the robot integrated management system 100. In S9-1, the operation unit 100c determines whether or not the end effector 220 is connected to the robot 200 of the target picking robot system 2.

[0175] If the end effector 220 is connected to the robot 200 (Y in S9-1), in S9-2, the operation unit 100c checks whether the end effector 220 connected to the robot 200 can process the next item to be processed among the items specified in the picking order. If the end effector 220 is primary or capable for the target item, the operation unit 100c determines that it can process the target item, and issues an instruction to the picking robot system 2 to execute picking using the end effector 220 (S9-3).

[0176] If it is confirmed that the end effector 220 is not connected to the robot 200 (N in S9-1), or if the end effector 220 connected in S9-2 is not primary or capable (N in S9-2), the procedure proceeds to S9-4. In S9-4, the data analysis unit 100b first picks out the primary end effector and capable end effector for the items that have not yet been processed among the items specified in the picking order from the end effector information in the integrated item database. Then, the data analysis unit 100b determines whether there is a primary end effector that can process all the specified items.

[0177] If it is determined in S9-4 that there is a primary end effector capable of processing all items (Y in S9-4), the procedure proceeds to step S9-5. In S9-5, the operation unit 100c instructs the picking robot system 2 to connect the end effector 220 determined to be the primary end effector, and confirms the completion of the connection. Then, the operation unit 100c instructs the picking robot system 2 to perform picking using the end effector 220 (S9-9).

[0178] On the other hand, if it is found in S9-4 that there is no end effector 220 capable of processing all the items (N in S9-4), the procedure proceeds to S9-6. In S9-6, the data analyzer 100b determines whether there is a capable end effector that can process all the unprocessed items specified in the picking order.

[0179] In S9-6, if it is found that there is a capable end effector capable of processing all items (Y in S9-6), the procedure proceeds to S9-7. In S9-7, the operation unit 100c selects the end effector 220 with the highest priority, such as the one with the largest number of determinations as a primary end effector, from among the end effectors 220 capable of processing all unprocessed items among the items specified in the picking order, and issues a connection instruction to the picking robot system 2 to connect the end effector 220 with the highest priority to the robot 200. The operation unit 100c also instructs the picking robot system 2 to perform picking using the connected end effector 220 (S9-9).

[0180] If it is found in S9-7 that there is no capable end effector capable of processing all items (Y in S9-7), the procedure proceeds to S9-8. In S9-8, the operation unit 100c instructs the picking robot system 2 to connect the primary end effector of the item to be processed next in the picking order. Then, the operation unit 100c instructs the picking robot system 2 to perform picking using the end effector 220 (S9-9).

[0181] FIG. 20A shows an example of a picking order, and FIG. 20B shows an example of primary end effectors and capable end effectors for unprocessed items specified in the picking order based on end effector information in the integrated item database.

[0182] When S9-4 is executed in the state of Fig. 20B, there is no primary end effector capable of processing all four types of products, so the process proceeds to S9-6. In S9-6, it is determined that the end effector 220 of type B is the end effector 220 capable of processing all four types of items. In this case, in S9-7 and S9-10, the operation unit 100c instructs the picking robot system 2 to connect the end effector 220 (type B) to the robot 200 and to perform picking using the connected end effector 220 of type B.

[0183] In this way, the robot integrated management system 100 utilizes information from the integrated item database to switch the end effector 220 connected to the picking robot system 2 (robot 200), thereby minimizing the number of times the end effector 220 is switched, even when processing multiple items, and enabling more efficient processing.

[0184] [Using database information in picking robots: Grasping process] FIG. 21 shows a procedure for switching parameters for the item grasping process by the picking robot system 2 using a database.

[0185] When performing an object handling process such as picking, the picking robot system 2 (controller 201) updates the internal database 201k in the controller 201 using the latest information in the integrated item database of the robot integrated management system 100. Then, the controller 201 utilizes the information in the updated internal database 201k when performing an object handling process such as picking.

[0186] In S10-1, when starting an item gripping operation, the integration unit 201a judges the fragile item flag of the target item based on the item information in the internal database 201k.

[0187] In S10-1, if the fragile item flag is "1" (N in S10-1), in S10-2, the robot control unit 201g controls the suction force by switching the suction pressure, and sets the pressing force of the suction pad 220a on the item or the clamping pressure when the item is clamped by the fingers 220c to the minimum value within the range in which the item can be held, and continues the item gripping operation.

[0188] On the other hand, if the easily damaged item flag is "0" in S10-1 (Y in S10-1), the integration unit 201a determines the item type from the item information in the internal database 201k in S10-3.

[0189] If the item type is determined to be a "box" in S10-3, the procedure proceeds to S10-4; if the item type is determined to be a "solid", the procedure proceeds to S10-5; and if the item type is determined to be a "deformable" the procedure proceeds to S10-6.

[0190] In S10-4, the robot control unit 201g sets the target value of the pressing force of the suction pad 220a against the item or the insertion force of the fingers 220c that pinch the item to a relatively large value and continues the item gripping operation. This is because if the item is a box, the item is placed in a stable state, and even if the end effector 220 is pressed against it with a certain amount of force, there is a low possibility that the item will break or the placed item will collapse.

[0191] In S10-5, the robot control unit 201g sets the target value of the pressing force of the suction pad 220a against the item or the insertion force of the fingers 220c that pinch the item to a relatively small value, in other words, a value smaller than that in S10-4, and continues the item gripping operation. This is because if the item is solid, the placement state of the item is unstable, and there is a risk that the placed item will collapse if it is pushed in with too much force.

[0192] In S10-6, the robot control unit 201g sets the target value of the pressing force of the suction pad 220a against the item or the insertion force of the fingers 220c that pinch the item to approximately 0 (zero), and continues the item gripping operation. This is because if the item is deformable, pressing it with a strong force will change the shape of the item, and the pressing effect will not be achieved.

[0193] In this way, the picking robot system 2 of this embodiment is able to execute object handling processing with more effective parameters by utilizing item type data from the database, and by using flag data such as a fragile item flag, it is possible to prevent inconveniences such as damage to items that occur during the handling processing.

[0194] [Use of database information in picking robots: box packing process] FIG. 22 shows the procedure for packing items into boxes by the picking robot system 2 using a database.

[0195] When performing an object handling process such as picking, the picking robot system 2 (controller 201) updates the internal database 201k in the controller 201 using the latest information in the integrated item database of the robot integrated management system 100. Then, the controller 201 utilizes the information in the updated internal database 201k when performing an object handling process such as boxing.

[0196] In S11-1, when starting to pack items, the integration unit 201a determines the item type of the target item from the item information in the internal database 201k.

[0197] In S11-1, if the item type is determined to be a "box", the procedure proceeds to S11-2; if it is determined to be a "solid", the procedure proceeds to S11-3; and if it is determined to be a "deformable", the procedure proceeds to S11-4.

[0198] In S11-2, the integration unit 201a sets parameters of an algorithm for packing the item determined to be a box into the container 11b. The algorithm takes into consideration that the item is box-shaped, and for example, a packing algorithm such as the BL method is used. This algorithm calculates a more appropriate packing position and places the item at the packing position by a pressing operation such as force control (first packing mode).

[0199] In S11-3, the integration unit 201a sets parameters for an algorithm for packing an item determined to be solid into the container 11b. The algorithm takes into consideration that the item is not box-shaped, and for example, calculates a packing position where the target object will fit using circumscribed rectangular parallelepiped information of the item, and places the item by pressing it near the calculated packing position using a pressing operation such as force control (second packing mode).

[0200] In S11-4, the integration unit 201a sets parameters for an algorithm for packing an item determined to be deformable into a deformable area in the container 11b. The algorithm takes into consideration that the item has a deformable shape, and for example, calculates a packing position where the target object fits using circumscribed rectangular parallelepiped information of the item, and places the item in the packing position by position control that does not rely on force control, so that no force is applied to the packing position or its surroundings (third packing mode).

[0201] The pressing force in the third packing mode is less than the pressing force in the second packing mode, which in turn is less than the pressing force in the first packing mode. The difference in pressing force is the difference in acceleration of the moving item. Thus, the acceleration at the end of the transfer in the third packing mode is less than the acceleration at the end of the transfer in the second packing mode, and the acceleration at the end of the transfer in the second packing mode is less than the acceleration at the end of the transfer in the first packing mode.

[0202] In S11-5, the integration unit 201a checks whether or not there is an item already placed inside the container 11b, and if there is an item, obtains the position information and orientation information of the existing item. This position information and orientation information are referred to as existing item information.

[0203] In S11-6, the integration unit 201a judges the small item flag for the item to be packed from the item information in the internal database 201k. If the small item flag is "1" (N in S11-6), the procedure proceeds to S11-7. If the small item flag is "0" (Y in S11-6), the procedure proceeds to S11-8.

[0204] In S11-7, the integration unit 201a overwrites the information set in the predetermined storage area in S11-2, S11-3, or S11-4 with information for the boxing process dedicated to small items.

[0205] In S11-10, the integration unit 201a searches for an area where small items can be placed based on the information overwritten in S11-7. Here, the search based on information dedicated to small items means a search under the condition that, for example, an item with a small item flag of "1" can be superimposed on an existing item in the deformable area even if the item type is a box.

[0206] In S11-8, the integration unit 201a judges the heavy item flag of the target item from the item information in the internal database 401k. If the heavy item flag is "0" (Y in S11-8), the procedure proceeds to S11-9. On the other hand, if the heavy item flag is "1" (N in S11-8), the procedure proceeds to S11-13.

[0207] In S11-9, the integration unit 201a checks whether or not there is a fragile item among the existing items from the existing item information. If there is a fragile item (S11-9), the answer is Y, and the procedure proceeds to S11-11. On the other hand, if there is no fragile item (N in S11-9), the procedure proceeds to S11-12.

[0208] In S11-11, the integration unit 201a searches for an item placement position by the set packing algorithm in an area that does not come into contact with fragile items among the areas specified in S11-2, S11-3, or S11-4. This process of S11-11 is a process for preventing the destruction of existing items that are fragile items by placing or pushing the item to be placed on top of the existing items.

[0209] In S11-12, the integration unit 201a searches for a location to place an item in the area specified in S11-2, S11-3, or S11-4 using the set packing algorithm.

[0210] In S11-13, the integration unit 201a searches for an installation position of an item by the set packing algorithm for an area where no existing items exist among the areas specified in S11-2, S11-3, or S11-4. The process of S11-13 is a process for preventing the existing items from being damaged by placing a heavy item on top of the existing items.

[0211] In S11-14, the integration unit 201a checks whether the search in S11-10, S11-11, S11-12, or S11-13 was successful or unsuccessful. If the search is successful and the item placement position has been determined (Y in S11-14), the procedure proceeds to S11-15. On the other hand, if the search is unsuccessful and the item placement position has not been determined (N in S11-14), the packing process is temporarily stopped and the process proceeds to the next step, such as switching the container 11b.

[0212] In S11-15, the integration unit 201a packs the items into the installation positions found using the parameters suitable for each item set in S11-2, S11-3, S11-4, or S11-7, and completes the packing process.

[0213] In this way, the picking robot system 2 of this embodiment is able to execute object handling processing with more effective parameters by utilizing item type data from the database, and by using flag data such as a fragile item flag, it is possible to prevent inconveniences such as damage to items that occur during the handling processing.

[0214] Figure 23A shows an item 10a with an item type of a box, e.g., a cardboard or cardboard box, Figure 23B shows an item 10b with an item type of a solid, e.g., a liquid container or a blister pack, Figure 23C shows an item 10c with an item type of a deformable, e.g., an article of clothing or a refill container.

[0215] Also, Fig. 23D is a plan view of the container 11b when the picking robot system 2 packs items according to the flow of Fig. 22. As shown in Fig. 23D, in the container 11b, the items 10a in Fig. 23A, the items 10b in Fig. 23B, and the items 10c in Fig. 23C are grouped by type and arranged in different positions so as not to interfere with each other. Thus, according to the handling system 1 of this embodiment, more efficient packing can be achieved by packing operations that utilize information on the characteristics of the items.

[0216] [Offline use of databases in an integrated system: On-site improvement prediction] 24 shows the procedure for the logistics system improvement proposal process. In S12-1, the data management unit 100a updates the integrated management database 130 with new robot data and end effector data acquired through the CAE function of the business management unit 100e. The data analysis unit 100b acquires the new robot data and end effector data.

[0217] In S12-2, the data analysis unit 100b acquires item information and the like from the integrated item database, which is necessary for evaluating the compatibility of the end effector 220.

[0218] In S12-3, the data analysis unit 100b uses the acquired item information and end effector data to evaluate the compatibility of the new end effector 220 with multiple items registered in the integrated item database, and updates the end effector information table in the integrated item database for each item, as in the case of Figure 14B.

[0219] In S12-4, the data analysis unit 100b judges whether each registered item can be processed or not from the updated end effector information table. In this case, if there is even one surface judged as primary or capable among multiple gripping surfaces, the item may be judged as a processable item, or the ratio of the number of processable gripping surfaces to the total number of gripping surfaces may be used as the result of the judgment.

[0220] In S12-5, the data analysis unit 100b counts the number of items determined to be processable by the new end effector 220 and the number of items determined to be inoperable among the items registered in the database or the items handled within a specified period such as the current day or the most recent year from the operation status database based on the determination result of S12-4, and calculates the ratio of items that can be processed using the new end effector 220 to the total number of items. This ratio is called the estimated item coverage rate.

[0221] In S12-6, the data analysis unit 100b compares the estimated item coverage rate for the new end effector 220 with the estimated item coverage rate for the existing end effector 220 to calculate the improvement rate of the processable item ratio when the new end effector 220 is introduced.

[0222] In S12-7, the service UI unit 100d displays the results calculated in S12-5 and S12-6 on a UI (display unit) such as a display. The SoS_API 100f also notifies the external system 310 or the higher-level system 320 of the results calculated in S12-5 and S12-6, and completes the process.

[0223] FIG. 25 shows a display example in the UI as the logistics system improvement proposal function. As shown in FIG. 25, the image Im shows the result of the logistics system improvement proposal function processing for end effector C, which is a new end effector 220. In the table in the image Im, the first column shows the processing lines on which the handling system 1 is performing processing, and the second column shows the end effectors 220 that can be selected by the object handling robots used on each line. Here, it can be seen that the robot 200 that can be selected as end effector A is arranged on line 1. Similarly, it can be seen that the picking robot system 2 (robot 200) that can switch and select between end effector A and end effector B is arranged on line 2.

[0224] Moreover, the third column of the table shows the estimated ratio of processable items to the items flowing on each line, estimated by the data analysis unit 100b of the robot integrated management system 100. The fourth column shows the actual ratio of items successfully processed based on the actual operating status of the items flowing on the line. The difference between the third and fourth columns is the difference between the estimate and the actual result.

[0225] Moreover, the fifth column of the table indicates the number of items that were actually processed, and the sixth column indicates the installation status of end effector 220 on each line. Moreover, the fourth row indicates the processing results of the logistics system improvement proposal function when end effector C is newly introduced to line 2. From these results, it can be seen that when end effector C is introduced to line 2, the ratio of items that can be processed automatically will increase from the current approximately 80% to approximately 88%. It can also be seen that the actual number of items processed by end effector C is estimated to be 900.

[0226] In this way, the handling system 1 can estimate the introduction effect of a new end effector 220 before actually introducing it by using the logistics system improvement proposal function of the robot integrated management system 100. Furthermore, by displaying the result on the UI, it becomes possible to effectively communicate with customers.

[0227] As described above, according to this embodiment, the handling system 1 can process a new item (object 10) more appropriately by automatically measuring and analyzing the new item even when data for the new item is not available.

[0228] Furthermore, according to this embodiment, the handling system 1 operates in a shape registration mode (second mode) in which the item is measured while being grasped, thereby making it possible to measure changes in the actual state of the item caused by grasping, thereby making it easier to handle the item more appropriately or with greater accuracy.

[0229] Furthermore, according to this embodiment, the handling system 1 operates in a shape registration mode in which the item is measured while being phased, thereby making it possible to measure changes in the actual state of the item caused by the phase, which makes it easier to process the items more appropriately or with greater accuracy, and also enables a higher throughput compared to a system that simply measures the items without transporting them.

[0230] Furthermore, according to this embodiment, the data analysis unit 100b can classify items according to their ease of deformation, for example, the amount of deformation caused by an external force of a predetermined magnitude, and can pack the items in a box according to the classification, thereby obtaining a less inconvenient storage state for the items in the container 11b, for example, a storage state that causes less damage to the items.

[0231] Furthermore, according to this embodiment, the data analysis unit 100b classifies items according to their external shape, for example, whether or not they are approximately rectangular, for example, whether or not the outer surface fits within a specified area of ​​an approximately rectangular parallelepiped, and can perform packing according to the classification. Therefore, it is possible to obtain a less inconvenient storage state of the items in the container 11b, for example, a storage state with a higher concentration of items.

[0232] Furthermore, in this embodiment, the robot integrated management system 100 (robot management system) is shared by multiple picking robot systems 2 (robot systems), so a handling system 1 with higher processing efficiency can be realized compared to a system in which each picking robot system 2 measures and analyzes items.

[0233] In the embodiment, the robot integrated management system 100, the master device 110, the slave device 120, and the controller 201 are each configured with one or more computers. The arithmetic processing and control of the robot integrated management system 100, the master device 110, the slave device 120, and the controller 201 may be executed by software or by hardware. Furthermore, the arithmetic processing and control by the robot integrated management system 100, the master device 110, the slave device 120, and the controller 201 may include arithmetic processing and control by software and arithmetic processing and control by hardware.

[0234] In the case of processing by software, the robot integrated management system 100, the master device 110, the slave device 120, and the controller 201 are each configured by a computer 1000 as shown in Fig. 26. The computer 1000 has at least a processor 1001 (circuit), a RAM 1002, a ROM 1003, and an auxiliary storage device 1004 such as a HDD or SSD, and the processor 1001 reads and executes a program (application) stored in the ROM 1003 or the auxiliary storage device 1004. The processor 1001 operates according to the program to operate as each functional block included in the robot integrated management system 100, the master device 110, the slave device 120, and the controller 201. In this case, the program includes a program module corresponding to each of the above-mentioned functional blocks.

[0235] The programs may be provided in the form of installable or executable files recorded on a recording medium readable by computer 1000, such as a CD-ROM, FD, CD-R, DVD, or USB memory. The recording medium may also be referred to as a program product. The programs may be stored in a storage unit of a computer connected to a communications network, and may be introduced into computer 1000 by being downloaded via the network. The programs may also be incorporated in advance into ROM 1003 or the like.

[0236] Furthermore, when at least a portion of the computer 1000 is configured from hardware, the computer 1000 may include, for example, a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC).

[0237] Although not shown, external interfaces and various UI devices can be connected to the computer 1000 as appropriate and used.

[0238] In the robot integrated management system 100 of the above embodiment, the master device 110 functions as the data management unit 100a, the data analysis unit 100b, the service / UI unit 100d, the business management unit 100e, and the SoS_API 100f, and the slave device 120 functions as the operation unit 100c. However, the division of functions in the robot integrated management system 100 is not limited to this embodiment. In addition, the robot integrated management system 100 may be configured by one computer, or may be configured by three or more computers.

[0239] Although the embodiment of the present invention has been illustrated above, the embodiment is merely an example and is not intended to limit the scope of the invention. The embodiment can be implemented in various other forms, and various omissions, substitutions, combinations, and modifications can be made without departing from the gist of the invention. These embodiments are included in the scope and gist of the invention, and are included in the scope of the invention and its equivalents described in the claims. In addition, the specifications of each configuration and shape (structure, type, direction, model, size, length, width, thickness, height, number, arrangement, position, material, etc.) can be appropriately changed and implemented. For example, the handling system can be applied to handling systems other than the object handling system that transports an object from one container to another container described above.

[0240] The inventions described in the claims of the original patent application are set forth below.

[0241] [Technical proposal 1] a robot system including a robot having a movable part and an end effector attached to the movable part and configured to grasp an object, and configured to grasp and move the object, and a sensor that measures the object, and capable of processing the object in a plurality of operation modes; a data analysis unit that analyzes first data of the object obtained by measuring the object with the sensor and calculates second data of the object; a database that stores the second data of the object in association with a status of the data of the object that is determined corresponding to the operation mode of the robot system when the first data of the object is obtained; an operation instruction unit that gives an operation instruction to the robot system that processes the object, in the operation mode determined according to the status of the object stored in the database, according to an operation command determined according to the second data of the object; A handling system equipped with

[0242] [Technical proposal 2] The handling system described in Technical Proposal 1, wherein the multiple operating modes include a first mode in which the sensor measures the object not grasped by the end effector, a second mode in which the sensor measures the object grasped by the end effector, or a third mode in which the robot transports the object.

[0243] [Technical proposal 3] the status of the object is set to a first state if the first data of the object was obtained in the first mode; the status of the object is set to a second state if the first data of the object was obtained in the second mode; When the status of the object stored in the database is the first state, the operation instruction unit issues the operation instruction to process the object in the second mode; A handling system as described in Technical Proposal 2, wherein when the status of the object stored in the database is the second state, the operation instruction unit gives the operation instruction to process the object in the third mode.

[0244] [Technical proposal 4] The handling system according to technical proposal 2 or 3, wherein in the second mode, the robot transports the object grasped by the end effector.

[0245] [Technical proposal 5] The handling system according to Technical Solution 4, wherein the moving speed of the object in the second mode is slower than the moving speed of the object in the third mode.

[0246] [Technical proposal 6] The handling system according to any one of Technical Schemes 1 to 5, wherein the data analysis unit classifies the object according to the amount of deformation caused by an external force.

[0247] [Technical proposal 7] The handling system described in Technical Proposal 6, wherein the data analysis unit classifies the objects according to their external shapes.

[0248] [Technical proposal 8] The handling system described in Technical Proposal 6 or 7, wherein the robot transports the objects so that objects of the same classification are gathered together.

[0249] [Technical proposal 9] The robot is capable of changing the acceleration when the object arrives at an end point of the transfer; A handling system described in any one of Technical Proposals 6 to 8, wherein the operation command is set so that the acceleration or a control target value related to the acceleration differs depending on the classification.

[0250] [Technical proposal 10] A plurality of robots having different end effectors are connected, The handling system according to any one of Technical Proposals 6 to 9, wherein the operation command specifies a predetermined robot according to the second data.

[0251] [Technical proposal 11] The robot is capable of replacing a plurality of the end effectors; The handling system according to any one of technical proposals 6 to 10, wherein the operation command specifies the end effector selected according to the classification.

[0252] [Technical proposal 12] A handling system described in any one of Technical Proposals 1 to 11, wherein the data analysis unit predicts the processing performance of the first end effector based on specification data of a first end effector that is an unused end effector for the object, specification data of a second end effector that is an end effector that has been used for the object, and processing performance data of the second end effector.

[0253] [Technical proposal 13] A robot management system included in a handling system according to any one of technical proposals 1 to 12, A robot management system comprising the data analysis unit and the operation instruction unit.

[0254] [Technical proposal 14] A robot management system as described in technical proposal 13, having the database.

[0255] [Technical proposal 15] A robot management system according to technical proposal 13 or 14 for controlling a plurality of the robot systems.

[0256] [Technical proposal 16] A robot system included in a handling system according to any one of technical proposals 1 to 12, A robot system comprising the robot and the sensor.

[0257] [Technical proposal 17] a robot having a movable part and an end effector attached to the movable part and configured to grasp and move an object; A sensor that measures the object; Equipped with A robotic system operable in a second mode in which the robot transports the object while the sensor measures the object grasped by the end effector, and a third mode in which the robot transports the object at a higher speed than in the second mode.

[0258] [Technical proposal 18] The robot system described in Technical Proposal 17 further comprises a first mode in which the sensor is operable to measure the object not being grasped by the end effector. [Explanation of symbols]

[0259] 1... handling system, 2... picking robot system (robot system), 10... object, 10a, 10b, 10c... items (objects), 100... robot integrated management system (robot management system), 100b... data analysis section, 100c... operation section (operation instruction section), 130... integrated management database (database), 200... robot, 210... arm (moving section), 220... end effector, 230... sensor.

Claims

1. a robot system including a robot having a movable part and an end effector attached to the movable part and configured to grasp an object, the robot gripping and moving the object, and a sensor configured to measure the object; and the robot system being capable of processing the object; a database that stores an item ID that identifies the object and item information that indicates characteristics of the object in association with each other; a plan generating unit that generates robot operation information representing operations and parameters of the robot; a robot control unit that controls the robot in accordance with the generated robot operation information; Equipped with When the robot is caused to grasp the object, the plan generation unit Calculating a gripping position and a gripping orientation that represent a position and an orientation of the robot at which the robot grips the object based on the position and the orientation of the object; generating robot operation information including a path of the robot from a current position and posture of the robot to the gripping position and the gripping posture; The item information includes an item type indicating a shape of the object and a degree of ease of shape change; When the robot is caused to grasp a first object which is one of the objects, the robot control unit: A target value of a force to be applied to the first object when the first object is grasped is set based on at least one of a shape of the item type of the first object stored in the database in correspondence with an item ID of the first object and a degree of ease of shape change. Handling system.

2. further identifying whether the item type is amorphous; When causing the robot to grasp the first object, the robot control unit If the item type of the first object indicates the shapeless, set the target value to 0. Handling system according to claim 1 .

3. When the robot gripping the object is caused to release the object, The plan generating unit calculating a release position and a release orientation representing a position and orientation at which the object is released; generating robot motion information representing motions and parameters of the robot from a current position and posture of the robot to the release position and the release posture; When causing the robot gripping the first object to release the first object, the robot control unit: When the item type of the first object indicates that the shape of the object is a rectangular parallelepiped, an acceleration of the movement of the first object immediately before the first object arrives at the release position is defined as a first acceleration; when the item type of the first object indicates an object that is difficult to change shape, an acceleration of the movement of the first object immediately before the first object arrives at the release position is set to a second acceleration that is smaller than the first acceleration; When the item type of the first object indicates an object with an indefinite shape, an acceleration of the movement of the first object immediately before the first object arrives at the release position is set to a third acceleration that is smaller than the second acceleration. Handling system according to claim 2.

4. a data analysis unit that acquires data obtained by measuring the object by the sensor when the robot is made to perform at least one of an action for grasping the object, an action for transporting the object, and an action for releasing the object, and analyzes the acquired data to generate the item type of the object; a data management unit that updates the database based on the generated item type; The handling system according to claim 2 or 3, further comprising:

5. The item information includes a fragile item flag indicating whether the object has been determined to be a fragile item based on previously input information, When causing the robot to grasp the first object, the robot control unit When the fragile item flag of the item information of the first object indicates that the first object is the fragile item, the first object is held with a minimum force within a range in which the first object can be held, regardless of the item type.

5. A handling system according to any one of claims 1 to 4.

6. The item information further includes a small item flag indicating whether the sum of the three side sizes of a circumscribing rectangular parallelepiped of the object is equal to or less than a specified value and the weight of the object is equal to or less than a specified value, or whether the sum of the three side sizes of a circumscribing rectangular parallelepiped of the object is equal to or less than a specified value and the weight of the object is equal to or more than a specified value.

6. A handling system according to claim 5.

7. A control device connected to a robot having a movable part and an end effector attached to the movable part and configured to grasp an object, the robot grasping and moving the object, and a sensor configured to measure the object, a database that stores an item ID that identifies the object and item information that indicates characteristics of the object in association with each other; a plan generating unit that generates robot operation information representing operations and parameters of the robot; a robot control unit that controls the robot in accordance with the generated robot operation information; With When the robot is caused to grasp the object, the plan generation unit Calculating a gripping position and a gripping orientation that represent a position and an orientation of the robot at which the robot grips the object based on the position and the orientation of the object; generating robot operation information including a path of the robot from a current position and posture of the robot to the gripping position and the gripping posture; The item information includes an item type indicating a shape of the object and a degree of ease of shape change; When the robot is caused to grasp a first object which is one of the objects, the robot control unit: A target value of a force to be applied to the first object when the first object is grasped is set based on at least one of a shape of the item type of the first object stored in the database in correspondence with an item ID of the first object and a degree of ease of shape change. Control device.

8. The control device according to claim 7 , which controls a plurality of the robots.

9. A robot management system for managing a robot system, The operations department and a data analysis unit that performs data analysis necessary for the operation unit to execute an operation; having the robot system includes a robot having a movable part and an end effector attached to the movable part and configured to grasp an object, the robot grasping and moving the object, and a sensor configured to measure the object, and is capable of processing the object; The operation unit includes: a database that stores an item ID that identifies the object and item information that indicates characteristics of the object in association with each other; a plan generating unit that generates robot operation information representing operations and parameters of the robot; a robot control unit that controls the robot in accordance with the generated robot operation information; Equipped with When the robot is caused to grasp the object, the plan generation unit Calculating a gripping position and a gripping orientation that represent a position and an orientation of the robot at which the robot grips the object based on the position and the orientation of the object; generating robot operation information including a path of the robot from a current position and posture of the robot to the gripping position and the gripping posture; The item information includes an item type indicating a shape of the object and a degree of ease of shape change; When causing the robot to grasp a first object that is one of the objects, the robot control unit sets a target value of a force to be applied to the first object when grasping the first object, based on at least one of a shape of the item type of the first object stored in the database corresponding to an item ID of the first object and a degree of ease of shape change. Robot management system.

Images