Robot management system

The robot management system addresses low productivity and control burden in humanoid robots by selectively attaching robot tools and using autonomous tool delivery, ensuring efficient task execution and adaptability in handling diverse items.

JP2024170247APending Publication Date: 2024-12-06SOFTBANK GROUP CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2023087305
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-26
Publication Date
2024-12-06

AI Technical Summary

Technical Problem

Conventional humanoid robots struggle with low productivity in picking tasks due to slow finger movements and high control burden when handling items of varying hardness and fragility, and require indirect management of grasping tools for diverse tasks.

Method used

A robot management system that selectively attaches robot tools based on the type of work, utilizing a determination unit to identify the task, a control unit to manage tool execution, and a mobile body management server to transport necessary tools to the work site, enabling autonomous tool delivery by a flying object.

Benefits of technology

Reduces control burden and maintains work efficiency by allowing robots to perform multiple tasks without interruptions, including tool replacement through autonomous tool delivery, enhancing productivity and adaptability.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024170247000001_ABST
    Figure 2024170247000001_ABST
Patent Text Reader

Abstract

To reduce a control load by directly executing a plurality of kinds of work including gripping of an article by a grip part.SOLUTION: When robot tools 21 when a humanoid robot 1 performs work are not enough in a work site 62 (when necessary robot tools 21 are not present in the robot tools 21 held as extras), necessary robot tools 21 are conveyed from a storage chamber 76 by a flying body 60A so that the humanoid robot 1 does not need to interrupt work and move to take the robot tool 21 by itself. Therefore, the work can be continued without decreasing work efficiency of the body of the humanoid robot 1. Basically, the humanoid robot 1 holds extra robot tools 21. Even when irregular work takes place, the humanoid robot can continue the work without leaving the work site 62 by cooperating with the flying body 60A.SELECTED DRAWING: Figure 6
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a robot management system. [Background technology]

[0002] Humanoid robots are used in factory production lines to perform tasks automatically. Patent Document 1 describes posture control of a humanoid robot.

[0003] Furthermore, Patent Document 2 describes a robot arm having multiple joints that is driven by an elastic actuator, and which is controlled by a hand support member that is arranged at the hand of the robot arm and supports the robot arm by contacting a support surface, and a control unit that controls the contact force between the hand support member and the support surface while also controlling the position and posture of the hand of the robot arm. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-093506 A [Patent Document 2] WO2011 / 001569 publication Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional picking tasks in warehouses using humanoid robots currently rely on human hands when picking up items (such as shampoo, conditioner, cosmetics, toothpaste, instant ramen, snack bags, etc., which vary in shape, weight, hardness, and fragility) from shelves on which they are displayed, and then packing them into the appropriate packaging (boxes, etc.).

[0006] Even if the structure of the robot's gripping section is changed to a finger type, the fingers and arms move slowly, resulting in low productivity. Furthermore, when performing tasks other than gripping luggage, the gripping section must grip the necessary tools (e.g., drills, screwdrivers, etc.), which is an indirect task that requires management control of the gripping state, and this can result in a large control burden.

[0007] In consideration of the above, an object of the present invention is to provide a robot management system that can reduce the control burden by directly performing multiple types of work, including gripping cargo using a gripper. [Means for solving the problem]

[0008] The robot management system of the present invention is a robot management system that manages work performed by a robot to which a plurality of robot tools are selectively attached and which performs work on an object according to a work type, and includes a robot control device having a determination unit that determines the work type of the object, and a first control unit that controls the execution of work using the robot tool corresponding to the work type determined by the determination unit, and when the robot tool corresponding to the work type is insufficient, issues a request notification to request the missing robot tool, and a mobile object management server having a second control unit that receives the request notification, loads the required robot tool in a storage facility that stores the robot tools, and controls the operation of a mobile object that transports the robot to a work site.

[0009] According to the present invention, the robot control device controls the execution of work using a robot tool corresponding to the work type determined by the determination unit, and when a robot tool corresponding to the work type is in short supply, issues a request notification to request the missing robot tool.

[0010] The mobile object management server receives the request and controls the operation of the mobile object which loads the necessary robot tool in a warehouse storing the robot tool and transports the robot tool to the work site.

[0011] This allows the robot to continue working without reducing its work efficiency. For example, the robot basically holds the robot tool, but even if an irregular task occurs, the robot can continue working without leaving the work site by cooperating with the mobile object.

[0012] In the present invention, the mobile body is a flying body capable of autonomous flight based on identification information of the robot that sent the request notification, location information of the robot's work site, and information on the type of robot tool required.

[0013] In the present invention, the robot tool has a sensor unit equipped with a camera that takes an image of the object and identifies the type of the object, and a motion processing unit that identifies the position of the object, and is characterized in that the information necessary for the flying object to fly independently is obtained from the sensor unit.

[0014] The camera identifies the photographed object (hereinafter, sometimes referred to as baggage) based on the photographed image information. In other words, the camera has the role of acquiring information for identifying the type of object (shape, size, hardness, etc.).

[0015] The motion processing unit (MoPU) outputs vector information of the movement of a point indicating the location of an object along a predetermined coordinate axis as position information together with the movement information. In other words, the movement information output from the MoPU contains only information indicating the movement (movement direction and movement speed) of the center point (or center of gravity) of the object on the coordinate axes (x-axis, y-axis, z-axis). In other words, it is possible to accurately guide the trajectory of the gripping unit when it approaches the object.

[0016] The flying object according to the present invention can acquire information necessary for the flying object to fly autonomously from a sensor unit mounted on the robot tool.

[0017] The above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions. Effect of the Invention

[0018] As described above, according to the present invention, the control burden can be reduced by directly executing a plurality of types of work including gripping an object with a gripper. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a front view of a humanoid robot according to a first embodiment. [Diagram 2] FIG. 1 is a side view of a humanoid robot according to a first embodiment. [Diagram 3] FIG. 2 is a front view of the palm side of the grip part according to the first embodiment. [Figure 4] FIG. 1A is a front view of a robot tool stored in a holder according to a first embodiment, and FIGS. 1B to 1D are front views showing states of the robot tool according to the first embodiment performing work. [Diagram 5] FIG. 2 is a diagram illustrating an example of a functional configuration of a humanoid robot according to a first embodiment. [Figure 6] This is an overhead view showing a work site where a humanoid robot is working and an air vehicle that can move around the site. [Figure 7] 1 is a perspective view of an aircraft according to a first embodiment. FIG. [Figure 8] 13 is a flowchart showing a procedure of gripping control when gripping an object with a gripper in conjunction with the overall movement of the humanoid robot. [Figure 9] 13 is a control flowchart showing details of a robot tool application processing subroutine. [Figure 10] 5 is a flowchart showing a robot tool transport control routine according to the first embodiment. [Figure 11]FIG. 11 is a front view of a gripping portion according to a second embodiment, and FIG. 11B is a table showing a list of work types of the gripping portion according to the second embodiment. [Figure 12] FIG. 1 is a diagram illustrating an example of computer hardware that functions as an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Also, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0021] (First embodiment)

[0022] Fig. 1 is a front view of a humanoid robot 1 according to a first embodiment. As shown in Fig. 1, the humanoid robot 1 according to the first embodiment includes an upper body 2, legs 3, and a connector 4 that rotatably connects the upper body 2 to the legs 3, and is placed, for example, on a production line in a factory to perform work on a line including shelves on which packages 100 (see Fig. 3) or the like that are to be picked are displayed, or on objects (such as fallen objects) on the floor. Note that the work includes not only picking, which is to grasp packages 100 from shelves, but also packing, which is to place the grasped packages 100 in a predetermined case (such as a cardboard box).

[0023] The upper body 2 has two arms 5 and 6. The arms 5 and 6 are attached to the left and right of the upper body 2 so as to be freely rotatable. Furthermore, a gripping part 20 (described in detail later) for gripping luggage 100 is attached to the tip of the arms 5 and 6. The number of arms is not limited to two, and may be one or three or more.

[0024] The leg 3 has two wheels 7, 8 attached to its lower portion, enabling the humanoid robot 1 to move on the floor on which it is placed.

[0025] The connecting portion 4 rotatably connects the upper body portion 2 and the legs 3. Therefore, the upper body portion 2 can lean forward and backward with respect to the legs 3. Therefore, the humanoid robot 1 according to the first embodiment can lean the upper body portion 2 forward with respect to the legs 3, as shown in Fig. 2, and pick up luggage 100 placed on a shelf, luggage 100 placed on the floor, and luggage 100 that has fallen to the floor during work.

[0026] In addition, the legs 3 have a balancing function to prevent the humanoid robot 1 from falling over when the upper body 2 leans forward or backward relative to the legs 3 or when the humanoid robot 1 moves.

[0027] 1, the connecting part 4 has a function of changing the distance between the upper body part 2 and the legs 3. Therefore, the vertical position of the upper body part 2 relative to the legs 3 can be adjusted as shown by the arrow A so as to match the height of the workbench in the production line.

[0028] Moreover, the driving of the humanoid robot 1 according to the first embodiment is controlled by a control system 10 implemented within the humanoid robot 1.

[0029] (Structure of gripping portion 20) 3, the gripping unit 20 attached to the tip of the arms 5, 6 has a structure similar to that of a human hand (Intelligent Hand System). The gripping unit 20 is attached to the arms 5, 6 so as to be freely rotatable. The gripping unit 20 is a portion beyond the wrist that connects to the arms 5, 6, and the gripping unit 20 is detachable from this wrist portion and can be replaced with a robot tool 21EX, which will be described later.

[0030] (Gripping part 20) 3, the grip portion 20 according to the first embodiment has a palm portion as a base corresponding to a so-called human palm, and five fingers 22A, 22B, 22C, 22D, and 22E, each having a plurality of joints, are attached to the palm portion. Note that, although the number of fingers on the grip portion 20L is five in the first embodiment, the number of fingers may be different, such as three fingers.

[0031] A palm sensor 26 is attached to the palm of the hand. The high-resolution camera constituting the palm sensor 26 according to the first embodiment identifies the photographed baggage 100 based on the captured image information, i.e., whether it is a care product such as shampoo, conditioner, cosmetics, or toothpaste, or whether it is food such as instant ramen or a bag of snacks.

[0032] In other words, the high-resolution camera serves to obtain information for identifying the type of luggage 100 (shape, size, hardness, etc.).

[0033] Meanwhile, the MoPU constituting palm sensor 26 of the first embodiment together with the high-resolution camera outputs, from images of luggage 100 captured at a frame rate of 1000 frames / second or more, motion information indicating the motion of the captured luggage 100 (in this case, the relative motion between arms 5, 6) at a frame rate of, for example, 1000 frames / second or more. Note that the frame rate may be increased when detecting a moving luggage 100, and may be decreased when detecting a fixed object (a luggage 100 that does not move).

[0034] The MoPU outputs, as motion information, vector information of the movement along a predetermined coordinate axis of a point indicating the location of the luggage 100. In other words, the motion information output from the MoPU does not include information necessary to identify what the photographed luggage 100 is (the above-mentioned care product or food), but only includes information indicating the movement (movement direction and movement speed) of the center point (or center of gravity) of the luggage 100 on the coordinate axes (x-axis, y-axis, z-axis).

[0035] That is, the trajectory of the gripper 20 as it approaches the luggage 100 can be accurately guided.

[0036] Information output from the palm sensor 26, which includes a high-resolution camera and a MoPU, is supplied to the information processing device 14.

[0037] The information processing device 14 uses information from the palm sensor 26 including a high-resolution camera and an MoPU to pinpoint the position of the luggage 100 with high precision, calculates the degree of spread of the fingers 22A, 22B, 22C when gripping, the strength of the grip, and the suction force of the suction pad 24, and precisely controls the minute movements of the arms 5, 6 and the gripping portion 20, making it possible to handle the picking of various luggage 100.

[0038] (Robot Tool 21EX) In the first embodiment, the main purpose of the work type for the load 100 is to grip the load 100 with the gripping unit 20 .

[0039] On the other hand, there are cases where the type of work performed on the load 100 is not gripping, but other types of work (for example, painting, gripping to increase strength, drilling, etc.).

[0040] In this case, it is possible for the gripping unit 20 to grip tools corresponding to various tasks and face the load 100; however, when continuously performing the same type of task, the burden of maintaining and controlling the gripping state (such as controlling the relative position between the gripping unit and the tool being gripped) is large.

[0041] Therefore, in the first embodiment, instead of the gripping unit 20, a robot tool 21EX (in the first embodiment, three types, 21EXA, 21EXB, and 21EXC) is provided according to the type of work to be performed on the luggage 100, as shown in Figures 1 and 4, and, as necessary, the gripping unit 20 is replaced with the robot tool 21EX to perform work according to a type of work other than gripping the luggage 100.

[0042] (Storage example of Robot Tool 21EX) As shown in FIG. 1, the humanoid robot 1 has a belt 28 attached to the lower part of the upper body 2 (at the so-called waist position), and holders (not shown) are attached to the belt 28 to detachably hold each of the three robot tools 21EXA, 21EXB, and 21EXC.

[0043] In FIG. 1, three robot tools 21EX are shown, but the number of robot tools 21EX may be one type, two types, or four or more types, and the number to be attached can be determined according to the attributes of the luggage 100 described below.

[0044] FIG. 4 shows the detailed configurations of three types of robot tools 21 (21EXA, 21EXB, 21EX) that are equipped on the humanoid robot 1 of the first embodiment apart from the gripping part 20, and their relationships with their uses.

[0045] FIG. 4(A) is a front view of the belt 28 attached to the humanoid robot 1 of FIG.

[0046] As shown in FIG. 4(A), the robot tools 21EX are detachably attached.

[0047] 4(B) to (D) show states in which the robot tool 21EXA is gripping the load 100. FIG.

[0048] As shown in FIG. 4(B), the type of work of the robot tool 21EXA is painting, and a spray gun is attached as a tool.

[0049] A spray gun is a type of pistol-shaped painting device used for spray painting. It uses compressed air from a compressor (not shown) to spray the paint in a mist form from the tip of the spray gun, making it possible to paint the surface evenly.

[0050] As shown in FIG. 4(C), the type of work of the robot tool 21EXB is gripping, similar to the gripping unit 20, but it is structured to be able to grip special luggage 100 that is difficult for the gripping unit 20 to grip.

[0051] That is, the robot tool 21EXB has a structure similar to that of a fork used as an attachment for heavy machinery, for example. In the first embodiment, the robot tool 21EXB has a two-finger structure, and the two fingers open and close due to pressure supplied from a pressure source through piping, which increases the strength with which the tool can grip the load 100, although it has lower versatility compared to a gripping operation (gripping unit 20) using a motor or the like.

[0052] As shown in FIG. 4(D), the type of work of the robot tool 21EXC is drilling, and a drill is attached as a tool.

[0053] A drill bit of a predetermined diameter can be removably attached to the drill, and the drill bit is pre-installed according to the hole size to be drilled in the load 100.

[0054] Here, if the type of work (gripping) is different from what can be processed by the gripping unit 20, the most suitable robot tool 21 is selected based on the type of work to be performed on the luggage 100 (see Table 1), and the gripping unit 20 is replaced with the selected robot tool 21 to perform the processing.

[0055] [Table 1]

[0056] Although not shown in the figure, a sensor portion equivalent to the palm sensor 26 is attached to each robot tool 21EX.

[0057] 5 is a schematic diagram of an example of a control system for a humanoid robot according to the first embodiment. The control system 10 includes a sensor 12 mounted on the humanoid robot, a palm sensor 26 including a high-resolution camera and an MoPU, and an information processing device 14.

[0058] The sensor 12 sequentially acquires information indicating at least the distance and angle between the luggage 100 around the humanoid robot 1 and the arms 5 and 6 on which the humanoid robot 1 is working. The sensor 12 may be a high-performance camera, a solid-state LiDAR, a multi-color laser coaxial displacement meter, or a variety of other sensors. Other examples of the sensor 12 include a vibration meter, a thermo camera, a hardness meter, a radar, a LiDAR, a high-pixel, telephoto, ultra-wide-angle, 360-degree, high-performance camera, vision recognition, fine sound, ultrasound, vibration, infrared, ultraviolet, electromagnetic waves, temperature, humidity, spot AI weather forecast, high-precision multi-channel GPS, low-altitude satellite information, or long-tail incident AI data.

[0059] In addition to the above information, the sensor 12 detects images, distance, vibration, heat, smell, color, sound, ultrasonic waves, ultraviolet rays, infrared rays, etc. Other information detected by the sensor 12 includes the movement of the center of gravity of the humanoid robot 1, the material of the floor on which the humanoid robot 1 is placed, the outside air temperature, the outside air humidity, the up / down / side / diagonal inclination angle of the floor, the amount of moisture, etc.

[0060] The sensor 12 performs these detections, for example, every nanosecond.

[0061] Palm sensor 26 (high resolution camera and MoPU) is a sensor provided on gripping portion 20 of arm portions 5, 6, and has a camera function for photographing baggage 100 and a position identification function for identifying the position of baggage 100, separately from sensor 12.

[0062] When one MoPU 12 is used, it is possible to obtain vector information of the movement of a point indicating the location of the luggage 100 along each of two coordinate axes (x-axis and y-axis) in a three-dimensional orthogonal coordinate system. Using the principle of a stereo camera, two MoPUs 12 may be used to output vector information of the movement of a point indicating the location of the luggage 100 along each of three coordinate axes (x-axis, y-axis, z-axis) in a three-dimensional orthogonal coordinate system. The z-axis is the axis along the depth direction (vehicle travel).

[0063] The information processing device 14 includes an information acquisition unit 140 , a control unit 142 , and an information storage unit 144 .

[0064] The information acquisition unit 140 acquires information about the baggage 100 detected by the sensor 12 and the palm sensor 26 (high-resolution camera and MoPU).

[0065] The control unit 142 uses the information acquired by the information acquisition unit 140 from the sensor 12 and AI (Artificial Intelligence) to control the rotational movement of the connecting unit 4, the movement of the arms 5, 6, and the like.

[0066] Furthermore, the control unit 142 uses the information acquired by the information acquisition unit 140 from the palm sensor 26 (high resolution camera and MoPU) to grasp the type (shape, size, hardness, etc.) and position of the luggage 100 in detail, and controls the palm side 20A to face the luggage 100 according to the external shape and position, to be sucked by the suction pad 24, and to be gripped by the three finger portions 22A, 22B, 22C (grasping control). Note that the type of luggage 100 may be grasped based on the external shape information, and gripping control (only "suction", only "grasping", a combination of "suction" and "grasping", etc.) may be selected.

[0067] For example, the control unit 142 executes the following processes as an overall operation. (1) The connecting portion 4 is driven to tilt the upper body portion 2 forward or backward so that the luggage 100 on a shelf or the floor can be picked up. (2) The arms 5, 6 and the gripping portion are driven so that the load 100 can be grasped. (3) The upper body 2 is driven up and down relative to the legs 3 to match the height of the workbench on the production line. (4) Maintain balance to prevent humanoid robot 1 from falling over. (5) The driving of the wheels 7, 8 is controlled so that the humanoid robot 1 can push a cart or the like.

[0068] For example, when picking up luggage 100 lying on the floor, the information processing device 14 acquires information about the luggage 100 detected by the sensor 12, and using the acquired information about the luggage 100 and AI, controls the connecting part 4 and the arms 5 and 6 to pick up the luggage 100 lying on the floor and move the picked-up luggage 100 to a predetermined position.

[0069] (Changes in Robot Tool 21) Incidentally, in the first embodiment, the three robot tools 21EXA, 21EXB, 21EXC are removably held via a belt 28 (including a holder) on the lower part (so-called waist position) of the upper body 2 of the humanoid robot 1. In this case, theoretically there is no limit to the number that can be held in the holder (1 or 4 or more), but there is a physical limit to the number that can be held.

[0070] For this reason, when a job other than the job type of the robot tool 21 (for example, three types of robot tools 21 (21EXA, 21EXB, 21EX)) held on the belt 28 is requested, the humanoid robot 1 must be individually moved to a storage facility where the other types of robot tools 21 are kept and re-attached to the belt 28.

[0071] For this reason, there may be cases where the original work is hindered due to the time it takes for the humanoid robot 1 to be moved to the storage facility, as well as congestion of multiple humanoid robots 1 in the storage facility.

[0072] Therefore, in the first embodiment, a mobile body 60 is provided that can move with the robot tool 21 mounted between each of the multiple humanoid robots 1 working at the site and the storage facility.

[0073] The mobile body 60 may take any form, such as a humanoid robot specialized in transporting the robot tool 21, a transport vehicle that moves on the road, or an aircraft that moves through the air. In the first embodiment, however, an aircraft 60A that moves through the air is used as the mobile body 60.

[0074] FIG. 6 is an overhead view showing a work site 62 where the humanoid robot 1 is working, and an air vehicle 60A serving as a mobile body 60 capable of moving around the work site.

[0075] 7, the aircraft 60A has a propeller 66 attached to each of the four corners of a substantially rectangular body 64. The aircraft 60A is managed by an aircraft management server 68, and when the aircraft management server 68 instructs a flight route, etc. for a specific aircraft 60A, the rotation speed, direction, etc. of each propeller 66 of the supported aircraft 60A are controlled by a controller 70 built into the body 64, and the aircraft 60A moves to a destination along a predetermined route (so-called automatic driving).

[0076] The upper surface of the main body 64 serves as a mounting table 64A for mounting and holding the robot tool 21 to be transported.

[0077] In addition, a camera (not shown) is attached to the main body 64, and by transmitting captured images to the controller 70, when the aircraft 60A flies based on specified route information, in addition to position management control using a GPS (Global Positioning System) or the like, the surroundings of the route are monitored.

[0078] The camera may be only a photographing element, but preferably has the same functions (high-resolution camera function and MoPU function) as palm sensor 26 attached to gripping part 20. Also, instead of providing the camera in main body 64, palm sensor 26 provided in robot tool 21 placed on mounting table 64A can be used.

[0079] As shown in FIG. 6, at a work site 62, a humanoid robot 1 is performing work at a work desk 72 with a necessary robot tool 21 attached to it.

[0080] In addition to the work desk 72, the work site 62 is provided with a transfer table 74 for transferring the robot tool 21 between the flying body 60A and the humanoid robot 1.

[0081] When the humanoid robot 1 is required to perform work on the work desk 72 and a robot tool 21 of a different type from the robot tool 21 that the humanoid robot 1 is holding is required, the humanoid robot 1 requests the required robot tool 21.

[0082] In response to this request, the aircraft management server 68, which manages the aircraft 60A, selects a specific aircraft 60A, launches it from the base 78, and instructs it to fly between the storage facility 76 and the work site 62.

[0083] The storage 76 is provided with a plurality of storage sections 76A, and each of the storage sections 76A stores a robot tool 21 therein.

[0084] The aircraft management server 68 instructs the specific aircraft 60A that has been instructed to fly to retrieve the necessary robot tool 21 from the storage facility 76 and provide route information, etc. for transporting it to the work site 62 where the requesting humanoid robot 1 will be working.

[0085] This enables the humanoid robot 1 to replace the transported robot tool 21 and perform work. The flying object 60A returns the robot tool 21 originally attached to the humanoid robot 1, or the transported robot tool 21 (which will be replaced after work is completed) to the storage 76 and returns to the base 78, with the robot tool 21 placed on the mounting platform 64A.

[0086] The operation of the first embodiment will be described below. (Grip control of luggage 100) FIG. 8 is a flowchart showing a procedure of gripping control when the gripper 20 grips the cargo 100. As shown in FIG.

[0087] In step 150, it is determined whether or not an instruction to grasp the luggage 100 has been given. If a positive determination is made, the process proceeds to step 152, in which the humanoid robot 1 is moved (e.g., the arms 5 and 6 are operated) so that the palm side faces the target luggage 100, and the process proceeds to step 154.

[0088] In step 154, the palm side 20A is faced and information about the baggage 100 is detected by the palm sensor 26 (high resolution camera and MoPU).

[0089] In the next step 156, the information detected by the palm sensor 26 is analyzed to ascertain the type (shape, size, hardness, etc.) and position of the baggage 100 in detail, and the process proceeds to step 158.

[0090] In step 158, a selection process is executed for the type of work (gripping or a type of work other than gripping) of the gripper 20 according to the attributes of the luggage 100. The selection process will be described in detail with reference to Fig. 9, and if the type of work is gripping in the selection process of Fig. 9, the process proceeds to step 160.

[0091] In step 160, a task for gripping the load 100 is selected. For example, a task is selected from "suction" only, "grabbing" only, a combination of "suction" and "grabbing", and the process proceeds to step 162.

[0092] In step 162, gripping of the load 100 ("suction" only, "grab" only, "suction" and "grab") is performed.

[0093] In the next step 164, it is determined whether the gripping of the luggage 100 was successful, and if a positive determination is made, the gripped luggage 100 is carried to a predetermined location, and the process proceeds to step 150 to wait for an instruction to grip the next luggage 100.

[0094] If the determination in step 164 is negative, the process proceeds to step 166, where an error process is performed.

[0095] The gripper 20 is fitted with a palm sensor 26 including a high-resolution camera and an MoPU. By mounting the gripper 20 having the above structure on the arms 5 and 6 of the humanoid robot 1, an object can be reliably picked up by the suction surface, and even if the humanoid robot 1 moves quickly, the baggage 100 can be carried without dropping from the gripper 20.

[0096] In addition, since the palm sensor 26 (high-resolution camera and MoPU) is mounted on the palm side 20A, the baggage 100 can be captured with high accuracy, and the device can also be used for tasks that require minute movements.

[0097] Furthermore, very soft and fragile items can be gripped by the movement of the fingers 22A, 22B, 22C without using the suction pad 24, and damage to the soft cargo 100 can be prevented by adjusting the gripping force.

[0098] (Details of robot tool application process) FIG. 9 is a control flow chart showing details of the work type selection process (robot tool application process) subroutine in step 158 in FIG.

[0099] In step 198, it is determined whether the type of work is grasping or a work other than grasping. If the type of work is determined to be grasping in this step 198, an instruction is given to return to step 160 in Fig. 8, and this routine ends.

[0100] If it is determined in step 199 that the task type is other than grasping, the process proceeds to step 200, where the robot tool 21 is selected based on the task type for the load 100 (see Table 1).

[0101] That is, as shown in Table 1, the type of robot tool 21 required is determined according to the type of work to be performed on the load 100.

[0102] The selection of the type of robot tool based on the type of work for the luggage 100 shown in Table 1 is merely an example, and may be determined based on the type and number of robot tools possessed.

[0103] In the first embodiment, three types of robot tools 21 are attached to the belt 28, and a selection is made from these three types of robot tools 21. However, the number of robot tools attached to the belt 28 may be increased, or a variety of robot tools 21 may be selected and attached to the belt 28 in advance depending on the work site. Also, different types of robot tools 21 may be attached to each humanoid robot 1. In the next step 201A, it is determined whether or not the selected robot tool is equipped, and if the determination is negative, the process proceeds to step 201B, where a request is made to procure a robot tool. In the next step 201C, a robot tool procurement process is executed, and the process proceeds to step 202. Also, if the determination is positive in step 201A, the process proceeds to step 202.

[0104] In the next step 202, the tool attached to the replacement arm 6 (or 5) (usually the gripper 20, but there are also cases where another robot tool 21 has already been attached) is removed.

[0105] In the next step 204, the arm 6 (or 5) on the replacement side is moved to the position of the belt 28, and the robot tool 21 is attached.

[0106] In the next step 206, the tool originally attached to the replacement arm 6 (or 5) that is being held by the non-replacement arm 5 (or 6) is stored in a holder on the belt 28, and the process proceeds to step 208.

[0107] In step 208, the programmed tasks are executed for each robot tool. In the next step 210, a restart of the flow chart in Fig. 8 (transition to step 150) is instructed, and this routine ends.

[0108] As described above, in the first embodiment, in addition to controlling the gripping operation by the gripper 20, the robot tool 21 is prepared in advance according to the type of work to be performed on the load 100, for example.

[0109] As a result, the control burden (such as the burden associated with controlling the relative position between the gripper 20 and the tool) that arises when the gripper of the comparative example grips and works on the tool required for the type of work can be eliminated by directly attaching the robot tool 21.

[0110] (Robot tool 21 transport control) FIG. 10 is a flowchart showing a robot tool transport control routine that is executed when the humanoid robot 1 is working at the work site 62 and a robot tool 21 of a type different from the robot tool 21 that the humanoid robot 1 is holding becomes necessary.

[0111] This robot tool transport control routine is executed by the aircraft management server 68 in cooperation with the information processing device 14 (see FIG. 5) mounted on the humanoid robot 1 and the controller 70 (see FIG. 7) of the aircraft 60A. That is, the robot tool transport control is started by a request from the information processing device 14 to the robot tool 21. Moreover, the transport (flight control) of the robot tool 21 by the aircraft 60A itself is automatically performed by the controller 70 of the aircraft 60A.

[0112] In step 250 in FIG. 10, it is determined whether or not a request for the robot tool 21 has been made from the information processing device 14 of the humanoid robot 1, and if a negative determination is made, this routine is terminated.

[0113] Also, if a positive judgment is made in step 250, the process proceeds to step 252, where (1) the identification information of the humanoid robot 1, (2) the identification information of the work site 62, and (3) the transportation information including the type of the robot tool 21 are recognized, and the process proceeds to step 254.

[0114] In step 254, the air vehicle 60A is selected as the specific air vehicle 60A, and then the process proceeds to step 256, in which the controller 70 of the air vehicle 60A is notified of the delivery information, and the process proceeds to step 258.

[0115] In step 258, an instruction to start transportation is given to the controller 70 of the flying object 60A. In response to this instruction, the flying object 60A takes off from the base 78, moves to the storage 76, removes the specified robot tool 21 from the storage section 76A, places it on the mounting table 64A, and moves to the work site 62. Thereafter, the flying object 60A lands on the delivery table 74 and waits.

[0116] After this, if a normal operation is carried out, the operation is carried out in the following order. (Task 1) The humanoid robot 1 changes the robot tool 21 currently attached to it to the robot tool 21 mounted on the flying body 60A. (Operation 2) Once the replacement is complete, the aircraft 60A takes off from the transfer platform 74. (Task 3) The aircraft 60A returns the robot tool 21 at storage 76. (Task 4) Aircraft 60A returns to base 78.

[0117] The task time for each of the above tasks 1 to 4 can be roughly recognized on a task-by-task basis.

[0118] In the next step 260, it is determined whether or not a notification has been received from the controller 70 of the flying object 60A indicating that the handover operation of the robot tool 21 has been completed. Here, the above-mentioned operation 2 is taken as an example of the end of the operation, and once operation 2 is completed, the handover operation of the robot tool 21 to the humanoid robot 1 is completed, and the humanoid robot 1 can continue its original operation.

[0119] If a negative judgment is made in step 260, the process proceeds to step 262, where it is determined whether or not a trouble notification has been received from controller 70 of aircraft 60A. If a positive judgment is made in this step 262, it is predicted that some kind of malfunction has occurred between the series of tasks 1 to 4, causing the tasks to stop, and the process proceeds to step 270, where error processing is performed, and this routine ends if the error is resolved.

[0120] If the determination in step 262 is negative, the process proceeds to step 264 to determine whether or not a predetermined time has elapsed. Since the work time from Task 1 to Task 2 can be roughly determined, if the tasks are not completed even after a preset predetermined time has elapsed (positive determination in step 264), it is determined that an event that is interfering with the tasks has occurred, and the process proceeds to step 270 to execute error processing, and this routine ends if the error is resolved.

[0121] If the determination in step 264 is negative, it is determined that work is in progress, and the process returns to step 260, and steps 260, 262, and 264 are repeated until the determination in step 260 is positive.

[0122] If the determination in step 260 is affirmative, it is determined that tasks 1 to 2 have been completed without any problems, and the process proceeds to step 266, where it is determined whether or not a return notice has been received from flying object 60A and the return has been confirmed. If the determination in step 266 is affirmative, it is confirmed that all tasks have been completed, and this routine ends.

[0123] On the other hand, if the determination in step 266 is negative, the process proceeds to step 268 to determine whether or not a predetermined time has elapsed. The predetermined time in step 268 is the work time between task 1 and task 4, and is set longer than the predetermined time in step 264 (the work time between task 1 and task 2).

[0124] If the answer at step 268 is positive, it is determined that some kind of trouble occurred during the return of the flying object 60A, and the process proceeds to step 270, where error processing is performed, and this routine ends if the error is resolved.

[0125] If the determination in step 268 is negative, the process returns to step 266, and steps 266 and 268 are repeated until the determination in step 266 is positive.

[0126] As described above, in the first embodiment, when there is a shortage of robot tools 21 at the work site 62 for the humanoid robot 1 to work (when the required robot tool 21 is not present among the robot tools 21 held as extras), the necessary robot tool 21 is transported from the storage facility 76 by the aircraft 60A, so that the humanoid robot 1 does not have to interrupt its work and go to get the robot tool 21 by itself.

[0127] Therefore, the work can be continued without reducing the work efficiency of the main body of the humanoid robot 1.

[0128] For example, the humanoid robot 1 basically holds an extra robot tool 21EX, but even if an irregular task occurs, the humanoid robot 1 can continue the task without leaving the work site 62 by cooperating with the flying object 60A.

[0129] In the first embodiment, the flying object 60A has been described as an example of the moving object 60, but the moving object 60 may be a humanoid robot that specializes in transporting the robot tool 21, or a transport vehicle that moves on a road surface. The object to be transported is not limited to the missing robot tool, and disposable items (cotton swabs, cutter blades, adhesive tape, etc.) that are attached to the robot tool 21 for use may be transported, or a robot tool 21 of the same type may be transported to a robot tool 21 that needs to be repaired or replaced.

[0130] (Second embodiment: "Replacing finger tips") A second embodiment of the present invention will be described below. Note that the same components as those in the second embodiment are given the same reference numerals and the description of the components will be omitted.

[0131] The second embodiment is characterized in that the finger tips, which are the gripping portion 20, can be changed into a tool according to the type of work, like the robot tool described in the first embodiment.

[0132] As shown in FIG. 11(A), the hand tool 50 of the second embodiment is basically intended to grip a load 100, similar to the gripping portion 20 of the first embodiment.

[0133] Each finger can be attached or detached from the hand tool 50. In the state shown in Fig. 11(A), the first joint portion 50A (the "finger tip" shown in Fig. 11(B)) for the main purpose is attached.

[0134] Meanwhile, as shown in FIG. 11(B), each finger portion is replaceable with a first joint portion 50B to 50F having a shape according to the type of work.

[0135] The first joint portion 50B is a "tweezer," or what is commonly called tweezers, and is used to pinch and grasp thin members.

[0136] The first joint portion 50C is a "stick" and is used when holding a part with a rod-shaped member.

[0137] The first joint portion 50D is a "cotton swab" and is a spherical cotton swab that is used to wipe off dirt from parts or remove moisture.

[0138] The first joint portion 50E is a "camera" and is used when photographing a part (particularly a close-up shot).

[0139] The first joint portion 50F is a "screwdriver" and is used when tightening or loosening a screw.

[0140] The hand tool according to the second embodiment is particularly suitable for tasks such as assembling and disassembling small parts (electronic parts, precision parts, etc.).

[0141] 12 is a schematic diagram showing an example of a hardware configuration of a computer 1200 functioning as the information processing device 14. A program installed in the computer 1200 can cause the computer 1200 to function as one or more "parts" of the device according to the first embodiment, or cause the computer 1200 to execute operations associated with the device according to the first embodiment or one or more "parts", and / or cause the computer 1200 to execute a process according to the first embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0142] The computer 1200 according to the first embodiment includes a CPU 1212, a RAM 1214, and a graphic controller 1216, which are connected to each other by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive, a solid state drive, etc. The computer 1200 also includes input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0143] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 retrieves image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0144] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0145] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or a program that depends on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0146] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by implementing operations or processing of information according to the use of the computer 1200.

[0147] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0148] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0149] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may execute various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout the present invention and specified by the instruction sequence of the program, and write back the results to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0150] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0151] The blocks in the flowcharts and block diagrams in the first and second embodiments may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0152] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0153] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0154] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0155] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0156] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order.

[0157] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0158] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order. [Explanation of symbols]

[0159] 1 humanoid robot, 2 upper body, 3 legs, 4 connecting portion, 5, 6 arms, 7, 8 wheels, 10 control system, 12 sensor, 14 information processing device, 20 (20L, 20R) gripping portion, 20A palm side, 20B main body, 22A, 22B, 22C fingers, 24 (24X, 24Y) suction pad, 24A pad portion, 24B nipple, 26 palm sensor, 100 luggage, 50 hand tool, 50A first joint portion, 50B first joint portion, 50C first joint portion, 50D first joint portion, 50E first joint portion, 50F first joint portion, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 I / O controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 I / O chip

Claims

1. A robot management system that manages a task mainly performed by a robot to which a plurality of robot tools are selectively attached and which performs a task according to a type of work on an object, comprising: a robot control device including: a determination unit that determines a type of work of the object; and a first control unit that controls execution of a work in which the robot tool corresponding to the type of work determined by the determination unit is attached, and when the robot tool corresponding to the type of work is in short supply, transmits a request notification to request the missing robot tool; a mobile object management server including a second control unit that receives the request notification, loads a necessary robot tool in a storage facility that stores the robot tool, and controls an operation of a mobile object that transports the robot tool to a work site; A robot management system having the above configuration.

2. 2. A robot management system as described in claim 1, wherein the mobile body is an autonomous flying body based on the identification information of the robot that sent the request notification, the location information of the robot's work site, and the type information of the required robot tool.

3. 3. A robot management system as described in claim 2, wherein the robot tool has a sensor unit equipped with a camera that takes an image of the object and identifies the type of the object, and a motion processing unit that identifies the position of the object, and the information necessary for the flying object to fly independently is obtained from the sensor unit.

Citation Information

Patent Citations

  • Basic posture setting device and basic posture setting method

    JP2019093506A

  • Robot, control device for robot arm, and control program for robot arm

    WO2011001569A1