Robot control system and robot control program

Abstract

To reduce a control load by directly executing a plurality of kinds of work.SOLUTION: A data acquisition unit 70 acquires detection data of each sensor unit attached to a fingertip of a hand tool and transmits the detection data to a collation unit 72. A picking target object information acquisition unit 74 acquires information on trash to be picked (picking target object information) and sends the information to a search unit 76. The search unit 76 accesses an object-attribute information database 78, reads attribute information of the object specified to be picked, performs collation between the read attribute information and the attribute information received from the data acquisition unit 70, identifies position coordinates (three-dimensional coordinates) of the object (the trash) of the attribute information matching in the collation, and controls operation of a gripping unit for a picking operation. Thus, the trash specified to be picked can be picked in a pinpoint manner.SELECTED DRAWING: Figure 10

Description

[Technical field]

[0001] The present invention relates to a robot control system and a robot control program. [Background technology]

[0002] Humanoid robots are used in factory production lines to perform tasks automatically. Patent Document 1 describes posture control of a humanoid robot.

[0003] Furthermore, Patent Document 2 describes a robot arm having multiple joints that is driven by an elastic actuator, and which is controlled by a hand support member that is arranged at the hand of the robot arm and supports the robot arm by contacting a support surface, and a control unit that controls the contact force between the hand support member and the support surface while also controlling the position and posture of the hand of the robot arm. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-093506 A [Patent Document 2] WO2011 / 001569 publication Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional picking tasks in warehouses using humanoid robots currently rely on human hands when picking up items (such as shampoo, conditioner, cosmetics, toothpaste, instant ramen, snack bags, etc., which vary in shape, weight, hardness, and fragility) from shelves on which they are displayed, and then packing them into the appropriate packaging (boxes, etc.).

[0006] Even if the structure of the robot's gripping section is changed to a finger type, the fingers and arms move slowly, resulting in low productivity. Furthermore, when performing tasks other than gripping luggage, the gripping section must grip the necessary tools (e.g., drills, screwdrivers, etc.), which is an indirect task that requires management control of the gripping state, and this can result in a large control burden.

[0007] In consideration of the above, an object of the present invention is to provide a robot control system and a robot control program that can reduce the control burden by directly executing a plurality of types of work. [Means for solving the problem]

[0008] The robot control system of the present invention is a robot control system that equips a robot with a robot tool required for a work type to be performed on an object, and causes the robot to perform work corresponding to the work type, and has a selection unit that selects a robot tool equipped with a plurality of attribute information detection units capable of detecting each of a plurality of attribute information obtainable from the object, an attribute information acquisition unit that obtains, using the plurality of attribute information detection units, attribute information of a type that can identify the object under environmental conditions in which a plurality of the objects are mixed, and a position identification unit that identifies the position of the object for which the work type has been specified by comparing the attribute information of the object for which the work type has been specified with the attribute information acquired by the attribute information acquisition unit.

[0009] In the present invention, the attribute information acquisition unit acquires attribute information of a type capable of identifying the object under environmental conditions in which a plurality of the objects are mixed.

[0010] According to the present invention, the selection unit selects a robot tool equipped with multiple attribute information detection units capable of detecting each of multiple pieces of attribute information obtainable from an object. The attribute information acquisition unit obtains, by the multiple attribute information detection units, attribute information of a type capable of identifying an object, for example, under environmental conditions in which multiple objects are mixed. The position identification unit identifies the position of the object for which a work type has been specified by comparing the attribute information of the object for which a work type has been specified with the attribute information acquired by the attribute information acquisition unit. This makes it possible to reduce the control burden by directly executing tasks of multiple work types.

[0011] In the present invention, the attribute information detection unit detects attribute information for each of the senses of sight, hearing, smell, touch, and taste, and identifies the object based on a combination of a plurality of pieces of attribute information.

[0012] The present invention is characterized in that the type of work is picking work of picking specified waste from a storage area in which multiple types of recyclable waste and multiple types of hazardous waste are mixed.

[0013] In the present invention, the robot includes a camera that captures an image of the object and identifies the type of the object, and a motion processing unit that identifies the position of the object.

[0014] The camera identifies the photographed object (hereinafter, sometimes referred to as baggage) based on the photographed image information. In other words, the camera has the role of acquiring information for identifying the type of object (shape, size, hardness, etc.).

[0015] The motion processing unit (MoPU) outputs vector information of the movement of a point indicating the location of an object along a predetermined coordinate axis as position information together with the movement information. In other words, the movement information output from the MoPU contains only information indicating the movement (movement direction and movement speed) of the center point (or center of gravity) of the object on the coordinate axes (x-axis, y-axis, z-axis). In other words, it is possible to accurately guide the trajectory of the gripping unit when it approaches the object.

[0016] A robot control system according to the present invention is characterized in that a computer is operated as the selection unit, the attribute information acquisition unit, and the position identification unit of the robot control system described above.

[0017] The above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions. Effect of the Invention

[0018] As described above, according to the present invention, the control burden can be reduced by directly executing a plurality of types of work. [Brief description of the drawings]

[0019] [Figure 1] FIG. 1 is a front view of a humanoid robot according to a first embodiment. [Diagram 2] FIG. 1 is a side view of a humanoid robot according to a first embodiment. [Diagram 3] FIG. 2 is a front view of the palm side of the grip part according to the first embodiment. [Figure 4] FIG. 1A is a front view of a robot tool stored in a holder according to a first embodiment, and FIGS. 1B to 1D are front views showing states of the robot tool according to the first embodiment performing work. [Diagram 5] FIG. 2 is a diagram illustrating an example of a functional configuration of a humanoid robot according to a first embodiment. [Figure 6] 13 is a flowchart showing a procedure of gripping control when gripping an object with a gripper in conjunction with the overall movement of the humanoid robot. [Figure 7] 13 is a control flowchart showing details of a robot tool application processing subroutine. [Figure 8] FIG. 11 is a front view of a gripping portion according to a second embodiment, and FIG. 11B is a table showing a list of work types of the gripping portion according to the second embodiment. [Figure 9]FIG. 13 is a front view showing a sorting picking operation according to the third embodiment; [Figure 10] FIG. 13 is a functional block diagram of an information processing device for executing a sorting picking operation according to a third embodiment. [Figure 11] 13 is a control flowchart showing a processing procedure of a sorting picking operation according to a third embodiment. [Figure 12] FIG. 13 is a perspective view showing an engine room of a vehicle according to a modified example of the third embodiment. [Figure 13] FIG. 1 is a diagram illustrating an example of computer hardware that functions as an information processing device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0020] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Also, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0021] (First embodiment) Fig. 1 is a front view of a humanoid robot 1 according to a first embodiment. As shown in Fig. 1, the humanoid robot 1 according to the first embodiment includes an upper body 2, legs 3, and a connector 4 that rotatably connects the upper body 2 to the legs 3, and is placed, for example, on a production line in a factory to perform work on a line including shelves on which packages 100 (see Fig. 3) or the like that are to be picked are displayed, or on objects (such as fallen objects) on the floor. Note that the work includes not only picking, which is to grasp packages 100 from shelves, but also packing, which is to place the grasped packages 100 in a predetermined case (such as a cardboard box).

[0022] The upper body 2 has two arms 5 and 6. The arms 5 and 6 are attached to the left and right of the upper body 2 so as to be freely rotatable. Furthermore, a gripping part 20 (described in detail later) for gripping luggage 100 is attached to the tip of the arms 5 and 6. The number of arms is not limited to two, and may be one or three or more.

[0023] The leg 3 has two wheels 7, 8 attached to its lower portion, enabling the humanoid robot 1 to move on the floor on which it is placed.

[0024] The connecting portion 4 rotatably connects the upper body portion 2 and the legs 3. Therefore, the upper body portion 2 can lean forward and backward with respect to the legs 3. Therefore, the humanoid robot 1 according to the first embodiment can lean the upper body portion 2 forward with respect to the legs 3, as shown in Fig. 2, and pick up luggage 100 placed on a shelf, luggage 100 placed on the floor, and luggage 100 that has fallen to the floor during work.

[0025] In addition, the legs 3 have a balancing function to prevent the humanoid robot 1 from falling over when the upper body 2 leans forward or backward relative to the legs 3 or when the humanoid robot 1 moves.

[0026] 1, the connecting part 4 has a function of changing the distance between the upper body part 2 and the legs 3. Therefore, the vertical position of the upper body part 2 relative to the legs 3 can be adjusted as shown by the arrow A so as to match the height of the workbench in the production line.

[0027] Moreover, the driving of the humanoid robot 1 according to the first embodiment is controlled by a control system 10 implemented within the humanoid robot 1.

[0028] (Structure of gripping portion 20) 3, the gripping unit 20 attached to the tip of the arms 5, 6 has a structure similar to that of a human hand (Intelligent Hand System). The gripping unit 20 is attached to the arms 5, 6 so as to be freely rotatable. The gripping unit 20 is a portion beyond the wrist that connects to the arms 5, 6, and the gripping unit 20 is detachable from this wrist portion and can be replaced with a robot tool 21EX, which will be described later.

[0029] (Gripping part 20) 3, the grip portion 20 according to the first embodiment has a palm portion as a base corresponding to a so-called human palm, and five fingers 22A, 22B, 22C, 22D, and 22E, each having a plurality of joints, are attached to the palm portion. Note that, although the number of fingers on the grip portion 20L is five in the first embodiment, the number of fingers may be different, such as three fingers.

[0030] A palm sensor 26 is attached to the palm of the hand. The high-resolution camera constituting the palm sensor 26 according to the first embodiment identifies the photographed baggage 100 based on the captured image information, i.e., whether it is a care product such as shampoo, conditioner, cosmetics, or toothpaste, or whether it is food such as instant ramen or a bag of snacks.

[0031] In other words, the high-resolution camera serves to obtain information for identifying the type of luggage 100 (shape, size, hardness, etc.).

[0032] Meanwhile, the MoPU constituting palm sensor 26 of the first embodiment together with the high-resolution camera outputs, from images of luggage 100 captured at a frame rate of 1000 frames / second or more, motion information indicating the motion of the captured luggage 100 (in this case, the relative motion between arms 5, 6) at a frame rate of, for example, 1000 frames / second or more. Note that the frame rate may be increased when detecting a moving luggage 100, and may be decreased when detecting a fixed object (a luggage 100 that does not move).

[0033] The MoPU outputs, as motion information, vector information of the movement along a predetermined coordinate axis of a point indicating the location of the luggage 100. In other words, the motion information output from the MoPU does not include information necessary to identify what the photographed luggage 100 is (the above-mentioned care product or food), but only includes information indicating the movement (movement direction and movement speed) of the center point (or center of gravity) of the luggage 100 on the coordinate axes (x-axis, y-axis, z-axis).

[0034] That is, the trajectory of the gripper 20 as it approaches the luggage 100 can be accurately guided.

[0035] Information output from the palm sensor 26, which includes a high-resolution camera and a MoPU, is supplied to the information processing device 14.

[0036] The information processing device 14 uses information from the palm sensor 26 including a high-resolution camera and an MoPU to pinpoint the position of the luggage 100 with high precision, calculates the degree of spread of the fingers 22A, 22B, 22C when gripping, the strength of the grip, and the suction force of the suction pad 24, and precisely controls the minute movements of the arms 5, 6 and the gripping portion 20, making it possible to handle the picking of various luggage 100.

[0037] (Robot Tool 21EX) In the first embodiment, the main purpose of the work type for the load 100 is to grip the load 100 with the gripping unit 20 .

[0038] On the other hand, there are cases where the type of work performed on the load 100 is not gripping, but other types of work (for example, painting, gripping to increase strength, drilling, etc.).

[0039] In this case, it is possible for the gripping unit 20 to grip tools corresponding to various tasks and face the load 100; however, when continuously performing the same type of task, the burden of maintaining and controlling the gripping state (such as controlling the relative position between the gripping unit and the tool being gripped) is large.

[0040] Therefore, in the first embodiment, instead of the gripping unit 20, a robot tool 21EX (in the first embodiment, three types, 21EXA, 21EXB, and 21EXC) is provided according to the type of work to be performed on the luggage 100, as shown in Figures 1 and 4, and, as necessary, the gripping unit 20 is replaced with the robot tool 21EX to perform work according to a type of work other than gripping the luggage 100.

[0041] (Storage example of Robot Tool 21EX) As shown in FIG. 1, the humanoid robot 1 has a belt 28 attached to the lower part of the upper body 2 (at the so-called waist position), and holders (not shown) are attached to the belt 28 to detachably hold each of the three robot tools 21EXA, 21EXB, and 21EXC.

[0042] In FIG. 1, three robot tools 21EX are shown, but the number of robot tools 21EX may be one type, two types, or four or more types, and the number to be attached can be determined according to the attributes of the luggage 100 described below.

[0043] FIG. 4 shows the detailed configurations of three types of robot tools 21 (21EXA, 21EXB, 21EX) that are equipped on the humanoid robot 1 of the first embodiment apart from the gripping part 20, and their relationships with their uses.

[0044] FIG. 4(A) is a front view of the belt 28 attached to the humanoid robot 1 of FIG.

[0045] As shown in FIG. 4(A), the robot tools 21EX are detachably attached.

[0046] 4(B) to (D) show states in which the robot tool 21EXA is gripping the load 100. FIG.

[0047] As shown in FIG. 4(B), the type of work of the robot tool 21EXA is painting, and a spray gun is attached as a tool.

[0048] A spray gun is a type of pistol-shaped painting device used for spray painting. It uses compressed air from a compressor (not shown) to spray the paint in a mist form from the tip of the spray gun, making it possible to paint the surface evenly.

[0049] As shown in FIG. 4(C), the type of work of the robot tool 21EXB is gripping, similar to the gripping unit 20, but it is structured to be able to grip special luggage 100 that is difficult for the gripping unit 20 to grip.

[0050] That is, the robot tool 21EXB has a structure similar to that of a fork used as an attachment for heavy machinery, for example. In the first embodiment, the robot tool 21EXB has a two-finger structure, and the two fingers open and close due to pressure supplied from a pressure source through piping, which increases the strength with which the tool can grip the load 100, although it has lower versatility compared to a gripping operation (gripping unit 20) using a motor or the like.

[0051] As shown in FIG. 4(D), the type of work of the robot tool 21EXC is drilling, and a drill is attached as a tool.

[0052] A drill bit of a predetermined diameter can be removably attached to the drill, and the drill bit is pre-installed according to the hole size to be drilled in the load 100.

[0053] Here, if the type of work (gripping) is different from what can be processed by the gripping unit 20, the most suitable robot tool 21 is selected based on the type of work to be performed on the luggage 100 (see Table 1), and the gripping unit 20 is replaced with the selected robot tool 21 to perform the processing.

[0054] [Table 1]

[0055] Although not shown in the figure, a palm sensor 26 is also attached to each robot tool 21EX.

[0056] 5 is a schematic diagram of an example of a control system for a humanoid robot according to the first embodiment. The control system 10 includes a sensor 12 mounted on the humanoid robot, a palm sensor 26 including a high-resolution camera and an MoPU, and an information processing device 14.

[0057] The sensor 12 sequentially acquires information indicating at least the distance and angle between the luggage 100 around the humanoid robot 1 and the arms 5 and 6 on which the humanoid robot 1 is working. The sensor 12 may be a high-performance camera, a solid-state LiDAR, a multi-color laser coaxial displacement meter, or a variety of other sensors. Other examples of the sensor 12 include a vibration meter, a thermo camera, a hardness meter, a radar, a LiDAR, a high-pixel, telephoto, ultra-wide-angle, 360-degree, high-performance camera, vision recognition, fine sound, ultrasound, vibration, infrared, ultraviolet, electromagnetic waves, temperature, humidity, spot AI weather forecast, high-precision multi-channel GPS, low-altitude satellite information, or long-tail incident AI data.

[0058] In addition to the above information, the sensor 12 detects images, distance, vibration, heat, smell, color, sound, ultrasonic waves, ultraviolet rays, infrared rays, etc. Other information detected by the sensor 12 includes the movement of the center of gravity of the humanoid robot 1, the material of the floor on which the humanoid robot 1 is placed, the outside air temperature, the outside air humidity, the up / down / side / diagonal inclination angle of the floor, the amount of moisture, etc.

[0059] The sensor 12 performs these detections, for example, every nanosecond.

[0060] Palm sensor 26 (high resolution camera and MoPU) is a sensor provided on gripping portion 20 of arm portions 5, 6, and has a camera function for photographing baggage 100 and a position identification function for identifying the position of baggage 100, separately from sensor 12.

[0061] When one MoPU 12 is used, it is possible to obtain vector information of the movement of a point indicating the location of the luggage 100 along each of two coordinate axes (x-axis and y-axis) in a three-dimensional orthogonal coordinate system. Using the principle of a stereo camera, two MoPUs 12 may be used to output vector information of the movement of a point indicating the location of the luggage 100 along each of three coordinate axes (x-axis, y-axis, z-axis) in a three-dimensional orthogonal coordinate system. The z-axis is the axis along the depth direction (vehicle travel).

[0062] The information processing device 14 includes an information acquisition unit 140 , a control unit 142 , and an information storage unit 144 .

[0063] The information acquisition unit 140 acquires information about the baggage 100 detected by the sensor 12 and the palm sensor 26 (high-resolution camera and MoPU).

[0064] The control unit 142 uses the information acquired by the information acquisition unit 140 from the sensor 12 and AI (Artificial Intelligence) to control the rotational movement of the connecting unit 4, the movement of the arms 5, 6, and the like.

[0065] Furthermore, the control unit 142 uses the information acquired by the information acquisition unit 140 from the palm sensor 26 (high resolution camera and MoPU) to grasp the type (shape, size, hardness, etc.) and position of the luggage 100 in detail, and controls the palm side 20A to face the luggage 100 according to the external shape and position, to be sucked by the suction pad 24, and to be gripped by the three finger portions 22A, 22B, 22C (grasping control). Note that the type of luggage 100 may be grasped based on the external shape information, and gripping control (only "suction", only "grasping", a combination of "suction" and "grasping", etc.) may be selected.

[0066] For example, the control unit 142 executes the following processes as an overall operation. (1) The connecting portion 4 is driven to tilt the upper body portion 2 forward or backward so that the luggage 100 on a shelf or the floor can be picked up. (2) The arms 5, 6 and the gripping portion are driven so that the load 100 can be grasped. (3) The upper body 2 is driven up and down relative to the legs 3 to match the height of the workbench on the production line. (4) Maintain balance to prevent humanoid robot 1 from falling over. (5) The driving of the wheels 7, 8 is controlled so that the humanoid robot 1 can push a cart or the like.

[0067] For example, when picking up luggage 100 lying on the floor, the information processing device 14 acquires information about the luggage 100 detected by the sensor 12, and using the acquired information about the luggage 100 and AI, controls the connecting part 4 and the arms 5 and 6 to pick up the luggage 100 lying on the floor and move the picked-up luggage 100 to a predetermined position.

[0068] The operation of the first embodiment will be described below.

[0069] (Grip control of luggage 100) FIG. 6 is a flowchart showing a procedure of gripping control when the gripper 20 grips the cargo 100. As shown in FIG.

[0070] In step 150, it is determined whether or not an instruction to grasp the luggage 100 has been given. If a positive determination is made, the process proceeds to step 152, in which the humanoid robot 1 is moved (e.g., the arms 5 and 6 are operated) so that the palm side faces the target luggage 100, and the process proceeds to step 154.

[0071] In step 154, the palm side 20A is faced and information about the baggage 100 is detected by the palm sensor 26 (high resolution camera and MoPU).

[0072] In the next step 156, the information detected by the palm sensor 26 is analyzed to ascertain the type (shape, size, hardness, etc.) and position of the baggage 100 in detail, and the process proceeds to step 158.

[0073] In step 158, a selection process is performed for the type of work (gripping or a type of work other than gripping) of the gripper 20 according to the attributes of the luggage 100. The selection process will be described in detail with reference to Fig. 7, and if the type of work is gripping in the selection process of Fig. 7, the process proceeds to step 160.

[0074] In step 160, a task for gripping the load 100 is selected. For example, a task is selected from "suction" only, "grabbing" only, a combination of "suction" and "grabbing", etc., and the process proceeds to step 162.

[0075] In step 162, gripping of the load 100 ("suction" only, "grab" only, "suction" and "grab") is performed.

[0076] In the next step 164, it is determined whether the gripping of the luggage 100 was successful, and if a positive determination is made, the gripped luggage 100 is carried to a predetermined location, and the process proceeds to step 150 to wait for an instruction to grip the next luggage 100.

[0077] If the determination in step 164 is negative, the process proceeds to step 166, where an error process is performed.

[0078] The gripper 20 is fitted with a palm sensor 26 including a high-resolution camera and an MoPU. By mounting the gripper 20 having the above structure on the arms 5 and 6 of the humanoid robot 1, an object can be reliably picked up by the suction surface, and even if the humanoid robot 1 moves quickly, the baggage 100 can be carried without dropping from the gripper 20.

[0079] In addition, since the palm sensor 26 (high-resolution camera and MoPU) is mounted on the palm side 20A, the baggage 100 can be captured with high accuracy, and the device can also be used for tasks that require minute movements.

[0080] Furthermore, very soft and fragile items can be gripped by the movement of the fingers 22A, 22B, 22C without using the suction pad 24, and damage to the soft cargo 100 can be prevented by adjusting the gripping force.

[0081] (Details of robot tool application process) FIG. 7 is a control flow chart showing details of the work type selection process (robot tool application process) subroutine in step 158 in FIG.

[0082] In step 198, it is determined whether the type of work is grasping or a work other than grasping. If the type of work is determined to be grasping in this step 198, an instruction is given to return to step 160 in Fig. 6, and this routine ends.

[0083] If it is determined in step 199 that the task type is other than grasping, the process proceeds to step 200, where the robot tool 21 is selected based on the task type for the load 100 (see Table 1).

[0084] That is, as shown in Table 1, the type of robot tool 21 required is determined according to the type of work to be performed on the load 100.

[0085] The selection of the type of robot tool based on the type of work for the luggage 100 shown in Table 1 is merely an example, and may be determined based on the type and number of robot tools possessed.

[0086] In the first embodiment, three types of robot tools 21 are attached to the belt 28, and a selection is made from these three types of robot tools 21, but the number of robot tools attached to the belt 28 may be increased, or multiple types of robot tools 21 may be selected and attached in advance to the belt 28 depending on the work site. Also, a different type of robot tool 21 may be attached to each humanoid robot 1.

[0087] In the next step 201A, it is determined whether or not the selected robot tool is equipped, and if the determination is negative, the process proceeds to step 201B, where a request is made to procure the robot tool. In the next step 201C, the engraving process of the robot tool is executed, and the process proceeds to step 202. Also, if the determination is positive in step 201A, the process proceeds to step 202.

[0088] In the next step 202, the tool attached to the replacement arm 6 (or 5) (usually the gripper 20, but there are also cases where another robot tool 21 has already been attached) is removed.

[0089] In the next step 204, the arm 6 (or 5) on the replacement side is moved to the position of the belt 28, and the robot tool 21 is attached.

[0090] In the next step 206, the tool originally attached to the replacement arm 6 (or 5) that is being held by the non-replacement arm 5 (or 6) is stored in a holder on the belt 28, and the process proceeds to step 208.

[0091] In step 208, the programmed tasks are executed for each robot tool. In the next step 210, a restart of the flow chart in Fig. 6 (transition to step 150) is instructed, and this routine ends.

[0092] As described above, in the first embodiment, in addition to controlling the gripping operation by the gripper 20, the robot tool 21 is prepared in advance according to the type of work to be performed on the load 100, for example.

[0093] As a result, the control burden (such as the burden associated with controlling the relative position between the gripper 20 and the tool) that arises when the gripper of the comparative example grips and works on the tool required for the type of work can be eliminated by directly attaching the robot tool 21.

[0094] (Second embodiment: "Replacing finger tips") A second embodiment of the present invention will be described below. Note that the same components as those in the second embodiment are given the same reference numerals and the description of the components will be omitted.

[0095] The second embodiment is characterized in that the finger tips, which are the gripping portion 20, can be changed into a tool according to the type of work, like the robot tool described in the first embodiment.

[0096] As shown in FIG. 8(A), the hand tool 50 of the second embodiment is basically intended to grip a load 100, similar to the gripping portion 20 of the first embodiment.

[0097] Each finger can be attached or detached from the hand tool 50. In the state shown in Fig. 8(A), the first joint portion 50A (the "finger tip" shown in Fig. 8(B)) for the main purpose is attached.

[0098] Meanwhile, as shown in FIG. 8(B), each finger portion is replaceable with a first joint portion 50B to 50F having a shape according to the type of work.

[0099] The first joint portion 50B is a "tweezer," or what is commonly called tweezers, and is used to pinch and grasp thin members.

[0100] The first joint portion 50C is a "stick" and is used when holding a part with a rod-shaped member.

[0101] The first joint portion 50D is a "cotton swab" and is a spherical cotton swab that is used to wipe off dirt from parts or remove moisture.

[0102] The first joint portion 50E is a "camera" and is used when photographing a part (particularly a close-up shot).

[0103] The first joint portion 50F is a "screwdriver" and is used when tightening or loosening a screw.

[0104] The hand tool according to the second embodiment is particularly suitable for tasks such as assembling and disassembling small parts (electronic parts, precision parts, etc.).

[0105] (Third embodiment "sorting and picking work") A third embodiment of the present invention will be described below. In the third embodiment, the humanoid hand tool 50 described in the second embodiment is applied, but the humanoid hand tool 50 is not necessarily required, and may be positioned as the gripper 20 or the robot tool 21EX described in the first embodiment.

[0106] The sorting and picking work refers to the work of selecting and picking out garbage 62 of a specific attribute (type) from garbage 62 of different attributes (types) mixed together in a single box 60, as shown in Figure 9.

[0107] 9, six types of garbage 62 are mixed and stored in a box 60. The garbage includes plastic, paper, and glass as recyclable garbage, and batteries, lighters, and spray cans as hazardous garbage.

[0108] In the third embodiment, sensor units 51A to 51E (described in detail later) capable of detecting attributes of dirt 62 are attached to each fingertip of hand tool 50, and hand tool 50 is placed opposite box body 60 to detect the attributes using each of sensor units 51A to 51E.

[0109] An example of the sensor units 51A to 51E attached to each finger is shown below. (Thumb) Visual sensor 51A that detects visual information as attribute information (Index finger) Auditory sensor 51B that detects auditory information as attribute information (Middle finger) Olfactory sensor 51C that detects olfactory information as attribute information (Ring finger) Tactile sensor 51D that detects tactile information as attribute information (Little finger) Taste sensor 51E that detects taste information as attribute information

[0110] Note that the sensor units 51A to 51E are not limited to those described above, and the detection form is not particularly limited to contact, non-contact, etc. Furthermore, detection sensors with the same type of attribute may be attached as necessary.

[0111] In the hand tool 50, the finger joints can freely move relative to one another above the box body 60, so that the sensor units 51A to 51E at the fingertips can freely change their detection areas. Therefore, the entire area of ​​the box body 60 can be used as the detection area.

[0112] Although not shown in FIG. 9, the sensor units 51A to 51E on the fingertips can leave the fingertips and enter the storage space for the waste 62 in the box 60, making it possible to detect attribute information of waste in a visually hidden position.

[0113] Here, when the type of garbage 62 to be picked (object) is specified, the attributes of the garbage 62 to be picked (attribute information obtained through the five senses of sight, hearing, smell, touch, and taste) are read out from the object-attribute information database 78 (see FIG. 10) and compared with the attribute information detected by the sensor units 51A to 51E, thereby making it possible to identify the position coordinates (three-dimensional) of the garbage 62 to be picked.

[0114] Based on the position coordinates, for example, the gripper 20 shown in Fig. 3 is attached to perform the picking operation. In this case, the hand tool 50 of the third embodiment may be attached in advance to the right hand side, and the gripper 20 for the picking operation may be attached to the left hand side.

[0115] Fig. 10 is a functional block diagram specialized for object attribute determination control executed in the information processing device 14 (see Fig. 5). Note that each block shown in Fig. 10 is classified by function, and some or all of the object attribute determination control function may be operated by a software program using a microcomputer (including ASIC, etc.).

[0116] The object attribute determination control function of the information processing device 14 includes a data acquisition unit 70. The data acquisition unit 70 acquires detection data of the sensors 51A to 51E attached to the fingertips of the hand tool 50.

[0117] The data acquisition unit 70 is connected to the collation unit 72. The data acquisition unit 72 sends to the collation unit 72 the acquired detection data from each of the sensor units 51A to 51E.

[0118] Furthermore, the object attribute determination control function of the information processing device 14 includes a picking object information acquisition unit 74. The picking object information acquisition unit 74 acquires information on the garbage to be picked (picking object information). This picking object information acquisition unit 74 is connected to a search unit 76 and transmits the picking object information.

[0119] The search unit 76 accesses the object-attribute information database 78 and reads out the attribute information of the object designated for picking.

[0120] The attribute information read by the search unit 76 is sent to the collation unit 72 .

[0121] Here, the matching unit 72 matches the attribute information received from the search unit 76 with the attribute information received from the data acquisition unit 70, and sends the matching result to the picking position coordinate identifying unit 79. The picking position coordinate identifying unit 79 identifies the position coordinates (three-dimensional coordinates) of the object (garbage 62) in the box 60 (see FIG. 9) whose attribute information matches the match.

[0122] The position coordinates identified by the picking position coordinate identifying unit 79 are sent via a coordinate information output unit 80 to a picking control unit (not shown), which is another function of the information processing device 14.

[0123] As a result, the picking control unit controls the operation of the gripping unit 20 for the picking operation based on the received position information, thereby enabling pinpoint picking of the garbage 62 designated for picking that is located within the box 60.

[0124] The operation of the third embodiment will be described below with reference to the flowchart of FIG.

[0125] In step 82, a sensor function is attached to each fingertip of the hand tool 50. As an example, a visual sensor 51A is attached to the thumb, an auditory sensor 51B to the index finger, an olfactory sensor 51C to the middle finger, a tactile sensor 51D to the ring finger, and a taste sensor 51E to the little finger.

[0126] If necessary, the relationship between finger type and sensor type may be changed, and only a necessary and sufficient number of sensors may be attached.

[0127] In addition, in the third embodiment, the hand tool 50 as the gripping portion 20 is attached to the arms 5 and 6, and the sensor units 51A to 51E are attached to the fingertips. However, the arms 5 and 6 may be replaced with a robot tool 21EX that belongs to the robot tool 21EX shown in the first embodiment and has a sensor function in advance.

[0128] In the next step 84, each of the sensor units 51A to 51E is made to face the box body 60 in which the objects are mixed and housed, and the process proceeds to step 86.

[0129] In step 86, the attribute information of each of the objects is obtained by each of the sensor units 51A to 51E.

[0130] In the next step 88, information on the object to be picked is acquired. For example, when a command is given to pick a battery that belongs to hazardous waste, attribute information on the battery is acquired. Batteries have almost no sense of hearing, smell, or taste, but can be distinguished from other waste 62 based on sight and touch. For example, spray cans that belong to hazardous waste can be distinguished from other waste 62 by sight, touch, and smell (thinner smell, etc.).

[0131] In the next step 90, attribute information of the object to be picked is read from an object-attribute information database (DB), and the process proceeds to step 92 to execute a comparison of the attribute information. That is, the read attribute information is compared with the detected attribute information.

[0132] In the next step 94, the location that matches as a result of the comparison is set as the picking position, and the coordinates of that picking position (three-dimensional coordinates set in the space inside the box 60) are identified, and then the process proceeds to step 96, where the identified picking position coordinates are sent to the picking control unit, and this routine ends. The picking control unit controls the gripper 20 based on the identified picking position coordinates to pick up the target object, that is, the waste 62.

[0133] In the third embodiment, the task of sorting and picking garbage has been exemplified. However, an example of a task in which different sensor units 51A-51E are attached to each fingertip of the hand tool 50 is inspection of the inside of an engine compartment 98A of a vehicle 98, as shown in FIG. 12.

[0134] By having the humanoid robot 1 face the engine room 98A and use the hand tool 50 of the third embodiment, it is possible to detect, as well as locate, defects such as oil leaks from the engine, damage and cracks in resin and rubber products, loose bolts, poor contact in electrical components, and fallen off parts in a single inspection of the engine room 98A.

[0135] 13 is a schematic diagram showing an example of a hardware configuration of a computer 1200 functioning as the information processing device 14. A program installed in the computer 1200 can cause the computer 1200 to function as one or more "parts" of the device according to the first embodiment and the second embodiment (including examples), or can cause the computer 1200 to execute operations or one or more "parts" associated with the device according to the first embodiment and the second embodiment (including examples), and / or can cause the computer 1200 to execute a process or a step of the process according to the first embodiment and the second embodiment (including examples). Such a program can be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0136] The computer 1200 according to the first to third embodiments includes a CPU 1212, a RAM 1214, and a graphic controller 1216, which are connected to each other by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive, a solid state drive, etc. The computer 1200 also includes input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0137] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 retrieves image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0138] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0139] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or a program that depends on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0140] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by implementing operations or processing of information according to the use of the computer 1200.

[0141] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0142] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0143] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may execute various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout the present invention and specified by the instruction sequence of the program, and write back the results to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0144] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0145] The blocks in the flowcharts and block diagrams in the first to third embodiments may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements, such as field programmable gate arrays (FPGAs) and programmable logic arrays (PLAs).

[0146] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0147] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0148] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0149] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0150] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order.

[0151] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0152] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order. [Explanation of symbols]

[0153] 1 humanoid robot, 2 upper body, 3 legs, 4 connecting portion, 5, 6 arms, 7, 8 wheels, 10 control system, 12 sensor, 14 information processing device, 20 (20L, 20R) gripping portion, 20A palm side, 20B main body, 22A, 22B, 22C fingers, 24 (24X, 24Y) suction pad, 24A pad portion, 24B nipple, 26 palm sensor, 100 luggage, 50 hand tool, 50A first joint portion, 50B first joint portion, 50C first joint portion, 50D first joint portion, 50E first joint portion, 50F first joint portion, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphic controller, 1218 display device, 1220 I / O controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 I / O chip

Claims

1. A robot control system for mounting a robot tool required for a type of work to be performed on an object to a robot and causing the robot to perform a task corresponding to the type of work, comprising: a selection unit that selects a robot tool having a plurality of attribute information detection units attached thereto, the plurality of attribute information detection units being capable of detecting each of a plurality of pieces of attribute information that can be acquired from the object; an attribute information acquisition unit that acquires attribute information of a type capable of identifying the object by using the plurality of attribute information detection units; a position specifying unit that specifies a position of the object for which the work type is specified by comparing attribute information of the object for which the work type is specified with attribute information acquired by the attribute information acquiring unit; A robot control system having the following:

2. The attribute information acquisition unit, The robot control system according to claim 1 , further comprising: acquiring attribute information of a type capable of identifying the object under an environmental condition in which a plurality of the objects are mixed.

3. The attribute information detection unit, Detecting attribute information for each of the senses of vision, hearing, smell, touch, and taste, and identifying the object based on a combination of a plurality of pieces of attribute information; The control system for the robot according to claim 1.

4. 2. The robot control system according to claim 1, wherein the type of work is a picking task of picking specified waste from a storage area in which multiple types of recyclable waste and multiple types of hazardous waste are mixed.

5. The robot control system of claim 1 , wherein the robot is equipped with a camera that takes an image of the object and identifies the type of the object, and a motion processing unit that identifies the position of the object.

6. A robot control program that causes a computer to operate as the selection unit, the attribute information acquisition unit, and the position identification unit according to any one of claims 1 to 5.

Images