Robot control system, robot and robot control program

Abstract

To reduce control burdens, by directly executing a plurality of types of work including work for grasping luggage by a grasping part.SOLUTION: An arm part 5 pulls a webbing 80 while grasping the webbing in order to confirm a fitting state of a seat belt device 78. A camera closeup-photographs fitting bolts 81A, 82A and 83A at subtle points to perform monitoring. If the seat belt device is loosened, additional fastening work is performed. The fastening state of the seat belt device 78 directly relating to safety can be monitored precisely in stead of a person, by utilizing robot tools including multi joints, long-bodies, long-neck and fingers such as arm parts 5 and 6.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a robot control system, a robot, and a robot control program. [Background technology]

[0002] Humanoid robots are used in factory production lines to perform tasks automatically. Patent Document 1 describes posture control of a humanoid robot.

[0003] Furthermore, Patent Document 2 describes a robot arm having multiple joints that is driven by an elastic actuator, and which is controlled by a hand support member that is arranged at the hand of the robot arm and supports the robot arm by contacting a support surface, and a control unit that controls the contact force between the hand support member and the support surface while also controlling the position and posture of the hand of the robot arm. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-093506 A [Patent Document 2] WO2011 / 001569 publication Summary of the Invention [Problem to be solved by the invention]

[0005] However, conventional picking tasks in warehouses using humanoid robots currently rely on human hands when picking up items (such as shampoo, conditioner, cosmetics, toothpaste, instant ramen, snack bags, etc., which vary in shape, weight, hardness, and fragility) from shelves on which they are displayed, and then packing them into the appropriate packaging (boxes, etc.).

[0006] Even if the structure of the robot's gripping section is changed to a finger type, the fingers and arms move slowly, resulting in low productivity. Furthermore, when performing tasks other than gripping luggage, the gripping section must grip the necessary tools (e.g., drills, screwdrivers, etc.), which is an indirect task that requires management control of the gripping state, and this can result in a large control burden.

[0007] In consideration of the above, an object of the present invention is to provide a robot control system, a robot, and a robot control program that can reduce the control burden by directly performing multiple types of work, including gripping cargo using a gripper. [Means for solving the problem]

[0008] A robot control system according to a first aspect of the present invention is a robot control system that is attachable to and detachable from the wrist of an arm that moves in three dimensions, is equipped with a tool corresponding to the type of work to be done on an object, and performs work on the object, and has a determination unit that determines the type of work to be done on the object, and a control unit that controls the operation of attaching the tool corresponding to the work type determined by the determination unit and controls the operation corresponding to the work type, and is characterized in that the robot has a plurality of arms to which at least the tool can be attached, one arm is selectively equipped with a tool corresponding to the type of work to be done on the object, and is equipped with a monitoring device that monitors the work area in conjunction with the work performed with the attached tool, and the other arms are equipped with auxiliary work tools that can perform auxiliary work to the main work on the object.

[0009] According to the first aspect of the present invention, the control unit controls the operation of attaching a tool corresponding to the work type determined by the determination unit that determines the work type for the target object, and controls the operation according to the work type. This makes it possible to reduce the control burden by directly executing work of multiple work types, including gripping of an object by the gripper.

[0010] The robot also has a plurality of arms to which at least the tools can be attached, and one arm is selectively attached with a tool according to the type of work to be done on the object, and is equipped with a monitoring device that monitors the work area in conjunction with the work performed with the attached tool, and the other arms are equipped with auxiliary work tools that can perform auxiliary work to the main work on the object.

[0011] The multiple arms can each perform their own role (main task and auxiliary task), and the monitoring device can monitor the work area in conjunction with the task being performed by the attached tool.

[0012] A second aspect of the present invention is characterized in that in the second aspect, the auxiliary work tool has a function of a human hand and holds the object.

[0013] It can hold the object during the main task, allowing the main task to be performed stably.

[0014] A third aspect of the present invention is characterized in that, in the invention described in the first or second aspect, the determination unit makes a determination based on information from a sensor unit equipped with a camera mounted on the tool that takes an image of the object and identifies the type of the object, and a motion processing unit that specifies the position of the object.

[0015] The camera identifies the photographed object (hereinafter, sometimes referred to as baggage) based on the photographed image information. In other words, the camera has the role of acquiring information for identifying the type of object (shape, size, hardness, etc.).

[0016] The motion processing unit (MoPU) outputs vector information of the movement of a point indicating the location of an object along a specified coordinate axis as position information together with the movement information. In other words, the movement information output from the MoPU contains only information indicating the movement (movement direction and movement speed) of the center point (or center of gravity) of the object on the coordinate axes (x-axis, y-axis, z-axis). In other words, it is possible to provide accurate guidance on the trajectory of the gripping unit when it approaches the object.

[0017] A fourth aspect of the present invention is a robot that operates in three dimensions and has a plurality of arms to which at least work tools can be attached, and that performs work on an object, wherein one arm is selectively attached with a tool corresponding to the type of work to be performed on the object, and is attached with a monitoring device that monitors the work area in conjunction with the work performed with the attached tool, and the other arms are attached with auxiliary work tools that can perform auxiliary work to the main work on the object.

[0018] A robot control program according to a fifth aspect of the present invention is characterized in that in the invention described in any one of the first to third aspects, the program causes a computer to operate as the determination unit and the control unit.

[0019] The above summary of the invention does not list all of the necessary features of the present invention. Also, subcombinations of these features may also be inventions. Effect of the Invention

[0020] As described above, according to the present invention, the control burden can be reduced by directly executing a plurality of types of work including gripping an object with a gripper. [Brief description of the drawings]

[0021] [Figure 1] FIG. 1 is a front view of a humanoid robot according to a first embodiment. [Diagram 2] FIG. 1 is a side view of a humanoid robot according to a first embodiment. [Diagram 3] FIG. 2 is a plan view showing a state in which a robot tool is attached to the humanoid robot according to the first embodiment. [Figure 4] FIG. 4 is a perspective view showing a working state of a robot tool corresponding to FIGS. 3(A) to (D). [Diagram 5] FIG. 2 is a diagram illustrating an example of a functional configuration of a humanoid robot according to a first embodiment. [Figure 6] 13 is a flowchart showing a procedure of gripping control when gripping an object with a gripper in conjunction with the overall movement of the humanoid robot. [Figure 7] 13 is a control flowchart showing details of a robot tool application processing subroutine. [Figure 8] FIG. 11 is a plan view of a humanoid robot according to a modification (modification 1) of the first embodiment. [Figure 9] FIG. 10(A) is a front view of an arm according to a second embodiment, and (B) to (D) are front views of a robot tool that can be attached to the arm. [Figure 10] FIG. 13 is a perspective view of an arm according to a modified example (modified example 2) of the second embodiment. [Figure 11] FIG. 13 is a perspective view of an arm according to a modified example (modified example 3) of the second embodiment. [Figure 12] FIG. 11 is a perspective view showing a working situation in an EV car production line using an arm according to a second embodiment. [Figure 13] 13A and 13B are perspective views showing the working conditions in an EV car production line using an arm according to a first practical example of the second embodiment, in which (A) is a perspective view showing the working conditions in an engine room (drive unit room), and (B) is a perspective view showing the working conditions in a vehicle cabin. [Figure 14] 11A and 11B are schematic diagrams relating to a practical example 2 of the second embodiment, in which (A) is a perspective view of a process for inspecting the installation state after the seat belt device is installed in the vehicle body, and (B) is an enlarged view of the inspection area (pillar portion) in the vehicle body. [Figure 15] FIG. 2 is a diagram illustrating an example of a hardware configuration of a computer that functions as a Central Brain. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0022] The present invention will be described below through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. Also, not all of the combinations of features described in the embodiments are necessarily essential to the solution of the invention.

[0023] (First embodiment)

[0024] Fig. 1 is a front view of a humanoid robot 1 according to a first embodiment. As shown in Fig. 1, the humanoid robot 1 according to the first embodiment includes an upper body 2, legs 3, and a connector 4 that rotatably connects the upper body 2 to the legs 3, and is placed, for example, on a production line in a factory to perform work on a work target 100 (see Fig. 2).

[0025] When the work is a picking work for grasping a package which is the object 100, the object (such as a fallen object) on the production line or on the floor is grasped. Note that the grasping work includes not only picking for grasping the object 100 from a shelf, but also packing for storing the grasped object 100 in a predetermined case (such as a cardboard box).

[0026] The upper body 2 has two arms 5 and 6. The arms 5 and 6 are attached to the left and right of the upper body 2 so as to be freely rotatable. In addition, robot tools 20L and 20R (described in detail later) for performing a predetermined task on the target object 100 are attached to the tips of the arms 5 and 6. The number of arms is not limited to two, and may be one or three or more.

[0027] The leg 3 has two wheels 7, 8 attached to its lower portion, enabling the humanoid robot 1 to move on the floor on which it is placed.

[0028] The connecting unit 4 rotatably connects the upper body 2 and the legs 3. Therefore, the upper body 2 can lean forward and backward with respect to the legs 3. Therefore, the humanoid robot 1 according to the first embodiment can lean the upper body 2 forward with respect to the legs 3, as shown in Fig. 2, and pick up an object 100 placed on a shelf, an object 100 placed on the floor, or an object 100 that has fallen to the floor during work.

[0029] In addition, the legs 3 have a balancing function to prevent the humanoid robot 1 from falling over when the upper body 2 leans forward or backward relative to the legs 3 or when the humanoid robot 1 moves.

[0030] 1, the connecting part 4 has a function of changing the distance between the upper body part 2 and the legs 3. Therefore, the vertical position of the upper body part 2 relative to the legs 3 can be adjusted as shown by the arrow A so as to match the height of the workbench in the production line.

[0031] Moreover, the driving of the humanoid robot 1 according to the first embodiment is controlled by a control system 10 (see FIG. 5) implemented within the humanoid robot 1.

[0032] (Structure of Robot Tool 20L and 20R) As shown in FIG. 3A, the robot tool 20L attached to the tip of the arm 6, which corresponds to the left hand of a human, has a structure similar to that of a human hand (Intelligent Hand System). ) mainly performs auxiliary work such as holding the target object 100 so that it does not move when working with the robot tool 20R.

[0033] As shown in FIG. 3A, a robot tool 20R attached to the tip of the arm 5, which corresponds to a human right hand, is a tool for performing a specific task.

[0034] The robot tools 20L, 20R are connected to the arms 6, 5 by a universal joint structure, and are attached so as to be freely rotatable in three dimensions. More specifically, the wrists can rotate (twist) and move up and down (cocking), and further, the wrists can extend and retract as necessary.

[0035] In addition, at least the robot tool 20R attached to the arm 5 is detachable from the arm 5. It can be replaced with the Robot Tool 21EX, which will be described later.

[0036] The robot tool 20L attached to the arm 6 is not intended to be detachable, but may be detachable like the robot tool 20R.

[0037] The arm 5 is replaced with a tool for executing the instructed work. On the other hand, the arm 6 is positioned as an auxiliary work tool having a role of assisting in the execution of the instructed work (for example, supporting the object 100 to assist in grasping the object 100, or holding the object 100 so that it does not move when processing the object 100, etc.).

[0038] In the humanoid robot 1 of the first embodiment, the structure corresponds to right-handedness in humans, and the robot tool 20R is detachably attached to the right arm 5, which is the dominant arm for actual work, and the robot tool 20L (auxiliary work tool) having the structure of a human hand is attached to the left arm 6, which is the non-dominant arm. However, the structure may also correspond to left-handedness in humans (the auxiliary work tool (human hand) on the arm 5, and the work tool on the arm 6).

[0039] (Robot Tool 20L) 3, the robot tool 20L according to the first embodiment has a palm portion as a base corresponding to a so-called human palm, and five fingers each having a plurality of joints are attached to the palm portion. Note that, although the number of fingers of the robot tool 20L is five in the first embodiment, the number of fingers may be different, such as three fingers.

[0040] A palm sensor 26 is attached to the palm of the hand. The high-resolution camera constituting the palm sensor 26 according to the first embodiment identifies the photographed object 100 as a care product such as shampoo, conditioner, cosmetics, or toothpaste, or as food such as instant ramen or a snack bag, based on the photographed image information.

[0041] In other words, the high-resolution camera serves to obtain information for identifying the type of object 100 (shape, size, hardness, etc.).

[0042] Meanwhile, the MoPU constituting palm sensor 26 of the first embodiment together with the high-resolution camera outputs, from an image of object 100 captured at a frame rate of 1000 frames / second or more, motion information indicating the motion of captured object 100 (in this case, the relative motion between arms 5, 6) at a frame rate of, for example, 1000 frames / second or more. Note that when detecting a moving object 100, the frame rate may be increased, and when detecting a fixed object (non-moving object 100), the frame rate may be decreased.

[0043] The MoPU outputs, as motion information, vector information of the movement along a predetermined coordinate axis of a point indicating the location of the object 100. In other words, the motion information output from the MoPU does not include information necessary to identify what the photographed object 100 is (the above-mentioned care product or food), but only includes information indicating the movement (movement direction and movement speed) of the center point (or center of gravity) of the object 100 on the coordinate axes (x-axis, y-axis, z-axis).

[0044] In other words, the trajectory of the robot tool 20 when approaching the target object 100 can be guided with high accuracy.

[0045] Information output from the palm sensor 26 including the high-resolution camera and the MoPU is supplied to the information processing device 14. The information processing device 14 functions as a determination unit and a control unit of the present invention.

[0046] The palm sensor 26 (high-resolution camera and MoPU) is shown as the location closest to the work. However, it is not essential that the high-resolution camera and MoPU are attached to the palm of the hand. For example, the high-resolution camera and MoPU may be attached to the back of the hand or the wrist. In order to grasp the work from a bird's-eye view, the high-resolution camera and MoPU may be attached to the head of the humanoid robot 1 shown in Fig. 3. Furthermore, the high-resolution camera and MoPU may be attached to multiple positions.

[0047] The information processing device 14 uses information from the palm sensor 26, which includes a high-resolution camera and a MoPU, to pinpoint the position of the object 100 with high precision, calculates the degree to which the fingers are spread when grasping, the strength of the grip, and the suction force of the suction pad, etc., and precisely controls the minute movements of the arms 5, 6 and the robot tool 20, making it possible to handle picking operations of various objects 100.

[0048] (Robot Tool 20R and Robot Tool 21EX) In the first embodiment, the main purpose of the work type for the object 100 is to grip the object 100 with the robot tool 20R.

[0049] On the other hand, there are cases where the type of work performed on the object 100 is not grasping but other types of work (for example, grasping a heavy object, drilling a hole with a drill, wiping off dirt, handling a minute object (tweezers work), etc.).

[0050] In this case, it is possible for the robot tool 20R to grasp tools corresponding to various tasks and confront the target object 100; however, when continuously performing the same type of task, the burden of maintaining and controlling the grasping state (such as controlling the relative position between the grasping part and the tool being grasped) is large.

[0051] Therefore, in the first embodiment, instead of the robot tool 20R (see FIG. 3(A)), which is the basic tool attached to the arm 5, robot tools 21EX (three types, 21EXA, 21EXB, and 21EXC in the first embodiment) are provided according to the type of work to be performed on the object 100, as shown in FIGS. 3(B) to 3(C), and the robot tool 20R is replaced by the robot tool 21EX as necessary to perform work according to a type of work other than grasping the object 100.

[0052] (Storage example of Robot Tool 21EX) As shown in FIG. 1, the humanoid robot 1 has a belt 28 attached to the lower part of the upper body 2 (at the so-called waist position), and holders (not shown) are attached to the belt 28 to detachably hold each of the three robot tools 21EXA, 21EXB, and 21EXC.

[0053] In FIG. 1, three robot tools 21EX are shown, but the number of robot tools 21EX that can be attached in place of the robot tool 20R may be one type, two types, or four or more types, and the number to be attached can be determined according to the attributes of the target object 100 described later.

[0054] 3 and 4 show the detailed configurations of the robot tool 20R and the robot tools 21 (21EXA, 21EXB, 21EX) that can be attached to the arm portion 5 in place of the robot tool 20R, and their relationships with their uses.

[0055] 3(A) and 4(A) show a state in which the robot tool 20R, which is a basic tool of the robot tools, is gripping an object 100. The type of work of the robot tool 21EXB is gripping, like the robot tool 20L, but it is structured to be able to grip special objects 100 (overweight, etc.) that are difficult to grip with the robot tool 20L.

[0056] That is, the robot tool 20R has a structure similar to that of a fork applied as an attachment for heavy machinery, for example. In the first embodiment, the robot tool 20R has a two-finger structure, and the two fingers open and close due to pressure supplied from a pressure source through a pipe, so that the robot tool 20R has a lower versatility than the gripping operation (robot tool 20L) by a motor or the like, but has an increased strength for gripping the target object 100.

[0057] During the operation by the robot tool 20R, the robot tool 20L holds the target object 100 so as not to move.

[0058] 3(B) and 4(B), the type of work of the robot tool 21EXA is drilling, and a drill is attached as a tool. A drill bit having a predetermined diameter can be removably attached to the drill, and the drill bit is attached in advance according to the hole size to be drilled in the target object 100.

[0059] During the operation with the robot tool 21EXA, the robot tool 20L holds the target object 100 so that it does not move.

[0060] As shown in FIG. 3(C) and FIG. 4(C), the type of work of the robot tool 21EXB is wiping off dirt, and a cotton swab (for example, a cotton bud) is attached to it.

[0061] The robot tool 21EXB is configured to be able to wipe off the dirty area 100A adhering to the surface of the object 100 held down by the robot tool 20L by contacting it with a cotton swab and oscillating it back and forth.

[0062] During the operation by the robot tool 21EXB, the robot tool 20L holds the target object 100 so as not to move.

[0063] As shown in Figures 3(D) and 4(D), the type of work of the robot tool 21EXC is tweezers work to pick up and remove minute objects, and tweezers are attached as tools. Incidentally, examples of minute contaminants include solder paste when attaching electronic components to an electric circuit board, and welding waste after welding when welding metal.

[0064] The robot tool 21EXC is structured so that fine particles 100B such as dust particles adhering to the surface of the target object 100 held down by the robot tool 20L can be picked up and removed with tweezers. Incidentally, examples of dust particles include shavings generated when a hole is drilled. The tool can also be used to remove screws and the like that have become lodged in narrow grooves.

[0065] During the operation by the robot tool 21EXC, the robot tool 20L holds the target object 100 so that it does not move.

[0066] In addition, although not mounted on the humanoid robot 1 in the first embodiment, robot tools for other types of work, such as a spray gun for painting, an electric drill, a carving knife, a writing instrument, and a molding agent dispenser for 3D modeling, may also be attached.

[0067] For example, a spray gun is a type of pistol-shaped painting device used for spray painting. It uses compressed air from a compressor (not shown) to spray the paint in a mist form from the tip of the spray gun, making it possible to paint the surface evenly.

[0068] In the first embodiment, the optimum robot tool 20R, robot tool 21EX is selected based on the type of work (see Table 1), and the robot tool 20 is replaced with the selected robot tool 21 to execute the process.

[0069] [Table 1]

[0070] Although the palm sensor 26 is attached to the robot tool 20L, the sensor unit 26 may also be attached to each robot tool 21EX.

[0071] 5 is a schematic diagram of an example of a control system 10 for the humanoid robot 1 according to the first embodiment. The control system 10 includes a sensor 12 mounted on the humanoid robot, a palm sensor 26 including a high-resolution camera and an MoPU, and an information processing device 14.

[0072] The sensor 12 sequentially acquires information indicating at least the distance and angle between the target object 100 around the humanoid robot 1 on which the humanoid robot 1 is working and the arms 5 and 6. The sensor 12 may be a high-performance camera, a solid-state LiDAR, a multi-color laser coaxial displacement meter, or a variety of other sensors. Other examples of the sensor 12 include a vibration meter, a thermo camera, a hardness meter, a radar, a LiDAR, a high-pixel, telephoto, ultra-wide-angle, 360-degree, high-performance camera, vision recognition, fine sound, ultrasound, vibration, infrared, ultraviolet, electromagnetic waves, temperature, humidity, spot AI weather forecast, high-precision multi-channel GPS, low-altitude satellite information, or long-tail incident AI data.

[0073] In addition to the above information, the sensor 12 detects images, distance, vibration, heat, smell, color, sound, ultrasonic waves, ultraviolet rays, infrared rays, etc. Other information detected by the sensor 12 includes the movement of the center of gravity of the humanoid robot 1, the material of the floor on which the humanoid robot 1 is placed, the outside air temperature, the outside air humidity, the up / down / side / diagonal inclination angle of the floor, the amount of moisture, etc.

[0074] The sensor 12 performs these detections, for example, every nanosecond.

[0075] The palm sensor 26 (high resolution camera and MoPU) is a sensor provided on the robot tool 20L of the arms 5, 6, and has a camera function for photographing the object 100 and a position identification function for identifying the position of the object 100, separately from the sensor 12.

[0076] When one MoPU 12 is used, it is possible to obtain vector information of the movement of a point indicating the location of the object 100 along each of two coordinate axes (x-axis and y-axis) in a three-dimensional orthogonal coordinate system. Using the principle of a stereo camera, two MoPUs 12 may be used to output vector information of the movement of a point indicating the location of the object 100 along each of three coordinate axes (x-axis, y-axis, z-axis) in a three-dimensional orthogonal coordinate system. The z-axis is an axis along the depth direction, which is the direction of the optical axis of photography.

[0077] The information processing device 14 includes an information acquisition unit 140 , a control unit 142 , and an information storage unit 144 .

[0078] The information acquisition unit 140 acquires information about the target object 100 detected by the sensor 12 and the palm sensor 26 (high-resolution camera and MoPU).

[0079] The control unit 142 uses the information acquired by the information acquisition unit 140 from the sensor 12 and AI (Artificial Intelligence) to control the rotational movement of the connecting unit 4, the movement of the arms 5, 6, and the like.

[0080] In addition, the control unit 142 uses the information acquired by the information acquisition unit 140 from the palm sensor 26 (high-resolution camera and MoPU) to grasp the type (shape, size, hardness, etc.) and position of the object 100 in detail, and controls it to confront the object 100 according to its external shape and position, for example, to grasp the object 100 (grasping control).

[0081] For example, the control unit 142 executes the following processes as an overall operation. (1) The connecting portion 4 is driven to tilt the upper body portion 2 forward or backward so that an object 100 on a shelf or the floor can be picked up. (2) The arms 5, 6 and the gripping portion are driven so as to be able to grasp the object 100. (3) The upper body 2 is driven up and down relative to the legs 3 to match the height of the workbench on the production line. (4) Maintain balance to prevent humanoid robot 1 from falling over. (5) The driving of the wheels 7, 8 is controlled so that the humanoid robot 1 can push a cart or the like.

[0082] The operation of the first embodiment will be described below.

[0083] (Work control for object 100) FIG. 6 is a flowchart showing a procedure for controlling a job when the robot tool 20 executes a job on the target object 100.

[0084] In step 150, it is determined whether or not there has been an instruction to perform work on the target object 100. If a positive determination is made, the process proceeds to step 152, in which the humanoid robot 1 is moved (e.g., the arms 5 and 6 are operated) so that the palm side faces the target object 100, and the process proceeds to step 154.

[0085] In step 154, the robot tool 20L is made to face the target object 100, and information about the target object 100 is detected by the palm sensor 26 (high-resolution camera and MoPU).

[0086] In the next step 156, the information detected by the palm sensor 26 is analyzed to ascertain the type (shape, size, hardness, etc.) and position of the target object 100 in detail, and the process proceeds to step 158.

[0087] In step 158, a task (job type) for work on the object 100 is selected. Next, the process proceeds to step 160, where a selection process of the robot tools 20R, 21EXA, 21EXB, and 21EXC is performed according to the attributes of the object 100 (see FIG. 7 for details), and the process proceeds to step 162.

[0088] In step 162, a task for the target object 100 (for example, gripping a heavy object in the case of the robot tool 20R) is executed.

[0089] In the next step 164, it is determined whether the job on the object 100 was successful, and if the determination is affirmative, post-processing is performed (in the case of grasping, post-processing of transporting the grasped object 100 to a specified location), and then the process proceeds to step 150 to wait for instructions on the job on the next object 100.

[0090] If the determination in step 164 is negative, the process proceeds to step 166, where error processing is executed, and the process returns to step 150.

[0091] The robot tool 20L is equipped with a palm sensor 26 including a high-resolution camera and MoPU, so that the status of the work can be grasped in real time. Even if the work is unsuccessful, a quick response (troubleshooting) is possible.

[0092] In addition, since the palm sensor 26 (high-resolution camera and MoPU) is mounted on the palm side 20A, the target object 100 can be captured with high accuracy, and the device can also be used for tasks that require minute movements.

[0093] (Details of robot tool application process) FIG. 7 is a control flowchart showing details of the robot tool application processing subroutine.

[0094] In step 200, the robot tool 20R or 21EXA, 21EXB, 21EXC, . . . is selected based on the type of work for the object 100 (see Table 1).

[0095] That is, as shown in Table 1, the type of robot tool 20R or 21EXA, 21EXB, 21EXC, . . . required is determined according to the type of work to be performed on the target object 100.

[0096] The selection of the type of robot tool based on the type of work for the object 100 shown in Table 1 is merely an example, and may be determined based on the type and number of robot tools possessed.

[0097] In the first embodiment, three types of robot tools 21EXA, 21EXB, and 21EXC are attached to the belt 28, and thus, including the robot tool 20R, a total of four types of robot tools can be selected from 20, 21EXA, 21EXB, and 21EXC, but the number of robot tools attached to the belt 28 may be increased, or various types of robot tools 21 may be selected and attached to the belt 28 in advance depending on the work site. Also, a different type of robot tool 21EX may be attached to each humanoid robot 1.

[0098] In the next step 202, the tool attached to the replacement arm 5 (usually the robot tool 20R, but there may be another robot tool 21EXA, 21EXB, 21EXC already attached) is removed using the robot tool 20L attached to the arm 6.

[0099] In the next step 204, the arm 5 on the replacement side is moved to the position of the belt 28, and the robot tools 21EXA, 21EXB, 21EXC are attached.

[0100] In the next step 206, the robot tool originally attached to the arm 5 of the replacement side, which is being held by the arm 6 of the non-replacement side, is stored in the holder of the belt 28, and this routine ends.

[0101] As described above, in the first embodiment, in addition to work control by the robot tools 20, 21EXA, 21EXB, and 21EXC, the robot tools 20R, 21EXA, 21EXB, and 21EXC are prepared according to the type of work on the target object 100, for example.

[0102] As a result, the control burden (such as the burden associated with controlling the relative position between the robot tool 20 and the tool) that arises when a tool required for a particular type of work is held by a robot shaped like a human hand in the gripping portion of the comparative example can be eliminated by directly attaching the robot tools 21EXA, 21EXB, and 21EXC.

[0103] (Modification 1 "First embodiment") In the humanoid robot 1 of the first embodiment, like a so-called human, it has two arms 5, 6 (a right arm 5 and a left arm 6), each of which is equipped with a robot tool 20R (21EXA, 21EXB, 21EXC). However, as shown in FIG. 8, it is also possible to attach an arm 9 dedicated to monitoring, and to perform work while monitoring with a camera 50 attached to the tip of the arm.

[0104] The arm 9 has a so-called snake structure, and can approach the target object 100 through the gap between the arms 5 and 6, thereby covering the blind spot of the palm sensor 26.

[0105] (Second embodiment) A second embodiment of the present invention will be described below. Note that the same components as those in the second embodiment are given the same reference numerals and the description of the components will be omitted.

[0106] As shown in FIG. 9, the second embodiment is characterized in that, instead of a humanoid robot 1, the tip of the robot is a movable arm 52 that can be attached or detached depending on the type of work.

[0107] The movable arm 52 is attached to a base 54 via a rotatable cylindrical support 56, for example, on a line in an assembly factory.

[0108] The movable arm 52 has multiple connecting members 58 each connected via a joint shaft portion 60, and can move the robot tool 20R (21EXA, 21EXB, 21EXC) attached to the tip to the desired position in the x-axis, y-axis, and θ-axis directions, including rotational movement relative to the base 54.

[0109] Moreover, the robot tool 20R (21EXA, 21EXB, 21EXC) itself is capable of moving three-dimensionally.

[0110] According to the movable arm 52 of the second embodiment, in an assembly factory line, by changing the robot tool 20R (21EXA, 21EXB, 21EXC), it is possible to perform a plurality of work types, rather than a single work type.

[0111] In this case, a series of tasks can be performed in a single line process, such as using robot tool 20R to grasp object 100 and move it to a designated work table, then switching to robot tool 21EXA to perform drilling work on object 100, switching to robot tool 21EXB to wipe off dirt from the drilled area, and switching to robot tool 21EXC to remove shavings remaining around the drilled area, and by simple calculation, the work can be performed on a line length that is 1 / 4 of the conventional length.

[0112] (Modification 2 "Second embodiment") In the second embodiment, a tool corresponding to one work type is attached to one robot tool 20R (21EXA, 21EXB, 21EXC). However, as shown in FIG. 10, in a movable arm 52A according to a first modified example of the second embodiment, tools corresponding to two work types may be attached to one robot tool 20R (21EXA, 21EXB, 21EXC).

[0113] 10, one tool is a two-jaw tool (robot tool 20R) for gripping a heavy object attached to arm 5, and the other tool is a local sensor tool 62 attached to arm 6. Arms 5 and 6 each branch off from a main connector 64 that operates in common, and can operate independently.

[0114] This makes it possible to obtain, from the local sensor tool 62, image information and the like specialized for the work (eg, gripping a heavy object) of the robot tool 20R while the robot tool 20R is performing work.

[0115] In addition, in FIG. 10, the movable arm 52A is mounted on a vehicle 66 that can run. Therefore, rather than being fixedly arranged on a line in an assembly factory, the movable arm 52A can go to the site where work is required, thereby enabling more versatile work.

[0116] (Modification 3 "Second embodiment") FIG. 11 shows that a movable arm 52B relating to variant example 2 of the second embodiment is equipped with a robot tool 20L in the shape of a human hand and a robot tool 20R (21EXA, 21EXB, 21EXC) that is detachable and replaceable depending on a number of types of work, similar to the humanoid robot 1 (see FIG. 8) relating to variant example 1 of the first embodiment, and is also equipped with a camera (snake camera 68) that can transform into a snake shape.

[0117] As a result, while a task is being performed with the attached robot tool 20R (21EXA, 21EXB, 21EXC), video information and the like specialized for that task can be acquired from the snake camera 68.

[0118] Furthermore, since the movable arm 52B is mounted on a drivable vehicle 66, in an area where the vehicle 66 can travel (such as a factory with a flat floor), the movable arm 52B can have greater mobility and footwork than a humanoid robot.

[0119] (Example 1 "Second embodiment") FIG. 12 is a perspective view showing an outline of an assembly line for an EV (electric vehicle) car using movable arm 52 according to the second embodiment (in particular, movable arm 52A according to variant example 2 and movable arm 52B according to variant example 3).

[0120] The process shown in FIGS. 12 and 13 is a process for mounting a lithium ion battery 72 specific to EV vehicles on a vehicle body 70.

[0121] By utilizing robot tools with multiple joints, long torsos, long necks, fingers, etc., like movable arm 52, movable arm 52A, and movable arm 52B, it is possible to perform tasks that cannot be reached by conventional robot arms, such as complex cable connections and screw tightening inside the vehicle body 70, instead of humans, even in the manufacture of electric vehicles, etc.

[0122] In addition, it is well known that vehicle assembly robots, including welding robots, are computer-controlled to perform work quickly and accurately, but in recent years, the installation of lithium-ion batteries 72 and wiring work, such as in EVs, has become more precise and accurate than ever before, so in addition to programmed operation control (sequence control) like current work robots, it is necessary to strengthen confirmation and monitoring during work.

[0123] In this regard, the robot according to the present embodiment, which is capable of flexible camera work, can be said to be an optimal robot for EV vehicles.

[0124] FIG. 13(A) is a perspective view showing work carried out in an engine compartment of a vehicle body 70 (in an EV vehicle, this is a space in which a drive unit such as a motor is disposed, and is sometimes called a drive unit room).

[0125] Tasks performed by the robot rule 20R (21EXA, 21EXB, 21EXC) attached to the two arms 5, 6 are monitored in real time by a camera (MoPU) 76 attached to the tip of the arm 74 while being executed by an execution camera 68. The joints of the movable arm 52 (52A, 52B) can rotate within a range of 360°, and the control load can be reduced by determining the rotation range according to the nature of the task.

[0126] FIG. 13(B) is a perspective view showing the work performed inside the vehicle cabin of the vehicle body 70.

[0127] As also shown in FIG. 12, the lithium ion battery 72 may be arranged along the floor of the vehicle compartment. In this case, particularly when the vehicle body 70 is a monocoque body, the space available for inserting the working arm into the vehicle compartment from the outside may be limited.

[0128] Even in such cases, by utilizing a robot (movable arm 52, movable arm 52A and movable arm 52B) with multiple joints, a long torso, a long neck, fingers, etc., it becomes easier to perform tasks such as installing lithium ion battery 72 and wiring.

[0129] (Example 2 "Second embodiment" A second practical example of the second embodiment shows a configuration in which work and inspection are performed simultaneously, and defective products are monitored by image recognition from all directions (360 degrees in three dimensions).

[0130] For example, if a tool and a camera are attached to the right hand (arm 5), it is possible to grasp it with the left hand (arm 6) and monitor for defects while assembling with the right hand (arm 5). In particular, monitoring vehicle assembly requires entering the inside of the vehicle, which is difficult for anyone other than a human to do, but it is possible with a robotic hand, and compared to human monitoring, it can be done more accurately and without omissions. Of course, with regard to work and monitoring, the left and right hands can be reversed, or the arm 9 shown in Figure 8 can be used.

[0131] FIG. 14A is a perspective view of a process for inspecting the installation state of a seat belt device 78 after it has been installed on a vehicle body 70, and FIG. 14B is an enlarged view of an inspection site (pillar portion) on the vehicle body 70.

[0132] 14(A), a seat belt device 78 is attached along a pillar portion 70A of a vehicle body 70. The vehicle 70 is provided with a pillar member 70PM, a roof member 70RM, and a floor member 70FM as rigid members.

[0133] The pillar member 70PM has an upper end fixed to the roof member 70RM and a lower end fixed to the floor member 70FM.

[0134] As shown in FIG. 14(B), the seat belt device 78 includes a belt-shaped webbing 80 for restraining the body of a seated occupant on the seat, a shoulder anchor 81 that holds an upper end side of the webbing 80, a lap anchor 82 that holds the lower end side, a retractor 83 that holds the other end side of the webbing 80 so that it can be wound up and pulled out, and a slip joint 84 that pulls out the webbing 80 so that it can be worn around the waist, shoulders, etc. of the occupant.

[0135] The shoulder anchor 81 is fixed to the pillar member 70PM with a mounting bolt 81A. The pillar member 70PM is provided with a reinforcing member 85 that is provided so that the mounting portions of the lap anchor 82 and the retractor 83 of the seat belt device 78 are set adjacent to each other and reinforces both of the mounting portions, and the lap anchor 82 and the retractor 83 are both fixed to the reinforcing member 85 via mounting bolts 82A and 83A.

[0136] In order to check the installation state of the seat belt device 78, the right hand (arm 5) grasps and pulls the webbing 80, and monitors the installation state of the mounting bolts 81A, 82A, and 83A.

[0137] Specifically, the webbing 80 of the seat belt device 78 is pulled with a tool (gripping tool), and the mounting bolts 81A, 82A, and 83A in detail are photographed and monitored with a camera. If any looseness is found, additional tightening work is performed.

[0138] The monitoring mainly focuses on damage to the webbing 80 and the external condition of the mounting bolts 81A, 82A, 83A, but it is also possible to attach a bolt socket or the like to measure the tightening torque of the mounting bolts 81A, 82A, 83A.

[0139] According to practical example 2 of the second embodiment, by utilizing a robot tool with multiple joints, a long torso, a long neck, fingers, etc., like the arms 5 and 6, the installation state of the seat belt device 78, which is directly related to safety, can be monitored accurately instead of a human being. In particular, it is possible to check for defects while holding it with the left hand (arm 6) and assembling it with the right hand (arm 5), and by performing installation work and monitoring work simultaneously, it is possible to shorten the work time. In addition, since checking vehicle assembly requires entering the inside of the vehicle, it is a difficult task for anyone other than a human being, but with a robot hand, it can be performed more accurately and without omissions than a human check.

[0140] 15 is a schematic diagram showing an example of a hardware configuration of a computer 1200 functioning as the information processing device 14. A program installed in the computer 1200 can cause the computer 1200 to function as one or more "parts" of the device according to the first embodiment, or cause the computer 1200 to execute operations associated with the device according to the first embodiment or one or more "parts", and / or cause the computer 1200 to execute a process according to the first embodiment or steps of the process. Such a program can be executed by the CPU 1212 to cause the computer 1200 to execute specific operations associated with some or all of the blocks in the flowcharts and block diagrams described herein.

[0141] The computer 1200 according to the first embodiment includes a CPU 1212, a RAM 1214, and a graphic controller 1216, which are connected to each other by a host controller 1210. The computer 1200 also includes input / output units such as a communication interface 1222, a storage device 1224, a DVD drive, and an IC card drive, which are connected to the host controller 1210 via an input / output controller 1220. The DVD drive may be a DVD-ROM drive, a DVD-RAM drive, etc. The storage device 1224 may be a hard disk drive, a solid state drive, etc. The computer 1200 also includes input / output units such as a ROM 1230 and a keyboard, which are connected to the input / output controller 1220 via an input / output chip 1240.

[0142] The CPU 1212 operates according to a program stored in the ROM 1230 and the RAM 1214, thereby controlling each unit. The graphic controller 1216 retrieves image data generated by the CPU 1212 into a frame buffer or the like provided in the RAM 1214 or into itself, and causes the image data to be displayed on the display device 1218.

[0143] The communication interface 1222 communicates with other electronic devices via a network. The storage device 1224 stores programs and data used by the CPU 1212 in the computer 1200. The DVD drive reads programs or data from a DVD-ROM or the like and provides them to the storage device 1224. The IC card drive reads programs and data from an IC card and / or writes programs and data to an IC card.

[0144] The ROM 1230 stores therein a boot program or the like that is executed by the computer 1200 upon activation, and / or a program that depends on the hardware of the computer 1200. The input / output chip 1240 may also connect various input / output units to the input / output controller 1220 via a USB port, a parallel port, a serial port, a keyboard port, a mouse port, and the like.

[0145] The programs are provided by a computer-readable storage medium such as a DVD-ROM or an IC card. The programs are read from the computer-readable storage medium, installed in the storage device 1224, the RAM 1214, or the ROM 1230, which are also examples of computer-readable storage media, and executed by the CPU 1212. Information processing described in these programs is read by the computer 1200, and brings about cooperation between the programs and the various types of hardware resources described above. An apparatus or method may be constructed by implementing operations or processing of information according to the use of the computer 1200.

[0146] For example, when communication is performed between the computer 1200 and an external device, the CPU 1212 may execute a communication program loaded in the RAM 1214 and instruct the communication interface 1222 to perform communication processing based on the processing described in the communication program. Under the control of the CPU 1212, the communication interface 1222 reads transmission data stored in a transmission buffer area provided in the RAM 1214, the storage device 1224, a DVD-ROM, or a recording medium such as an IC card, and transmits the read transmission data to the network, or writes reception data received from the network to a reception buffer area or the like provided on the recording medium.

[0147] Furthermore, the CPU 1212 may cause all or a necessary portion of a file or database stored in an external recording medium such as the storage device 1224, a DVD drive (DVD-ROM), an IC card, etc. to be read into the RAM 1214, and perform various types of processing on the data on the RAM 1214. The CPU 1212 may then write back the processed data to the external recording medium.

[0148] Various types of information, such as various types of programs, data, tables, and databases, may be stored in the recording medium and undergo information processing. The CPU 1212 may execute various types of processing on the data read from the RAM 1214, including various types of operations, information processing, conditional judgment, conditional branching, unconditional branching, information search / replacement, etc., as described throughout the present invention and specified by the instruction sequence of the program, and write back the results to the RAM 1214. The CPU 1212 may also search for information in a file, database, etc. in the recording medium. For example, when a plurality of entries each having an attribute value of a first attribute associated with an attribute value of a second attribute are stored in the recording medium, the CPU 1212 may search for an entry whose attribute value of the first attribute matches a specified condition from among the plurality of entries, read the attribute value of the second attribute stored in the entry, and thereby obtain the attribute value of the second attribute associated with the first attribute that satisfies a predetermined condition.

[0149] The above-described programs or software modules may be stored in a computer-readable storage medium on the computer 1200 or in the vicinity of the computer 1200. In addition, a recording medium such as a hard disk or a RAM provided in a server system connected to a dedicated communication network or the Internet can be used as a computer-readable storage medium, thereby providing the programs to the computer 1200 via the network.

[0150] The blocks in the flowcharts and block diagrams in the first and second embodiments may represent stages of a process in which an operation is performed or "parts" of an apparatus responsible for performing the operation. Particular stages and "parts" may be implemented by dedicated circuitry, programmable circuitry provided with computer-readable instructions stored on a computer-readable storage medium, and / or a processor provided with computer-readable instructions stored on a computer-readable storage medium. The dedicated circuitry may include digital and / or analog hardware circuits, and may include integrated circuits (ICs) and / or discrete circuits. The programmable circuitry may include reconfigurable hardware circuits, such as field programmable gate arrays (FPGAs), programmable logic arrays (PLAs), and the like, including AND, OR, XOR, NAND, NOR, and other logical operations, flip-flops, registers, and memory elements.

[0151] A computer-readable storage medium may include any tangible device capable of storing instructions that are executed by a suitable device, such that a computer-readable storage medium having instructions stored thereon comprises an article of manufacture that includes instructions that can be executed to create means for performing the operations specified in the flowcharts or block diagrams. Examples of computer-readable storage media may include electronic storage media, magnetic storage media, optical storage media, electromagnetic storage media, semiconductor storage media, and the like. More specific examples of computer-readable storage media may include floppy disks, diskettes, hard disks, random access memories (RAMs), read-only memories (ROMs), erasable programmable read-only memories (EPROMs or flash memories), electrically erasable programmable read-only memories (EEPROMs), static random access memories (SRAMs), compact disk read-only memories (CD-ROMs), digital versatile disks (DVDs), Blu-ray disks, memory sticks, integrated circuit cards, and the like.

[0152] The computer readable instructions may include either assembler instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, state setting data, or source or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk®, JAVA®, C++, etc., and conventional procedural programming languages ​​such as the “C” programming language or similar programming languages.

[0153] Computer readable instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, either locally or over a local area network (LAN), a wide area network (WAN), such as the Internet, etc., to cause the processor of the general purpose computer, special purpose computer, or other programmable data processing apparatus, or to a programmable circuit, to execute the computer readable instructions to generate means for performing the operations specified in the flowcharts or block diagrams. Examples of processors include computer processors, processing units, microprocessors, digital signal processors, controllers, microcontrollers, etc.

[0154] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0155] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order.

[0156] Although the present invention has been described above using the embodiments, the technical scope of the present invention is not limited to the scope described in the above embodiments. It is clear to those skilled in the art that various modifications and improvements can be made to the above embodiments. It is clear from the description of the claims that such modifications and improvements can also be included in the technical scope of the present invention.

[0157] It should be noted that the order of execution of each process, such as operations, procedures, steps, and stages, in the devices, systems, programs, and methods shown in the claims, specifications, and drawings is not specifically stated as "before" or "prior to," and that the process may be performed in any order unless the output of a previous process is used in a later process. Even if the operational flow in the claims, specifications, and drawings is described using "first," "next," and the like for convenience, this does not mean that the process must be performed in this order. [Explanation of symbols]

[0158] 1 humanoid robot, 2 upper body, 3 legs, 4 connecting part, 5, 6 arms, 7, 8 wheels, 10 control system, 12 sensor, 14 information processing device, 20 (20L, 20R, 21EXA, 21EXB, 21EXC) robot tool, 26 palm sensor, 50 camera, 52 movable arm, 54 pedestal, 56 support, 58 connecting member, 60 joint shaft, 62 local sensor tool, 64 main connecting part, 66 vehicle, 68 snake camera, 70 vehicle body, 72 lithium ion battery, 100 object, 1200 computer, 1210 host controller, 1212 CPU, 1214 RAM, 1216 graphics controller, 1218 display device, 1220 input / output controller, 1222 communication interface, 1224 storage device, 1230 ROM, 1240 I / O chip

Claims

1. A control system for a robot that is detachable from a wrist of an arm that moves three-dimensionally and is equipped with a tool according to a type of work to be performed on an object, and that performs work on the object, comprising: A determination unit for determining a type of work for the object; a control unit that controls an operation of mounting the tool corresponding to the work type determined by the determination unit and controls an operation according to the work type, A plurality of arms to which at least the tool can be attached are provided, A tool corresponding to a type of work for the object is selectively attached to one arm, and a monitoring device is attached to monitor a work site in conjunction with a work performed by the attached tool, A robot control system in which an auxiliary work tool is attached to the other arm, which can perform an auxiliary task to the main task on the object.

2. The robot control system according to claim 1 , wherein the auxiliary work tool has a function of a human hand and holds the object.

3. 2. A robot control system as described in claim 1, wherein the determination unit makes determination based on information from a sensor unit mounted on the tool, the sensor unit having a camera that takes an image of the object and identifies the type of the object, and a motion processing unit that specifies the position of the object.

4. A robot that operates in three dimensions, has a plurality of arms to which at least a work tool can be attached, and performs a task on an object, A tool corresponding to a type of work for the object is selectively attached to one arm, and a monitoring device is attached to monitor a work site in conjunction with a work performed by the attached tool, The robot has an auxiliary work tool attached to the other arm, which can perform an auxiliary work to the main work on the object.

5. A robot control program that causes a computer to operate as the judgment unit and the control unit described in any one of claims 1 to 3.

Images