Robot control device, robot, and robot control system

Abstract

Provided is a robot that realizes operations for gripping and fitting onto an object even when the object is loaded in a bulk state. The present invention comprises: a storage unit that stores teaching data including object information relating to an object to be gripped, change information relating to a change in the position of the object in a teaching environment when the object is moved from a gripping work position to a target work position, and work environment information relating to the target work position; an operation generation unit that, on the basis of the object information, generates an operation for gripping an object of the same type as the object in the teaching data; and a calculation unit that calculates an object / hand relative relationship between the object, which is gripped by a work robot, and a hand of the work robot. The calculation unit calculates a robot / target-work-position relative relationship between the work robot and the target work position on the basis of the work environment information. On the basis of the change information, the object / hand relative relationship, and the robot / target-work-position relative relationship, the operation generation unit generates an operation trajectory of the work robot so that the position change of the object gripped by the work robot follows the position change included in the teaching data.

Description

Robot control device, robot, and robot control system

[0001] The present invention relates to a robot control device that controls the operation of a working robot using teaching data, a robot equipped with the same, and a robot control system.

[0002] While conventional work robots, which perform a variety of tasks such as pick-and-place, fitting, and assembly, are often only capable of performing routine tasks, in recent years work robots have been required to be adaptable and able to autonomously respond flexibly to their surroundings. Against this background, various technologies related to the control of autonomous robots have been made public.

[0003] For example, claim 1 of Patent Document 1 states, "A robot teaching device that teaches a task to a robot that grasps and moves a graspable object, the robot teaching device comprising: a teaching pose measurement unit that measures and / or calculates a teaching pose, which is the pose of a first graspable object grasped by an instructor during a teaching task; and a robot motion generation unit that generates a sequence of joint displacements of the robot so that the pose of a second graspable object grasped by the robot is the same as the teaching pose."

[0004] International Publication No. 2021 / 260898

[0005] However, when the object to be grasped is placed in a loose pile or when the position of the object to be grasped changes randomly, the robot's hand must grasp the object without colliding with the surrounding environment. Therefore, the robot may not always be able to grasp the object in the same pose as the teaching pose. Furthermore, when considering the task of grasping an object in a loose pile and then fitting the grasped object into a hole, the taught gripping pose may differ from the actual gripping pose. Therefore, even if the robot is controlled according to the teaching pose as in Patent Document 1, the positions of the fitting hole and the object to be grasped may be misaligned. To solve this problem, one method is to teach the fitting operation using the technology described in Patent Document 1 for each gripping pose. However, because the poses required for picking from a loose pile are diverse, another problem arises: the number of teaching operations becomes enormous. Thus, with the technology described in Patent Document 1, there was a high possibility that the taught fitting operation could not be reproduced in an environment where the objects to be grasped were placed in a loose pile.

[0006] Therefore, the present invention provides a robot control device, a robot, and a robot control system that enable a robot to grasp an object and realize a desired operation on the object to be grasped with a small number of teaching operations.

[0007] a motion generation unit that generates a motion to grasp an object of the same type as the object in the teaching data based on the object information stored in the memory unit; and a calculation unit that calculates an object-hand relative relationship between the object grasped by the work robot and the hand of the work robot based on the motion generated by the motion generation unit, wherein the calculation unit further calculates a robot target work position relative relationship between the work robot and the target work position based on the work environment information, and the motion generation unit generates a motion trajectory for the work robot based on the change information, the object-hand relative relationship, and the robot target work position relative relationship so that the position change of the object grasped by the work robot follows the position change included in the teaching data.

[0008] According to the robot control device of the present invention, even if the object to be grasped is placed in a loose pile or its position changes randomly, the robot can grasp the object and fit it into a hole with a small number of teaching operations.

[0009] Overall configuration of a robot control system in Example 1 Example configuration of a control device in Example 1 Process flow of teaching and reproduction in Example 1 Generation of teaching data in Example 1 Generation of picking motion in Example 1 Calculation of gripping posture in Example 1 Recognition of work environment in Example 1 Calculation of trajectory for reproducing teaching work in Example 1 Overall configuration of a robot control system in Example 2 Example configuration of remote control in Example 2 Process flow of teaching and reproduction in Example 2 Generation of teaching data in Example 2 Reproduction of teaching work in Example 2 Overall configuration of a robot control system in Example 3 Process flow of teaching and reproduction in Example 3 Picking motion generation unit in Example 3 Overall configuration of a robot control system in Example 4 Example of robot work

[0010] Hereinafter, an embodiment of the robot control device of the present invention will be described with reference to the drawings. Prior to that, an example of an operation that the robot control device causes the robot to perform will be described with reference to FIG.

[0011] Figure 18(a) shows a robot inserting a memory into a personal computer, Figure 18(b) shows a robot inserting a key into a door lock and turning it, Figure 18(c) shows a robot inserting a dropper into a flask, and Figure 18(d) shows a peg-in-hole task in which a robot inserts a peg into a hole it finds by pressing the robot against the environment.

[0012] Since all of these tasks require continuous control of the position and orientation of the gripped article from the task start position to the task end position, the taught task cannot be autonomously reproduced by a method that teaches only the final position and orientation of the gripped article. Therefore, in order to autonomously reproduce a task such as the example shown in Figure 18, the robot control device of the present invention is configured as shown in the following embodiment.

[0013] The work illustrated in FIG. 18 is one example of a work that can be reproduced by the robot control device of the present invention, and other work such as pick-and-place placement work (placing an object in a desired location), assembly work (matching one part with another), and connector mating (fitting connectors together) may also be tasks that can be reproduced by the robot control device of the present invention.

[0014] [Overall Configuration of Robot Control System] Fig. 1 is a schematic diagram of the overall configuration of a robot control system 1 of this embodiment. As shown here, the robot control system 1 of this embodiment is composed of a robot 2, a control device 3 that controls the robot 2, and a camera 4 that captures images of the robot 2 and its surrounding environment. Also arranged around the robot 2 are a tray 5 on which objects O are piled up in bulk, and a work table 6 having a hole H on its top surface for inserting the object O. Each component will be outlined below. Note that in the robot control system 1 of Fig. 1, the control device 3 is arranged external to the robot 2, but the control device 3 may also be built into the robot 2.

[0015] [Robot 2] The robot 2 is a multi-joint arm-type robot having an arm 21 and a hand 22, and having a function of outputting its own posture and external forces detected based on the outputs of encoders and force sensors provided at each movable part as robot information Dr. Note that the robot 2 in this embodiment may be another type of robot, such as a mobile manipulator.

[0016] [[Camera 4]] Camera 4 is an imaging device installed in a position where it can capture images of the robot 2, the tray 5, and the workbench 6, and captures images of the object O grasped by the robot 2, the object O in the tray 5, the individual holes H on the top surface of the workbench 6, obstacles around the robot 2, etc., and outputs the images as image information Dg. Note that camera 4 may be a mono camera or a stereo camera, and a sensor capable of measuring three-dimensional information around the robot, such as a TOF sensor or a laser sensor, may be used instead of camera 4.

[0017] [Control Device 3] The control device 3 is a device that controls the robot 2 based on the robot information Dr acquired from the robot 2 and the image information Dg acquired from the camera 4. Note that, hereinafter, the robot to be controlled when the control device 3 is set to the teaching mode will be referred to as the teaching robot 2A, and the robot to be controlled when the control device 3 is set to the reproduction mode will be referred to as the working robot 2B.

[0018] Specifically, the teaching mode in this embodiment is a direct teaching mode. In this mode, the instructor P applies an external force to the teaching robot 2A to guide the arm 21 and hand 22 of the teaching robot 2A to various positions and postures (hereinafter, for simplicity, positions and postures will be referred to as "positions"), thereby teaching the working robot 2B a series of actions to be reproduced in the reproduction mode. For example, the instructor P operates the control device 3 to teach the teaching robot 2A to use the hand 22 to grasp one of the objects O randomly stacked in the tray 5 and fit the grasped object O into one of the holes H on the top surface of the work table 6.

[0019] On the other hand, the reproduction mode of this embodiment is a mode that reproduces, in accordance with actual conditions, the same operation as that taught under the teaching mode, based on the teaching data Dt generated by the control device 3 under the teaching mode, and the image information Dg and robot information Dr input to the control device 3 under the reproduction mode.

[0020] 2, the hardware configuration of the control device 3 will be described. The control device 3 illustrated here is a computer including a CPU 3a (Central Processing Unit), a ROM 3b (Read Only Memory), a RAM 3c (Random Access Memory), a non-volatile storage 3d, a network interface 3e, an input unit 3f, a display unit 3g, and a bus 3h connecting the respective components.

[0021] The CPU 3a is a processing unit that reads out from the ROM 3b and executes the program code of software that executes the arithmetic processing related to the robot control, which will be described later. Variables, parameters, etc. that are generated during the arithmetic processing are temporarily written to the RAM 3c.

[0022] The nonvolatile storage 3d may be a large-capacity information storage unit such as a hard disk drive (HDD) or a solid state drive (SSD). The nonvolatile storage 3d constitutes a database in which the taught data is stored.

[0023] The network interface 3e may be, for example, a network interface card (NIC), etc. Instructions to the robot 2 are given via the network interface 3e.

[0024] The input unit 3f performs input processing for the robot control program by switching to a direct teaching mode and registering waypoints, which are necessary for teaching the robot.

[0025] The display unit 3g displays the operating status of the control device 3.

[0026] 2 is merely an example, and the control device 3 may be configured with a device that performs arithmetic processing other than a computer. For example, some or all of the functions performed by the control device 3 may be realized by hardware such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0027] Also, the configuration in which the control device 3 is equipped with an input unit 3f and a display unit 3g is also an example, and the control device 3 may be configured as a computer that does not have either or both of the input unit 3f and the display unit 3g.

[0028] [Teaching Process and Reproduction Process in First Embodiment] Next, the teaching phase and reproduction phase processes in the first embodiment will be described with reference to Fig. 3. Note that the thick arrows in Fig. 3 represent homogeneous transformation matrices, which are the relative positional relationships between the coordinate systems (hereinafter referred to as relative relationships).

[0029] [Step S1: Teaching Phase] When the control device 3 is set to the teaching mode, the robot 2 is treated as a teaching robot 2A in the direct teaching mode, which makes it ready to start the teaching phase in which teaching data Dt is generated.

[0030] In step S1 of Figure 3, the instructor P physically contacts the teaching robot 2A and applies an external force to cause the work robot 2B to perform the desired operation. This allows the control device 3 to acquire time-series image information Dg and robot information Dr when the teaching robot 2A performs the desired task, and to store teaching data Dt based on these in the non-volatile storage 3d. The teaching data Dt stored here may include, for example, change information (R env_o ), object information Do relating to the object O to be grasped, and environment information De of the work environment into which the object O is inserted.

[0031] In addition, this change information (R env_o) is the environmental coordinate system Σ fixed to the working environment. ｅｎｖ Object coordinate system Σ fixed to object O seen from Ｏ This change information (R env_o ) represents how the most important object O is inserted into the work environment when reproducing the insertion operation, and by saving this as teaching data Dt and controlling it in combination with the gripping posture of the actual object O described later, it is possible to reproduce the mating operation even in a variety of gripping postures.

[0032] 4 shows details of the method for generating the teaching data Dt in step S1. This process is executed while the instructor P is teaching the desired operation, and is a process for saving the teaching data Dt. Here, the target is a fitting operation, but it is not limited to fitting operations and may also be pick-and-place, peg-in-hole, assembly operations, etc. Note that the object position recognition unit 31a, work environment recognition unit 31b, and relative relationship calculation unit 31c in the figure are functional units realized by the CPU 3a of the control device 3 executing programs corresponding to each function.

[0033] The object position recognition unit 31a recognizes the position of the object O based on the image information Dg, and calculates the position of the object O in the camera coordinate system Σ ｃ Object coordinate system Σ in ｏ The relative relationship between (R c_o ), and the shape, type, etc. of the object O.

[0034] The work environment recognition unit 31b recognizes the work environment based on the image information Dg and calculates the camera coordinate system Σ ｃ In the environmental coordinate system Σ ｅｎｖ The relative relationship between (R c_env ) and the shape and type of the work environment.

[0035] The relative relationship calculation unit 31c calculates the relative relationship (R) between the work environment and the object O based on the object position output from the object position recognition unit 31a and the work environment position output from the work environment recognition unit 31b. env_o ) in the environmental coordinate system Σ ｅｎｖ Object coordinate system Σ ｏ Calculate the homogeneous transformation matrix of

[0036] The shape and type of object O, the shape and type of the work environment, and the relative relationship between the work environment and object O calculated by the above process are stored as teaching data Dt in a database in the nonvolatile storage 3d.

[0037] 4 is executed at a predetermined cycle while the instructor P is teaching a desired operation. That is, the teaching data Dt is time-series data on the relative relationship between the work environment and the object O, in other words, change information on the change in the position of the object O in the work environment when the object O is moved from the gripping work position to the target work position. This teaching data Dt represents, in time series, the trajectory along which the object O is inserted into the work environment, which is essential for reproducing the fitting work.

[0038] [Reproduction Phase] Next, the reproduction phase will be explained, divided into steps S2 to S5. When the control device 3 is set to reproduction mode, the robot 2 is treated as a working robot 2B. This completes preparations for the start of the reproduction phase, in which the working robot 2B reproduces a desired task.

[0039] [Step S2: Picking operation generation process] In step S2, the control device 3 recognizes the objects O stacked in the tray 5 based on the current image information Dg and object information Do, and then outputs a command to the work robot 2B to grasp the objects O in the stacked state.

[0040] The picking motion generation method in step S2 will now be described with reference to Fig. 5. Note that the candidate object recognition unit 32a, obstacle recognition unit 32b, gripping point position determination unit 32c, and robot motion generation unit 32d in the figure are functional units realized by the CPU 3a of the control device 3 executing programs corresponding to the respective functions.

[0041] The object candidate recognition unit 32a outputs a plurality of pieces of object candidate information Dc relating to the positions and shapes of object candidates based on the image information Dg and the shape and type information of the object O stored in the database. This processing may be performed using a method such as pattern matching or a machine learning method such as a convolutional neural network (CNN).

[0042] The obstacle recognition unit 32b outputs obstacle information Db about obstacles around the object candidate based on the image information Dg and the grasped object candidate information Dc. The obstacle information Db may be represented by a type and its shape, or may be point cloud information indicating the presence of a three-dimensional object even if the type is unknown.

[0043] The gripping point position determining unit 32c recognizes the packaging style in which each object O is placed based on the candidate object information Dc and the obstacle information Db, and determines the gripping point (position to grip) of an object that is easy for the work robot 2B to grip (for example, an object that is at the top and can be gripped without colliding with other objects). This gripping point position is determined by the posture in which the hand 22 grips the object O, i.e., the object coordinate system Σ ｏ and the TCP coordinate system Σ ｔｃｐ The relative relationship between (R o_tcp ) The TCP (Tool Center Point) is the tip position of the hand 22.

[0044] The robot motion generator 32d generates a trajectory for the hand 22 of the working robot 2B based on the gripping point position and geometric information about the arm 21. Then, a robot trajectory command is executed to realize the generated robot trajectory, causing the working robot 2B to grip the desired object O.

[0045] [[Step S3: Grasping Posture Calculation Process]] In step S3, the control device 3 calculates the relative relationship (R o_tcp The relative relationship between the hand 22 and the object O is calculated in the object coordinate system Σ Ｏ It is expressed as a homogeneous transformation matrix of the hand coordinates as seen from the

[0046] The grip posture calculation process in step S3 will now be described with reference to Fig. 6. The grip object recognition unit 33a, TCP calculation unit 33b, and grip posture calculation unit 33c in the figure are functional units realized by the CPU 3a of the control device 3 executing programs corresponding to the respective functions.

[0047] The object recognition unit 33a calculates the position of the object O being grasped by the work robot 2B from the image information Dg and the shape, type, etc. of the object O.

[0048] The TCP calculation unit 33b calculates the TCP, which is the tip position of the hand 22, from the encoder information included in the robot information Dr and geometric information of the working robot 2B.

[0049] The gripping posture calculation unit 33c calculates the relative relationship (R o_tcp This process is executed periodically during the reproduction phase, and the relative relationship (R o_tcp However, if it is assumed that the relative relationship between the object O and the hand does not change after the object O is grasped, the above process may be performed only once, or the relative relationship between the object O and the hand 22 (R o_tcp ) can be used as is.

[0050] [[Step S4: Work Environment Recognition Process]] In step S4, the relative relationship (R r_env The relative relationship between the work robot 2B and the work environment is calculated using the robot coordinate system Σ ｒ The environmental coordinate system Σ ｅｎｖ is expressed as a homogeneous transformation matrix.

[0051] The work environment recognition process in step S4 will now be described with reference to Fig. 7. The work environment candidate recognition unit 34a and the target work environment determination unit 34b in the figure are functional units realized by the CPU 3a of the control device 3 executing programs corresponding to the respective functions.

[0052] The work environment candidate recognition unit 34a recognizes candidates for the work environment (hole H for inserting the object O) based on the image information Dg and the shape and type information of the taught work environment. This process may be performed using a method such as pattern matching or a machine learning method such as CNN.

[0053] The target work environment determination unit 34b outputs the next target work environment position (hole position) based on the order of fitting into the work environment (which hole H to fit in order) previously set by the instructor P and the position information of the candidate work environment.

[0054] [[Step S5: Calculation process of trajectory for reproducing teaching work]] In step S5, the control device 3 calculates the relative relationship (R o_tcp ) and the relative relationship between the work robot 2B and the work environment (R r_env ) and change information (R env_o ) and calculate the trajectory of the TCP from the robot coordinate system. Note that each point of the trajectory of the TCP calculated in this step is calculated in the robot coordinate system Σ ｒ From the TCP coordinate system Σ ＴＣＰ That is, the TCP trajectory is time series data of the homogeneous transformation matrix.

[0055] Here, a method for calculating a trajectory for reproducing the teaching operation in step S5 will be described with reference to FIG. 8. First, the teaching data Dt is information on the change in position of the object O in the work environment when the object O is moved from the gripping operation position to the target operation position (R env_o ) and the current relative positional relationship between the work robot 2B and the environment (R r_env ) based on the position change of object O relative to the robot (R r_o ) is calculated.

[0056] Next, the position change of the object O relative to the work robot 2B (R r_o ) and the grasping posture of object O (R o_tcp ) based on the robot coordinate system Σ, which is the robot trajectory that reproduces the teaching task. ｒ Change in hand position (R r_tcp ) is calculated. Here, the grasping posture (R o_tcp ) may be time-series data, or may be a constant as long as the relative relationship between the object O and the hand does not change after the object O is grasped.

[0057] Then, at the end of step S5, the control device 3 controls the work robot 2B according to the calculated TCP trajectory, thereby causing the work robot 2B to reproduce, in a different environment, the same task as that taught to the teaching robot 2A. Note that the homogeneous transformation calculations are for calculating geometric relationships, and the order of calculation is not important, so the calculations may be performed in a different order.

[0058] In this way, the work taught via the teaching robot 2A is reproduced by the work robot 2B not only as the final fitting posture but also as the taught trajectory. This means that the work robot 2B moves so that the change in position of the object O grasped by the work robot 2B in the reproduction phase follows the change in position of the object O from the grasping work position to the target work position, which is included in the teaching data Dt.

[0059] Effect of First Embodiment As described above, when the robot teaching and reproduction method of the first embodiment is applied, during reproduction, bulk picking is performed in step S2, and the robot trajectory (how the robot moves to reproduce the insertion operation) is calculated in step S5 from the actual gripping posture (how the object O is grasped) calculated in step S3 and the teaching data Dt (how the object O is inserted into the work environment), and the work robot 2B can be controlled.

[0060] Therefore, even in a situation where the gripping posture changes each time the object O is picked, such as when the object O is piled up in a random state or the posture of the object O is not constant, the robot can reproduce the fitting operation using teaching data Dt that shows the object O being gripped and fitted in a different posture.

[0061] Furthermore, in step S4, the relative relationship between the work robot 2B and the work environment is determined, and in step S5, the robot's trajectory is calculated based on this relative relationship, so the taught task can be reproduced even if the position of the work environment changes between the time of teaching and the time of reproduction.In other words, a robot control system that is robust to a variety of environmental changes can be provided with few teaching steps.

[0062] Although the above has been described using the example of a fitting operation in which an object O is inserted into a hole H, the present invention is intended for general operations that change the relative distance between an object and the work environment or another object, and may be applied to pick-and-place placement operations in which an object is placed in a desired location, assembly operations in which certain parts are combined in a desired posture, connector fitting operations in which connectors are fitted together, and peg-in-hole operations in which an object is inserted into a desired position.

[0063] Next, a robot control system according to a second embodiment will be described with reference to Figures 9 to 13. The following description will focus on differences from the first embodiment, and a description of similar parts will be omitted.

[0064] 9 shows the overall configuration of a robot control system 1 of Example 2. In Example 1, one robot 2 was used as a teaching robot 2A in the teaching phase and as a working robot 2B in the reproduction phase, but in this example, one of two robots 2 is used as a dedicated teaching robot 2A and the other is used as a dedicated working robot 2B, and dedicated control devices 3A and 3B are connected to each robot.

[0065] The control devices 3A, 3B of this embodiment are equipped with bilateral control, which will be described in Fig. 10. Therefore, when the instructor P moves the teaching robot 2A, the movement of the working robot 2B is controlled to follow. Furthermore, the reaction force generated when the working robot 2B comes into contact with the environment is returned to the instructor P via the teaching robot 2A. Therefore, by using the robot control system 1 of this embodiment, which applies bilateral control, the instructor P can teach the fitting operation, including adjusting the amount of force.

[0066] [Configuration Example of Remote Control] Next, bilateral control implemented in the control device 3A and the control device 3B will be described with reference to the block diagram of FIG. ｌ is the position information of the teaching robot 2A, F ｌ is the force information of the teaching robot 2A, x ｆ is the position information of the work robot 2B, F ｆ is the force information of the work robot 2B.

[0067] Bilateral control is a well-known technology, as seen in, for example, International Publication No. 2005 / 109139, and involves controlling each joint so that the postures of the teaching robot 2A and the working robot 2B are aligned, while also controlling the forces between the robots 2A and 2B so that they have an action-reaction relationship. Therefore, the instructor P operating the teaching robot 2A can physically feel the reaction force that the working robot 2B receives from the environment. The reaction force does not necessarily have to be detected by a human; it can be detected by a force sensor or by a means for estimating the reaction force, such as a disturbance observer. Note that the bilateral control method is not limited to this, and a force feedback control system or other similar control system can also be used. In this way, by having the instructor P teach the robot using bilateral control to perform a certain action, the robot can be taught the amount of force applied by the human.

[0068] 11 shows an example of the flow of the teaching and reproduction processes in Example 2. The difference from Example 1 is that in the teaching phase, step S1 is replaced with step S1A and step S1B, and in the reproduction phase, step S2 is replaced with step S2A and step S5 is replaced with step S5A. Each of the replaced steps will be described in detail below.

[0069] In step S1A, the control device 3A records the change in the position of the TCP (the trajectory during picking) and the time series data of the force (the strength of the force during picking) as the teaching data Dt based on the encoder information and force sensor information of the teaching robot 2A and the object position information recognized from the image information Dg. At this time, the object coordinate system Σ ｏ By storing the position and force of the TCP as seen from the object O, it is possible to record as teaching data Dt how the teaching robot 2A grasps each object O (how to grasp an object differently for each object). By performing the process of step S1A multiple times to generate teaching data Dt for grasping items in a variety of postures, when reproducing the picking operation with the work robot 2B, it can respond more flexibly to the environment, even in situations such as bulk picking.

[0070] In step S1B, the control device 3A calculates the position change (R env_o ) and the object coordinate system Σ detected by bilateral control. ｏ The time series data of the force at the object O, i.e., how the robot applies force to the object O during the fitting operation, is recorded.

[0071] In step S2A, the control device 3B selects the object O to be grasped and a trajectory for picking the object without colliding with surrounding obstacles from the candidate object to be grasped, obstacle information, and multiple taught picking trajectories. Furthermore, the control device 3B calculates a force command to be applied to the object O to be grasped by the work robot 2B from the position of the object O to be grasped and information on changes in force as seen from the object O, and controls the work robot 2B based on the picking trajectory and grasp command information to grasp the object O.

[0072] In step S5A, the control device 3B determines the relative relationship (R o_tcp ) and the relative relationship between the work robot 2B and the work environment (R r_env ) and the position change of object O relative to the working environment (R env_o ) and time series data of the force during mating, the trajectory and force command of the TCP as viewed from the robot coordinates of the work robot 2B are calculated, and by executing that trajectory, the mating work taught in step S1B is reproduced, including the amount of force.

[0073] As described above, when teaching the picking operation in step S1A, the "posture and force to be used to pick the object" are memorized, and when reproducing the operation, in step S2A, the trajectory of the work robot 2B when reproducing the picking operation is calculated based on the picking teaching data Dt (the trajectories of multiple gripping operations) and the actual object position recognized by the camera, and the work robot 2B is controlled according to the calculated trajectory. This makes it possible to pick even when there are objects in a bulk pile that require a variety of picking postures. Furthermore, because the gripping position and force are adjusted for each item, it is possible to pick fragile items or items that have a specific location to hold.

[0074] Furthermore, when teaching the fitting operation in step S1B, the "force required to insert the object" is remembered, so when reproducing the taught operation, the force can also be reproduced. In particular, fitting operations such as pressing an object against a work table with an appropriate force to find the position of the hole can be properly reproduced, as shown in Figure 18(d).

[0075] [Generation of Teaching Data Dt] Next, a method for generating teaching data Dt in the second embodiment will be described with reference to Fig. 12. This method is based on the assumption that the relative relationship between the object O and the hand 22 does not change after the object O is grasped, and is based on the relative relationship (R o_tcp ) and the trajectory (R r_tcp time series data) and the relative relationship between the work environment and work robot 2B (R r_env ) based on the position change of the object in the working environment during the mating operation (R env_o ) is calculated.

[0076] According to this method, unlike in Example 1, when generating the teaching data Dt, it is not necessary to continuously photograph the position of the object O during the fitting operation with the camera 4, so there is no need to take care to ensure that the object O is not hidden in the shadow of the arm 21 or the hand 22 (there is no need to worry about occlusion), which has the effect of making teaching easier.

[0077] [Reproduction of teaching work] Next, a method of reproducing the teaching work in the second embodiment will be described with reference to Fig. 13. The difference from the first embodiment is that the teaching data Dt of the force is reproduced (the calculation in the upper half of Fig. 13). The force / grasping force F applied by the robot in the object coordinates, which is the teaching data Dt, ｏ is the change in the object's position as seen by the robot (R r_o ) to perform homogeneous transformation to obtain the force command F ｒ (how the robot applies force to the object during the fitting operation). The value of the position command is the change in the position of the hand in the robot coordinate system calculated in Example 1 (R r_tcp By using these as position and force command values ​​to operate the robot, it is possible to achieve a fitting operation that replicates the force exerted by a human.

[0078] Next, a robot control system according to a third embodiment will be described with reference to Figures 14 to 16. The following description will focus on differences from the first embodiment, and a description of similar parts will be omitted.

[0079] [Overall Configuration of Robot Control System] First, the overall configuration of the robot control system 1 of Example 3 will be described with reference to Fig. 14. In Example 1, the instructor P generated the teaching data Dt by directly moving each joint of the teaching robot 2A, but in this example, the instructor P operates the teaching robot 2A using an input / output device 7 such as a teaching pendant connected to the control device 3, thereby teaching the teaching data Dt to the control device 3.

[0080] Robot teaching using the input / output device 7 is a method commonly used for industrial robots. For example, the TCP of the teaching robot 2A is registered as a sequence of multiple waypoints in the input / output device 7. Alternatively, the robot may be taught to move at a predetermined speed until it detects a force sensor (not shown) using information from a force sensor (not shown) for only a predetermined coordinate axis. In this way, in this embodiment, robot operation may be taught using various teaching methods commonly used for industrial robots.

[0081] [Processing Flow of Teaching and Reproduction] Next, the processing flow of teaching and reproduction in Example 3 will be described with reference to Fig. 15. The difference from Example 1 is that step S2 in the reproduction phase is replaced with step S2B. The replaced step S2B will be described in detail below.

[0082] Step S2B is a process for determining the posture of the picking operation, taking into consideration collisions with the surrounding environment during the fitting operation. Therefore, in this step, the posture of the picking operation is determined based on the object information and image information Dg, as well as the position change of the object in the work environment during the fitting operation and information about the work environment, taking into consideration whether the object will collide with surrounding obstacles during the fitting operation. As a result, as shown in the left diagram of FIG. 15 , the gripping posture (a) that can grasp the object but will not collide with the environment during fitting is selected and picked, rather than the gripping posture (b) that can grasp the object but will collide with the environment during fitting. This makes it possible to realize picking and fitting operations with increased robustness and safety. More specific processing will be described in FIG. 16.

[0083] Next, a picking operation generation method in Example 3 will be described with reference to FIG. 16 . The difference from FIG. 5 in Example 1 is the processing of the gripping point position determination unit 32c′. The gripping point position determination unit 32c′ in this example determines whether a collision will occur during picking based on the position and shape of the candidate object to be gripped, obstacles around the object, as well as the change in the position of the object in the work environment during the fitting operation, and the shape and type of the work environment, as well as the position and shape of the candidate object to be gripped, which were also used in Example 1. Furthermore, the gripping point position determination unit 32c′ determines whether the hand will collide with the environment in the final posture of the fitting operation when gripped at the gripping point candidate, and selects an appropriate gripping posture that will not collide in both the picking and fitting operations based on another criterion (e.g., no collapse of the load) as the gripping point position.

[0084] The robot teaching and reproduction method in Example 3 described above makes it possible to teach the operation using the input / output device of the control device, and by performing the picking operation while taking into consideration collisions during the fitting operation, it is possible to realize picking and fitting operations with increased robustness and safety.

[0085] Next, a robot control system according to a fourth embodiment will be described with reference to Fig. 17. The following description will focus on differences from the first embodiment, and a description of similar parts will be omitted.

[0086] 17 is a schematic diagram of the overall configuration of a robot control system 1 according to Example 4. In Example 1, the instructor P generates the teaching data Dt by directly moving each joint of the teaching robot 2A, but in this example, the instructor himself grasps the object O and converts the action of fitting the object O into the hole H into data using the camera 4, and generates the teaching data Dt based on this data.

[0087] By applying Example 4, the same effect as in Example 1 can be obtained even in work teaching by the instructor P. That is, when reproducing, bulk picking is performed in step S2, and the robot trajectory (how the robot moves to reproduce the insertion operation) is calculated in step S5 from the actual gripping posture (how the object is gripped) calculated in step S3 and the teaching data Dt (how the object is inserted into the work environment), and the robot can be controlled. Therefore, the teaching data Dt can be generated more easily than in Example 1, and the same effect as in Example 1 can be obtained.

[0088] [Modifications] The above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, in the above-described embodiments, the device or system configuration may be changed, and some processing steps may be omitted or replaced, within the scope of the gist of the present invention. Furthermore, information such as programs for performing picking and fitting processes may be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or optical disk.

[0089] REFERENCE SIGNS LIST 1...robot control system, 2...robot, 2A...teaching robot 2B...working robot 21...arm, 22...hand, 3...control device, 3a...central processing unit (CPU), 3b...ROM, 3c...RAM, 3d...non-volatile storage, 3e...network interface, 3f...input section, 3g...display section, 31a...object position recognition section, 31b...work environment recognition section, 31c...relative relationship calculation section, 32a...grasped object candidate recognition section, 32b...obstacle recognition section, 32c...grasping point position determination section, 32d...robot motion generation section, 33a...grasped object recognition section, 33b...TCP calculation section, 33c...grasping posture calculation section, 34a...work environment candidate recognition section, 34b...target work environment determination section, 4...camera, 5...tray 6...work table, 7...input / output device, P...instructor, O...object, H...hole, Dr...robot information, Dg...image information, Dt...teaching data, Do...object information, De...environment information, Dc...grasp object candidate information, Db...obstacle information,

Claims

1. A robot control device that controls the movement of a work robot based on teaching data, comprising: a memory unit that stores teaching data including object information about an object to be grasped, change information about the change in position of the object in the teaching environment when an instructor or teaching robot moves the object from a grasping work position to a target work position, and work environment information about the target work position; a motion generation unit that generates a motion to grasp an object of the same type as the object in the teaching data based on the object information stored in the memory unit; and a calculation unit that calculates an object-hand relative relationship between the object grasped by the work robot and the hand of the work robot based on the motion generated by the motion generation unit, wherein the calculation unit further calculates a robot target work position relative relationship between the work robot and the target work position based on the work environment information, and the motion generation unit generates a motion trajectory for the work robot based on the change information, the object-hand relative relationship, and the robot target work position relative relationship so that the change in position of the object grasped by the work robot follows the position change included in the teaching data.

2. A robot control device according to claim 1, wherein the object hand relative relationship and the robot target work position relative relationship are relative positional relationships.

3. A robot control device as described in claim 1, characterized in that the calculation unit recognizes the position change of the object and the work environment position when the instructor or the teaching robot moves the object from the grasping work position to the target work position based on image information captured by a camera, and calculates the change information based on the position change of the object and the work environment position.

4. A robot control device as described in claim 1, wherein the calculation unit calculates the object-hand relative relationship at the moment the object is grasped based on image information captured by a camera and the object information, and further calculates a work robot work environment relative relationship between the work robot and the work environment based on the image information and the work environment information, and calculates the change information based on the object-hand relative relationship, the work robot work environment relative relationship, and the movement trajectory of the work robot.

5. A robot control device as described in claim 3, wherein the operation generation unit calculates candidate objects to be grasped based on the image information and the object information stored in the memory unit, detects obstacles around the candidate objects to be grasped from the image information and the object information of the candidate objects to be grasped, determines a grasp point position based on the candidate objects to be grasped and the obstacles around them, and controls the work robot based on the grasp point position to grasp an object of the same type as the object in the teaching data.

6. A robot control device as described in claim 1, characterized in that the operation generation unit determines a gripping point position that will not collide with obstacles during a fitting operation based on candidate objects to be grasped, information about obstacles around the candidate objects, change information about changes in the position of the object related to the fitting operation, and information about the target work position, and grasps an object of the same type as the object in the teaching data.

7. A robot control device as described in claim 1, wherein the memory unit further stores the TCP picking trajectory in the object coordinate system when the instructor or the teaching robot picks an object and information on changes in force applied to the object grasped by the teaching robot, and the operation generation unit selects a trajectory for picking the object without colliding with the object to be grasped and surrounding obstacles from position information of the candidate object to be grasped and information on obstacles around it, and multiple taught picking trajectories, and calculates a force command to be applied to the item to be grasped by the work robot from the position of the object to be grasped and the force change information, and grasps the item based on the picking trajectory and force command information.

8. A robot control device as described in claim 1, wherein the memory unit further stores teaching data including information on changes in force applied to the object by the teaching robot in the teaching environment, and the operation generation unit further generates a force for the working robot so as to reproduce the direction and magnitude of the force applied to the object by the teaching robot contained in the teaching data, based on the force change information, change information regarding changes in the position of the object, and the robot target work position relative relationship.

9. A robot control device as described in claim 3, characterized in that the calculation unit calculates the relative relationship of the robot's target work position between the camera and the target work position based on the image information and the work environment information, and calculates the relative relationship between the target work position and the robot based on the relative relationship of the robot's target work position and the relative relationship between the camera and the robot.

10. A robot control device according to claim 1, wherein an instructor operates the teaching robot by physical contact and teaches it how to move.

11. A robot control device according to claim 1, wherein remote control is applied to two robots, and the instructor operates one teaching robot and teaches the other teaching robot how to operate.

12. A robot control device according to claim 1, wherein the instructor uses a teach pendant of the teaching robot to teach the teaching robot how to operate.

13. A robot control device according to claim 3, characterized in that the teaching data is generated based on image information captured by the camera of the instructor grasping the object and moving it to the target work position.

14. A robot characterized by incorporating a robot control device according to any one of claims 1 to 13.

15. A robot control system comprising a robot control device according to any one of claims 1 to 13, and a robot controlled by said robot control device.

Images