Robot control device, robot, and robot control system

Abstract

To provide a robot that realize operation for gripping and fitting an object even if the object is in bulk state.SOLUTION: The robot control device comprises: a storage part for storing teaching data that includes object information concerning an object to be gripped, change information concerning positional change of the object in teaching environment when moving the object from a gripping work position to a target work position, and work environment information concerning the target work position; an operation generating part which generates an operation for gripping an object of a same kind as the object in the teaching data on the basis of the object information; and a calculation part which calculates an object-hand relative relationship between the object gripped by a work robot and a hand of the work robot. The calculation part calculates robot target position relative relationship between the work robot and the target work position on the basis of the work environment information. The operation generating part generates a motion trajectory of the work robot so that positional change of the object gripped by the work robot is in accordance with positional change included in the teaching data on the basis of the change information, the object-hand relative relationship, and the robot target work position relative relationship.SELECTED DRAWING: Figure 3

Description

[Technical Field]

[0001] The present invention relates to a robot control device that controls the operation of a working robot using teaching data, a robot equipped with the same, and a robot control system. [Background technology]

[0002] While conventional work robots, which perform a variety of tasks such as pick-and-place, fitting, and assembly, are often only capable of performing routine tasks, in recent years work robots have been required to be adaptable and able to autonomously respond flexibly to their surroundings. Against this background, various technologies related to the control of autonomous robots have been made public.

[0003] For example, claim 1 of Patent Document 1 states, "A robot teaching device that teaches a task to a robot that grasps and moves a graspable object, the robot teaching device comprising: a teaching pose measurement unit that measures and / or calculates a teaching pose, which is the pose of a first graspable object grasped by an instructor during a teaching task; and a robot motion generation unit that generates a sequence of joint displacements of the robot so that the pose of a second graspable object grasped by the robot is identical to the teaching pose." [Prior art documents] [Patent documents]

[0004] [Patent Document 1] International Publication No. 2021 / 260898 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when the object to be grasped is placed in a loose pile or when the position of the object to be grasped changes randomly, the robot's hand must grasp the object without colliding with the surrounding environment. Therefore, the robot may not always be able to grasp the object in the same pose as the teaching pose. Furthermore, when considering the task of grasping an object in a loose pile and then fitting the grasped object into a hole, the taught gripping pose may differ from the actual gripping pose. Therefore, even if the robot is controlled according to the teaching pose as in Patent Document 1, the position of the fitting hole and the object to be grasped may not be aligned. To solve this problem, one method is to teach the fitting operation using the technology described in Patent Document 1 for each gripping pose. However, because the poses required for picking from a loose pile are diverse, another problem arises: the number of teaching operations becomes enormous. Thus, the technology described in Patent Document 1 was highly likely to be unable to reproduce the taught fitting operation in an environment where the objects to be grasped are placed in a loose pile.

[0006] Therefore, the present invention provides a robot control device, a robot, and a robot control system that enable a robot to grasp an object and realize a desired operation on the object to be grasped with a small number of teaching operations. [Means for solving the problem]

[0007] a motion generation unit that generates a motion to grasp an object of the same type as the object in the teaching data based on the object information stored in the memory unit; and a calculation unit that calculates an object-hand relative relationship between the object grasped by the work robot and the hand of the work robot based on the motion generated by the motion generation unit, wherein the calculation unit further calculates a robot target work position relative relationship between the work robot and the target work position based on the work environment information, and the motion generation unit generates a motion trajectory for the work robot based on the change information, the object-hand relative relationship, and the robot target work position relative relationship so that the position change of the object grasped by the work robot follows the position change included in the teaching data. [Effects of the Invention]

[0008] According to the robot control device of the present invention, even if the object to be grasped is placed in a loose pile or its position changes randomly, the robot can grasp the object and fit it into a hole with a small number of teaching operations. [Brief explanation of the drawings]

[0009] [Figure 1] Overall configuration of the robot control system in the first embodiment [Figure 2] Configuration example of the control device in the first embodiment [Figure 3] Flow of teaching and reproduction processing in the first embodiment [Figure 4] Generation of teaching data in the first embodiment [Figure 5] Picking operation generation in Example 1 [Figure 6] Calculation of gripping posture in Example 1 [Figure 7]Recognition of the work environment in Example 1 [Figure 8] Calculation of a trajectory that reproduces the teaching work in Example 1 [Figure 9] Overall configuration of the robot control system in Example 2 [Figure 10] Configuration example of remote control in embodiment 2 [Figure 11] Teaching and reproduction process flow in the second embodiment [Figure 12] Generation of teaching data in the second embodiment [Figure 13] Reproduction of the teaching work in Example 2 [Figure 14] Overall configuration of the robot control system in the third embodiment [Figure 15] Teaching and reproduction process flow in the third embodiment [Figure 16] Picking operation generation unit in the third embodiment [Figure 17] Overall configuration of the robot control system in the fourth embodiment [Figure 18] Robot work example DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the robot control device of the present invention will be described with reference to the drawings. Prior to that, an example of an operation that the robot control device causes the robot to perform will be described with reference to FIG.

[0011] Figure 18(a) shows a robot inserting a memory into a personal computer, Figure 18(b) shows a robot inserting a key into a door lock and turning it, Figure 18(c) shows a robot inserting a dropper into a flask, and Figure 18(d) shows a peg-in-hole task in which the robot inserts a peg into a hole it finds by pressing the robot against the environment.

[0012] Since all of these tasks require continuous control of the position and orientation of the gripped article from the task start position to the task end position, the taught task cannot be autonomously reproduced by teaching only the final position and orientation of the gripped article. Therefore, in order to autonomously reproduce the task shown in Figure 18, the robot control device of the present invention is configured as shown in the following embodiment.

[0013] The work illustrated in FIG. 18 is one example of a work that can be reproduced by the robot control device of the present invention, and other work such as pick-and-place placement work (placing an object in a desired location), assembly work (matching one part with another), and connector mating (fitting connectors together) may also be the work to be reproduced by the robot control device of the present invention. [Example]

[0014] [Overall configuration of the robot control system] FIG. 1 is a schematic diagram of the overall configuration of a robot control system 1 of this embodiment. As shown here, the robot control system 1 of this embodiment is composed of a robot 2, a control device 3 that controls the robot 2, and a camera 4 that captures images of the robot 2 and its surrounding environment. Also arranged around the robot 2 are a tray 5 on which objects O are piled up in bulk, and a work table 6 with a hole H on its top surface for inserting the object O. Each component will be outlined below. Note that in the robot control system 1 of FIG. 1, the control device 3 is arranged external to the robot 2, but the control device 3 may also be built into the robot 2.

[0015] [[Robot 2]] The robot 2 is an articulated arm-type robot having an arm 21 and a hand 22, and having a function of outputting its own posture and external forces detected based on the outputs of encoders and force sensors provided at each movable part as robot information Dr. Note that the robot 2 in this embodiment may be another type of robot, such as a mobile manipulator.

[0016] [[Camera 4]] The camera 4 is an imaging device installed in a position where it can capture images of the robot 2, the tray 5, and the workbench 6, and captures images of the object O held by the robot 2, the object O in the tray 5, the individual holes H on the top surface of the workbench 6, obstacles around the robot 2, etc., and outputs the images as image information Dg. The camera 4 may be a mono camera or a stereo camera, and a sensor capable of measuring three-dimensional information around the robot, such as a TOF sensor or a laser sensor, may be used instead of the camera 4.

[0017] [[Controller 3]] The control device 3 is a device that controls the robot 2 based on the robot information Dr acquired from the robot 2 and the image information Dg acquired from the camera 4. In the following, the robot to be controlled when the control device 3 is set to the teaching mode will be referred to as the teaching robot 2A, and the robot to be controlled when the control device 3 is set to the reproduction mode will be referred to as the working robot 2B.

[0018] Specifically, the teaching mode in this embodiment is a direct teaching mode. In this mode, an instructor P applies an external force to the teaching robot 2A to guide the arm 21 and hand 22 of the teaching robot 2A to various positions and postures (hereinafter, for simplicity, positions and postures will be referred to as "positions"), thereby teaching a series of actions to be reproduced by the working robot 2B in the reproduction mode. For example, through operation by the instructor P, the control device 3 is taught to use the hand 22 of the teaching robot 2A to grasp one of the objects O randomly piled in the tray 5 and fit the grasped object O into one of the holes H on the top surface of the workbench 6.

[0019] On the other hand, the reproduction mode of this embodiment is a mode that reproduces, in accordance with actual conditions, the same operation as that taught under the teaching mode, based on the teaching data Dt generated by the control device 3 under the teaching mode, and the image information Dg and robot information Dr input to the control device 3 under the reproduction mode.

[0020] [[Hardware configuration of control device 3]] Next, the hardware configuration of the control device 3 will be described with reference to Fig. 2. The control device 3 illustrated here is a computer including a CPU 3a (Central Processing Unit), a ROM 3b (Read Only Memory), a RAM 3c (Random Access Memory), a non-volatile storage 3d, a network interface 3e, an input unit 3f, a display unit 3g, and a bus 3h connecting the respective components.

[0021] The CPU 3a is a processing unit that reads out from the ROM 3b and executes the program code of the software that executes the arithmetic processing related to the robot control described later. Variables and parameters generated during the arithmetic processing are temporarily written to the RAM 3c.

[0022] The nonvolatile storage 3d may be, for example, a large-capacity information storage unit such as a hard disk drive (HDD) or a solid state drive (SSD). The nonvolatile storage 3d constitutes a database in which the instructed data is stored.

[0023] The network interface 3e may be, for example, a network interface card (NIC), etc. Instructions to the robot 2 are given via the network interface 3e.

[0024] The input unit 3f performs input processing for the robot control program by switching to a direct teaching mode and registering waypoints, which are necessary for teaching the robot.

[0025] The operating status of the control device 3 is displayed on the display section 3g.

[0026] 2 is merely an example, and the control device 3 may be configured with a device that performs arithmetic processing other than a computer. For example, some or all of the functions performed by the control device 3 may be realized by hardware such as an FPGA (Field Programmable Gate Array) or an ASIC (Application Specific Integrated Circuit).

[0027] Furthermore, the configuration in which the control device 3 is equipped with an input unit 3f and a display unit 3g is also an example, and the control device 3 may be configured as a computer that does not have either or both of the input unit 3f and the display unit 3g.

[0028] [Teaching process and reproduction process in Example 1] Next, the processing of the teaching phase and the reproduction phase in the first embodiment will be described with reference to Fig. 3. Note that the thick arrows in Fig. 3 represent homogeneous transformation matrices, which are the relative positional relationships between the coordinate systems (hereinafter referred to as relative relationships).

[0029] Step S1: Teaching Phase When the control device 3 is set to the teaching mode, the robot 2 is treated as a teaching robot 2A in the direct teaching mode, which makes it ready to start the teaching phase in which teaching data Dt is generated.

[0030] In step S1 of FIG. 3, the instructor P physically contacts the teaching robot 2A and applies an external force to make the work robot 2B execute the desired operation. This allows the control device 3 to acquire time-series image information Dg and robot information Dr when the teaching robot 2A executes the desired task, and to store teaching data Dt based on these in the non-volatile storage 3d. The teaching data Dt stored here may include, for example, change information (R) relating to the change in the position of the object O in the work environment when the object O is moved from the grasping task position to the target task position. env_o ), object information Do regarding the object O to be grasped, and environmental information De regarding the work environment into which the object O is inserted.

[0031] In addition, this change information (R env_o ) is the environmental coordinate system Σ fixed to the working environment. env Object coordinate system Σ fixed to object O as seen from O This change information (R env_o ) represents how the object O, which is the most important factor when reproducing the insertion work, is inserted into the work environment. By saving this as teaching data Dt and controlling it in combination with the gripping posture of the actual object O described later, it is possible to reproduce the mating operation even in a variety of gripping postures.

[0032] FIG. 4 shows details of the method for generating the teaching data Dt in step S1. This process is executed while the instructor P is teaching the desired operation, and the teaching data Dt is saved. Here, the target is a fitting operation, but it is not limited to fitting operations and may also be pick-and-place, peg-in-hole, assembly operations, etc. Note that the object position recognition unit 31a, work environment recognition unit 31b, and relative relationship calculation unit 31c in the figure are functional units realized by the CPU 3a of the control device 3 executing programs corresponding to each function.

[0033] The object position recognition unit 31a recognizes the position of the object O based on the image information Dg, and calculates the position of the object O in the camera coordinate system Σ c Object coordinate system Σ in o The relative relationship (R c_o ) and the shape and type of object O.

[0034] The work environment recognition unit 31b recognizes the work environment based on the image information Dg and calculates the image in the camera coordinate system Σ c In the environmental coordinate system Σ env The relative relationship (R c_env ) and the shape and type of the working environment.

[0035] The relative relationship calculation unit 31c calculates the relative relationship (R) between the work environment and the object O based on the object position output from the object position recognition unit 31a and the work environment position output from the work environment recognition unit 31b.env_o ) in the environmental coordinate system Σ env Object coordinate system Σ o Calculate the homogeneous transformation matrix of

[0036] The shape and type of object O, the shape and type of the work environment, and the relative relationship between the work environment and object O calculated by the above process are stored as teaching data Dt in a database in the nonvolatile storage 3d.

[0037] The above process in Fig. 4 is executed at a predetermined cycle while the instructor P is teaching the desired operation. That is, the teaching data Dt is time-series data on the relative relationship between the work environment and the object O, in other words, change information on the change in the position of the object O in the work environment when the object O is moved from the gripping work position to the target work position. This teaching data Dt represents in time series the trajectory along which the object O is inserted into the work environment, which is essential for reproducing the fitting work.

[0038] [[Reproduction Phase]] Next, the reproduction phase will be explained, divided into steps S2 to S5. When the control device 3 is set to reproduction mode, the robot 2 is treated as a working robot 2B. This completes preparations for the start of the reproduction phase, in which the working robot 2B reproduces the desired work.

[0039] Step S2: Picking motion generation process In step S2, the control device 3 recognizes the objects O stacked randomly in the tray 5 based on the current image information Dg and object information Do, and then outputs a command to the working robot 2B to grasp the objects O in the random state.

[0040] Here, the picking motion generation method in step S2 will be described with reference to Fig. 5. Note that the candidate object recognition unit 32a, obstacle recognition unit 32b, gripping point position determination unit 32c, and robot motion generation unit 32d in the figure are functional units realized by the CPU 3a of the control device 3 executing programs corresponding to each function.

[0041] The object candidate recognition unit 32a outputs a plurality of pieces of object candidate information Dc relating to the positions and shapes of object candidates based on the image information Dg and the shape and type information of the object O stored in the database. This process may be performed using a method such as pattern matching or a machine learning method such as a CNN (Convolutional Neural Network).

[0042] The obstacle recognition unit 32b outputs obstacle information Db about obstacles around the object candidate based on the image information Dg and the grasped object candidate information Dc. The obstacle information Db may be represented by the type and shape, or may be point cloud information indicating the presence of a three-dimensional object even if the type is unknown.

[0043] The gripping point position determining unit 32c recognizes the packaging style in which each object O is placed based on the gripping object candidate information Dc and the obstacle information Db, and determines the gripping point (the position to grip) of an object that is easy for the work robot 2B to grip (for example, an object that is at the top and can be gripped without colliding with other objects). This gripping point position is determined by the posture in which the hand 22 grips the object O, i.e., the object coordinate system Σ o and the TCP coordinate system Σ tcp The relative relationship (R o_tcp ) The TCP (Tool Center Point) is the tip position of the hand 22.

[0044] Robot motion generator 32d generates a trajectory for hand 22 of working robot 2B based on the gripping point position and geometric information of arm 21. Then, a robot trajectory command that realizes the generated robot trajectory is executed, causing working robot 2B to grip desired object O.

[0045] Step S3: Grasp posture calculation In step S3, the control device 3 determines the relative relationship (R o_tcp The relative relationship between the hand 22 and the object O is calculated in the object coordinate system Σ OIt is expressed as a homogeneous transformation matrix of the hand coordinates as seen from the

[0046] Here, the grip posture calculation process in step S3 will be described with reference to Fig. 6. Note that the grip object recognition unit 33a, TCP calculation unit 33b, and grip posture calculation unit 33c in the figure are functional units realized by the CPU 3a of the control device 3 executing programs corresponding to each function.

[0047] The object recognition unit 33a calculates the position of the object O being grasped by the work robot 2B from the image information Dg and the shape, type, etc. of the object O.

[0048] TCP calculation unit 33b calculates TCP, which is the tip position of hand 22, from the encoder information included in robot information Dr and geometric information of working robot 2B.

[0049] The gripping posture calculation unit 33c calculates the relative relationship (R o_tcp This process is executed periodically during the reproduction phase, so the relative relationship (R o_tcp ) is updated at a predetermined interval. However, if it is assumed that the relative relationship between the object O and the hand does not change after the object O is grasped, the above process may be performed only once, or the relative relationship between the object O and the hand 22 (R o_tcp ) can be used as is.

[0050] Step S4: Recognition of the working environment In step S4, the relative relationship (R) between the work robot 2B and the work environment is calculated based on the image information Dg and the work environment information De. r_env ) is calculated. The relative relationship between the work robot 2B and the work environment is expressed in terms of the robot coordinate system Σ r The environmental coordinate system Σ env is expressed as a homogeneous transformation matrix.

[0051] The work environment recognition process in step S4 will now be described with reference to Fig. 7. Note that the work environment candidate recognition unit 34a and the target work environment determination unit 34b in the figure are functional units realized by the CPU 3a of the control device 3 executing programs corresponding to the respective functions.

[0052] The work environment candidate recognition unit 34a recognizes candidates for the work environment (hole H for inserting object O) based on the image information Dg and the shape and type information of the taught work environment. This process may be performed using a method such as pattern matching or a machine learning method such as CNN.

[0053] The target work environment determination unit 34b outputs the next target work environment position (hole position) based on the order of fitting into the work environment (which hole H to fit in order) previously set by the instructor P and the position information of the candidate work environment.

[0054] Step S5: Calculation of the trajectory that reproduces the teaching task In step S5, the control device 3 calculates the relative relationship (R o_tcp ) and the relative relationship between the work robot 2B and the work environment (R r_env ) and change information (R env_o ) and calculate the trajectory of the TCP from the robot coordinate system. Note that each point of the trajectory of the TCP calculated in this step is calculated in the robot coordinate system Σ r From the TCP coordinate system Σ TCP represents the homogeneous transformation matrix of the TCP. In other words, the TCP trajectory is a time series data of the homogeneous transformation matrix.

[0055] Here, a method for calculating a trajectory for reproducing the teaching work in step S5 will be described with reference to FIG. 8. First, the teaching data Dt is information (R env_o ) and the current relative positional relationship between the work robot 2B and the environment (R r_env) based on the homogeneous transformation process, the position change of object O relative to the robot (R r_o ) is calculated.

[0056] Next, the position change of the object O relative to the work robot 2B (R r_o ) and the grasping posture of object O (R o_tcp ) based on the robot coordinate system Σ, which is the robot trajectory that reproduces the teaching task. r Change in hand position (R r_tcp ) is calculated. Here, the grasping posture (R o_tcp ) may be time-series data, or may be a constant as long as the relative relationship between the object O and the hand does not change after the object O is grasped.

[0057] Then, at the end of step S5, the control device 3 controls the work robot 2B according to the calculated TCP trajectory, causing the work robot 2B to reproduce, in a different environment, the same task as that taught to the teaching robot 2A. Note that the calculation of the homogeneous transformation is a calculation of geometric relationships, and the order of calculation is not important, so the calculations may be performed in a different order.

[0058] In this way, the work taught via the teaching robot 2A is reproduced by the working robot 2B not only as the final fitting posture but also as the taught trajectory. This means that the working robot 2B moves so that the change in position of the object O grasped by the working robot 2B in the reproduction phase follows the change in position of the object O from the grasping work position to the target work position, which is included in the teaching data Dt.

[0059] [Effects of Example 1] As described above, when the robot teaching and reproduction method of Example 1 is applied, during reproduction, bulk picking is performed in step S2, and the robot trajectory (how the robot moves to reproduce the insertion operation) is calculated in step S5 from the actual grasping posture (how the object O is grasped) calculated in step S3 and the teaching data Dt (how the object O is inserted into the work environment), and the work robot 2B can be controlled.

[0060] Therefore, even in situations where the gripping posture changes each time the object O is picked, such as when the object O is piled up in a random state or when the object O is not placed in a constant posture, the robot can reproduce the fitting operation using teaching data Dt that shows the object O being gripped and fitted in a different posture.

[0061] Furthermore, in step S4, the relative relationship between the work robot 2B and the work environment is determined, and in step S5, the robot's trajectory is calculated based on this relative relationship, so the taught task can be reproduced even if the position of the work environment changes between the time of teaching and the time of reproduction.In other words, a robot control system that is robust to a variety of environmental changes can be provided with few teaching steps.

[0062] Although the above has been described using the example of a fitting operation in which an object O is inserted into a hole H, the present invention is intended for general operations that change the relative distance between an object and the work environment or another object, and may be applied to pick-and-place placement operations in which an object is placed in a desired location, assembly operations in which certain parts are combined in a desired posture, connector fitting operations in which connectors are fitted together, and peg-in-hole operations in which an object is inserted into a desired position. [Example]

[0063] Next, a robot control system according to the second embodiment will be described with reference to Figures 9 to 13. The following description will focus on differences from the first embodiment, and a description of similar parts will be omitted.

[0064] 9 shows the overall configuration of a robot control system 1 of Example 2. In Example 1, one robot 2 was used as a teaching robot 2A in the teaching phase and as a working robot 2B in the reproduction phase, but in this example, one of two robots 2 is used as a dedicated teaching robot 2A and the other is used as a dedicated working robot 2B, and dedicated control devices 3A and 3B are connected to each robot.

[0065] The control devices 3A and 3B of this embodiment are equipped with bilateral control, which is described in FIG. 10. Therefore, when the instructor P moves the teaching robot 2A, the movement of the working robot 2B is controlled to follow. Furthermore, the reaction force generated when the working robot 2B comes into contact with the environment is returned to the instructor P via the teaching robot 2A. Therefore, by using the robot control system 1 of this embodiment, which applies bilateral control, the instructor P can teach the fitting operation, including adjusting the amount of force.

[0066] [Remote control configuration example] Next, the bilateral control implemented in the control device 3A and the control device 3B will be described with reference to the block diagram of FIG. l is the position information of the teaching robot 2A, F l is the force information of the teaching robot 2A, x f is the position information of work robot 2B, F f is the force information of the work robot 2B.

[0067] Bilateral control is a well-known technology, as seen in, for example, International Publication No. 2005 / 109139. It controls each joint so that the postures of the teaching robot 2A and the working robot 2B are aligned, while controlling the forces between the robots 2A and 2B so that they have an action-reaction relationship. Therefore, the instructor P operating the teaching robot 2A can physically feel the reaction force that the working robot 2B receives from the environment. The reaction force does not necessarily have to be detected by a human; it can be detected by a force sensor or by a means for estimating the reaction force, such as a disturbance observer. Note that the bilateral control method is not limited to this, and a force feedback control system or other similar control system can also be used. In this way, by having the instructor P teach the robot using bilateral control, it is possible to teach the robot the amount of force applied by the human.

[0068] [Teaching and Reproduction Process Flow] 11 shows an example of the flow of the teaching and reproduction processes in Example 2. The difference from Example 1 is that in the teaching phase, step S1 is replaced with step S1A and step S1B, and in the reproduction phase, step S2 is replaced with step S2A and step S5 is replaced with step S5A. Each of the replaced steps will be described in detail below.

[0069] In step S1A, the control device 3A records the change in the position of the TCP (the trajectory during picking) and the time series data of the force (the strength of the force during picking) as teaching data Dt based on the encoder information and force sensor information of the teaching robot 2A and the object position information recognized from the image information Dg. At this time, the object coordinate system Σ o By storing the position and force of the TCP as seen from the object O, it is possible to record as teaching data Dt how the teaching robot 2A grasps each individual object O (how to grasp an object differently for each object). By repeating the process of step S1A many times and generating teaching data Dt for grasping items in a variety of postures, when reproducing the picking operation with the work robot 2B, it can respond more flexibly to the environment, even in situations such as bulk picking.

[0070] In step S1B, the control device 3A calculates the position change (R env_o ) and the object coordinate system Σ detected by bilateral control. o The time series data of the force at object O, i.e., how the robot applies force to object O during the mating task, is recorded.

[0071] In step S2A, the control device 3B selects the object O to be grasped and a trajectory for picking the object without colliding with surrounding obstacles from the candidate object to be grasped, obstacle information, and multiple taught picking trajectories. Furthermore, the control device 3B calculates a force command to be applied to the object O to be grasped by the work robot 2B from the position of the object O to be grasped and information on changes in force as seen from the object O, and controls the work robot 2B based on the picking trajectory and grasp command information to grasp the object O.

[0072] In step S5A, the control device 3B determines the relative relationship (R o_tcp ) and the relative relationship between the work robot 2B and the work environment (R r_env ) and the position change of object O relative to the working environment (R env_o ) and time-series data of the force during mating, the trajectory and force command of the TCP as seen from the robot coordinates of the work robot 2B are calculated, and by executing that trajectory, the mating work taught in step S1B is reproduced, including the amount of force.

[0073] As described above, when teaching the picking operation in step S1A, the "posture and force to be used to pick the object" are memorized, and when reproducing the operation, in step S2A, the trajectory of the work robot 2B when reproducing the picking operation is calculated based on the picking teaching data Dt (the trajectories of multiple gripping operations) and the actual object position recognized by the camera, and the work robot 2B is controlled according to the calculated trajectory. This makes it possible to pick even when there are objects in a bulk pile that require a variety of picking postures. In addition, because the gripping position and force are adjusted for each item, it is possible to pick fragile items or items that have a specific location to hold.

[0074] Furthermore, when teaching the mating operation in step S1B, the "force required to insert the object" is remembered, so when reproducing the taught operation, the force can be included in the reproduction. In particular, it is possible to properly reproduce a mating operation such as that shown in Figure 18(d), in which the position of the hole is searched for while pressing an object against a work table with an appropriate force.

[0075] [Generation of teaching data Dt] Next, a method for generating teaching data Dt in the second embodiment will be described with reference to Fig. 12. This method assumes that the relative relationship between the object O and the hand 22 does not change after the object O is grasped, and uses the relative relationship (R o_tcp ) and the trajectory of the TCP of the work robot 2B during the mating operation (Rr_tcp time series data) and the relative relationship between the work environment and work robot 2B (R r_env ) based on the position change of the object in the working environment during the mating operation (R env_o ) is calculated.

[0076] According to this method, unlike in Example 1, when generating the teaching data Dt, it is not necessary to continuously photograph the position of the object O during the fitting operation with the camera 4, so there is no need to take care to ensure that the object O is not hidden in the shadow of the arm 21 or the hand 22 (there is no need to worry about occlusion), which has the effect of making teaching easier.

[0077] [Reproduction of teaching work] Next, a method for reproducing the teaching work in the second embodiment will be described with reference to FIG. 13. The difference from the first embodiment is that the teaching data Dt of the force is reproduced (the calculation in the upper half of FIG. 13). The force applied by the robot in the object coordinates, which is the teaching data Dt, is expressed as the gripping force F o is the change in the object's position as seen by the robot (R r_o ) to perform homogeneous transformation and obtain the force command F r (how the robot applies force to the object during the fitting operation). The value of the position command is the change in the position of the hand (R r_tcp By using these as position and force command values ​​to operate the robot, it is possible to achieve a fitting operation that replicates the force exerted by a human. [Example]

[0078] Next, a robot control system according to a third embodiment will be described with reference to Figures 14 to 16. The following description will focus on differences from the first embodiment, and a description of similar parts will be omitted.

[0079] [Overall configuration of the robot control system] First, the overall configuration of the robot control system 1 of Example 3 will be described with reference to Fig. 14. In Example 1, the instructor P generates the teaching data Dt by directly moving each joint of the teaching robot 2A, but in this example, the instructor P operates the teaching robot 2A using an input / output device 7 such as a teaching pendant connected to the control device 3, thereby teaching the teaching data Dt to the control device 3.

[0080] Teaching the robot using the input / output device 7 is a method commonly used for industrial robots. For example, the TCP of the teaching robot 2A is registered as a sequence of multiple waypoints in the input / output device 7. Alternatively, the robot may be taught to move at a predetermined speed until it detects a force sensor (not shown) using information from a force sensor (not shown) only along a predetermined coordinate axis. In this way, in this embodiment, robot operation may be taught using various teaching methods commonly used for industrial robots.

[0081] [Teaching and Reproduction Process Flow] Next, the flow of the teaching and reproduction process in the third embodiment will be described with reference to Fig. 15. The difference from the first embodiment is that step S2 in the reproduction phase is replaced with step S2B. The replaced step S2B will be described in detail below.

[0082] Step S2B is a process for determining the posture of the picking operation, taking into consideration collisions with the surrounding environment during the mating operation. Therefore, in this step, the posture of the picking operation is determined based on the object information and image information Dg, as well as the position change of the object in the work environment during the mating operation and information about the work environment, taking into consideration whether there will be collisions with surrounding obstacles during the mating operation. As a result, as shown in the left diagram of Figure 15, the gripping posture (a) that can grasp the object and will not collide with the environment during mating is selected and picked, rather than the gripping posture (b) that can grasp the object but will collide with the environment during mating. This makes it possible to achieve picking and mating operations with improved robustness and safety. More specific processing will be described in Figure 16.

[0083] Next, a method for generating a picking operation in the third embodiment will be described with reference to FIG. 16. The difference from FIG. 5 of the first embodiment is the processing of the gripping point position determination unit 32c'. The gripping point position determination unit 32c' in this embodiment determines whether a collision will occur during picking based on the position and shape of the candidate object to be gripped, obstacles around the object, as well as the change in the position of the object in the working environment during the fitting operation, and the shape and type of the working environment, as well as the position and shape of the candidate object to be gripped, which were also used in the first embodiment. Furthermore, the gripping point position determination unit 32c' determines whether the hand will collide with the environment in the final posture of the fitting operation when gripped at the gripping point candidate, and selects an appropriate gripping posture that will not collide in both the picking and fitting operations based on another criterion (for example, no collapse of the load) as the gripping point position.

[0084] The robot teaching and reproduction method in Example 3 described above makes it possible to teach the operation using the input / output device of the control device, and by performing the picking operation while taking into consideration collisions during the fitting operation, it is possible to realize picking and fitting operations with increased robustness and safety. [Example]

[0085] Next, a robot control system according to a fourth embodiment will be described with reference to Fig. 17. The following description will focus on differences from the first embodiment, and a description of similar parts will be omitted.

[0086] 17 is a schematic diagram of the overall configuration of a robot control system 1 according to Example 4. In Example 1, the instructor P generates the teaching data Dt by directly moving each joint of the teaching robot 2A, but in this example, the instructor himself grasps the object O and converts the action of fitting it into the hole H into data using a camera 4, and generates the teaching data Dt based on this data.

[0087] By applying the fourth embodiment, the same effect as in the first embodiment can be obtained even in the work teaching by the instructor P. That is, when reproducing, bulk picking is performed in step S2, and the robot trajectory (how the robot moves to reproduce the insertion operation) is calculated in step S5 from the actual gripping posture (how the object is gripped) calculated in step S3 and the teaching data Dt (how the object is inserted into the work environment), and the robot can be controlled. Therefore, the teaching data Dt can be generated more easily than in the first embodiment, and the same effect as in the first embodiment can be obtained.

[0088] [Variations] The above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, in the above-described embodiments, the device or system configuration may be changed, and some processing steps may be omitted or replaced, within the scope of the gist of the present invention. Information such as programs for performing picking and fitting processes may be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or optical disk. [Explanation of symbols]

[0089] 1...Robot control system, 2...Robot, 2A...Teaching robot 2B...Work robot 21...Arm, 22...hand, 3...control device, 3a...Central processing unit (CPU), 3b…ROM, 3c…RAM, 3d...non-volatile storage, 3e...Network interface, 3f...input section, 3g...display section, 31a...Object position recognition unit, 31b...Working environment recognition unit, 31c…Relative relationship calculation unit, 32a ... grasped object candidate recognition unit, 32b…Obstacle Recognition Department, 32c…holding point position determination unit, 32d…Robot action generation unit, 33a…object recognition unit, 33b…TCP calculation unit, 33c…holding posture calculation part, 34a…Work Environment Candidate Recognition Department, 34b…Target work environment decision department, 4…Kara, 5…トレイ 6…Workbench, 7...Input and output device, P…Teacher, O…object, H…hole Dr...Robot information, Dg...portrait information, Dt…teaching データ, Do... Object information, De…Environmental information, DC... Candidate information on controlled objects, Db...Obstacle information,

Claims

1. A robot control device that controls the operation of a work robot based on teaching data, a storage unit that stores teaching data including object information regarding an object to be grasped, change information regarding a position change of the object in a teaching environment when an instructor or a teaching robot moves the object from a grasping work position to a target work position, and work environment information regarding the target work position; an action generating unit that generates an action to grasp an object of the same type as the object in the teaching data based on the object information stored in the storage unit; a calculation unit that calculates an object-hand relative relationship between an object grasped by the working robot by the motion generated by the motion generation unit and a hand of the working robot, the calculation unit further calculates a robot target work position relative relationship between the work robot and the target work position based on the work environment information; the movement generation unit generates a movement trajectory of the work robot based on the change information, the object-hand relative relationship, and the robot target work position relative relationship so that the position change of the object grasped by the work robot conforms to the position change included in the teaching data.

2. The robot control device according to claim 1, 10. A robot control device according to claim 9, wherein the object hand relative relationship and the robot target work position relative relationship are relative positional relationships.

3. The robot control device according to claim 1, The calculation unit recognizes the position change of the object and the work environment position when the instructor or the teaching robot moves the object from the grasping work position to the target work position based on image information captured by a camera, and calculates the change information based on the position change of the object and the work environment position.

4. The robot control device according to claim 1, the calculation unit calculates the relative relationship between the object and the hand at the moment the object is grasped based on image information captured by a camera and the object information; Furthermore, a relative relationship between the work robot and the work environment is calculated based on the image information and the work environment information; A robot control device characterized in that the change information is calculated based on the object-hand relative relationship, the working robot-working environment relative relationship, and the motion trajectory of the working robot.

5. The robot control device according to claim 3, the action generation unit calculates candidates for the object to be grasped based on the image information and the object information stored in the storage unit; Detecting obstacles around the candidate object to be grasped based on the image information and object information of the candidate object to be grasped; determining a gripping point position based on the gripping object candidate and obstacles around the candidate; A robot control device characterized in that the work robot is controlled based on the gripping point position to grip an object of the same type as the object in the teaching data.

6. The robot control device according to claim 1, The action generation unit Based on the candidate object to be grasped and information on obstacles around it, change information on the positional changes of the object related to the fitting operation, and information on the target work position, the system determines a gripping point position that will not collide with obstacles during the fitting operation, and grasps an object of the same type as the object in the teaching data.

7. The robot control device according to claim 1, The storage unit further stores a picking trajectory of the TCP in an object coordinate system when the instructor or the teaching robot picks up an object and information on a change in force applied to the object grasped by the teaching robot, the operation generation unit selects a trajectory for picking the object without colliding with the object to be grasped and surrounding obstacles from position information of the candidate object to be grasped, information on obstacles around the candidate object, and a plurality of taught picking trajectories; and calculating a force command to be applied to the item to be grasped by the working robot based on the position of the object to be grasped and the force change information; A robot control device characterized in that it grasps an item based on information on the picking trajectory and force command.

8. The robot control device according to claim 1, the storage unit further stores teaching data including information on a change in force applied to the object by the teaching robot in a teaching environment; The robot control device is further characterized in that the action generation unit generates a force for the working robot so that the direction and magnitude of the force applied by the teaching robot to the object, which is included in the teaching data, are reproduced based on the force change information, change information related to the position change of the object, and the robot target working position relative relationship.

9. The robot control device according to claim 3, the calculation unit calculates a relative relationship between the target work position of the robot and the camera based on the image information and the work environment information, and calculates the relative relationship between the target work position of the robot and the robot based on the relative relationship between the target work position of the robot and the camera.

10. The robot control device according to claim 1, A robot control device characterized in that a teacher operates a teaching robot by physically contacting it and teaching it how to move.

11. The robot control device according to claim 1, A robot control device characterized in that remote control is applied to two robots, and the instructor operates one teaching robot and teaches the other teaching robot how to operate.

12. The robot control device according to claim 1, A robot control device characterized in that the instructor uses a teach pendant of the teaching robot to teach the teaching robot an operation.

13. The robot control device according to claim 3, A robot control device characterized in that the teaching data is generated based on image information obtained by capturing, with the camera, the action of the instructor grasping the object and moving the object to the target work position.

14. A robot comprising a built-in robot control device according to any one of claims 1 to 13.

15. A robot control device according to any one of claims 1 to 13, A robot control system comprising a robot controlled by the robot control device.

Images