Robot remote operation control system, robot remote operation control device, robot remote operation control method, program

Abstract

To provide a robot remote operation control system, a robot remote operation control device, a robot remote operation control method, and a program which can improve workability more than before, in remotely operating three or more arms each having an end effector.SOLUTION: In a robot remote operation control system, a robot comprises: three or more end effectors; movement mechanisms that move the end effectors respectively; a first control part that controls motion of at least one end effector, in accordance with movement of an operator; a motion designating part that determines an action starting point for the end effectors on the basis of movement of the operator, and designates motion from the action starting point for the end effectors; a track generating part that generates restricted tracks for the end effectors, from the motion designated by the motion designating part; and a second control part that makes at least one end effector of the end effectors that are not controlled by the first control part operate on the track.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a robot remote operation control system, a robot remote operation control device, a robot remote operation control method, and a program. [Background technology]

[0002] For example, there is technology in which robots with three or more arms work together to assemble on a factory line, or in which medical robots perform surgery by remotely switching between multiple arms (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-107721 Summary of the Invention [Problem to be solved by the invention]

[0004] However, conventional technology systems that control multiple arms fully automatically have great difficulty in manipulating flexible objects. Furthermore, with conventional technology, switching between arms remotely results in confusion and time loss, and the system cannot handle tasks that require the simultaneous movement of three or more arms. Thus, with conventional technology, remote operation of three or more arms, each with an end effector, is difficult and inefficient.

[0005] The present invention has been made in consideration of the above-mentioned problems, and aims to provide a robot remote operation control system, a robot remote operation control device, a robot remote operation control method, and a program that can improve workability compared to conventional methods when remotely operating three or more arms, each having an end effector. [Means for solving the problem]

[0006] (1) In order to achieve the above-mentioned object, a robot remote operation control system according to one embodiment of the present invention is a robot remote operation control system that recognizes the movements of an operator and transmits the movements of the operator to a robot to operate the robot, wherein the robot includes three or more end effectors, a movement mechanism that moves each of the end effectors, a first control unit that controls the movement of at least one of the end effectors in accordance with the movement of the operator, a movement designation unit that determines a starting point of action for the end effector based on the movement of the operator and designates the movement of the end effector from the starting point of action, a trajectory generation unit that generates a constrained trajectory for the end effector from the movement designated by the movement designation unit, and a second control unit that operates at least one of the end effectors not controlled by the first control unit on the trajectory.

[0007] (2) In a robot remote operation control system according to one aspect of (1) above, when there are two gripping points at which the object to be operated is grasped based on the movement of the operator, the operation designation unit may set one of the two gripping points as the action starting point and the other of the two gripping points as the action end point, and the trajectory generation unit may set a constrained trajectory of the end effector between the action starting point and the action end point.

[0008] (3) In the robot remote operation control system according to one aspect of (1) or (2) above, the trajectory generation unit may generate a constrained trajectory by preventing movement in at least one of a plurality of axial directions based on the target object and the work content estimated based on the movement of the operator.

[0009] (4) In a robot remote operation control system according to any one of the above (1) to (3), an intention estimation unit may be provided that estimates a target object and a task content based on the movement of the operator, and the first control unit and the second control unit may control the end effectors to be controlled, respectively, using the estimation results obtained by the intention estimation unit.

[0010] (5) In the robot remote operation control system according to one aspect of (4) above, the first control unit may be configured to control the operation of the end effector to support the operation of the operator based on the operation intention estimated by the intention estimation unit.

[0011] (6) In order to achieve the above object, a robot remote operation control device according to one embodiment of the present invention is a remote operation control device that remotely controls a robot having three or more end effectors and a movement mechanism that moves each of the end effectors, and is equipped with: a first control unit that controls the movement of at least one of the end effectors in accordance with the movement of an operator; a movement designation unit that determines a starting point of action for the end effector based on the movement of the operator and designates the movement of the end effector from the starting point of action; a trajectory generation unit that generates a constrained trajectory for the end effector from the movement designated by the movement designation unit; and a second control unit that operates at least one of the end effectors not controlled by the first control unit on the trajectory.

[0012] (7) In order to achieve the above-mentioned object, a robot remote operation control method according to one embodiment of the present invention is a control method for a remote operation control device that remotely controls a robot having three or more end effectors and a movement mechanism for moving each of the end effectors, comprising: a first control unit that controls the movement of at least one of the end effectors in accordance with the movement of an operator; a movement designation unit that determines a starting point of action for the end effector based on the movement of the operator and designates the movement of the end effector from the starting point of action; a trajectory generation unit that generates a constrained trajectory for the end effector from the movement designated by the movement designation unit; and a second control unit that operates at least one of the end effectors that is not controlled by the first control unit on the trajectory.

[0013] (8) In order to achieve the above object, one aspect of the present invention provides a program that causes a computer of a remote control device that remotely controls a robot having three or more end effectors and a movement mechanism for moving each of the end effectors to control the movement of at least one of the end effectors in accordance with the movement of an operator, determines a starting point of action for the end effector based on the movement of the operator, specifies the movement of the end effector from the starting point of action, generates a constrained trajectory for the end effector from the specified movement, and moves at least one of the end effectors other than the end effector controlled in accordance with the movement of the operator on the trajectory. [Effects of the Invention]

[0014] According to the above (1) to (8), when three or more arms each having an end effector are remotely controlled, workability can be improved compared to conventional methods. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a diagram illustrating a schematic configuration example of a robot remote operation control system according to an embodiment. [Figure 2]1 is a diagram illustrating an example of the configuration of a robot remote operation control system according to an embodiment. [Figure 3] 4 is a flowchart of a process of the remote control control device according to the embodiment. [Figure 4] FIG. 1 is a diagram for explaining a first embodiment. [Figure 5] 3 is a flowchart of a process according to the first embodiment. [Figure 6] FIG. 10 is a diagram for explaining a second embodiment. [Figure 7] 10 is a flowchart of a process according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings used in the following description, the scale of each component is appropriately changed so that each component can be recognized. In all the drawings for explaining the embodiments, the same reference numerals are used for components having the same functions, and repeated explanations will be omitted. Furthermore, in this application, "based on XX" means "based on at least XX," and includes cases where it is based on other elements in addition to XX. Furthermore, "based on XX" is not limited to cases where XX is used directly, but also includes cases where it is based on XX that has been calculated or processed. "XX" is any element (for example, any information).

[0017] [overview] 1 is a diagram showing a schematic configuration example of a robot remote operation control system according to this embodiment. The robot remote operation control system 1 includes, for example, a first robot 2-1, a second robot 2-2, a third robot 2-3, a remote operation control device 3 (robot remote operation control device), and an environmental sensor 4. The nth robot 2-n (n is an integer between 1 and 3) includes, for example, a moving mechanism 21-n and an end effector 22-n. Note that one robot 2 may include three moving mechanisms and three end effectors. Alternatively, the first robot may include two moving mechanisms and two end effectors, and the second robot may include one moving mechanism and one end effector. Furthermore, the number of robots each having a combination of moving mechanisms and end effectors may be four or more.

[0018] In this embodiment, for example, in a combination of three sets of arms and end effectors, two sets are controlled based on remote control instructions, and the remaining set is automatically controlled based on the operation content and operation intention. This allows this embodiment to efficiently perform tasks that are difficult to automate.

[0019] [Configuration of a robot remote control system] Fig. 2 is a diagram showing an example of the configuration of a robot remote operation control system according to this embodiment. As shown in Fig. 2, the robot remote operation control system 1 includes, for example, a first robot 2-1, a second robot 2-2, a third robot 2-3, ..., a remote operation control device 3, an environmental sensor 4, an operation input unit 5, and an image display device 6.

[0020] The robots 2 (first robot 2-1, ..., n-th robot-n) include, for example, moving mechanisms 21 (21-1, ..., 21-n), end effectors 22 (22-1, ..., 22-n), sensors 23 (23-1, ..., 23-n), actuators 24 (24-1, ..., 24-n), drive units 25 (25-1, ..., 25-n), and communication units 26 (26-1, ..., 26-n). Note that the configurations of the robots 2-n may be the same or different. The remote control control device 3 includes, for example, an acquisition unit 31, an action designation unit 32, a trajectory generation unit 33, a first control unit 34, a second control unit 35, a memory unit 36, a communication unit 37, an image generation unit 38, and an intention estimation unit 39. The environmental sensor 4 includes, for example, an imaging device 41 and a communication unit 42. The robot 2, the remote control device 3, the environmental sensor 4, the operation input unit 5, and the image display device 6 are equipped with a power supply and the like (not shown).

[0021] The operation input unit 5 detects operation input information from the operator and outputs it to the remote operation control device 3. The operation input unit 5 is, for example, a data glove. The operation input information includes information such as the position of each hand, the angle of the finger joints, the position of the wrist, and the angle of the wrist joint.

[0022] The image display device 6 displays the display image required for remote operation generated by the image generation unit 38 of the remote operation control device 3. The image display device 6 is, for example, an HMD (head mounted display). The image display device 6 may also include a line of sight detection device that detects the line of sight of the operator.

[0023] (environmental sensor) The image capturing device 41 is, for example, an RGB (red, green, blue)-D image capturing device that can also acquire information on depth D. Note that the image capturing device 41 may be, for example, an RGB image capturing device and a distance sensor. The communication unit 42 outputs environmental data (image data, distance data) captured by the image capturing device 41 to the remote control device 3.

[0024] (robot) The moving mechanism 21 is, for example, an arm of the robot 2, with one end connected to the end effector 22 via a joint and the other end connected to the body or a base via a joint. The moving mechanism 21 may also include, for example, a cart. Each joint is equipped with a sensor 23 and an actuator.

[0025] The end effector 22 has, for example, two or more fingers. The end effector 22 may be, for example, a multi-fingered hand having three or more fingers, or may be a gripper or the like. The fingers have joints, and each joint has a sensor 23 and an actuator.

[0026] The sensor 23 is, for example, an encoder, a force sensor, a pressure sensor, a six-axis sensor, or the like, and is attached to each joint of the moving mechanism 21 and the end effector 22.

[0027] The actuator 24 is driven in response to the output of the drive unit 25 and is attached to each joint of the moving mechanism 21 and the end effector 22 .

[0028] The driving unit 25 drives the actuator 24 in response to a control instruction or an operation instruction from the remote control device 3, thereby causing the movement mechanism 21 and the end effector 22 to perform an operation. Note that the control instruction is an instruction based on an instruction from the operator. Also, the operation instruction is an instruction to automatically operate the movement mechanism 21 and the end effector 22 according to the trajectory of the movement mechanism 21 and the end effector 22 that are operated based on the operation instruction.

[0029] The communication unit 26 adds identification information for identifying the robot 2 to the detection value detected by the sensor 23 and outputs the result to the remote control device 3. The communication unit 26 acquires a control instruction or an operation instruction from the remote control device 3.

[0030] (Remote control device) The acquisition unit 31 acquires operation input information from the operation input unit 5. The acquisition unit 31 acquires environmental data from the environmental sensor 4. The acquisition unit 31 outputs the detection values ​​detected by each sensor 23 from each robot 2.

[0031] The intention estimation unit 39 estimates, for example, the object to be worked on and the intended work of the operator based on the operation input information and the environmental data. Note that the intention estimation is performed using, for example, a method described in Japanese Patent Laid-Open No. 2022-157101.

[0032] The action designation unit 32 determines the action starting point of the end effector 22 based on the operation input information, and designates the action from the action starting point of the end effector 22. Note that the action designation unit 32 may also use the result of estimation by the intention estimation unit 39 to determine the action starting point of the end effector 22 and designate the action from the action starting point of the end effector 22.

[0033] The trajectory generation unit 33 generates a constrained trajectory for the end effector 22 using the motion specified by the motion specification unit 32. The constraints are imposed, for example, by fixing the motion in at least one of the x, y, and z axis directions or at least one of the rotation axis directions (roll, pitch, and yaw). This allows the object to be grasped stably, and the work to be performed stably. The constraints on the moving mechanism 21 and the end effector 22 may be imposed based on the estimated work content and the object to be investigated, for example, using the method described in Japanese Patent Application No. 2023-219221.

[0034] The first control unit 34 determines the robot whose operation is to be controlled in accordance with the operation input information, for example, based on the operation input information and the estimated work content and intention. For example, when performing work using two sets of moving mechanisms 21 and end effectors 22, the first control unit 34 determines the robot whose operation is to be controlled in accordance with the operation input information, based on the distance between the object to be operated and the end effector 22, the positional relationship between the end effector 22 and its surroundings (walls and other objects), etc. The first control unit 34 generates a first operation instruction based on the operation input information and the estimated work content and intention, and outputs the generated first operation instruction to the determined robot 2 via the communication unit 37. Note that the first control unit 34 may control the robot 2 to support (assist) the operator's operation based on the estimated operation intention, for example, using a method described in Japanese Patent Application No. 2023-045616.

[0035] The second control unit 35 generates a second operation instruction to operate at least one end effector 22 other than the robot 2 determined by the first control unit 34 on the trajectory generated by the trajectory generation unit 33. The second control unit 35 generates the second operation instruction based on, for example, the task content and operation intention of the operator estimated by the intention estimation unit 39, and the operation target object.

[0036] The storage unit 36 ​​stores programs, mathematical expressions, threshold values, predetermined values, identification information of the robot 2, and the like used by each unit of the remote control device 3.

[0037] The communication unit 37 outputs the first operation instruction generated by the first control unit 34 to, for example, the first robot 2-1 and the second robot 2-2. The communication unit 37 outputs the second operation instruction generated by the second control unit 35 to, for example, the third robot 2-3.

[0038] The image generating unit 38 generates a display image required for remote control to be displayed on the image display device 6 using the environmental data.

[0039] [Processing Procedure] Next, an example of the processing procedure will be described with reference to Fig. 3, which is a flowchart of the processing of the remote control control device in this embodiment.

[0040] (Step S1) The acquisition unit 31 acquires operation input information from the operation input unit 5.

[0041] (Step S2) The acquisition unit 31 acquires environmental data from the environmental sensor 4.

[0042] (Step S3) The intention estimation unit 39 estimates, for example, the object to be worked on and the work content intended by the operator based on the operation input information and the environmental data.

[0043] (Step S4) The first control unit 34 determines the robot 2 to be controlled by the first operation instruction, for example, based on the operation input information and the result of estimation by the intention estimation unit 39. The second control unit 35 determines the robot 2 to be controlled by the second operation instruction from among the robots 2 other than the robot 2 determined by the first control unit 34, for example, based on the result of estimation by the intention estimation unit 39.

[0044] (Step S5) The action designation unit 32 obtains a gripping point of the end effector 22 relative to the object based on, for example, operation input information. The action designation unit 32 determines a starting point of action of the end effector 22 controlled by the second action instruction based on the obtained gripping point, and designates an action of the end effector 22 from the starting point of action.

[0045] (Step S6) The trajectory generating unit 33 uses the action designated by the action designating unit 32 to generate a constrained trajectory of the end effector 22 controlled by the second action instruction.

[0046] (Step S7) The first control unit 34 generates the determined first action instruction for the robot 2 based on the input operation information, the estimated operation intention, and the like.

[0047] (Step S8) The second control unit 35 generates a second operation instruction for at least one end effector 22 other than the one determined by the first control unit 34, based on the trajectory generated by the trajectory generation unit 33.

[0048] (Step S9) The first control unit 34 outputs a first operation instruction to the robot 2 that it controls via the communication unit 37. The second control unit 35 outputs a second operation instruction to the robot 2 that it controls via the communication unit 37.

[0049] The first control unit 34 and the second control unit 35 may switch between the robot 2 controlled by the first operation instruction and the robot controlled by the second operation instruction during work.

[0050] (First Example) The first embodiment will be described with reference to Fig. 4. Fig. 4 is a diagram for explaining the first embodiment. 4, reference numeral g11 denotes a first object, and reference numeral g12 denotes a second object. The second object is, for example, a string or the like that binds the first object g11. In this example, the second robot 2-2 and the third robot 2-3 are controlled by a first operation instruction in accordance with operation input information, and the operation of the first robot 2-1 is automatically controlled by a second operation instruction. That is, in the first embodiment, two robots are remotely controlled, and the remaining robot is automatically controlled.

[0051] In this example, the intention estimation unit 39 estimates that the task is to cut the object g11 with the tool g15. The gripping point may be input or selected by the operator.

[0052] The processing procedure of the work in Fig. 4 will be described with reference to Fig. 4 and Fig. 5. Fig. 5 is a flowchart of the processing in the first embodiment.

[0053] (Step S11) The operator operates the operation input units 5 of both hands to input an input to grasp the object g11. Next, based on the operation input information, the first control unit 34 determines the robots to be used for the task as the first robot 2-1 and the second robot 2-2, because the task is performed using two end effectors 22. Next, based on the operation input information, the first control unit 34 determines the grasping points (points g12 and g13 in FIG. 4) of the first object g11, and grasps the object g11. The intention estimation unit 39 estimates the operation intention and the object based on the operation input information and the acquired environmental information.

[0054] (Step S12) The first control unit 34 pulls the object g11 to both sides with an appropriate force at the gripped position using the support function, based on, for example, the operation input information, the estimated operation intention, and the first object.

[0055] (Step S13) The first control unit 34 generates a first operation instruction to move the first object g11 to a position where it is easy to cut the object g11 while pulling the object g11 to both sides, and controls the corresponding first robot 2-1 and second robot 2-2.

[0056] (Step S14) After the object g11 has moved to a position where it is easy to cut, the action specifying unit 32 determines the start point g12 (action starting point) and end point g13 (action end point) of the trajectory from the grip point. Note that the grip point, the start point, and the end point may be the same or different. In this way, when there are two grip points, the action specifying unit 32 may set one of the two grip points as the start point and the other of the two grip points as the end point.

[0057] (Step S15) The trajectory generating unit 33 generates a trajectory g14 from the start point g12 and end point g13 determined by the operation specifying unit 32. Subsequently, the second control unit 35 generates a second operation instruction based on the generated trajectory g14 and controls the corresponding operation of the robot 2. Specifically, the second control unit 35 controls the end effector 22-1 of the first robot 2-1 to grasp a tool (for example, a cutter or a knife) g15 and cut the object g11.

[0058] The information about the object to be operated may be stored in advance in the storage unit 36, or may be selected or input by operating the operation input unit 5 or the like. Alternatively, the first control unit 34 or the like may acquire the information about the object to be operated based on the detection value of the sensor 23 provided in the end effector 22.

[0059] (Second Example) A second embodiment will be described with reference to Fig. 6. Fig. 6 is a diagram for explaining the second embodiment. The example in Fig. 6 is an example of a task in which three robots 2 are used to unfold an object and, for example, place it over another object. In this case, the operator first operates the operation input units 5 of both hands to determine the grasping points of the two robots 2, and then operates the operation input units 5 of one hand to determine the grasping point of the remaining robot 2 before performing the operation.

[0060] The processing procedure of the work in Fig. 6 will be described with reference to Fig. 6 and Fig. 7. Fig. 7 is a flowchart of the processing in the second embodiment.

[0061] (Step S21) The operator operates the operation input units 5 of both hands to input an instruction to grasp the object g21 using, for example, markers g22 and g23 as landmarks. The operator operates the operation input unit 5 of one hand to input an instruction to grasp the object g21 using, for example, the markers as landmarks. Based on this, for example, the first control unit 34 controls the first robot 2-1 and the second robot 2-2 based on the operation input information to grasp both sides of the object g21. Furthermore, the second control unit 35 controls the third robot 2-3 based on the operation input information to grasp the object g21. Note that the first control unit 34 and the second control unit 35 may automatically grasp three points on the object g21 using the markers as landmarks. The intention estimation unit 39 estimates the operation intention and the object based on the operation input information and the acquired environmental information.

[0062] (Step S22) After the processing of step S21, the first control unit 34 controls, for example, the first robot 2-1 and the second robot 2-2 based on the operation input information of the operator so that they move along a trajectory restricted to the direction in which the object g22 is pulled. In other words, the two robots 2 are controlled based on remote operation.

[0063] (Step S23) Simultaneously with the processing of step S22, the second control unit 35 controls, for example, the third robot 2-3 to move in a direction in which the object g21 is pulled and spread.

[0064] (Step S24) The operator operates the operation input units 5 with both hands, for example, to place the object g22 over another object. In response to this, the first control unit 34 controls the first robot 2-1 and the second robot 2-2 to place them over the other object. In addition, the second control unit 35 controls the third robot 2-3 to operate on a trajectory that will place it over the other object.

[0065] The work examples and work procedures described with reference to FIGS. 4 to 6 are merely examples, and the present invention is not limited to these. For example, if there are four or more robots 2, the first control unit 34 and the second control unit 35 may select the robot 2 to be used for each task, and perform the task using, for example, three of the four robots 2.

[0066] As described above, in this embodiment, for example, two sets of moving mechanisms 21 and end effectors are remotely controlled in accordance with instructions from an operator, and the remaining set of moving mechanisms 21 and end effectors are remotely controlled, so that they operate according to the trajectory of movement.

[0067] As a result, according to this embodiment, when three or more arms each having an end effector are remotely controlled, workability can be improved compared to conventional methods. Furthermore, according to this embodiment, tasks that are difficult to automatically generate trajectories for, such as manipulating flexible objects, are performed by humans through remote control, and the reproducibility of operations can be improved by automatically generating trajectories for constrained workpieces.

[0068] A program for implementing some or all of the functions of the remote control device 3 of the present invention may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be loaded into a computer system and executed to perform all or part of the processing performed by the remote control device 3. Note that the term "computer system" as used herein includes hardware such as an OS and peripheral devices. The term "computer system" also includes a WWW system equipped with a homepage provision environment (or display environment). The term "computer-readable recording medium" refers to portable media such as flexible disks, optical magnetic disks, ROMs, and CD-ROMs, as well as storage devices such as hard disks built into computer systems. The term "computer-readable recording medium" also includes devices that retain a program for a certain period of time, such as volatile memory (RAM) within a computer system that acts as a server or client when the program is transmitted via a network such as the Internet or a communication line such as a telephone line. Alternatively, some or all of these components may be realized by LSI (Large Scale Integration) hardware (including circuitry) such as an ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), GPU (Graphics Processing Unit), or SOC (System On Chip), or may be realized by a combination of software and hardware.

[0069] The program may also be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by transmission waves in the transmission medium. Here, the "transmission medium" that transmits the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may also be a program that realizes part of the above-mentioned functions. Furthermore, the program may be a so-called differential file (differential program) that can realize the above-mentioned functions in combination with a program already recorded in the computer system.

[0070] The above describes the form for carrying out the present invention using an embodiment, but the present invention is not limited to such an embodiment, and various modifications and substitutions can be made within the scope that does not deviate from the gist of the present invention. [Explanation of symbols]

[0071] 1...robot remote operation control system, 2-1...first robot, 2-2...second robot, 22-3...third robot, 2 (2-1, 2-2, 2-3,...)...robot, 3...remote operation control device, 4...environment sensor, 5...operation input unit, 6...image display device, NW...network, 21, 21-1,..., 21-n...movement mechanism, 22, 22-1,..., 22-n...end effector, 23, 2 3-1,...,23-n...sensors, 24,24-1,...,24-n...actuators, 25,25-1,...,25-n...drive units, 26,26-1,...,26-n...communication units, 31...acquisition unit, 32...action designation unit, 33...trajectory generation unit, 34...first control unit, 35...second control unit, 36...storage unit, 37...communication unit, 38...image generation unit, 39...intention estimation unit, 41...imaging device, 42...communication unit

Claims

1. A robot remote operation control system that recognizes an operator's movement and transmits the operator's movement to a robot to operate the robot, The robot has three or more end effectors; a movement mechanism for moving each of the end effectors; a first control unit that controls an operation of at least one of the end effectors in response to a movement of the operator; an action designation unit that determines a starting point of action of the end effector based on a movement of the operator and designates an action of the end effector from the starting point of action; a trajectory generation unit that generates a constrained trajectory of the end effector from the motion designated by the motion designation unit; a second control unit that operates at least one of the end effectors that is not controlled by the first control unit on the trajectory; A robot remote operation control system comprising:

2. when there are two gripping points at which the operation target object is gripped based on the movement of the operator, the action designation unit sets one of the two gripping points as the action start point and the other of the two gripping points as the action end point; the trajectory generation unit sets a constrained trajectory of the end effector between the action starting point and the action ending point; The robot remote operation control system according to claim 1 .

3. the trajectory generation unit generates a constrained trajectory by preventing movement in at least one of a plurality of axial directions based on a target object and a task content estimated based on the movement of the operator.

3. The robot remote operation control system according to claim 1 or 2.

4. an intention estimation unit that estimates a target object and a task content based on the movement of the operator; the first control unit and the second control unit control the end effectors to be controlled, respectively, using the estimation results obtained by the intention estimation unit.

3. The robot remote operation control system according to claim 1 or 2.

5. the first control unit controls an operation of the end effector to support an operation by the operator, based on the operation intention estimated by the intention estimation unit.

5. The robot remote operation control system according to claim 4.

6. A remote operation control device that remotely controls a robot having three or more end effectors and movement mechanisms that move each of the end effectors, a first control unit that controls the operation of at least one of the end effectors in response to a movement of an operator; an action designation unit that determines a starting point of action of the end effector based on a movement of the operator and designates an action of the end effector from the starting point of action; a trajectory generation unit that generates a constrained trajectory of the end effector from the motion designated by the motion designation unit; a second control unit that operates at least one of the end effectors that is not controlled by the first control unit on the trajectory; A robot remote control device comprising:

7. A control method for a remote operation control device that remotely controls a robot having three or more end effectors and movement mechanisms that move each of the end effectors, comprising: a first control unit that controls the operation of at least one of the end effectors in response to a movement of an operator; an action designation unit that determines a starting point of action of the end effector based on the movement of the operator and designates an action of the end effector from the starting point of action; a trajectory generation unit generates a constrained trajectory of the end effector from the motion designated by the motion designation unit; a second control unit that causes at least one of the end effectors that is not controlled by the first control unit to move along the trajectory; A robot remote control control method comprising:

8. A computer of a remote operation control device that remotely controls a robot having three or more end effectors and movement mechanisms that move each of the end effectors, Controlling the movement of at least one of the end effectors in response to a movement of an operator; determining a starting point of action of the end effector based on the movement of the operator, and specifying an operation of the end effector from the starting point of action; generating a constrained trajectory for the end effector from the specified motion; moving at least one of the end effectors other than the end effector controlled in response to the movement of an operator on the trajectory; program.

Images