Remote control system, remote control method, and program

Abstract

The present invention provides a remote control system, a remote control method, and a program that enable an operator to precisely control an object remotely. [Solution] The remote control system comprises: an input unit into which an operator's operation command for an object to be remotely controlled is input; an acquisition unit that acquires environmental data indicating the environment in which the object exists and / or state data indicating the state of the object; a determination unit that determines the target position of the object based on the environmental data and / or state data and the operation command; and a reaction force application unit that applies a reaction force to the operator based on a position error which is the difference between the target position and the position requested by the operator as the operation command.

Description

[Technical Field]

[0001] This invention relates to a remote control system, a remote control method, and a program. [Background technology]

[0002] Technology for remotely controlling robots using controllers is under development (see, for example, Patent Documents 1-3). [Prior art documents] [Patent Documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-164028 [Patent Document 2] Japanese Patent Publication No. 2003-25259 [Patent Document 3] Japanese Patent Publication No. 2023-157679 [Overview of the project] [Problems that the invention aims to solve]

[0004] However, with conventional technology, there were cases where the operator could not control the remotely controlled object (for example, a robot) as intended.

[0005] This invention has been made in consideration of these circumstances, and one of its objectives is to provide a remote control system, a remote control method, and a program that enable an operator to accurately control an object to be remotely controlled. [Means for solving the problem]

[0006] The remote control system, remote control method, and program according to the present invention employ the following configuration. (1) A first aspect of the present invention is a remote control system comprising: an input unit into which an operator's operation command for a remotely controlled object is input; an acquisition unit that acquires environmental data indicating the environment in which the object exists and / or state data indicating the state of the object; a determination unit that determines the target position of the object based on the environmental data and / or state data and the operation command; and a reaction force application unit that applies a reaction force to the operator based on a position error which is the difference between the target position and the position requested by the operator as the operation command.

[0007] (2) A second aspect of the present invention is that, in the first aspect, the reaction force applying unit applies a force in the opposite direction to the force applied to the input unit by the operator as a reaction force, via the input unit to the operator.

[0008] (3) A third aspect of the present invention further comprises a calculation unit for calculating the reaction force in the first or second aspect, wherein the reaction force application unit applies the calculated reaction force to the operator.

[0009] (4) A fourth aspect of the present invention is the third aspect, wherein the calculation unit calculates the reaction force using at least impedance control.

[0010] (5) A fifth aspect of the present invention is, in the fourth aspect, the determination unit further determines the target velocity of the object, and the calculation unit calculates the reaction force as impedance control based on the velocity error, which is the difference between the target velocity and the velocity requested by the operator as the operation command, and the position error.

[0011] (6) A sixth aspect of the present invention is the fourth aspect, wherein the calculation unit further calculates the reaction force using admittance control.

[0012] (7) A seventh aspect of the present invention, in the sixth aspect, further determines the target velocity of the object and the target value of the force to be applied to the input unit, and the calculation unit calculates the reaction force as admittance control based on the velocity error, which is the difference between the target velocity and the velocity requested by the operator as the operation command, the force error, which is the difference between the target value of the force and the force applied to the input unit by the operator, and the position error.

[0013] (8) An eighth aspect of the present invention is the seventh aspect, in which the determination unit updates the target position based on the force error.

[0014] (9) A ninth aspect of the present invention is a fourth aspect, in which the determination unit determines a target trajectory which is a trajectory in which a plurality of target positions are arranged in a time series, and the calculation unit calculates the first reaction force which is the reaction force in the normal direction of the target trajectory and the second reaction force which is the reaction force in the tangential direction of the target trajectory using position control and calculates the second reaction force using impedance control.

[0015] (10) A tenth aspect of the present invention is the ninth aspect, in which the calculation unit calculates the first reaction force based on the position error as the position control.

[0016] (11) An eleventh aspect of the present invention is a remote control method using a computer, comprising: acquiring environmental data indicating the environment in which an object to be remotely controlled exists and / or state data indicating the state of the object; determining a target position of the object based on the environmental data and / or state data and an operation command from an operator to the object input to an input unit; and applying a reaction force to the operator based on a position error which is the difference between the target position and the position requested by the operator as the operation command.

[0017] (12) A twelfth aspect of the present invention is a program for a computer to be executed, which includes acquiring environmental data indicating the environment in which an object to be remotely controlled exists and / or state data indicating the state of the object; determining a target position of the object based on the environmental data and / or the state data and an operator's operation command for the object input to an input unit; and applying a reaction force to the operator based on a position error which is the difference between the target position and the position requested by the operator as the operation command. [Effects of the Invention]

[0018] According to the above embodiment, the operator can operate the remotely controlled object with high precision. [Brief explanation of the drawing]

[0019] [Figure 1] This is a diagram illustrating the outline of the remote control system 1 in the embodiment. [Figure 2] This diagram illustrates the reaction force output from the remote control interface 20. [Figure 3] This diagram illustrates the reaction force output from the remote control interface 20. [Figure 4] This diagram illustrates the reaction force that operator OP inputs to the remote control interface 20. [Figure 5] This diagram illustrates the reaction force that operator OP inputs to the remote control interface 20. [Figure 6] This figure shows an example of the configuration of the remote control system 1 in the embodiment. [Figure 7] This flowchart shows the sequence of processes of the remote control system 1 in the embodiment. [Figure 8] This diagram illustrates an example of position control. [Figure 9] This diagram illustrates an example of impedance control. [Figure 10]This diagram illustrates an example of control when admittance control and impedance control are connected in series. [Modes for carrying out the invention]

[0020] Hereinafter, embodiments of the remote control system, remote control method, and program of the present invention will be described with reference to the drawings.

[0021] [Overview of the remote control system] Figure 1 is a diagram illustrating the outline of the remote control system 1 in an embodiment. As shown in Figure 1, the remote control system 1 includes, for example, an avatar robot 10 (hereinafter simply referred to as robot 10) remotely controlled by an operator OP, a teleoperation interface 20 operated by the operator OP when remotely controlling robot 10, and a remote control device 100 for controlling robot 10 and the teleoperation interface 20. These are connected via a network NW. The network NW includes LAN (Local Area Network) and WAN (Wide Area Network), etc.

[0022] Robot 10 includes, for example, an end effector 11 (also called a robot hand or manipulator). Robot 10 is a humanoid robot that can grasp or manipulate a target TR using the end effector 11. The target TR is the object on which the desired task is performed. Robot 10 is an example of a "remotely controlled object".

[0023] Tasks include, for example, grabbing a target TR with one end effector 11, transferring the target TR to the other end effector 11, or moving the target TR. However, tasks are not limited to these; any task can be set.

[0024] Robot 10 is not limited to humanoid robots; it can be any type of robot as long as it can perform the intended task. For example, robot 10 may be a quadrupedal animal-type robot, an industrial robot, a military robot, or any other type of robot.

[0025] The end effector 11 may be provided with several fingers F (thumb, index finger, middle finger, ring finger, etc.) as grippers.

[0026] The remote control interface 20 is a human-machine interface to which operator OP inputs operation commands for the robot 10. The remote control interface 20 includes, for example, a handle 21 (which may be read as a grip or lever, etc.). By grasping the handle 21 and applying force, operator OP can move the remote control interface 20 vertically (Y direction in the figure), horizontally (X direction in the figure), or horizontally (Z direction in the figure). The movement of the remote control interface 20 is linked to the movement of the robot 10, and the robot 10 operates in conjunction with the operator OP's operation of the remote control interface 20. The remote control interface 20 detects the force applied to the handle 21 by operator OP, as well as the position, velocity, and acceleration of the handle 21, and transmits data indicating these detection results to the remote control control device 100 as operation commands for operator OP. The remote control interface 20 is an example of an "input unit".

[0027] Furthermore, the remote control interface 20 applies a reaction force to the operator OP while it is being operated by the operator OP (generates a reaction force on the operator OP). The reaction force is a force in the opposite direction to the force applied by the operator OP to the handle 21. The reaction force may also be interpreted as a resistance force or an assist force. Details of the application of the reaction force will be described later.

[0028] The remote control device 100 may be installed at a location far away from both the robot 10 and the remote control interface 20, and may remotely control the robot 10 and the remote control interface 20 via a network NW. Alternatively, the remote control device 100 may be mounted on the robot 10 and directly control the robot 10 based on operation commands received from the remote control interface 20, or remotely control the remote control interface 20. Conversely, the remote control device 100 may be mounted on the remote control interface 20 and remotely control the robot 10 based on operation commands input to the remote control interface 20, or directly control the remote control interface 20.

[0029] [Overview of reaction forces] Figures 2 and 3 illustrate the reaction forces output from the remote control interface 20. Figures 4 and 5 illustrate the reaction forces input by the operator OP to the remote control interface 20. All figures show the vector decomposition of the reaction forces as viewed from above the remote control interface 20, that is, the vector decomposition of the reaction forces on the horizontal plane (XZ plane).

[0030] P represents the target trajectory of the handle 21. The target trajectory P is a time-series arrangement of multiple target positions that the handle 21 should pass through at each time point, and its details will be described later. E represents the threshold (allowable distance) that determines how far the handle 21 can deviate from the target trajectory P.

[0031] F out This represents the reaction force output from the remote control interface 20. Reaction force F out It is decomposed into the nominal direction and the tangential direction of the target trajectory P. F n This represents the reaction force in the direction normal to the target trajectory P, and F t This represents the reaction force tangential to the target trajectory P. The normal reaction force F. nis an example of the "first reaction force", the tangential reaction force F t is an example of the "second reaction force."

[0032] x0 represents the target position at a certain time t0 among the plurality of target positions included in the target trajectory P. x0 (·) represents the first derivative of the target position x0, that is, the target velocity. For convenience, the symbol (·) represents the first derivative, and the symbol (··) represents the second derivative. x represents the current position of the handle 21. x (·) represents the first derivative of the current position x, that is, the current velocity.

[0033] e represents the error (that is, the distance between two points) between the current position x of the handle 21 and the target position x0. For example, as shown in FIGS. 4 and 5, when the operator OP operates the handle 21, a force F ext is applied in a direction away from the target trajectory P. In this case, the teleoperation interface 20 has a force F ext to move the handle 21 away from the target trajectory P. In the normal direction of the target trajectory P (the opposite direction of the force F ext ), a reaction force F n is generated, and a reaction force F t is generated in the tangential direction (traveling direction) of the target trajectory P. As a result, for example, a tube (or tunnel) having a radius of the threshold value E centered on the target trajectory P and generating a stronger reaction force F out as it approaches the outside of the diameter is virtually formed. Since the reaction force F out is generated so that the handle 21 does not deviate from this tube, the operation of the operator OP can be induced to move the handle 21 along the target trajectory P.

[0034] [Configuration of the Teleoperation System] FIG. 6 is a diagram showing an example of the configuration of the teleoperation system 1 in the embodiment. In the embodiment, in addition to the end effector 11 described above, the robot 10 further includes an environment sensor 12, a first state sensor 13, a first actuator 14, and a first processing unit 15.

[0035] The environmental sensor 12 senses the environment in which the robot 10 is placed. For example, the environmental sensor 12 may include a visual sensor, a tactile sensor, a force sensor, and so on.

[0036] The vision sensor is installed on a part of the robot 10's body (typically the head). The vision sensor may be, for example, a depth camera (3D camera). For example, the vision sensor captures a scene in which the target TR is grasped or manipulated by the end effector 11 and generates image data of that scene. The image data is transmitted to the remote control device 100. The vision sensor is not limited to a depth camera; for example, it may be a sensor that images the environment in which the robot 10 is placed by irradiating it with electromagnetic waves, such as radar or lidar.

[0037] The tactile sensors are arranged, for example, on the palm of the end effector 11. The tactile sensors detect the force applied to the end effector 11.

[0038] The force sensors are placed, for example, at the fingertips of the end effector 11 and detect the force (load) applied to each fingertip in three axes (X, Y, Z) and the moment (torque) around each axis.

[0039] The first state sensor 13 is a sensor that detects the state of the robot 10 (for example, joint angles, angular velocity, torque, etc.). The first state sensor 13 includes, for example, a rotary encoder that detects the degree of rotation of the joints of the robot 10 or the end effector 11, a tension sensor that detects the tension of the wires that rotate the joints, a torque sensor that detects the torque applied to the joint axis, and an acceleration sensor or gyro sensor for detecting the posture of the robot 10.

[0040] The first actuator 14 drives various parts of the robot 10 (such as each finger F of the end effector 11, or the arm connecting the torso of the robot 10 to the end effector 11) under the control of the first processing unit 15. The first actuator 14 includes, for example, an electromagnetic motor, gears, or artificial muscle.

[0041] The first processing unit 15 transmits data indicating the environment detected by the environment sensor 12 (hereinafter referred to as environment data) and data indicating the state of the robot 10 detected by the first state sensor 13 (hereinafter referred to as first state data) to the remote control device 100. The first processing unit 15 also controls the first actuator 14 based on the information output from the remote control device 100. The first processing unit 15 may be implemented by a CPU (Central Processing Unit) or a GPU (Graphics Processing Unit), etc. Furthermore, the first processing unit 15 may be implemented by hardware such as an LSI (Large Scale Integration), ASIC (Application Specific Integrated Circuit), FPGA (Field-Programmable Gate Array), or SOC (System On Chip), or it may be implemented by the cooperation of software and hardware.

[0042] In this embodiment, the remote control interface 20 further includes a second state sensor 22, a second actuator 23, and a second processing unit 24, in addition to the handle 21 described above.

[0043] The second state sensor 22 is a sensor that detects the state of the remote control interface 20 (e.g., force, position, velocity, acceleration, etc.). The second state sensor 22 detects, for example, the force F applied to the handle 21 by the operator OP. ext A force sensor to detect the current position x of the handle 21, a position sensor to detect the current speed x of the handle 21 (·) Speed ​​sensor to detect the current acceleration x of handle 21 (··) This includes acceleration sensors and other devices that detect [something].

[0044] The second actuator 23 drives various parts of the remote control interface 20 under the control of the second processing unit 24. The second actuator 23 includes, for example, an electromagnetic motor or a gear.

[0045] The second processing unit 24 transmits data indicating the state of the remote control interface 20 detected by the second state sensor 22 (hereinafter referred to as "second state data") to the remote control control device 100 as an operation command for operator OP. The second processing unit 24 also controls the second actuator 23 based on the information output from the remote control control device 100. The second processing unit 24 may be implemented by a CPU or GPU. Furthermore, the second processing unit 24 may be implemented by hardware such as an LSI, ASIC, FPGA, or SOC, or by the cooperation of software and hardware.

[0046] The remote control device 100 in this embodiment includes, for example, a communication interface 110, a third processing unit 120, and a storage unit 130.

[0047] The communication interface 110 communicates with an external device, for example, via a network NW. The external device may be, for example, a robot 10 or a remote control interface 20. The communication interface 110 may include, for example, a wireless communication module including a receiver and a transmitter, or a NIC (Network Interface Card). If the remote control control device 100 is mounted on the robot 10, the communication interface 110 may communicate with the robot 10 via a communication line such as a bus. Similarly, if the remote control control device 100 is mounted on the remote control interface 20, the communication interface 110 may communicate with the remote control interface 20 via a communication line such as a bus.

[0048] The third processing unit 120 includes, for example, an acquisition unit 121, an estimation unit 122, a determination unit 123, a calculation unit 124, and a communication control unit 125.

[0049] The above-mentioned components of the third processing unit 120 are realized, for example, by a CPU or GPU executing a program or instructions stored in the memory unit 130. Some or all of these components may be realized by hardware such as an LSI, ASIC, or FPGA, or by the cooperation of software and hardware.

[0050] The storage unit 130 is implemented using, for example, an HDD (Hard Disk Drive), flash memory, EEPROM (Electrically Erasable Programmable Read Only Memory), ROM (Read Only Memory), or RAM (Random Access Memory). The storage unit 130 stores firmware, application programs, and calculation results from the third processing unit 120.

[0051] The acquisition unit 121 acquires environmental data and first state data from the robot 10 via the communication interface 110, and second state data (operation commands) from the remote control interface 20.

[0052] The estimation unit 122 estimates the operator OP's operational intent based on at least one or both of the environmental data and the first state data, and the second state data (operation command). Operational intent is the operator OP's intention, representing what kind of operation the operator OP intends to perform on the robot 10 via the remote control interface 20.

[0053] For example, the estimation unit 122 may estimate the operator OP's intention to operate based on the size, shape, and type of the target TR detected as part of the environment by the environmental sensor 12. Specifically, if the target TR is a plastic bottle, the estimation unit 122 may estimate that the operator OP intends to perform operations such as grasping the plastic bottle or opening the bottle cap, which are common tasks for plastic bottles.

[0054] Furthermore, if the environmental sensor 12 detects that there is another object (hereinafter referred to as an obstacle OB) different from the target TR in front of the target TR as seen from the robot 10, the estimation unit 122 may estimate that the operator OP intends to avoid the obstacle OB before grasping or touching the target TR.

[0055] Furthermore, the estimation unit 122 may estimate the operator OP's intention to operate based on the operation command (i.e., second state data) input by the operator OP to the remote control interface 20. For example, the estimation unit 122 may estimate the force F indicated by the second state data. ext or, current position x, current velocity x (·) , current acceleration x (··) Therefore, it is reasonable to infer the operator's intention regarding what operation the operator OP is currently attempting to perform on the remote control interface 20.

[0056] The determination unit 123 determines the target trajectory P of the handle 21 based on the operator OP's operational intent. The target trajectory P includes at least a plurality of target positions arranged in chronological order, as described above. For example, the determination unit 123 may connect the position of the end effector 11 to the position of the target TR using a spline curve or the like, and determine the trajectory from the end effector 11 to the target TR as the target trajectory P of the handle 21.

[0057] In this process, the determination unit 123 determines the target position x0 of the handle 21 at each time point, and arranges these multiple target positions x0 in chronological order to form the target trajectory P. In addition to the target position x0, the determination unit 123 further determines the target velocity x0 of the handle 21 at each time point. (·) or target acceleration x0 (··) Determine the target position x0 and target velocity x0. (·) , target acceleration x0 (··) The sequence of events arranged in chronological order may be used as the target trajectory P.

[0058] The calculation unit 124 calculates the reaction force F applied to the operator OP via the remote control interface 20.out The calculation unit 124 calculates the error e (hereinafter referred to as the position error) between the target position x0 and the current position x of the handle 21. Then, based on the position error e, the calculation unit 124 calculates the reaction force F in the direction normal to the target trajectory P. n And the reaction force F in the tangential direction of the target trajectory P. t You may calculate each of these separately.

[0059] Furthermore, the calculation unit 124 calculates the target speed x0 of the handle 21 in addition to the position error e. (·) and current speed x (·) Error e (·) The calculation unit 124 may calculate the position error e and the speed error e. (·) Based on this, reaction force F n and reaction force F t You may calculate this.

[0060] The communication control unit 125 transmits the operator OP's operation command to the robot 10 via the communication interface 110. When the robot 10's first processing unit 15 receives the operation command from the remote control device 100, it controls the first actuator 14 based on that command. As a result, the robot 10 operates according to the operation command input to the remote control interface 20.

[0061] Furthermore, the communication control unit 125 receives the calculated reaction force F. out (Reaction force F) n F t The remote control interface 20 transmits an operation command (hereinafter referred to as the second operation command) corresponding to the operation command from the remote control device 100 via the communication interface 110. When the second processing unit 24 of the remote control interface 20 receives the second operation command from the remote control device 100, it controls the second actuator 23 based on the second operation command. This transmits a reaction force F to the operator OP via the remote control interface 20. out A force is applied. The communication control unit 125 and the remote control interface 20 described above are an example of a "reaction force application unit".

[0062] [Processing flow of the remote control system] The following describes the sequence of processes of the remote control system 1 in the embodiment using a flowchart. Figure 7 is a flowchart of the sequence of processes of the remote control system 1 in the embodiment.

[0063] First, the operator OP operates the robot 10 via the remote control interface 20 (step S100).

[0064] In response to the operator OP operating the handle 21 of the remote control interface 20, the second state sensor 22 of the remote control interface 20 determines the state of the remote control interface 20 (e.g., force F). ext , current position x, current speed x (·) , current acceleration x (··) Detects.

[0065] Next, the determination unit 123 determines the target trajectory P of the handle 21 (step S102). For example, the estimation unit 122 may estimate the operator OP's intention to operate, and the determination unit 123 may determine the target trajectory P based on this intention.

[0066] Next, the remote control interface 20 provides the operator OP with a reaction force F out Grant (step S104).

[0067] As described above, the calculation unit 124 calculates the reaction force F to the operator OP. out To apply this, first calculate the position error e between the target position x0 of handle 21 and its current position x, and then, based on this position error e, calculate the reaction force F in the normal direction of the target trajectory P. n And the reaction force F in the tangential direction of the target trajectory P. t Each of these is calculated. Specifically, the calculation unit 124 uses a control method called position control to calculate the reaction force F n F t You may calculate this.

[0068] [Calculation of reaction force: position control] Figure 8 shows an example of position control. In this embodiment, position control can be represented by a spring system that returns a constant reaction force to the position error e, and this spring system can be represented, for example, by equations (1) and (2).

[0069]

number

[0070]

number

[0071] K is an arbitrary weighting coefficient (constant), n (→) is a unit vector in the normal direction, and t (→) is a unit vector in the tangential direction.

[0072] As shown in equation (1), the larger the position error e, the greater the reaction force F in the normal direction. n It becomes larger. Also, as shown in equation (2), the tangential reaction force F t The force in the direction of propulsion decreases as the position error e increases, and increases as the position error e decreases. When the position error e exceeds the threshold E, a reaction force F in the tangential direction is applied. t It becomes zero.

[0073] Furthermore, the calculation unit 124 calculates not only the position error e, but also the velocity error e. (·) Calculate the position error e and the velocity error e (·) Based on this, the reaction force F in the direction normal to the target trajectory P. n And the reaction force F in the tangential direction of the target trajectory P. t The position error e and velocity error e may be calculated separately. In this case, the calculation unit 124 uses impedance control to calculate the position error e and velocity error e. (·) Therefore, reaction force F n F t Calculate.

[0074] [Calculation of reaction force: Impedance control] Figure 9 shows an example of impedance control. In this embodiment, impedance control is performed on position error e and velocity error e. (·) It can be represented by a spring-mass-damper system that returns a reaction force to the given forces, and this spring-mass-damper system can be represented, for example, by equations (3) and (4).

[0075]

number

[0076]

number

[0077] m represents the expected force of the handle 21, including the operator OP's fingertips. d,i This represents a hypothetical mass. d,i This represents the damping constant. K d,i This represents the spring constant. These represent mass m and virtual mass M. d,i Damping constant D d,i , spring constant K d,i This is determined in advance by the designer, etc.

[0078] As shown in equation (3), the reaction force F in the normal direction n This includes position error e and velocity error e (·) The larger the value, the larger the reaction force F in the normal direction. In other words, the further the current position x of handle 21 deviates from the target position x0, the greater the reaction force F in the normal direction. n As it increases, the current speed of handle 21 x (·) Target speed x0 (·) The reaction force F in the normal direction increases as the deviation from the normal direction increases. n It gets bigger.

[0079] Furthermore, as shown in equation (4), the tangential reaction force F t The force in the direction of propulsion decreases as the position error e increases, and increases as the position error e decreases. Furthermore, there is a reaction force F in the tangential direction. t (Force in the direction of propulsion) is the velocity error e(·) The larger the value, the larger the speed error e (·) The smaller the value, the smaller it becomes.

[0080] Furthermore, in addition to impedance control, the calculation unit 124 also uses admittance control to calculate the position error e and the velocity error e. (·) Therefore, reaction force F n F t You may calculate this.

[0081] [Calculation of reaction force: Admittance control] Figure 10 shows an example of control when admittance control and impedance control are connected in series. In this embodiment, admittance control controls position error e and velocity error e. (·) , and force error (F ref -F ext The system returns a reaction force while updating the target position in response to the force. Equation (5) represents admittance control, and equations (6) and (7) represent impedance control when admittance control is considered. Note that in the example in Figure 10, the velocity term is omitted for simplification.

[0082]

number

[0083]

number

[0084]

number

[0085] M d,a This represents a hypothetical mass. d,a This represents the damping constant. K d,a This represents the spring constant. F ref This represents the target force that should be applied to the handle 21. These are the mass m and virtual mass M. d,a Damping constant Dd,a 、Spring constant K d,a 、Target value of force F ref is determined in advance by designers or the like.

[0086] x d represents the new target position calculated from the force F applied to the handle 21 by the operator OP. ext x d (·) represents the new target speed calculated from the force F applied to the handle 21 by the operator OP. ext

[0087] e d represents the position error calculated from the new target position x d Specifically, the new position error e d is calculated as the error between the sum (x0 + x d ) of the old target position x0 and the new target position x d and the current position x.

[0088] e d (·) represents the speed error calculated from the new target speed x d (·) Specifically, the new speed error e d (·) is calculated as the error between the sum (x0 (·) + x d (·) ) of the old target speed x0 (·) and the new target speed x d (·) ) and the current speed x (·) .

[0089] As shown in the mathematical formula (5), the calculation unit 124 calculates the new target position x ext and the new target speed x ref from the difference between the force F d applied to the handle 21 by the operator OP under admittance control and the predetermined target value of force F d (·) . In response to this, the determination unit 123 determines the new target position x dand new target speed x d (·) The target trajectory P is updated based on this.

[0090] Furthermore, the calculation unit 124, as shown in equations (6) and (7), calculates the new target position x, similar to impedance control. d Reaction force F in the direction normal to the vector n In addition to calculating the new target speed x d (·) Tangential reaction force F t Calculate the mx0 of formulas (6) and (7). (··) This is the target force value F ref It is equivalent to this.

[0091] The communication control unit 125 calculates the reaction force F using the calculation unit 124. n F t Once these are calculated, these reaction forces F n F t A second operation command corresponding to this is transmitted to the remote control interface 20 via the communication interface 110. When the second processing unit 24 of the remote control interface 20 receives the second operation command from the remote control control device 100, it controls the second actuator 23 based on the second operation command. As a result, a reaction force F is transmitted to the operator OP via the remote control interface 20. out It will be granted.

[0092] Returning to the flowchart explanation, the estimation unit 122 then determines whether the task has been completed (step S106), and if the task has been completed, it terminates the process shown in this flowchart.

[0093] On the other hand, if the task has not been accomplished, the estimation unit 122 further determines that the operator OP is reacting to the reaction force F n F t Determine whether or not you feel uncomfortable with the situation (Step S108).

[0094] For example, the estimation unit 122 determines the target force value F ref Compared to the force F applied by operator OP to handle 21 ext If the reaction force F is large, operator OP will react.n F t It was determined that there was something wrong with that, and conversely, the target force value F ref Compared to the force F applied by operator OP to handle 21 ext If it is small, operator OP is the reaction force F n F t It can be concluded that they do not feel any discomfort regarding this.

[0095] Furthermore, the estimation unit 122 detects when operator OP's eyes are darting around or when a predetermined button or switch is pressed when the operator OP feels discomfort, and the operator OP generates a reaction force F n F t It would be fair to conclude that they feel uncomfortable with this.

[0096] The estimation unit 122 determines that the operator OP is the reaction force F n F t If no discomfort is felt, the process returns to S104. This allows the operator OP to receive the reaction force F via the remote control interface 20. out The granting of this status will continue.

[0097] On the other hand, the estimation unit 122 determines that the operator OP is the reaction force F n F t If the operator OP feels something is wrong with the current direction, they estimate the direction in which they want to move the remote control interface 20 (hereinafter referred to as the desired direction) (step S110). The estimation unit 122 then returns the process to S102. In response, the determination unit 123 newly determines the target trajectory P based on the desired direction.

[0098] According to the embodiments described above, the remote control device 100 acquires environmental data indicating the environment in which the robot 10 (an example of the "object to be remotely controlled") exists and / or first state data indicating the state of the robot 10. The remote control device 100 further acquires second state data indicating the state of the remote control interface 20 as an operation command for the operator OP.

[0099] The remote control device 100 determines a target trajectory P that includes multiple target positions x0 based on environmental data and / or first state data and second state data (operation command). Based on the position error e between the target positions x0 included in the target trajectory P and the current position x included in the second state data (operation command), the remote control device 100 applies a reaction force F to the operator OP via the remote control interface 20. out (=F n F t The remote control device 100 calculates the reaction force F. out (=F n F t A second operation command corresponding to ) is transmitted to the remote control interface 20. This transmits the reaction force F to the operator OP via the remote control interface 20. out It will be granted.

[0100] This configuration allows the operator OP to precisely control remotely operated objects such as robot 10.

[0101] In particular, regarding the operator OP, if the trajectory deviates from the target trajectory P, a reaction force F is generated from the remote control interface 20. out To receive this, the end effector 11 can be moved toward the target TR while avoiding obstacles OB, taking into account the state of the robot 10. Also, under admittance control, the force F applied by the operator OP to the handle 21 ext As the target trajectory P is updated accordingly, the excessive reaction force F out This suppresses interference and enables support for the operation of the remote control interface 20 with minimal discomfort.

[0102] With respect to the robot 10, even when an obstacle OB is present, it can avoid the obstacle OB or stop before reaching the obstacle OB when bringing the end effector 11 closer to the target TR, taking into account the operation commands of the operator OP.

[0103] <Other Embodiments> Other embodiments will be described below. In the embodiments described above, operator OP is reaction force F n F t If there is any discomfort with the above, the calculation unit 124 further calculates the parameters used in admittance control (virtual mass M) d,a , spring constant K d,a Damping constant D d,a (etc.) may be updated dynamically.

[0104] For example, the calculation unit 124 calculates that the reaction force F is determined by the operator OP. n F t If you feel something is wrong with this, consider the virtual mass M. d,a and spring constant K d,a You may update the damping constant D to a smaller value. Alternatively, or in addition to this, the calculation unit 124 may use the damping constant D d,a You may update it to a smaller value. This will result in a reaction force F being applied to operator OP. n F t Because the size is reduced, the discomfort for the operator can be lessened.

[0105] Furthermore, in the above-described embodiment, the position control represented by equations (1) and (2) is used to control the reaction force F in the normal direction. n and the reaction force F in the tangential direction t Both of these can be calculated, or the normal reaction force F can be calculated using impedance control represented by equations (3) and (4). n and the reaction force F in the tangential direction t Although it was explained as calculating both of these, it is not limited to this. For example, the reaction force F in the normal direction. n and the reaction force F in the tangential direction t Of these, the reaction force F in the normal direction n This is calculated using position control, and the tangential reaction force F t This can be calculated using impedance control.

[0106] Furthermore, in the embodiments described above, the robot 10 operated by the remote control system 1 was described as a tangible robot, but it is not limited to this. For example, the robot 10 operated by the remote control system 1 may be an intangible robot, or avatar, existing in a virtual space such as a VR (Virtual Reality) space, an AR (Augmented Reality) space, or an MR (Mixed Reality) space. A robot (avatar) operated in a virtual space is another example of a "remotely controlled object."

[0107] Furthermore, although the remote control interface 20 was described as a handle-type (lever-type) human-machine interface in the above-described embodiment, it is not limited to this. For example, the remote control interface 20 may be a glove-type human-machine interface worn on the operator OP's hand, or an exoskeleton-type human-machine interface worn on the shoulder, arm, leg, etc.

[0108] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0109] 1...Remote control system, 10...Robot, 11...End effector, 12...Environmental sensor, 13...First state sensor, 14...First actuator, 15...First processing unit, 20...Remote control interface, 21...Handle, 22...Second state sensor, 23...Second actuator, 24...Second processing unit, 100...Remote control device, 110...Communication interface, 120...Third processing unit, 121...Acquisition unit, 122...Estimation unit, 123...Determination unit, 124...Calculation unit, 125...Communication control unit, NW...Network, OP...Operator

Claims

1. An input unit into which the operator's control commands for the remotely controlled object are input, An acquisition unit that acquires environmental data indicating the environment in which the object exists and / or state data indicating the state of the object, A determination unit that determines the target position of the object based on the environmental data and / or the state data and the operation command, A reaction force applying unit that applies a reaction force to the operator based on a position error which is the difference between the target position and the position requested by the operator as the operation command, A remote control system equipped with the following features.

2. The reaction force application unit applies a force in the opposite direction to the force applied to the input unit by the operator as a reaction force, via the input unit to the operator. The remote control system according to claim 1.

3. The system further includes a calculation unit for calculating the reaction force, The reaction force application unit applies the calculated reaction force to the operator. The remote control system according to claim 1 or 2.

4. The calculation unit calculates the reaction force using at least impedance control. The remote control system according to claim 3.

5. The determination unit further determines the target velocity of the object, The calculation unit calculates the reaction force based on the speed error, which is the difference between the target speed and the speed requested by the operator as the operation command, and the position error, as impedance control. The remote control system according to claim 4.

6. The calculation unit further calculates the reaction force using admittance control. The remote control system according to claim 4.

7. The determination unit further determines the target velocity of the object and the target value of the force to be applied to the input unit. The calculation unit calculates the reaction force as admittance control based on the speed error, which is the difference between the target speed and the speed requested by the operator as an operation command; the force error, which is the difference between the target force value and the force applied to the input unit by the operator; and the position error. The remote control system according to claim 6.

8. The determination unit updates the target position based on the force error. The remote control system according to claim 7.

9. The determination unit determines a target trajectory which is a trajectory in which the multiple target positions are arranged in chronological order. The calculation unit calculates the first reaction force, which is the reaction force in the normal direction of the target trajectory, and the second reaction force, which is the reaction force in the tangential direction of the target trajectory, using position control, and calculates the second reaction force using impedance control. The remote control system according to claim 4.

10. The calculation unit calculates the first reaction force based on the position error as part of the position control. The remote control system according to claim 9.

11. A method of remote control using a computer, To acquire environmental data indicating the environment in which the object to be remotely controlled exists and / or state data indicating the state of the said object, Based on the aforementioned environmental data and / or state data, and the operator's operation commands for the object input to the input unit, the target position of the object is determined. Based on the position error, which is the difference between the target position and the position requested by the operator as the operation command, a reaction force is applied to the operator. A remote control method including

12. A program to be executed by a computer, To acquire environmental data indicating the environment in which the object to be remotely controlled exists and / or state data indicating the state of the said object, Based on the aforementioned environmental data and / or state data, and the operator's operation commands for the object input to the input unit, the target position of the object is determined. Based on the position error, which is the difference between the target position and the position requested by the operator as the operation command, a reaction force is applied to the operator. A program that includes this.

Images