Robot remote control device, robot remote control method, and program

Abstract

To provide a robot remote operation control device, a robot remote operation control method, and a program that are able to present information on an operation without blocking a field of view of an operator.SOLUTION: A robot remote operation control device for robot remote operation control for an operator to remotely operate a robot, comprises: a spatial information sensor configured to acquire spatial information of a space where the robot is present; an image output unit configured to present, to the operator, a spatial reproduction image in which the space where the robot is present is reproduced from the spatial information; a sensor information acquisition unit configured to acquire sensor information of a sensor mounted on the robot; a robot posture calculation unit configured to calculate a posture of the robot, based on the sensor information; and a spatial reproduction image drawing unit configured to draw the sensor information at a position associated with a body of the robot, based on the spatial information and based on information on the calculated posture of the robot, to generate a spatial reproduction image.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present invention relates to a robot remote operation control device, a robot remote operation control method, and a program.

Background Art

[0002] Robots are equipped with, for example, force sensors, contact sensors, etc. Information obtained from these sensors is displayed on a display device connected to a personal computer or the like connected to operate the robot (see, for example, Patent Document 1). In the technology described in Patent Document 1, this information is presented as numerical values and charts on a separate screen that can be confirmed by the operator.

Prior Art Documents

Patent Documents

[0003] [[ID=2"]]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, in the conventional technology described in Patent Document 1 and the like, when numerical values and charts are displayed on a separate screen, there is a problem that the user does not pay attention to the sensor information displayed on the separate screen. In particular, 、U there is a problem that the operability deteriorates as the number of I (user interfaces) increases.

[0005] The present invention has been made in view of the above problems, and an object thereof is to provide a robot remote operation control device, a robot remote operation control method, and a program that can present information related to an operation without obstructing the operator's field of view.

Means for Solving the Problems

[0006] (1) To achieve the above objective, a robot remote control device according to one aspect of the present invention is a robot remote control device that, in a robot remote control in which an operator remotely controls a robot, comprises: a spatial information sensor that acquires spatial information of the space in which the robot is located; an image output unit that presents to the operator a spatial reproduction image which reproduces the space in which the robot is located from the spatial information; a sensor information acquisition unit that acquires sensor information of sensors mounted on the robot; a robot posture calculation unit that calculates the posture of the robot based on the sensor information; and a spatial reproduction image drawing unit that draws the sensor information at positions linked to the body of the robot, based on the spatial information and the calculated posture information of the robot, to form a spatial reproduction image.

[0007] (2) A robot remote control device according to one aspect of the present invention further comprises an information acquisition unit that acquires operator status information of the state of the operator operating the robot, a robot posture calculation unit that calculates the posture of the robot based on the three-dimensional model of the robot, the spatial information sensor and the operator status information, and a spatial reproduction image drawing unit that draws the control command value information included in the operator status information at a position linked to the body of the robot, as described in (1) above.

[0008] (3) Furthermore, a robot remote control device according to one aspect of the present invention is a robot remote control device according to (1) or (2) above, wherein the spatial reproduction image drawing unit calculates how the robot appears from the operator's viewpoint based on the spatial information and the information calculated by the robot posture calculation unit, and draws the sensor information at a position where the robot's body does not obstruct the operator's field of view.

[0009] (4) Furthermore, a robot remote control device according to one aspect of the present invention is a robot remote control device according to any one of (1) to (3) above, wherein the spatial reproduction image drawing unit draws the command values ​​of each joint angle of the robot at positions associated with the body of the robot based on the sensor information.

[0010] (5) Furthermore, a robot remote control device according to one aspect of the present invention is a robot remote control device according to any one of (1) to (4) above, wherein the spatial information sensor includes a shooting device, the shooting device is attached to the wrist of the robot, and the spatial reproduction image drawing unit draws the image taken by the shooting device at a position linked to the wrist of the robot.

[0011] (6) Furthermore, a robot remote control device according to one aspect of the present invention is the robot remote control device described in (5) above, further comprising an operation amount determination unit that determines the amount of operation of the robot based on the operator status information.

[0012] (7) To achieve the above objective, a robot remote control method according to one aspect of the present invention is a robot remote control control device in which an operator remotely controls a robot, acquires spatial information of the space in which the robot is located, presents the operator with a spatial reproduction image which is

[0013] (8) To achieve the above objective, a program according to one aspect of the present invention causes the computer of a robot remote control device, which is used to remotely control a robot by an operator, to acquire spatial information of the space in which the robot is located, to present to the operator a spatially reproduced image which is [Effects of the Invention]

[0014] According to the above (1) to (8), it is possible to present information regarding operations without obstructing the operator's field of view.

Brief Description of the Drawings

[0015] [Figure 1] It is a diagram showing a configuration example of a robot remote operation control system according to an embodiment. [Figure 2] It is a diagram showing an example of a robot according to an embodiment and an example of an installation location of a camera. [Figure 3] It is a flowchart of a processing procedure performed by a robot remote operation control system according to an embodiment. [Figure 4] It is a diagram showing an example of displaying a command value to a robot on the contour. [Figure 5] It is a diagram of an example of displaying an image captured by a photographing unit attached to a wrist in association with the wrist position. [Figure 6] It is a diagram showing an example of drawing force sense information in association with a forearm. [Figure 7] It is a diagram for explaining an example of a method for generating an image presented to an operator according to an embodiment.

Modes for Carrying Out the Invention

[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the drawings used in the following description, the scales of each member are appropriately changed in order to make each member recognizable in size. In all the drawings for explaining the embodiments, those having the same function are denoted by the same reference numerals, and repeated explanations are omitted. In addition, “based on XX” as used in the present application means “based on at least XX”, and includes cases where it is based on another element in addition to XX. Also, “based on XX” is not limited to the case of directly using XX, and includes cases where it is based on something obtained by performing arithmetic operations or processing on XX. “XX” is an arbitrary element (for example, arbitrary information).

[0017] [Configuration of Robot Remote Operation Control System] An example configuration of a robot remote operation control system will be described. FIG. 1 is a diagram showing an example configuration of a robot remote operation control system according to the present embodiment. As shown in FIG. 1, the robot remote operation control system 9 includes, for example, a robot 1, a second imaging unit 2 (spatial information sensor), an HMD 3, an operation detection unit 4, a robot remote operation control device 5, and a robot model storage unit 6.

[0018] The robot 1 includes, for example, a first imaging unit 11 (spatial information sensor), a sensor 12 (spatial information sensor), an arm 13, an end effector 14, and a drive unit 15.

[0019] The robot remote operation control device 5 includes, for example, an acquisition unit 51, an operation amount determination unit 52, a robot posture calculation unit 53, a spatial reproduction image drawing unit 54, a control unit 55, a storage unit 56, and an image output unit 57. The acquisition unit 51 includes, for example, an image acquisition unit 511, a sensor information acquisition unit 512, and an operator information acquisition unit 513 (information acquisition unit).

[0020] The robot 1 is, for example, any one of a single-arm robot, a double-arm robot, a multi-arm robot, an arm robot, etc.

[0021] The first imaging unit 11 is, for example, a CCD (Charge Coupled Device) imaging device or a CMOS (Complementary Metal-Oxide-Semiconductor) imaging device. The first imaging unit 11 includes, for example, a fish-eye lens or a wide-angle lens, etc. The first imaging unit 11 is attached to, for example, the head of the robot 1 or the end effector 14. The first imaging unit 11 includes, for example, an RGB (red, green, blue) camera that captures RGB images and a depth camera that captures depth images. The image captured by the first imaging unit 11 is the spatial information of the space where the robot exists. Note that the position of the first imaging unit 11 attached to the robot 1 is known and is stored, for example, in the robot model storage unit 6. The first imaging unit 11 acquires data in time series and assigns a time stamp.

[0022] Sensor 12 is, for example, a 6-axis sensor. Sensor 12 also detects the joint angles of each joint of the arm 13 and each joint of the end effector 14. Sensor 12 is a contact sensor that is also attached to, for example, a fingertip. Sensor 12 is a joint torque sensor attached to the joints of the robot 1. In this way, sensor 12 detects sensor information used by the robot 1. The first imaging unit 11 may also be one of the sensors 12. Sensor 12 acquires data in chronological order and assigns a timestamp.

[0023] The arm 13 has an end effector 14 connected to its tip.

[0024] The end effector 14 has at least two finger parts. The number of fingers on the end effector 14 may be three or more.

[0025] The second imaging unit 2 is installed, for example, in the working environment of robot 1. The second imaging unit 2 includes, for example, an RGB camera that captures RGB (red, green, blue) images and a depth camera that captures depth images. The images captured by the second imaging unit 2 are spatial information of the space in which the robot is located. The installation location of the second imaging unit 2 is known and is stored, for example, in the robot model memory unit 6. The second imaging unit 2 acquires data in chronological order and assigns a timestamp.

[0026] HMD3 is a head-mounted display, for example, worn on the operator's head. HMD3 includes, for example, an image display unit 31 that presents images to the operator, an eye-tracking unit 32 that detects the operator's gaze, and a communication unit (not shown) that transmits and receives information with the robot remote control device 5.

[0027] The operation detection unit 4 detects the operator's operation instructions. The operation detection unit 4 is, for example, a data glove.

[0028] The robot model memory unit 6 stores, for example, a three-dimensional shape model (including dimensions, etc.) of the robot 1.

[0029] The robot remote control device 5 uses spatial information of the space in which the robot is located, operator state information indicating the state of the operator controlling the robot 1, and the robot model to draw a spatially reproduced image in which the spatial occlusion caused by the robot 1's body is removed.

[0030] The image acquisition unit 511 acquires images taken by the first shooting unit 11 and images taken by the second shooting unit 2.

[0031] The sensor information acquisition unit 512 acquires the joint angles of each joint of the end effector 14 detected by the sensor 12 from the sensor 12. Alternatively, the sensor information acquisition unit 512 may acquire images captured by the imaging unit (first imaging unit 11, second imaging unit 2) as sensor information. Or, the sensor information acquisition unit 512 may acquire operator status information detected by the operation detection unit 4 as sensor information.

[0032] The operator information acquisition unit 513 acquires operator status information (including control command value information (commands)) from the operation detection unit 4, for example, which indicates the status of the operator operating the robot 1. The operator status information allows the operator to determine how they want the robot 1 to operate or how they want the robot 1 to operate. The operator information acquisition unit 513 may also acquire gaze information detected by the gaze detection unit 32.

[0033] The control amount determination unit 52 determines the control amount of the robot 1 based on the operator status information.

[0034] The robot posture calculation unit 53 calculates the robot's posture using the three-dimensional model of robot 1, the information acquired by the sensor information acquisition unit 512, and the operator status information acquired by the operator information acquisition unit 513. Alternatively, the robot posture calculation unit 53 calculates the robot's posture using the three-dimensional model of robot 1 and the information acquired by the sensor information acquisition unit 512, and calculates how the robot appears from the operator's perspective.

[0035] The spatial reproduction image rendering unit 54 uses the calculated posture of the robot model to calculate how the robot model appears from the operator's viewpoint. The spatial reproduction image rendering unit 54 displays sensor information on the body parts of the robot 1 in the image presented to the operator. to The associated image is generated. The spatial reproduction image rendering unit 54 provides the generated image to the HMD3.

[0036] The control unit 55 controls the movement of the robot 1 based on the manipulated amount determined by the manipulated amount determination unit 52.

[0037] The memory unit 56 stores, for example, values, programs, etc., necessary for controlling the robot 1. The memory unit 56 temporarily stores the acquired information.

[0038] The image output unit 57 outputs a spatially reconstructed image, which is a reconstruction of the space in which the robot is located, to the HMD3 for the operator to see.

[0039] [Example of robot configuration] First, I will explain examples of robots and camera placement locations. Figure 2 shows an example of a robot according to this embodiment and an example of where the camera is installed. Robot 1 comprises, for example, a body 16, a head 17, arms 13 (13L, 13R), and a hand (end effector) 14 (14L, 14R). In the following description, the end effector 14 will also be referred to as the hand 14. The hand 14 has, for example, first imaging units 11a and 11b installed on the wrist on the back of the palm. The head 17 includes first imaging units 11c and 11d. Additionally, a second imaging unit 2 is installed in the workspace of robot 1.

[0040] Note that the first imaging units 11c and 11d of the head of robot 1 may be replaced by the second imaging unit 2. Note that while Figure 2 shows an example where the hand 14 has five fingers, it is sufficient to have two or more fingers. Also, while Figure 2 shows an example of a robot with two arms, it may also have one arm. Furthermore, the number of imaging units is just one example and is not limited to this. In addition, the imaging units can capture the entire palm, fingertips, and wrist. hand Hand 14 may have at least one of these, and may have multiple of them.

[0041] [Example of processing performed by the manipulated variable determination unit] Next, an example of the processing performed by the manipulated variable determination unit 52 will be explained. The control amount determination unit 52 determines the control amount of the robot 1 based on the operator status information acquired by the operator information acquisition unit 513. Alternatively, the control amount determination unit 52 may also use, for example, images acquired by the image acquisition unit 511 to estimate the target object, its position, its shape, etc., and use the estimated information about the target object to determine the control amount of the robot 1. Furthermore, the control amount determination unit 52 may also estimate taxonomy information related to the task (see, for example, Reference 1) based on the operator status information acquired by the operator information acquisition unit 513, and use the estimated taxonomy information to determine the control amount of the robot 1.

[0042] Reference 1; Thomas Feix, Javier Romero, et al., “The GRASP Taxonomy of Human GraspTypes” IEEE Transactions on Human-Machine Systems (Volume: 46, Issue: 1, Feb.2016), IEEE, p66-77

[0043] [Example of processing performed by the robot posture calculation unit] Next, we will explain an example of the processing performed by the robot posture calculation unit 53. The robot posture calculation unit 53 uses the joint angle information (including timestamps) acquired by the sensor information acquisition unit 512 and the three-dimensional shape model stored in the robot model storage unit 6 to calculate the posture of the robot 1, for example, at predetermined intervals. The robot posture calculation unit 53 calculates the posture of the arms 13 and end effectors 14 according to the task, and does not need to calculate the posture of the body 16, head 17, etc.

[0044] [Example of processing procedure] Next, an example of a processing procedure performed by the robot remote control system 9 will be described. Figure 3 is a flowchart of the processing procedure performed by the robot remote control system according to this embodiment.

[0045] (Step S1) The image acquisition unit 511 acquires the image taken by the first shooting unit 11 and the image taken by the second shooting unit 2.

[0046] (Step S2) The sensor information acquisition unit 512 acquires the sensor information (including joint angle information) detected by the sensor 12.

[0047] (Step S3) The operator information acquisition unit 513 acquires operator status information from the HMD3 and the operation detection unit 4.

[0048] (Step S4) The robot posture calculation unit 53 calculates the posture of the robot 1 based on the information acquired by the sensor information acquisition unit 512. Alternatively, the robot posture calculation unit 53 may calculate the posture of the robot 1 based on the operator status information.

[0049] (Step S5) The spatial reproduction image rendering unit 54 uses the acquired information and the robot's three-dimensional shape model to calculate how the robot appears from the operator's viewpoint.

[0050] (Step S6) The spatial reproduction image drawing unit 54 generates an image by relating the sensor information to how the robot appears from the operator's viewpoint, which has been calculated.

[0051] (Step S7) The image output unit 57 provides the operator with the image generated by the spatial reproduction image drawing unit 54.

[0052] Note that the operating procedure shown in Figure 3 is just one example and is not limited to this. For example, the processes in steps S1 to S3 may be performed in any order, or they may be performed simultaneously or in parallel.

[0053] [Example of processing performed by the spatial reproduction image rendering unit] Next, we will explain an example of the processing performed by the spatial reproduction image rendering unit 54. The spatial reconstruction image rendering unit 54 renders a spatial reconstruction image based on spatial information and robot posture information, removing the spatial occlusion caused by the robot 1's body. More specifically, the spatial reconstruction image rendering unit 54 first calculates how the robot would appear from the operator's viewpoint if sensor information were added. The spatial reconstruction image rendering unit 54 then adds sensor information to the calculated image of how the robot would appear from the operator's viewpoint, at positions that do not obstruct the operator's field of view.

[0054] [Examples of processes performed by a robot remote control device] Next, we will explain an example of the processing performed by the robot remote control device 5. Since the robot posture calculation unit 53 has a known detailed three-dimensional model of the robot 1 to be operated, it can calculate occlusion in the view from the operator by the robot's body. Furthermore, the robot posture calculation unit 53 can sequentially acquire information on each joint of the robot 1 being operated, and therefore can calculate the robot's posture information. The spatial reproduction image rendering unit 54 can obtain control command value information of the robot 1 to be operated from the operation detection unit 4, and uses the results calculated by the robot posture calculation unit 53 to display the operator's view. direction In this system, sensor information can be displayed in conjunction with the body of robot 1.

[0055] Next, we will explain an example of drawing sensor information linked to the body of robot 1. Figure 4 shows an example of displaying command values ​​for a robot as outlines. In the example in Figure 4, the command values ​​for each joint angle of robot 1 are drawn and displayed. As shown in Figure 4, instead of displaying the command values ​​as values, the outline lines showing the position of the command values, the outline lines showing the current position due to the delay, and the amount of delay are displayed with arrows. Furthermore, the outline lines showing the position of the command values ​​and the outline lines showing the position due to the delay are represented by different colors or line types, for example. The objects represented by the outline lines are, for example, arms, end effectors, etc., as shown in Figure 4. In this embodiment, such delay amounts are also sensor information based on information obtained from sensors. This allows the operator to understand the delay between the command and the action.

[0056] Figure 5 shows an example of displaying images captured by a wrist-mounted imaging unit in relation to the wrist position. In the example in Figure 5, the image captured by the first imaging unit mounted on the wrist is composited and displayed within circle g11. Furthermore, in the example in Figure 5, sensor information for the contour, as explained using Figure 4, is also composited and displayed. Note that the position where the captured image is displayed does not obstruct the operator's field of view.

[0057] Figure 6 shows the forearm. Put it on Force information to draw This figure shows an example of this. In the example in Figure 6, information showing the movement of the fingertips in six axes is overlaid on the forearm. The display is positioned so as not to obstruct the operator's field of view. The display shows, for example, the force applied to each fingertip when gripping, broken down into the x, y, and z axes. This allows the operator to visually understand how much force is being applied when gripping.

[0058] The location to which sensor information (including images captured by the imaging unit) is associated is a location linked to the body of robot 1. A location linked to the body of robot 1 is, for example, the outline of robot 1's body, a location in contact with robot 1's body, or the vicinity of robot 1's body. Thus, the location to which the added sensor information is added does not necessarily have to be in contact with robot 1; it is sufficient if it is a location linked to robot 1's body. Furthermore, the display may be shown like a tattoo on, for example, the palm of the hand or on the arm, or it may be displayed in space in contact with the palm or arm. The location to which it is added is preferably within the range that the operator is looking at, i.e., the range of the robot 1's work object. Furthermore, the body of robot 1 (for example, fingertips) of When the position changes, the displayed sensor information's position will adjust to reflect the change in the body's position. Follow You may make them obey.

[0059] In this embodiment, for example, the first imaging unit 11 attached to the head 17 of the robot 1 captures an RGB image, and then combines it with an image in which command values ​​are drawn as outlines, the captured image, a graph showing force information, etc., as shown in Figures 4 to 6, and presents it.

[0060] Note that the information to be combined shown in Figures 4 to 6 is just an example and is not limited to this. Information related to other operations (for example, control command value information included in the operator status information) and information related to robot 1 may also be combined and presented. In addition, two or more additional sensor information items shown in Figures 4 to 6 may be presented simultaneously, or they may be switched between. Furthermore, the robot remote control device 5 may switch the display and hiding of the sensor information, for example, according to the operator's operation, or it may switch between display and hiding automatically. The robot remote control device 5 may also switch the additional sensor information depending on the work content.

[0061] Furthermore, the method for illustrating information related to operations is not limited to the methods described above. The illustrating method may include, for example, showing command values ​​numerically, or illustrating them using tables, graphs, etc.

[0062] By displaying these indicators, operators can understand the amount of manipulation, the actual operation in response to that amount of manipulation, and the amount of delay without their vision being obstructed.

[0063] [Example of a method for generating the displayed image] Here, we will explain one example of a method for generating images to be presented to the operator. Figure 7 is a diagram illustrating an example of a method for generating an image presented to the operator according to this embodiment. The example in Figure 7 is the image generation method shown in Figure 6. The robot remote control device 5 recognizes and extracts the hand portion g112 from the image g111 captured by the first imaging unit 11 using a well-known image recognition process. The robot remote control device 5 uses the three-dimensional model of the robot 1 stored in the robot model memory unit 6 to render, for example, the fingertip force information contained in the sensor information detected by the sensor 12 (image g121). The robot remote control device 5 combines the extracted image g112 with the drawn image g121 to create a spatial reconstruction image. At this time, the robot remote control device 5 displays the drawn image g121 superimposed on the fingertip of the robot 1 (image g131).

[0064] As described above, in this embodiment, sensor information is presented by synthesizing it with the captured image in a position that does not obstruct the operator's field of view and is associated with the body of the robot 1.

[0065] Therefore, according to this embodiment, it is possible to provide an interface that does not obstruct the operator's view.

[0066] In the example described above, sensor information is presented by compositing it onto an image, but this is not the only example. The robot remote control device 5 may generate an image from the operator's viewpoint using the information acquired by the sensor information acquisition unit 512 and the three-dimensional model of the robot 1 stored in the robot model storage unit 6, and then composite the sensor information onto the generated image. This makes it possible to provide an image with sensor information from any viewpoint.

[0067] Furthermore, a program to implement all or part of the functions of the robot remote control device 5 in this invention may be recorded on a computer-readable recording medium, and all or part of the processing performed by the robot remote control device 5 may be performed by loading the program recorded on this recording medium into a computer system and executing it. Herein, "computer system" includes hardware such as an OS and peripheral devices. Furthermore, "computer system" also includes a WWW system equipped with a homepage provisioning environment (or display environment). Furthermore, "computer-readable recording medium" refers to portable media such as flexible disks, magneto-optical disks, ROMs, CD-ROMs, and storage devices such as hard disks built into a computer system. Moreover, "computer-readable recording medium" also includes volatile memory (RAM) inside a computer system that acts as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line, which holds the program for a certain period of time.

[0068] Furthermore, the above program may be transmitted from a computer system that stores the program in a memory device or the like to another computer system via a transmission medium or by transmission waves within the transmission medium. Here, the "transmission medium" for transmitting the program refers to a medium that has the function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. In addition, the above program may be for the purpose of realizing a part of the functions described above. Furthermore, it may be a so-called differential file (differential program) that can realize the functions described above in combination with a program already recorded in the computer system.

[0069] Although embodiments for carrying out the present invention have been described above using examples, the present invention is not limited in any way to these embodiments, and various modifications and substitutions can be made without departing from the spirit of the present invention. [Explanation of Symbols]

[0070] 9...Robot remote control system, 1...Robot, 2...Second imaging unit, 3...HMD, 4...Operation detection unit, 5...Robot remote control device, 6...Robot model memory unit, 11...First imaging unit, 12...Sensor, 13...Arm, 14...End effector, 15...Drive unit, 31...Image display unit, 32...Gaze detection unit, 51...Acquisition unit, 52...Operation amount determination unit, 53...Robot posture calculation unit, 54...Spatial reproduction image drawing unit, 55...Control unit, 56...Memory unit, 57...Image output unit, 511...Image acquisition unit, 512...Sensor information acquisition unit, 513...Operator information acquisition Department

Claims

1. In robot remote control, where an operator remotely controls a robot, An image output unit presents to the operator a spatial reproduction image, which is a spatial reproduction of the space in which the robot is located, obtained from spatial information of the space in which the robot is located by a spatial information sensor. A sensor information acquisition unit that acquires sensor information from sensors mounted on the robot, A robot posture calculation unit that calculates the posture of the robot, A spatial reproduction image drawing unit draws the sensor information at positions associated with the robot's body, based on the spatial information and the calculated robot's posture information, to create a spatial reproduction image. An information acquisition unit that acquires operator status information of the operator operating the robot, Equipped with, The robot posture calculation unit calculates the posture of the robot based on the sensor information, the three-dimensional model of the robot, and the operator status information. The spatial reproduction image rendering unit renders the control command value information included in the operator state information at a position associated with the robot's body. Robot remote control device.

2. In robot remote control, where an operator remotely controls a robot, An image output unit presents to the operator a spatial reproduction image, which is a spatial reproduction of the space in which the robot is located, obtained from spatial information of the space in which the robot is located by a spatial information sensor. A sensor information acquisition unit that acquires sensor information from sensors mounted on the robot, A robot posture calculation unit calculates the posture of the robot based on the sensor information, A spatial reproduction image drawing unit draws the sensor information at positions associated with the robot's body, based on the spatial information and the calculated robot's posture information, to create a spatial reproduction image. Equipped with, The spatial reproduction image rendering unit calculates how the robot appears from the operator's viewpoint based on the spatial information and the information calculated by the robot posture calculation unit, and renders the sensor information in a position where the robot's body does not obstruct the operator's field of view. Robot remote control device.

3. In robot remote control, where an operator remotely controls a robot, An image output unit presents to the operator a spatial reproduction image, which is a spatial reproduction of the space in which the robot is located, obtained from spatial information of the space in which the robot is located by a spatial information sensor. A sensor information acquisition unit that acquires sensor information from sensors mounted on the robot, A robot posture calculation unit calculates the posture of the robot based on the sensor information, A spatial reproduction image drawing unit draws the sensor information at positions associated with the robot's body, based on the spatial information and the calculated robot's posture information, to create a spatial reproduction image. Equipped with, The spatial reproduction image rendering unit renders the command values ​​of each joint angle of the robot at positions associated with the robot's body, based on the sensor information. Robot remote control device.

4. In robot remote control, where an operator remotely controls a robot, An image output unit presents to the operator a spatial reproduction image, which is a spatial reproduction of the space in which the robot is located, obtained from spatial information of the space in which the robot is located by a spatial information sensor. A sensor information acquisition unit that acquires sensor information from sensors mounted on the robot, A robot posture calculation unit calculates the posture of the robot based on the sensor information, A spatial reproduction image drawing unit draws the sensor information at positions associated with the robot's body, based on the spatial information and the calculated robot's posture information, to create a spatial reproduction image. Equipped with, The spatial information sensor includes an imaging device, The aforementioned imaging device is attached to the robot's wrist. The spatial reproduction image rendering unit renders the image captured by the shooting device at a position attached to the robot's wrist. Robot remote control device.

5. The system further includes an operation amount determination unit that determines the amount of operation of the robot based on the operator status information. The robot remote control device according to claim 1.

6. A robot remote control device, which allows an operator to remotely control a robot, The spatial information of the space in which the robot exists is acquired, The operator is presented with a spatially reproduced image, which is a reconstruction of the space in which the robot exists, based on the spatial information. The sensor information of the sensor mounted on the robot is acquired, The posture of the robot is calculated, Based on the spatial information and the calculated posture information of the robot, the sensor information is plotted at positions associated with the robot's body to create a spatially reconstructed image. A method for remotely controlling a robot, The aforementioned robot remote control device further, The operator status information of the operator operating the robot is acquired, Based on the sensor information, the three-dimensional model of the robot, and the operator status information, the posture of the robot is calculated. The control command value information included in the operator status information is drawn at a position associated with the robot's body. A method for remotely controlling a robot.

7. The computer in the robot remote control device, which allows an operator to remotely control the robot, The robot acquires spatial information of the space in which it exists. The operator is presented with a spatial reproduction image, which is a reconstruction of the space in which the robot exists, based on the spatial information. The robot is made to acquire sensor information from the sensors mounted on it. The robot's posture is calculated, Based on the spatial information and the calculated posture of the robot, the sensor information is plotted at positions associated with the robot's body to create a spatially reconstructed image. It is a program, The aforementioned computer further, The system acquires operator status information regarding the status of the operator operating the robot. Based on the sensor information, the three-dimensional model of the robot, and the operator status information, the posture of the robot is calculated. The control command value information included in the operator status information is drawn at a position associated with the robot's body. program.

Images