Remote operation device, remote operation method, and program

Abstract

To improve operability from remote of a robot arm.SOLUTION: A remote operation device reflects motion of an operation body on operation of a robot arm through network. The remote operation device includes an input part, a calculation part, a prediction part, and a communication part. The input part acquires space positional information of the operation body. The calculation part calculates moving speed of the operation body on the basis of the space positional information. The prediction part predicts the space position of the operation body in the future for only a prediction period which is variably set according to the moving speed. The communication part includes a communication part for transmitting the predicted space position to a device for controlling the robot arm through the network.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] One aspect of the present invention relates to a technology for assisting in remotely operating a robot arm. [Background technology]

[0002] Robot arms, or manipulators, are becoming used in many fields, such as advanced medical care and factory automation (FA). In recent years, systems that operate robot arms remotely in combination with communication technologies such as 5G are being developed. Attempts are also being made to support remote operation of robot arms with intuitive operation interfaces.

[0003] An operator remotely controls a robot arm by moving a control object held in his / her hand or by pressing a button. Generally, a certain degree of delay occurs before the operator's operation is reflected in the movement of the remote robot arm. There are various causes for this, including communication delays and delays in drive processing, and since both of these lead to reduced operability, technologies to resolve this problem are being sought. For example, a technology has been disclosed that predicts operation signals when remotely operating a humanoid robot (see Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] “Prescient whole-body teleoperation of humanoid robots”Luigi Penco, Jean-Baptiste Mouret, Serena Ivaldi, Prescient teleoperation of humanoid robots, 2022 Summary of the Invention [Problem to be solved by the invention]

[0005] When working with remote control, delays caused by various factors impair operability, which causes discomfort and fatigue for the operator, creating issues that need to be resolved. The present invention has been made in light of the above circumstances, and aims to provide a technique that makes it possible to improve the remote operability of a robot arm. [Means for solving the problem]

[0006] A remote control device according to one aspect of the present invention reflects the movement of a control body in the operation of a robot arm via a network. The remote control device includes an input unit, a calculation unit, a prediction unit, and a communication unit. The input unit acquires spatial position information of the control body. The calculation unit calculates a moving speed of the control body based on the spatial position information. The prediction unit predicts the spatial position of the control body in the future for a prediction period that is variably set according to the moving speed. The communication unit includes a communication unit that transmits the predicted spatial position to a device that controls the robot arm via the network. Effect of the Invention

[0007] According to one aspect of the present invention, it is possible to provide a technique that enables improved remote operability of a robot arm. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a remote operation system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the remote operation system shown in FIG. [Diagram 3] FIG. 3 is a flowchart showing an example of a processing procedure in the operation device 100. As shown in FIG. [Figure 4] FIG. 4 is a functional block diagram showing an example of variable prediction section 152. As shown in FIG. [Diagram 5] FIG. 5 is a graph showing an example of a function for determining the number of predicted frames. [Figure 6]FIG. 6 is a diagram showing an example of an operation screen on which the predicted position of the operating object 11 is superimposed. [Figure 7] FIG. 7 is a diagram for explaining extraction of a planned work area. [Figure 8] FIG. 8 is a block diagram showing an example of a hardware configuration of the operation device 100 according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. (composition) Fig. 1 is a diagram showing an example of a remote operation system according to an embodiment. In Fig. 1, an operating device 100 is installed at a base A (master side), and a robot arm 240 (manipulator) and a robot device 200 that controls it are installed at a base B (slave side). Bases A and B are geographically separated, and the operating device 100 and the robot device 200 communicate with each other via a network NW such as the Internet.

[0010] When the operator at site A holds the operating body 11 in his / her hand and moves it, the robot arm 240 moves in response to that movement. That is, the operation device 100 reflects the movement of the operating body 11 in the operation of the robot arm 240 via the network NW. This state is photographed by the display camera 221 at site B, and the image data is transmitted to the operation device 100 via the network NW. The image data is visualized by the operation device 100 and displayed on the operation screen of the display unit 12 together with various markers.

[0011] That is, the operating body 11 generates operation information such as three-dimensional position information of the operating body 11 and a button press state in response to the operation of the operator, and inputs it to the operation device 100 (a). The operation device 100 calculates a predicted three-dimensional position of the operating body 11 from the operation information, and transmits it to the robot device 200 together with the button press state via the network NW (b).

[0012] The robot device 200 has a communication processing function, a robot arm simulation function, a robot arm control function, etc. The robot device 200 generates a robot arm control command from the received information and inputs it to the robot arm 240 (c). In response to this, the robot arm 240 moves so as to grab the target 21, for example. The movement is captured by the display camera 221, and image data is generated and input to the robot device 200 (d). In addition, manipulator information such as the position, posture, and sensor values ​​is generated along with the movement of the robot arm 240 and input to the robot device 200 (e).

[0013] The robot device 200 generates image data and robot arm information such as position, posture, sensor values, gripping state information / release state information based on the manipulator information, and transmits the information to the operation device 100 via the network NW (f).

[0014] The operation device 100 is an example of a remote operation device, and includes a communication processing function, a variable amount three-dimensional position prediction processing function, and a camera image display function. In particular, the variable amount three-dimensional position prediction processing function will be described in detail later. The operation device 100 reproduces the camera image from the image data received via the network NW, and displays it on the operation screen of the display unit 12 (g). The operation screen is generated by superimposing, for example, a cross-shaped symbol mark (marker) on the actual image captured by the display camera 221. Delays in camera images that occur due to network issues impair operability. Therefore, the technology for improving operability is explained in detail below.

[0015] Fig. 2 is a block diagram showing an example of the remote operation system shown in Fig. 1. The remote operation system includes an operation device 100 and a robotic device 200. The operation device 100 accepts operations by an operator. The robotic device 200 operates remotely based on operation information from the operation device 100.

[0016] (Regarding the operation device 100) The operation device 100 is a device that is directly used by an operator of the system, and includes an input unit 110 , a display control unit 120 , a calculation unit 130 , a communication unit 140 , and a prediction unit 150 .

[0017] The input unit 110 acquires operation information from the operating body 11. Here, as the operating body 11, a motion glove worn on the operator's hand, a motion sensor worn on the arm, or a controller held in the hand such as a joystick can be used. A mechanical device having the same degree of freedom as the slave robot arm 240 may be used as the operating body 11.

[0018] The operation information is, for example, information indicating the three-dimensional position coordinates (spatial position information) or tilt information of the operating object 11, or information indicating the state of various buttons (on / off, etc.), and is collected, for example, at a predetermined rate.

[0019] The type of three-dimensional position coordinates varies depending on the input device constituting the operating body 11. For example, a motion sensor attached to the joints of the operator's hand or arm can acquire position coordinate information of multiple locations such as the hand position, wrist, elbow, and upper arm. In the embodiment, the three-dimensional position coordinates and button state of a controller that can be held in one hand are described as operation information. Note that the spatial position information is not limited to three dimensions, and may be two-dimensional data or one-dimensional data depending on the degree of freedom of the joints of the robot arm 240, for example.

[0020] The display control unit 120 visualizes image data of the robot arm 240 captured at the site B to generate an operation screen, displays the screen on the display unit 12, and presents the current work status to the operator. The display control unit 120 also displays, for example, the work status at a remote location obtained by the sensor unit 220 of the robot device 200 as an image. The display unit 12 may be not only a display installed on a desk or the like, but also a head-mounted display or smart glasses worn on the operator's head. In other words, the technology of the embodiment has a high affinity with so-called XR technology, such as virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[0021] The calculation unit 130 acquires the operation information acquired from the input unit 110 and the sensor information sent from the communication unit 140, extracts and formats data required for the prediction unit 150, and sends it to the prediction unit 150. The calculation unit 130 also extracts and formats data required for the calculation unit 230 of the robot device 200 based on the prediction result calculated by the prediction unit 150, and sends it to the communication unit 140. Furthermore, the calculation unit 130 extracts and formats data required for the display control unit 120 based on the acquired sensor information, generates an operation screen, and sends it to the display control unit 120.

[0022] The communication unit 140 communicates with the communication unit 210 of the robot device 200, and transmits and receives information generated by the operation device 100 and the robot device 200, respectively, via the network.

[0023] The prediction unit 150 acquires the operation information of the operating body 11 and various sensor information related to the robot arm 240 from the calculation unit 130, and performs a predictive calculation regarding the three-dimensional position of the operating body 11. That is, the prediction unit 150 calculates the moving speed of the operating body 11 based on the spatial position information of the operating body 11 included in the operation information acquired by the input unit 110. Then, a prediction period whose length changes according to the moving speed is set, and the prediction unit 150 predicts the three-dimensional position of the operating body 11 in the future by this prediction period. That is, the prediction unit 150 sets the prediction period when the moving speed of the operating body 11 is fast to be longer than the prediction period when the moving speed is slow. The predicted three-dimensional position is transmitted to the robot device 200 by the communication unit 140 via the network NW.

[0024] Incidentally, the prediction unit 150 includes a delay measurement unit 151 and a variable prediction unit 152 as functional blocks according to the embodiment. The delay measurement unit 151 acquires operation information and various sensor information from the calculation unit 130, measures the delay time until the operation of the operator is reflected in the slave side image on the display unit 12, and outputs the result to the variable prediction unit 152. In other words, the delay measurement unit 151 measures the delay time until the movement of the operating body 11 is reflected in the operation of the robot arm 240.

[0025] The variable prediction unit 152 predicts the three-dimensional position of the operating object 11 based on the delay time information and operation information from the delay measurement unit 151, and outputs the prediction result to the calculation unit 130. That is, the variable prediction unit 152 adaptively sets a prediction period based on the delay time information and spatial position information of the operating object 11, and predicts the spatial position of the operating object 11 according to the prediction period.

[0026] (Regarding the robot device 200) The robot device 200 operates at a remote location based on control information generated by the operation device 100. The robot device 200 includes a communication unit 210, a sensor unit 220, a calculation unit 230, and a robot arm 240. In other words, the robot device 200 is a device that remotely controls the robot arm as viewed from the operation device 100.

[0027] The communication unit 210 communicates with the communication unit 140 of the operation device 100, and transmits and receives information between the input unit 110 and the robot device 200 via the network.

[0028] The sensor unit 220 senses information related to the working status on the side of the robot device 200, acquires sensor data such as the position, posture, and operation state of the robot arm 240, and sends it to the communication unit 210 and the calculation unit 230. The sensor unit 220 includes a display camera 221. The display camera 221 acquires image data such as RGB, RGB-D, and IR (Infra-Red). The sensor unit 220 acquires sensor data such as images of the robot arm 240 and an operation target (target), infrared rays, and depth using the image data acquired by the display camera 221. Furthermore, the sensor unit 220 may also include sensors for sensing the position and posture of the robot arm 240, operation states such as "grabbing" and "releasing", and states of torque, temperature, and force sensors.

[0029] The calculation unit 230 generates a control signal for controlling the position of an end effector, which is the tip of the robot arm 240, based on the control signal received from the communication unit 210 and information on the work status on the robot device 200 side obtained by the sensor unit 220. This control signal is sent from the calculation unit 230 to the robot arm 240. The calculation unit 230 also acquires various sensor data including image data obtained from the sensor unit 220, extracts and formats data required for the calculation unit 130 and prediction unit 150 of the operation device 100, and sends it to the communication unit 210.

[0030] The robot arm 240 operates based on a control signal from the calculation unit 230 to perform a task. The end effector may be in the shape of a human hand so that it can perform general-purpose tasks, or it may have a shape specialized for a specific task. The robot arm may be installed alone at the work site, or may be mounted on a mechanism for movement, such as wheels or legs. It may also be installed as an arm on the torso of a humanoid robot.

[0031] (action) FIG. 3 is a flowchart showing an example of a processing procedure of the operation device 100. As shown in FIG. (Step S101) In step S101, the operation device 100 acquires operation information and sensor data. The operation information is generated, for example, at regular time intervals in the operation object 11, and is acquired by the input unit 110. In the embodiment, it is assumed that the operation information is generated at a frame rate of 60 Hz.

[0032] (Step S102) In step S102, the operating device 100 measures a communication delay time. In the embodiment, the communication delay time is defined as the time it takes from when the input unit 110 acquires operation information from the operating body 11, to when the robot arm moves through communication with the robot device 200, and when the sensor data indicating the movement is returned from the robot device 200 and reaches the operating device 100.

[0033] To measure the communication delay time, a change in state or movement of the operating body 11 can be used. For example, the gripping and releasing actions of the robot arm 240 can be associated with pressing and releasing a button on the operating body 11, respectively. That is, the communication time can be measured by measuring the time from when the operator presses the button on the operating body 11 until the robot arm is in a gripping state in the sensor information received from the robot device 200. Alternatively, a change in the movement direction of the operating body 11 may be detected from the history of the three-dimensional position information of the operating body 11, and the time until this change is reflected may be used as the communication delay time.

[0034] (Step S103) In step S103, the operation device 100 predicts a future three-dimensional position of the operation object 11 based on the time series of the three-dimensional position of the operation object 11 and the measured communication delay time. The variable amount three-dimensional position prediction process will be described with reference to FIG.

[0035] (Variable 3D position prediction processing function) 4 is a functional block diagram showing an example of the variable prediction unit 152. The variable prediction unit 152 includes a prediction filter (PredictionFilter) 152a, a speed calculation unit (SpeedCalc) 152b, and an adaptive prediction selector (AdaptivePredictionSelector) 152c.

[0036] In Fig. 4, three-dimensional position information (x, y, z) of the operating object 11 and its acquisition time (t) are acquired from the operating object 11 and input to the variable prediction unit 152. In Fig. 4, the operating object 11 continuously acquires its own position information at, for example, 60 Hz. In addition, the following description will be given assuming that the position is predicted up to 250 milliseconds into the future based on the communication delay time. If the data acquired at each time is expressed in frame units, 250 milliseconds into the future corresponds to predicting 15 frames into the future.

[0037] Prediction filter 152a receives the time series of input three-dimensional position information and predicts future three-dimensional positions over frames 1 to 15. That is, prediction filter 152a holds input coordinates historically (in a time series manner) and predicts three-dimensional positions for the future 15 frames using several past three-dimensional points as input.

[0038] To predict 3D position, for example, one method is to calculate the speed and acceleration from time-series coordinate information and then use them to calculate the future position. Alternatively, a deep learning model can be used.

[0039] When using a deep learning model, the input can be time-series coordinates as is, the amount of movement from a certain time (coordinate difference value), or data converted into distance, speed, or acceleration. In addition, it is possible to use models with various structures for neural networks, such as FNN, CNN, RNN, and combinations of these.

[0040] The velocity calculation unit 152b uses the three-dimensional position information and the time information of each frame to calculate the moving velocity of the operating object 11. That is, the velocity calculation unit 152b calculates the velocity from the three-dimensional position and time of the past few frames.

[0041] The speed can be calculated by differentiating the three-dimensional position information of at least the past two frames, but the position information may contain noise depending on the accuracy of the sensor equipped in the operating body 11. Therefore, the speed may be calculated using any frame, or the number of frames for calculating the speed next may be dynamically changed using a previously calculated speed. In other words, by dynamically changing the frames used for speed calculation, noise resistance can be improved.

[0042] The adaptive prediction selector 152c uses the future predicted position calculated by the prediction filter 152a and the speed calculated by the speed calculation unit 152b to determine how many frames into the future the predicted position should be used. That is, the adaptive prediction selector 152c sets the prediction period as the number of frames. To set the number of frames, for example, a function such as that shown in FIG. 5 can be used.

[0043] Fig. 5 is a graph showing an example of a function for determining the number of predicted frames. The graph in Fig. 5 is a graph of the function of equation (1), with the horizontal axis indicating speed and the vertical axis indicating the number of frames.

number

[0044] In the low speed range of 0.0 to 0.4 m / sec, a function with a gentle slope is used to determine a small value. In the speed range of 0.4 m / sec to 1.0 m / sec, the slope is made steeper to increase the number of frames determined. In the speed range of 1.0 m / sec or higher, it is fixed at 15 frames (250 milliseconds at 60 Hz). This is because it shows an example of predicting a position up to 250 milliseconds into the future. By using such a function, it is possible to shorten the future prediction period when the speed is slow, and to lengthen the future prediction period when the speed is fast.

[0045] Here, 0.4 m / sec and 1.0 m / sec given as the reference values ​​of the speed are merely examples, and any value can be used as the reference value. After these processes, the adaptive prediction selector 152c returns the coordinate information of the predicted position of the calculated number of frames.

[0046] Returning to FIG. (Step S104) In step S104, the operating device 100 transmits the predicted future position acquired from the adaptive prediction selector 152c and information on the button state of the operating object 11 to the robot device 200. Upon receiving this, the robot device 200 generates a control signal by the calculation unit 230 and inputs it to the robot arm 240. The robot arm 240 operates in response to the control signal from the calculation unit 230 and performs a desired task. The position information used for calculating the control signal is predicted future position information of the operating object 11.

[0047] (Step S105) In step S105, the operating device 100 displays the working situation at the remote location obtained by the sensor unit 220 of the robot device 200 as an image on the display unit 12. The predicted future position of the operating body 11 is reflected in the movement of the robot arm in the displayed image.

[0048] That is, in the embodiment, a control is realized in which the future position of the operating body 11 is predicted, the predicted position is notified to the robot device 200, and the robot arm 240 is operated in advance. This makes it possible to reduce a delay in reflecting an operation due to a communication delay.

[0049] (effect) As described above, in the embodiment, the immediate future three-dimensional position of the operating body 11 is predicted based on the time series of the three-dimensional position of the operating body 11, and this predicted position is notified to the slave side. At that time, the prediction period is dynamically changed according to the speed of the operating body 11. That is, the prediction period is set long for fast movement (high speed) of the operating body 11, and the amount of movement of the spatial position of the operating body 11 is predicted to be large, thereby reducing the delay felt by the operator. On the other hand, the prediction period is set short for slow movement (low speed) of the operating body 11, and the amount of movement of the spatial position of the operating body 11 is predicted to be small, thereby reducing deviation and disturbance of the predicted position and enabling precise work. Therefore, accurate operation can be performed even in an environment with delay.

[0050] Existing technologies have been proposed to measure communication delay times and perform simulations and predictions based on those delay times. For example, a relay node is installed in the network, and the delay between the master site and the relay node, and the delay between the relay node and the slave site are measured. Then, a technology has been proposed in which a future state is simulated based on the measured values ​​and notified to the master side. Alternatively, a technology is known in which a probability distribution of communication time is generated by taking into account the jitter of delay times, multiple arrival patterns (corresponding to predictions of operation results) are generated, and a control input is determined by statistical processing. However, there has been no consideration of changing the prediction period according to the operation speed of the operating object.

[0051] Technology has also been proposed to predict the operation position of a robot arm. However, it is technically difficult to accurately predict human operations, and in cases that require precise operation, such as pick-and-place and medical use cases, there is a possibility that operability will deteriorate due to deviations from the predicted position or fluctuations in the predicted position over time.

[0052] In contrast, according to the embodiment, by inputting the predicted position of the operating body 11 to the robot arm at a remote location, it is possible to shorten the delay in input to the robot due to communication delays and quickly reflect the operation. In addition, it is possible to reduce the sense of delay felt by the operator when operating in remote work, improve work efficiency, and reduce the physical burden on the operator. From these points, according to the embodiment, it is possible to improve the remote operability of the robot arm. As a result, it is possible to reduce the sense of delay felt by the operator in remote work, promote improvement of work efficiency, and reduce the physical burden on the operator.

[0053] (Regarding the first modified example) In the embodiment, the control for moving the robot arm 240 in advance is realized by transmitting the predicted future position of the operating body 11 to the robot device 200. Alternatively, the predicted position may be superimposed on the operation screen and displayed as a movement guide for the operator.

[0054] Fig. 6 is a diagram showing an example of an operation screen on which the predicted position of the operating body 11 is superimposed. In Fig. 6, predicted coordinates for seven frames are indicated by dots (·). By displaying the future predicted position on the operation screen in this way, the operator can understand the path that the robot arm 240 will take in the future. This type of display can be realized by converting the predicted three-dimensional position coordinates into two-dimensional coordinates in accordance with the image received by the calculation unit 130.

[0055] (Regarding the second modified example) The attention of the operator performing remote operation is focused on the planned work area where the work is about to be performed. Therefore, it is possible to incorporate a mechanism that extracts the planned work area based on the predicted position and reduces the processing frequency of information other than the planned work area. This reduces the processing load of the entire system.

[0056] 7 is a diagram for explaining extraction of the planned work area. The area to be sensed is divided into a plurality of areas in advance, and the area including the predicted position is extracted as the planned work area (hatched area). Normally, all information such as images and point clouds sensed by the sensor unit 220 is sent to the operation device 100. In contrast, the processing load of the system can be reduced by transmitting data of areas not included in the planned work area less frequently than data of the work area.

[0057] (Hardware configuration) 8 is a block diagram showing an example of a hardware configuration of an operation device 100 according to an embodiment. The operation device 100 includes a processor 1 such as a CPU (Central Processing Unit), and a storage 2, a memory 3, and an interface (I / F) unit 4 are connected to the processor 1 via a bus 5. In other words, the operation device 100 is a computer having a processor and a storage unit.

[0058] The I / F unit 4 has a communication interface function and communicates with the robot device 200 via a network NW. The storage 2 is a storage device configured by combining a non-volatile memory such as a solid state drive (SSD) that can be written / read at any time as a storage medium, and a non-volatile memory such as a read only memory (ROM). The storage 2 stores a program 2a and data 2b required to execute various control processes according to an embodiment, in addition to an operating system (OS) and the like.

[0059] The memory 3 is, for example, a combination of a non-volatile memory such as an SSD, which can be written to and read from at any time, and a volatile memory such as a RAM (Random Access Memory), as a storage medium.

[0060] The processor 1 includes an input unit 110, a display control unit 120, a calculation unit 130, a communication unit 140, and a prediction unit 150 in FIG. 2 as processing functions necessary to carry out an embodiment. These functional blocks are realized by the processor 1 executing a program 2a. In other words, the program 2a includes instructions that cause a computer to function as the operation device 100 of the embodiment. Some or all of these functional blocks may be realized using hardware such as a large scale integration (LSI) or an application specific integrated circuit (ASIC).

[0061] In the above disclosure, the embodiment of the present invention has been described in detail. The above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, a specific configuration according to the embodiment may be appropriately adopted.

[0062] In short, this invention is not limited to the above-mentioned embodiment as it is, and in the implementation stage, the components can be modified and embodied without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining multiple components disclosed in the above-mentioned embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0063] 1…Processor 2. Storage 2a…Program 2b…Data 3. Memory 4. Interface section 5. Bus 11...Operation body 12...Display section 21…Target 100...Operating device 110...Input section 120...Display control unit 130...Calculation section 140…Communications Department 150…Prediction Section 151…Delay measurement unit 152…Variable prediction unit 152a…Prediction filter 152b…Speed ​​calculation section 152c…Adaptive Prediction Selector 200...Robot device 210…Communications Department 220…Sensor section 221...Display camera 230...Calculation section 240...Robot arm.

Claims

1. A remote control device that reflects the movement of a control body in the operation of a robot arm via a network, An input unit that acquires spatial position information of the operating object; A calculation unit that calculates a moving speed of the operating object based on the spatial position information; a prediction unit that predicts a spatial position of the operating object in the future for a prediction period that is variably set according to the moving speed; A communication unit that transmits the predicted spatial position to a device that controls the robot arm via the network.

2. The remote control device according to claim 1 , wherein the prediction unit sets the prediction period when the moving speed is fast to be longer than the prediction period when the moving speed is slow.

3. The prediction unit is a delay measurement unit that measures a delay time until a movement of the operating body is reflected in an operation of the robot arm; The remote control device according to claim 1 , further comprising: a variable prediction unit that adaptively sets the prediction period based on the delay time and the spatial position information, and predicts the spatial position according to the prediction period.

4. The communication unit receives image data from a camera that captures an area including the robot arm, The remote control device according to claim 1 , further comprising a display control unit that visualizes the received image data to generate an operation screen and displays the operation screen on a display device.

5. The remote control device according to claim 4 , wherein the display control unit displays the predicted spatial position in a superimposed manner on the operation screen.

6. A remote control method executed by a computer that controls a robot arm via a network, comprising: A remote control method executed by a computer for reflecting a movement of an operating body in an operation of a robot arm via a network, comprising: acquiring spatial position information of the operating object; calculating a moving speed of the operating object based on the spatial position information; predicting a spatial position of the operating object in the future for a prediction period that is variably set according to the moving speed; and transmitting the predicted spatial position over the network to a device controlling the robot arm.

7. A program comprising instructions for causing a computer to function as the remote control device according to any one of claims 1 to 5.

Images