Remote operation device, remote operation method, and program

Abstract

To improve operability from remote of a robot arm.SOLUTION: A remote operation device operates a robot arm through network. The remote operation device includes an input part, a display part, a calculation part, and a display correction part. The input part acquires operation information reflecting the operation of an operator. The display part displays an operation screen including an actual video image of the robot arm. The calculation part calculates the operation position of the robot arm from the operation information. The display correction part displays a visual user interface indicating the operation position of the robot arm on the operation screen in a superimposed state.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] One aspect of the present invention relates to a technology for assisting in remotely operating a robot arm. [Background technology]

[0002] Robot arms, or manipulators, are becoming used in many fields, such as advanced medical care and factory automation (FA). In recent years, systems that operate robot arms remotely in combination with communication technologies such as 5G are being developed. Attempts are also being made to support remote operation of robot arms with intuitive operation interfaces.

[0003] An operator remotely controls a robot arm by moving a control object held in his / her hand or by pressing a button. Generally, a certain degree of delay occurs before the operator's operation is reflected in the movement of the remote robot arm. There are various causes for this, including communication delays and delays in drive processing, and since both of these lead to reduced operability, technologies to resolve this problem are being sought. For example, a technology has been disclosed that predicts operation signals when remotely operating a humanoid robot (see Non-Patent Document 1). [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] “Prescient whole-body teleoperation of humanoid robots”Luigi Penco, Jean-Baptiste Mouret, Serena Ivaldi, Prescient teleoperation of humanoid robots, 2022 Summary of the Invention [Problem to be solved by the invention]

[0005] When working with remote control, delays caused by various factors impair operability, which causes discomfort and fatigue for the operator, creating issues that need to be resolved. The present invention has been made in light of the above circumstances, and aims to provide a technique that makes it possible to improve the remote operability of a robot arm. [Means for solving the problem]

[0006] A remote control device according to one aspect of the present invention operates a robot arm via a network. The remote control device includes an input unit, a display unit, a calculation unit, and a display correction unit. The input unit acquires operation information reflecting an operation by an operator. The display unit displays an operation screen including an actual image of the robot arm. The calculation unit calculates an operation position of the robot arm from the operation information. The display correction unit superimposes and displays a visual user interface indicating the operation position of the robot arm on the operation screen. Effect of the Invention

[0007] According to one aspect of the present invention, it is possible to provide a technique that enables improved remote operability of a robot arm. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating an example of a remote operation system according to an embodiment. [Diagram 2] FIG. 2 is a block diagram showing an example of the remote operation system shown in FIG. [Diagram 3] FIG. 3 is a flowchart showing an example of a processing procedure of the operation device 100. As shown in FIG. [Figure 4] FIG. 4 is a diagram showing an example of an image in which a UI is superimposed on a video of the robot arm 240. As shown in FIG. [Diagram 5] FIG. 5 is a diagram showing an example of a change in the UI when the robot arm 240 moves. [Figure 6] FIG. 6 is a diagram showing an example of a display form of the UI. [Figure 7] FIG. 7 is a diagram showing an example of a UI according to the modified example. [Figure 8] FIG. 8 is a block diagram showing an example of a hardware configuration of the operation device 100 according to the embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. (composition) Fig. 1 is a diagram showing an example of a remote operation system according to an embodiment. In Fig. 1, an operating device 100 is installed at a base A (master side), and a robot arm 240 (manipulator) and a robot device 200 that controls it are installed at a base B (slave side). Bases A and B are geographically separated, and the operating device 100 and the robot device 200 communicate with each other via a network NW such as the Internet.

[0010] When the operator at site A holds the operating body 11 in his / her hand and moves it, the robot arm 240 moves in response to that movement. That is, the operation device 100 reflects the movement of the operating body 11 in the operation of the robot arm 240 via the network NW. This state is photographed by the display camera 221 at site B, and the image data is transmitted to the operation device 100 via the network NW. The image data is visualized by the operation device 100 and displayed on the operation screen of the display unit 12 together with various markers.

[0011] That is, the operating body 11 generates operation information such as three-dimensional position information of the operating body 11 and a button press state in response to the operation of the operator, and inputs it to the operation device 100 (a). The operation device 100 calculates a predicted three-dimensional position of the operating body 11 from the operation information, and transmits it to the robot device 200 together with the button press state via the network NW (b).

[0012] The robot device 200 has a communication processing function, a robot arm simulation function, a robot arm control function, etc. The robot device 200 generates a robot arm control command from the received information and inputs it to the robot arm 240 (c). In response to this, the robot arm 240 moves so as to grab the target 21, for example. The movement is captured by the display camera 221, and image data is generated and input to the robot device 200 (d). In addition, manipulator information such as the position, posture, and sensor values ​​is generated along with the movement of the robot arm 240 and input to the robot device 200 (e).

[0013] The robot device 200 generates image data and robot arm information such as position, posture, sensor values, gripping state information / release state information based on the manipulator information, and transmits the information to the operation device 100 via the network NW (f).

[0014] The operation device 100 is an example of a remote operation device, and includes a communication processing function, a variable amount three-dimensional position prediction processing function, and a camera image display function. In particular, the variable amount three-dimensional position prediction processing function will be described in detail later. The operation device 100 reproduces the camera image from the image data received via the network NW, and displays it on the operation screen of the display unit 12 (g). The operation screen is generated by superimposing, for example, a cross-shaped symbol mark (marker) on the actual image captured by the display camera 221.

[0015] Delays in camera images that occur due to network issues impair operability. Therefore, the technology for improving operability is explained in detail below.

[0016] Fig. 2 is a block diagram showing an example of the remote operation system shown in Fig. 1. The remote operation system includes an operation device 100 and a robotic device 200. The operation device 100 accepts operations by an operator. The robotic device 200 operates remotely based on operation information from the operation device 100.

[0017] (Regarding the operation device 100) The operation device 100 operates the robot arm via a network NW. The operation device 100 is a device that is directly used by an operator of the system, and includes an input unit 110, a display control unit 120, a calculation unit 130, a communication unit 140, and a display correction unit 150.

[0018] The input unit 110 acquires operation information from the operating body 11. Here, as the operating body 11, a motion glove worn on the operator's hand, a motion sensor worn on the arm, or a controller held in the hand such as a joystick can be used. A mechanical device having the same degree of freedom as the slave robot arm 240 may be used as the operating body 11.

[0019] The operation information is, for example, information indicating the three-dimensional position coordinates (spatial position information) or tilt information of the operating object 11, or information indicating the state of various buttons (on / off, etc.), and is collected, for example, at a predetermined rate.

[0020] The type of three-dimensional position coordinates varies depending on the input device constituting the operating body 11. For example, a motion sensor attached to the joints of the operator's hand or arm can acquire position coordinate information of multiple locations such as the hand position, wrist, elbow, and upper arm. In the embodiment, the three-dimensional position coordinates and button state of a controller that can be held in one hand are described as operation information. Note that the spatial position information is not limited to three dimensions, and may be two-dimensional data or one-dimensional data depending on the degree of freedom of the joints of the robot arm 240, for example.

[0021] The display control unit 120 visualizes image data of the robot arm 240 captured at the site B, and generates an operation screen including a live-action image of the robot arm 240. The display control unit 120 displays the operation screen on the display unit 12 to present the current work situation to the operator. The display control unit 120 also displays, for example, the work situation at a remote location obtained by the sensor unit 220 of the robot device 200 as an image. The display unit 12 may be not only a display installed on a desk or the like, but also a head-mounted display or smart glasses worn on the operator's head. In other words, the technology of the embodiment has a high affinity with so-called XR technology, such as virtual reality (VR), augmented reality (AR), and mixed reality (MR).

[0022] The calculation unit 130 acquires the operation information acquired from the input unit 110 and the sensor data sent from the communication unit 140, extracts and formats data required for the display correction unit 150, and sends it to the display correction unit 150. The calculation unit 130 also extracts and formats data required for the display control unit 120 based on the acquired sensor data, generates an operation screen, and sends it to the display control unit 120. Furthermore, the calculation unit 130 calculates the operation position of the robot arm 240 from the input coordinates of the operating body 11 included in the operation information acquired from the input unit 110.

[0023] The communication unit 140 communicates with the communication unit 210 of the robot device 200, and transmits and receives information generated by the operation device 100 and the robot device 200, respectively, via the network.

[0024] The display correction unit 150 acquires operation information of the operating object 11 and various sensor data related to the robot arm 240 from the calculation unit 130, and generates a visual user interface (UI) for superimposing and displaying on the operation screen.

[0025] Incidentally, the display correction unit 150 includes a delay measurement unit 151 and an operation information superimposition unit 152 as functional blocks according to the embodiment.

[0026] The delay measurement unit 151 acquires operation information and various sensor data from the calculation unit 130, measures the delay time until the operation of the operator is reflected in the slave side image on the display unit 12, and outputs the result to the operation information superimposition unit 152. In other words, the delay measurement unit 151 measures the delay time until the movement of the operating body 11 is reflected in the operation of the robot arm 240.

[0027] The operation information superimposition unit 152 extracts and formats the data necessary for the display unit 12 based on the delay time information and operation information from the delay measurement unit 151, superimposes it on the UI operation screen showing the operation position of the robot arm 240, generates video data, and sends it to the display unit 12.

[0028] (Regarding the robot device 200) The robot device 200 operates at a remote location based on control information generated by the operation device 100. The robot device 200 includes a communication unit 210, a sensor unit 220, a calculation unit 230, and a robot arm 240. In other words, the robot device 200 is a device that remotely controls the robot arm as viewed from the operation device 100.

[0029] The communication unit 210 communicates with the communication unit 140 of the operation device 100, and transmits and receives information between the input unit 110 and the robot device 200 via the network.

[0030] The sensor unit 220 senses information related to the working status on the side of the robot device 200, acquires sensor data such as the position, posture, and operation state of the robot arm 240, and sends it to the communication unit 210 and the calculation unit 230. The sensor unit 220 includes a display camera 221. The display camera 221 acquires image data such as RGB, RGB-D, and IR (Infra-Red). The sensor unit 220 acquires sensor data such as images of the robot arm 240 and an operation target (target), infrared rays, and depth using the image data acquired by the display camera 221. Furthermore, the sensor unit 220 may also include sensors for sensing the position and posture of the robot arm 240, operation states such as "grabbing" and "releasing", and states of torque, temperature, and force sensors.

[0031] The calculation unit 230 generates a control signal for controlling the position of an end effector, which is the tip of the robot arm 240, based on the control signal received from the communication unit 210 and information on the work status on the robot device 200 side obtained by the sensor unit 220. This control signal is sent from the calculation unit 230 to the robot arm 240. The calculation unit 230 also acquires various sensor data including image data obtained from the sensor unit 220, extracts and formats data required for the calculation unit 130 and display correction unit 150 of the operation device 100, and sends the data to the communication unit 210.

[0032] The robot arm 240 operates based on a control signal from the calculation unit 230 to perform a task. The end effector may be in the shape of a human hand so that it can perform general-purpose tasks, or it may have a shape specialized for a specific task. The robot arm may be installed alone at the work site, or may be mounted on a mechanism for movement, such as wheels or legs. It may also be installed as an arm on the torso of a humanoid robot.

[0033] (action) FIG. 3 is a flowchart showing an example of a processing procedure of the operation device 100. As shown in FIG. (Step S101) In step S101, the operation device 100 acquires operation information and sensor data. The operation information is generated, for example, at regular time intervals in the operation object 11, and is acquired by the input unit 110. In the embodiment, it is assumed that the operation information is generated at a frame rate of 60 Hz.

[0034] (Step S102) In step S102, the operating device 100 measures a communication delay time. In the embodiment, the communication delay time is defined as the time it takes from when the input unit 110 acquires operation information from the operating body 11, to when the robot arm moves through communication with the robot device 200, and when the sensor data indicating the movement is returned from the robot device 200 and reaches the operating device 100.

[0035] To measure the communication delay time, a change in state or movement of the operating body 11 can be used. For example, the gripping and releasing actions of the robot arm 240 can be associated with pressing and releasing a button on the operating body 11, respectively. That is, the communication time can be measured by measuring the time from when the operator presses the button on the operating body 11 until the robot arm 240 is in a gripping state in the sensor data received from the robot device 200. Alternatively, a change in the movement direction of the operating body 11 may be detected from the history of the three-dimensional position information of the operating body 11, and the time until this change is reflected may be used as the communication delay time.

[0036] (Step S103) In step S103, the operation device 100 analyzes the operation information and the operation state. That is, the operation device 100 calculates the movement amount and speed for the communication delay time from the latest three-dimensional position coordinates of the operation object 11. The movement amount and speed can be calculated from the input three-dimensional position coordinates of the operation object 11 and the time information of the time series of the coordinate data.

[0037] The operation device 100 also calculates the distance between the tip of the robot arm 240 at a remote location and an object from the sensor data received from the robot device 200. By performing alignment (calibration) between the display camera 221 and the robot arm 240 in advance, it is possible to acquire the position of the tip of the robot arm 240 at the time of capturing an image and posture information of the robot arm 240.

[0038] The operation device 100 recognizes an object from image data, infrared rays, depth, and other information of the sensor data received from the robot device 200, and detects the position and posture of the target object. The difference between the position of the target object obtained by this process and the position of the tip of the robot arm 240 is the above-mentioned distance. Furthermore, the operation device 100 retains the states of "not grasping", "grasping motion", "still grasping", and "releasing motion" from the motion information obtained from the sensor data of the robot arm 240.

[0039] (Step S104) In step S104, the operation device 100 synthesizes the input three-dimensional position onto the image of the remote robot arm 240. That is, the operation device 100 superimposes a visual user interface (UI) onto the image obtained from the sensor data based on the information acquired in step S103.

[0040] 4 is a diagram showing an example of an image in which a UI is superimposed on an image of the robot arm 240. The robot arm 240 appears in the image of the remote location, and the position of the robot arm 240 is represented, for example, by the center of the arm tip. This representative position of the robot arm 240 is indicated by a dashed circle (α).

[0041] On the other hand, the latest three-dimensional position from the operating body 11 is indicated by a hatched circle (β). The circle (β) as the first widget is the latest input position, that is, it indicates the operation position of the robot arm 240. This is equivalent to indicating the movement destination of the robot arm 240 for the image delayed by the communication delay.

[0042] Furthermore, a trajectory (γ) is displayed as a second widget using a history of input coordinates for the communication delay time from the latest input. The trajectory (γ) indicates the difference between the current position of the robot arm 240 and the operation position. In other words, the UI superimposed on the image of the robot arm includes the latest input position and the movement trajectory from the current position to the latest input position.

[0043] By displaying the movement trajectory, the operation system for the operator is replaced with the conventional operation system. In other words, instead of operating the robot arm directly, the operation system is changed to one in which the operator operates the UI of the circle (β) and pulls the robot arm to move it. This is because the latest input position corresponding to the circle (β) is data acquired on the master side and does not include communication delays. This change in the operation system allows the operator to operate with reduced sense of delay.

[0044] Fig. 5 is a diagram showing an example of a change in the UI when the robot arm 240 moves. Fig. 5(a) shows an operation screen when the operator stops the operation body 11 without moving it. At this time, the remote robot arm 240 does not move either, and therefore the tip of the robot arm 240 and the hatched circle coincide with each other.

[0045] 5(b) shows the operation screen when the operator moves the operating body 11. Due to a communication delay, the robot arm 240 has not yet started moving on the image on the display unit 12, but the hatched circle moves ahead of it. In addition, a trajectory indicating the history of input positions is displayed between the robot arm 240 and the hatched circle.

[0046] Fig. 6 is a diagram showing an example of the display form of the UI. For example, based on the information obtained by the analysis in step S103 (Fig. 3), the transparency of the UI can be dynamically changed according to the state of the operating body 11 or the robot arm 240. For example, when the moving speed of the operating body 11 is fast, when the moving amount is large, when there is no target object near the robot arm 240, or when the arm is in a "not gripping" state or a "gripping" state, the visibility can be improved by clearly displaying the trajectory (γ) as shown in Fig. 6(a).

[0047] On the other hand, the UI may overlap with the object that the operator wants to operate. In order to solve this problem, it is possible to increase the transparency of the UI.

[0048] As shown in Fig. 6(b), when the speed of the robot arm 240 is slow and the amount of movement is small, it is advisable to display the trajectory (γ) transparently to emphasize the difference from the case of Fig. 6(a). Fig. 6(c) shows a case where the object to be operated is near the robot arm 240, and in such a case, the trajectory (γ) and the circle (β) are also displayed transparently. Fig. 6(d) shows a case where the robot arm 240 performs a grasping motion and a releasing motion, and in this case too, it is advisable to display the circle (β) transparently. In this way, by varying the transparency of the UI, it is possible to increase visibility and improve operability.

[0049] In this way, in step S104, the operation device 100 generates an operation screen on which the UI is superimposed, using the information analyzed in step S103, and the processing procedure proceeds to step S105.

[0050] (Step S105) Returning to Fig. 3, the explanation will be continued. In step S105, the operation device 100 displays the operation screen generated in S104. The image of the robot arm displayed at this time includes a communication delay, whereas the superimposed UI is displayed including the latest position of the input device. This allows the operator to reduce the sense of delay in the reflection of the operation caused by the communication delay.

[0051] (Step S106) In step S106, the operating device 100 transmits the input position information and the button state information of the operating body 11 to the robot device 200. The robot arm 240 operates based on the control signal from the calculation unit 230 to perform a task. (Step S107) The procedure from step S102 to step S106 is repeated until it is determined in step S107 that the process is finished.

[0052] (effect) As described above, according to the embodiment, the three-dimensional position information of the operating body 11 and the image of the slave side are input, and a UI that makes it look like the operator is pulling the rubber band and operating the robot arm 240 is superimposed on the operation screen that displays the image of the slave side (remote location) where the robot arm 240 is installed. In addition, the UI display form is dynamically changed according to the working state on the slave side. This reduces the sense of delay felt by the operator when operating the robot arm in remote work, improves work efficiency, and reduces the physical burden on the operator.

[0053] Existing technologies include methods to measure communication delays and simulate the future by the amount of delay measured, as well as prediction technologies to predict the operation position. However, it has been technically difficult to accurately simulate or predict the movement of a robot arm operated by a human. For this reason, especially in cases where precision operation is required, such as medical use cases, there is a possibility that operability will deteriorate due to deviations from the predicted position, fluctuations in the predicted position, and errors between the simulation results and the actual situation.

[0054] In contrast, in the embodiment, the input position information of the operating body 11 is used to express a trajectory that expands and contracts like a rubber band, and the trajectory is superimposed on the image. That is, a UI (which may be called a "rubber UI") using the position information of the operating body 11 on the operating side (local) is synthesized with the image of the robot in a remote location. The rubber UI is realized by the operation information superimposing unit 152 giving an operation feel that imitates an elastic body (rubber) to the trajectory (γ) as the second widget. Controlling the operation feel of the widget is not limited to the method of the embodiment, and can also be realized by using an existing image processing framework.

[0055] With this technology, the operator feels as if he or she is pulling and moving the robot arm using the UI, rather than operating a robot arm at a remote location, and this has the effect of reducing the sense of delay. Therefore, according to the embodiment, it is possible to improve the operability of the robot arm from a remote location.

[0056] (About modified examples) In the above embodiment, an example was shown in which the difference between the operation position of the operator and the position of the arm of the real machine was displayed as a UI. Instead of this, when the operation information includes the posture of the robot arm 240, the difference between the operation and the posture of the arm of the real machine may also be displayed as a UI. An example of overlaying a UI based on posture information is shown in Figure 7.

[0057] FIG. 7 is a diagram showing an example of a UI according to a modified example. In FIG. 7, a triangle (q) as a third widget indicates the latest posture of the robot arm 240 based on the input operation information. A triangle (p) as a fourth widget indicates the posture of the robot arm 240 on the operation screen. That is, the latest posture information based on the operation information is indicated by a triangle (q), and the posture of the actual arm that changes with a delay from the operation is indicated by a triangle (p). In this way, by presenting a UI that immediately reflects the information operated on the posture in addition to the position, the operator can operate the actual arm without checking the image presented with a delay, and the sense of delay can be reduced.

[0058] (Hardware configuration) 8 is a block diagram showing an example of a hardware configuration of an operation device 100 according to an embodiment. The operation device 100 includes a processor 1 such as a CPU (Central Processing Unit), and a storage 2, a memory 3, and an interface (I / F) unit 4 are connected to the processor 1 via a bus 5. In other words, the operation device 100 is a computer having a processor and a storage unit.

[0059] The I / F unit 4 has a communication interface function and communicates with the robot device 200 via a network NW.

[0060] The storage 2 is a storage device configured by combining a non-volatile memory such as a solid state drive (SSD) that can be written / read at any time as a storage medium, and a non-volatile memory such as a read only memory (ROM). The storage 2 stores a program 2a and data 2b required to execute various control processes according to an embodiment, in addition to an operating system (OS) and the like.

[0061] The memory 3 is, for example, a combination of a non-volatile memory such as an SSD, which can be written to and read from at any time, and a volatile memory such as a RAM (Random Access Memory), as a storage medium.

[0062] The processor 1 includes an input unit 110, a display control unit 120, a calculation unit 130, a communication unit 140, and a display correction unit 150 in FIG. 2 as processing functions necessary to carry out an embodiment. These functional blocks are realized by the processor 1 executing a program 2a. In other words, the program 2a includes instructions that cause a computer to function as the operation device 100 of the embodiment. Some or all of these functional blocks may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0063] The present invention is not limited to the above embodiment. For example, in Fig. 6, an example in which the transparency is variable is shown, but the size of the circle (β), the thickness of the trajectory (γ), the line type, color, etc. may be dynamically changed according to the state of the operating body 11 or the robot arm 240.

[0064] In the above disclosure, the embodiment of the present invention has been described in detail. The above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, a specific configuration according to the embodiment may be appropriately adopted.

[0065] In short, this invention is not limited to the above-mentioned embodiment as it is, and in the implementation stage, the components can be modified and embodied without departing from the gist of the invention. In addition, various inventions can be formed by appropriately combining multiple components disclosed in the above-mentioned embodiment. For example, some components may be deleted from all the components shown in the embodiment. Furthermore, components from different embodiments may be appropriately combined. [Explanation of symbols]

[0066] 1…Processor 2. Storage 2a…Program 2b…Data 3. Memory 4. Interface section 5. Bus 11...Operation body 12...Display section 21…Target 100...Operating device 110...Input section 120...Display control unit 130...Calculation section 140…Communications Department 150…Display correction section 151…Delay measurement unit 152...operation information superimposition unit 200...Robot device 210…Communications Department 220…Sensor section 221...Display camera 230...Calculation section 240...Robot arm.

Claims

1. A remote control device for controlling a robot arm via a network, an input unit that acquires operation information that reflects an operation by an operator; A display unit that displays an operation screen including an actual image of the robot arm; A calculation unit that calculates an operation position of the robot arm from the operation information; and a display correction unit that superimposes and displays a visual user interface indicating the operation position of the robot arm on the operation screen.

2. The display correction unit is a delay measurement unit that measures a delay time until a movement of an operating body for operating the robot arm is reflected in the operation of the robot arm; The remote operation device according to claim 1 , further comprising: an operation information superimposing unit that generates the visual user interface based on the operation information and the delay time, and displays the generated visual user interface in a superimposed manner on the operation screen.

3. The operation information superimposing unit A first widget corresponding to the operation position of the robot arm; The remote operation device according to claim 2 , which generates a visual user interface including a second widget indicating a difference between a current position of the robot arm on the operation screen and the operation position.

4. The operation information superimposing unit The remote control device according to claim 3 , wherein the second widget is generated as a trajectory of the robot arm from a history of input coordinates of the operating object over the delay time.

5. The operation information superimposing unit a third widget indicating a posture of the robot arm based on the operation information; and a fourth widget indicating a posture of the robot arm on the operation screen.

6. The operation information superimposing unit The remote control device according to claim 3 , wherein the second widget is provided with a feeling of operation simulating an elastic body.

7. A remote control method executed by a computer that controls a robot arm via a network, comprising: obtaining operation information reflecting an operation by an operator; displaying an operation screen including a live image of the robot arm; calculating an operation position of the robot arm from the operation information; and a step of superimposing a visual user interface indicating the operation position of the robot arm on the operation screen.

8. A program comprising instructions for causing a computer to function as the remote control device according to any one of claims 1 to 5.

Images