Remote operation control system

Abstract

Provided is a remote operation control system.　A remote operation control system 1 comprises a remote manipulation device 9 that has a manipulator 5 and that operates the manipulator 5, and an operation information output device 11 that outputs, to the remote manipulation device 9, operation information for operating the remote manipulation device 9. The operation information output device 11 and the remote manipulation device 9 use UDP as a communication protocol to carry out quantum encryption communication using an encryption key provided by quantum key distribution. When a data packet for communicating the operation information is an operation-related data packet, the header of the operation-related data packet includes, as sequence information, a temporal sequence number assigned in order of time. When a plurality of operation-related data packets are received, the remote manipulation device 9 performs a process for controlling the manipulator 5 to stop the operation of the manipulator 5 if the temporal sequence numbers are reversed, and outputting an alert to the operation information output device 11 if the intervals of time at which the plurality of operation-related data packets are received are equal to or greater than a prescribed threshold value.

Description

Remote operation control system 【0001】 This invention relates to a remote operation control system. 【0002】 In recent years, due to the spread of viruses and the need to support medical services in remote areas due to a shortage of doctors, the need for remote diagnosis and remote treatment has been increasing. To meet such needs, for example, Japanese Patent No. 6157785 describes a medical manipulator system for remote surgery. The system for remote surgery exchanges various types of information such as diagnosis, surgery, and personal information between the terminal on the operator side and the medical device on the patient side. Since the information transmitted and received in remote surgery includes personal information, it is desirable to encrypt the communication information so that it is difficult to decrypt. 【0003】 In addition, medical-related devices and closed networks for remote surgery are based on the premise that they are always safe and stable, and in particular, the occurrence of unauthorized communication due to cyberattacks during surgery or vulnerabilities of medical devices is not assumed. On the other hand, high-precision medical devices generally have an embedded OS, applications, etc. installed, and the risk of unauthorized communication due to cyberattacks including the supply chain and the increasing vulnerability of software is increasing. Since surgeons have little expertise in network technology, they also need to consider the possibility of communication abnormalities due to cyberattacks and unauthorized operations in remote medical devices, and operations that can safely protect patients in the event of sudden communication abnormalities, so there is a problem that they cannot concentrate on the surgery. 【0004】 Japanese Patent No. 6157785 【0005】 An object of this invention is to provide a system that can encrypt communication information in remote operations, quickly detect unauthorized operations occurring within a network, perform appropriate actions, and notify the operator of the unauthorized operations. 【0006】This invention is based on the understanding that, fundamentally, when the time sequence numbering of multiple operation-related data packets is reversed, the manipulator's operation is stopped, allowing for early detection of malicious activity. Furthermore, when the time interval between receiving multiple operation-related data packets exceeds a predetermined threshold, an alert is output, enabling the operator to be notified of delay attacks by third parties. 【0007】 The first invention described in this specification relates to a remote control system 1 for controlling remote surgery. This remote control system 1 comprises a remote operating device 9 and an operation information output device 11. The operation information output device 11 is an element for outputting operation information from an operator to the remote operating device 9 in order to operate the remote operating device 9. The remote operating device 9 has a manipulator 5 for treating the affected area and is an element that operates the manipulator 5 in response to instructions from the operator. In other words, the remote operating device 9 is a device for operating the manipulator 5 based on the operation information output from the operation information output device 11 and performing a procedure on a patient. 【0008】 The operation information output device 11 and the remote operation device 9 communicate using UDP as the communication protocol and encrypted communication using quantum cryptography with encryption keys obtained by quantum key distribution. When data packets for communicating operation information are called operation-related data packets, the header of the operation-related data packet includes a time-sequence number, which is a number assigned in chronological order as sequence information. When the remote operation device 9 receives multiple operation-related data packets, if the time-sequence numbers are reversed, it controls the manipulator 5 to stop its operation. When the time interval between receiving multiple operation-related data packets exceeds a predetermined threshold, the remote operation device 9 outputs an alert to the operation information output device 11. 【0009】When the remote control device 9 receives multiple operation-related data packets and their time-sequence numbers are reversed, it controls the manipulator 5 to stop its operation, thus enabling early detection of malicious operation and appropriate action. Furthermore, when the time interval between receiving multiple operation-related data packets exceeds a predetermined threshold, the remote control device 9 outputs an alert to the operation information output device 11, allowing the operator to be notified of malicious operation, such as when a delay attack is being carried out by a third party. 【0010】 Figure 1 is a conceptual diagram illustrating the configuration of a remote control system. Figure 2 is a conceptual diagram illustrating an example of an operation-related data packet. Figure 3 is a flowchart illustrating an example of manipulator control related to a time sequence number. Figure 4 is a flowchart illustrating an example of a process that outputs an alert regarding the time an operation-related data packet is received. Figure 5 is a conceptual diagram illustrating another example of an operation-related data packet. Figure 6 is a conceptual diagram showing an example of the display unit of an operation information output device. Figure 7 is a conceptual diagram showing another example of a remote control system. 【0011】 The following describes a remote control system 1 for controlling remote operation. Remote control systems themselves are already known. Therefore, the remote control system of this invention can appropriately incorporate elements of known remote control systems. Examples of remote operation include remote surgery and remote operation of devices with manipulation capabilities. Examples of devices with manipulation capabilities include construction equipment, work equipment, and equipment for handling hazardous materials. The following description will use remote surgery as an example, but this invention is not limited to remote surgery. This invention can be particularly preferably used for remote operations where careful operation and confidentiality are required. 【0012】Figure 1 is a conceptual diagram illustrating the configuration of a remote control system. As shown in Figure 1, the remote control system 1 comprises a remote operating device 9 and an operation information output device 11. The operation information output device 11 is an element for outputting operation information from an operator to the remote operating device 9 in order to operate the remote operating device 9. An example of an operator is a doctor. When a doctor performs an operation process at the operation information output device 11, information related to that operation process is input to the operation information output device 11. This information is converted into digital information and output to the remote operating device 9 as operation information. Various types of information, which will be described later, may be added to the operation information. 【0013】 The remote control device 9 has a manipulator 5. An example of a manipulator 5 is one used to treat an affected area and performs a predetermined task using a medical instrument instead of the operator's hand. The remote control device 9 is a device that operates the manipulator 5 based on operation information output from the operation information output device 11 and performs a procedure on a patient. The manipulator is an element that is operated based on the operation information. This invention relates to a remote control system. For this reason, the manipulator 5 has an element for treating an affected area (for example, a medical instrument such as a scalpel) and performs a procedure such as treating an affected area. 【0014】 The operation information output device 11 and the remote operation device 9 communicate using UDP as the communication protocol. UDP (User Datagram Protocol) is one of the standard protocols (communication rules) used in IP networks such as the Internet, as it is the transport layer protocol, which is a higher layer than IP (Internet Protocol). UDP is simple and has low latency, but it is unreliable. In other words, the operation information output device 11 and the remote operation device 9 have elements that appropriately transmit and receive digital information in accordance with UDP. 【0015】Furthermore, the operation information output device 11 and the remote operation device 9 communicate encrypted using a quantum cryptography scheme with an encryption key obtained through quantum key distribution. In other words, the information transmitted and received between the operation information output device 11 and the remote operation device 9 is basically encrypted and decrypted using an encryption key. Quantum cryptography using an encryption key obtained through quantum key distribution is already known. Therefore, the operation information output device 11 and the remote operation device 9 can appropriately utilize known quantum key distribution technologies. 【0016】 It is preferable that the operation information output device 11 and the remote operation device 9 can also exchange information via a classical channel. A classical channel is, for example, a public communication channel. Classical channels are used, for example, to send and receive some information about an encryption key, or for various verifications and authentications. Generally, the encryption key itself is not sent over a classical channel, and the communication content is not encrypted. 【0017】The remote operation device 9 and the operation information output device 11 may be implemented by a computer. The computer has an input unit, an output unit, a control unit, an arithmetic unit, and a storage unit, and each element is connected by a bus or the like to enable the exchange of information. Computers usually handle digital information. For example, the computer may store programs or various kinds of information in its storage unit. When predetermined information is input from the input unit, the control unit reads the program stored in the storage unit. The control unit then reads the information stored in the storage unit as appropriate and transmits it to the arithmetic unit. The control unit also transmits the input information to the arithmetic unit as appropriate. The arithmetic unit performs calculations using the received information and stores it in the storage unit. The control unit reads the calculation results stored in the storage unit and outputs them from the output unit. In this way, various processes and steps are executed. Each unit and each means is responsible for executing these various processes. The computer may have a processor, and the processor may implement various functions and steps. The computer may be standalone. Some of the computer's functions may be distributed between a server and terminals. In this case, it is preferable that the server and terminal can exchange information via a network such as the internet or an intranet. The computer may include a processor and memory connected to the processor. The memory may store instructions, and when these instructions are executed by the processor, they may cause the computer to perform various processes and function as various components. The computer may build a learning model by providing various training data and perform various calculations using machine learning. In this case, the computer may perform various analyses and interpretations using the learning model created by AI (artificial intelligence) machine learning and deep learning. 【0018】As described above, the present invention may implement a remote control system using a computer. In other words, the present invention also provides a method for controlling remote operation by a computer, and a remote control system for a computer. Furthermore, the present invention also provides a program that causes a computer to execute a method for controlling remote operation, and a non-temporary information recording medium that stores such a program. Examples of non-temporary information recording media include DVDs, CDs, hard disks, USB memory, and SD cards. 【0019】 The operation information output device 11 generates operation-related data packets based on the operation information and stores them in the storage unit as appropriate, in order to output operation information to the remote operation device 9. Specifically, the operation information output device 11 divides the operation information into packets of a certain size and processes them to be sent to the remote operation device 9 so that the remote operation device 9 can restore them. Operation-related data packets are data packets for communicating operation information. A packet is a small unit that is divided when data is sent and received over a network. A packet includes, for example, a header and a data portion (Payload). Some packets also have a footer. The header usually includes the source address, destination address, protocol information (UDP), and packet number. 【0020】 Figure 2 is a conceptual diagram illustrating an example of an operation-related data packet. An operation-related data packet may have a different configuration than this example. For example, in Figure 2, the destination address may be omitted. 【0021】The operation information output device 11 processes the UDP header of the operation-related data packet to include a time-sequence number, which is a number assigned in chronological order as checksum information. As a result, the operation-related data packet will have a time-sequence number in its header (UDP header). In this invention, for example, the header may include a time-sequence number in addition to the packet number. Alternatively, the header may include a time-sequence number instead of the packet number. The operation information output device 11 outputs an operation-related data packet having such a header. The output operation-related data packet is transmitted to the remote operation device 9 via the communication channel. 【0022】 It is preferable that the operation information output device 11 continues to output UDP-based data packets including a time sequence number to the remote operation device 9, even when there is no operation information. This makes it easy to synchronize the time of the data packets. Furthermore, this allows the practitioner to detect attacks by third parties even when they are not performing a procedure. This enables the practitioner to take necessary countermeasures. 【0023】 The remote control device 9 receives operation-related data packets. The remote control device 9 then analyzes the received operation-related data packets, decodes the operation information, and operates the manipulator 5 based on the operation information. For example, the remote control device 9 performs a procedure on a patient using the manipulator 5. At this time, the remote control device 9 decodes the operation-related data packets using an encryption key shared with the operation information output device 11. The remote control device 9 stores the order in which the operation-related data packets were received in its memory unit. The remote control device 9 then stores the order in which the operation-related data packets were received in its memory unit in association with a time sequence number. 【0024】Figure 3 is a flowchart illustrating an example of manipulator control related to time sequence numbers. When the remote operating device 9 receives a new operation-related data packet (S101), it stores the reception order Rn and the time sequence number Tn in the storage unit. Then, it reads the time sequence number Tm of the operation-related data packet received immediately before from the storage unit using the reception order Rm (S102). The remote operating device 9 performs a calculation comparing the time sequence number Tn and the time sequence number Tm (S103). If the time sequence number Tn is greater than the time sequence number Tm, the remote operating device 9 processes the manipulator 5 to continue operation (S104). If the time sequence number Tn is less than (not greater than) the time sequence number Tm, the remote operating device 9 controls the operation of the manipulator 5 to stop (S105). The remote operating device 9 may also output information regarding the stopping of the operation of the manipulator 5 to the operation information output device 11. 【0025】 Furthermore, the remote control device 9 may associate the time sequence number contained in the decoded UDP header with the divided operation information and store it in the memory unit. When the remote control device 9 receives multiple operation-related data packets, it reads the time sequence number and the divided operation information from the memory unit. The remote control device 9 may then analyze the order relationship between the time sequence number and the divided operation information. If the time sequence numbers for multiple operation-related data packets are reversed, it may indicate that an attack by a third party has occurred. Therefore, when the remote control device 9 detects that the time sequence numbers for multiple operation-related data packets have been reversed, it creates a command to stop the operation of the manipulator 5 and controls the manipulator 5 to stop its operation. On the other hand, the remote control device 9 may output information regarding the cessation of the operation of the manipulator 5 to the operation information output device 11. 【0026】The remote operation device 9 outputs an alert to the operation information output device 11 when the time interval between receiving multiple operation-related data packets is greater than or equal to a predetermined threshold. Figure 4 is a flowchart showing an example of the process of outputting an alert regarding the time of receiving operation-related data packets. The remote operation device 9 stores a threshold for outputting this alert in its storage unit. When the remote operation device 9 receives an operation-related data packet (S201), it stores the time ty of the time the operation-related data packet was received in its storage unit. Following the process of decoding the operation-related data packets described earlier, the time sequence number and the divided operation information are decoded. Then, the time the packets were received can be read out in the order of the time sequence numbers. The remote operation device 9 reads out the time tx of the previous operation-related data packet from the storage unit (S202). The remote operation device 9 performs a calculation to find the difference in the time the packets were received before and after the time sequence number (S203). In this way, the remote operation device 9 can find the time interval Δt between receiving operation-related data packets. Then, the remote operation device 9 reads out the threshold tt from the storage unit. The remote operating device 9 performs a calculation comparing the calculated time interval with a predetermined threshold (S204). As a result, if the time interval for receiving related data packets with consecutive time sequence numbers is greater than or equal to the predetermined threshold, it outputs an alert to the operation information output device 11 (S205). The alert is transmitted to the operation information output device 11 via the communication path described earlier. The operation information output device 11 then performs a process to display the alert on the display unit based on the received alert. In this way, the operator (e.g., a doctor) operating the operation information output device 11 can grasp that some kind of abnormality has occurred. If the time interval for receiving related data packets with consecutive time sequence numbers is less than the predetermined threshold, the device may proceed with the operation of the manipulator 5 (S206). 【0027】 Furthermore, if the time interval between receiving related data packets with consecutive time sequence numbers is greater than or equal to a predetermined threshold, the remote operating device 9 may create a command to stop the operation of the manipulator 5 and control the manipulator 5 to stop its operation. 【0028】 The remote operation device 9 may further include an image capture unit 7 for capturing images and an image output unit 13 for outputting image information, which is information about the images captured by the image capture unit 7, to the operation information output device 11. The image capture unit 7 may be any known image capture unit used in remote control systems. The image capture unit 7 generally captures the area being treated, the affected area, or the area including the manipulator 5. The images captured by the image capture unit 7 are usually images related to the treatment (for example, images of the affected area or the area being treated). The remote operation device 9 performs the same processing as the operation information output device 11 described above and outputs image-related data packets to the operation information output device 11. 【0029】 Image-related data packets are data packets for communicating image information. The UDP header in an image-related data packet includes a second chronological number, which is a number assigned in chronological order as checksum information. This process is the same as described earlier. The operation information output device 11 receives multiple image-related data packets. When the second chronological number is reversed, it outputs a control signal to the remote operation device 9 to control the manipulator 5 so that it stops operating the manipulator 5. Furthermore, if the time interval between receiving multiple image-related data packets is greater than or equal to a predetermined threshold, it outputs a second alert. The second alert may be displayed, for example, on the display unit of the operation information output device 11. These processes are the same as the manipulator control regarding the chronological number described earlier, and the process of outputting alerts regarding the time of receiving operation-related data packets described earlier. 【0030】The operation information output device 11 and the remote operation device 9 may share a standard time using synchronized clocks. The remote operation device 9 may control the manipulator 5 to stop operation if it receives multiple operation-related data packets and the interval between the receipt times of operation-related data packets corresponding to the time sequence number exceeds a predetermined data transmission delay time determined from the time sequence number. This process can be performed in the same manner as described above. In addition, the communication status between the operation information output device 11 and the remote operation device 9 may be analyzed using the time sequence number, the second time sequence number, and the standard time, and an alert regarding the communication status may be output to the operator according to the analyzed communication status. 【0031】 Up to this point, we have primarily described examples where the UDP header contains a time sequence number, timestamp, packet number, etc. In this example, the timing of detecting malicious activity can be accelerated. However, this invention also allows this information to be contained within the payload. 【0032】 Figure 5 is a conceptual diagram illustrating an example of an operation-related data packet in which the payload contains a time sequence number, time, etc. In this example, an application data header is provided in the payload portion (UDP data octet) following the UDP header. Authentication information, such as a time sequence number and time, is then added to this header. In this case, since the authentication information is not present in the UDP header, even if malicious data exists, it may pass through Layer 4 (UDP) and the malicious activity may only be detected at Layer 7. Therefore, compared to the previous example, the detection of malicious activity may be slightly delayed. However, with this packet, by deliberately including a time sequence number, time, packet number, etc., in the payload (UDP data octet), standard UDP communication can be performed based on the RFC standard. 【0033】Figure 6 is a conceptual diagram showing an example of the display unit of the operation information output device. In this example, three levels of display are shown depending on the communication status. Of course, the display unit (monitor) may also display the alerts described earlier and information necessary for the procedure. By using the time sequence number, the second time sequence number, and the standard time, the interval between arrival times of consecutive packets to the receiver can be analyzed. By using this information on time intervals, the degree of congestion in the communication path between the operation information output device 11 and the remote operation device 9, and whether or not there is an attack by a third party can be analyzed. Alerts can be displayed, for example, on the display unit of the operation information output device 11. In this case, the operation information output device 11 may perform a process to categorize the communication status into multiple levels, calculate which of the multiple levels the current status belongs to, and display it on the display unit. In this way, even in a remote surgery control system using quantum cryptography, the status of the communication path can be shown to the operator. 【0034】 Figure 7 is a conceptual diagram showing another example of a remote control system. In this example, the operator controls the construction equipment using their terminal. Based on the operator's operation information, the operator controls the manipulator of the construction equipment. In this case, if the construction equipment handles hazardous materials, information regarding the presence or absence of hazardous materials may be transmitted to the operator's terminal. 【0035】1. To prevent erroneous operation due to manipulator delays caused by data buffering, UDP was used, and buffering was set to almost zero. 2. Data packets of the manipulator were numbered in chronological order, and while missing numbers were permitted, operation stopped if the numbers were reversed. 3. When a packet was received, if a certain amount of time had passed since the previous packet was received, an alert was output to ask the operator whether to continue. This alert was displayed on the operator's terminal monitor. 4. Time information (sequential numbering) was transmitted even when there was no or little operation. 5. Image data was also handled according to the same specifications as above. 6. Synchronized clocks were provided on both sides, and the packet transceivers managed the numbers. If a discrepancy in packet numbers exceeding the data transmission delay occurred, the manipulator's operation stopped, and the operator was asked whether to continue. 7. To make it easier for the operator (doctor) to understand the conditions described in items 1 through 6 above, a simplified score was calculated and displayed using a three-level system (blue, yellow, red) similar to a traffic light, either as a lamp or on a monitor used by the operator. 【0036】 This invention can be used in the field of medical devices. 【0037】 1 Remote control system 5 Manipulator 9 Remote operation device 11 Operation information output device

Claims

1. A remote control system (1) for controlling remote operation, comprising: a remote operation device (9) having a manipulator (5) and operating the manipulator (5) in response to instructions from an operator; and an operation information output device (11) that outputs operation information from the operator to the remote operation device (9) for operating the remote operation device (9), wherein the operation information output device (11) and the remote operation device (9) use UDP as the communication protocol and perform quantum cryptographic communication using encryption keys by quantum key distribution, wherein the data packets for communicating the operation information are called operation-related data packets, and the header of the operation-related data packets includes a time-sequence number which is a number assigned in chronological order as sequence information, and the remote operation device (9), when it receives multiple operation-related data packets, controls the manipulator (5) to stop operation when the time-sequence number is reversed, and outputs an alert to the operation information output device (11) when the time interval for receiving multiple operation-related data packets is greater than or equal to a predetermined threshold. Remote control system.

2. A remote control system according to claim 1, wherein the operation information output device (11) outputs a UDP-based data packet including the time sequence number to the remote operation device (9) even when there is no operation information.

3. A remote control system according to claim 1, wherein the remote operating device (9) further comprises an imaging unit (7) for capturing an image, and an image output unit (13) for outputting image information, which is information relating to an image captured by the imaging unit (7), to the operation information output device (11), wherein, if the data packets for communicating the image information are called image-related data packets, the header of the image-related data packets includes a second time-sequence number, which is a number assigned in chronological order as sequence information, and the operation information output device (11), upon receiving a plurality of image-related data packets, (i) outputs a control signal to the remote operating device (9) to control the manipulator (5) to stop its operation if the second time-sequence number is reversed, and (ii) outputs a second alert if the time interval between receiving a plurality of image-related data packets is greater than or equal to a predetermined threshold.

4. A remote control system according to claim 1, wherein the operation information output device (11) and the remote operation device (9) share a standard time using synchronized clocks, and the remote operation device (9) receives a plurality of operation-related data packets, and when the interval between the reception times of operation-related data packets corresponding to the time sequence number exceeds a predetermined time or more for data transmission delay time determined from the time sequence number, the remote control system controls the manipulator (5) to stop operating the manipulator (5).

5. A remote control system according to claim 4, wherein the system analyzes the communication status between the operation information output device (11) and the remote operation device (9) using the time sequence number, the second time sequence number, and the standard time, and outputs an alert regarding the communication status to the operator according to the analyzed communication status.

Images