Robot control device, method, and program for remote operation system

Abstract

In one aspect of the present invention, in a remote operation system, when a remote-side robot control device controls a catheter of a robot in accordance with various control signals transmitted from an operation-side remote control terminal to perform a prescribed treatment on a treatment target site, the remote-side robot control device recognizes the internal shape of the treatment target site, estimates a treatment direction for the treatment target site on the basis of the recognized internal shape of the treatment target site, and sets a posture including the position and orientation of the catheter with respect to the treatment target site in accordance with the estimated treatment direction. In accordance with an insertion control signal sent from the remote control terminal, the catheter is moved from the state of the posture along the treatment direction and inserted into the treatment target site. After the insertion, the catheter is operated to perform the treatment on the treatment target site in accordance with a treatment control signal sent from the remote control terminal. After the treatment is finished, the catheter is autonomously retracted along the treatment direction and is removed from the inside of the treatment target site.

Description

Robot control device, method and program for remote operation system

[0001] One aspect of the present invention relates to a robot control device, method, and program used in a remote operation system in which, for example, a medical professional remotely controls a robot to perform medical treatment on a patient.

[0002] In medical settings, patients who are unable to cough up phlegm on their own require the periodic insertion of a suction catheter into their airways to aspirate the phlegm. When performing sputum suction, a catheter is inserted into the nasal or oral cavity, but the insertion of the catheter not only causes discomfort and pain to the patient, but also affects breathing and may lead to suffocation. For this reason, sputum suction procedures must be performed quickly and smoothly, and it is considered desirable for each suction to take no more than 10 to 15 seconds. Non-Patent Document 1, for example, is known as a guide for performing suction procedures.

[0003] In an ultra-aging society, the number of patients requiring sputum suction is increasing, and it is expected that there will be a shortage of qualified personnel, including medical professionals, who can perform this procedure. Therefore, in order to enable a limited number of qualified personnel to treat a large number of patients across regions, remote control of a robot to perform sputum suction is being considered. This type of system is realized, for example, by attaching a catheter to the tip of the arm of a robot device, and a qualified person operating the catheter with a controller while watching the video from a camera.

[0004] FY2012 Sputum Suction Instructor Training Program, "Sputum Suction Training Textbook (Third Training)," Ministry of Health, Labour and Welfare, Internet <URL: https: / / www.mhlw.go.jp / seisakunitsuite / bunya / hukushi_kaigo / shougaishahukushi / kaigosyokuin / >

[0005] However, when performing suction procedures using a remotely operated robot, medical professionals must operate the catheter while viewing the patient's video, inevitably encountering transmission delays due to the network and equipment. This makes it difficult to gauge the distance between the patient and the catheter, and can lead to misperceptions of the catheter's position, compared to when performing procedures while holding the catheter directly. Therefore, careful operation is required, which inevitably prolongs the procedure. Additionally, since the patient is awake during the procedure, the affected area may move. Therefore, considerable skill is required to complete the procedure quickly without injuring the patient.

[0006] The present invention has been made in light of the above-mentioned circumstances, and aims to provide a technique that enables treatment using a catheter to be performed by remote control with high accuracy in a short time.

[0007] In order to solve the above problems, one aspect of the present invention is a remote operation system in which a remote control terminal located on an operating side and a robot control device located on a remote side are connected via a network, and when the robot control device controls a catheter of a robot in accordance with various control signals sent from the remote control terminal to perform a predetermined treatment on a treatment target site, the robot control device recognizes the internal shape of the treatment target site, estimates a treatment direction relative to the treatment target site based on the recognized internal shape of the treatment target site, and sets a posture of the catheter including its position and orientation relative to the treatment target site based on the estimated treatment direction.The catheter is then inserted into the treatment target site by moving it from the posture along the treatment direction in accordance with an insertion control signal sent from the remote control terminal, and after insertion, the catheter is operated in accordance with a treatment control signal sent from the remote control terminal to perform the treatment on the treatment target site, and after the treatment is completed, the catheter is retracted along the treatment direction to be removed from the treatment target site.

[0008] That is, according to one aspect of the present invention, it is possible to provide a technique that enables treatment using a catheter to be performed by remote control with high accuracy in a short time.

[0009] FIG. 1 is a diagram showing an example of the configuration of a remote operation system according to an embodiment of the present invention. FIG. 2 is a block diagram showing an example of the hardware configuration of a remote control terminal provided on the operating side in the system shown in FIG. 1. FIG. 3 is a block diagram showing an example of the software configuration of the remote control terminal provided on the operating side in the system shown in FIG. 1. FIG. 4 is a block diagram showing an example of the hardware configuration of a robot control device provided on the remote side in the system shown in FIG. 1. FIG. 5 is a block diagram showing an example of the software configuration of the robot control device provided on the remote side in the system shown in FIG. 1. FIG. 6 is a flowchart showing an example of the procedure and content of control processing related to robot remote operation performed by the remote control terminal shown in FIG. 3 and the robot control device shown in FIG. 5. FIG. 7 is a flowchart showing an example of the procedure and content of robot remote control processing performed by the remote control terminal in FIG. 6 and robot control processing performed by the robot control device. FIG. 8 is a diagram for explaining an example of processing for setting the attitude of a catheter relative to a nostril. FIG. 9 is a diagram for explaining the insertion direction of a catheter relative to a nostril. FIG. 10 is a diagram for explaining the removal direction of a catheter from a nostril.

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] [One Embodiment] (Configuration Example) (1) System FIG. 1 is a diagram showing an example of the configuration of a remote control system according to one embodiment of the present invention.

[0012] The remote control system of one embodiment enables data transmission via a network NW between an operating side A, such as a hospital where an operator US, such as a medical professional, is present, and a remote side B, such as a home or facility where a patient PT is present.

[0013] A remote control terminal UT is placed on the operating side A, and an output device OU including a display device and an input device IN functioning as a controller are connected to this remote control terminal UT. The operator US operates the input device IN while watching the image of the patient PT displayed on the display device to remotely control the robot on the remote side B.

[0014] At the remote side B, there are installed a suction device AS that performs the suction operation of phlegm, a robot RB that performs the suction operation of the phlegm of the patient PT on behalf of the operator US, a robot control device RT that controls the robot RB, and a camera CM. A catheter CT is attached to the tip of the arm of the robot RB, and the base end of the catheter CT is connected to the suction device AS via a suction tube AT. The camera CM is, for example, an RGB-D camera (depth camera), and captures an image of a predetermined range including the patient's face.

[0015] The network NW is composed of a wide area network such as the Internet and an access network for accessing the wide area network. The access network may be a wired or wireless local area network (LAN), an optical transmission network, or a mobile communication network that adopts the 5G standard.

[0016] (2) Remote Control Terminal UT FIGS. 2 and 3 are block diagrams showing an example of the hardware configuration and software configuration of the remote control terminal UT, respectively.

[0017] The remote control terminal UT is, for example, a personal computer, and has a control unit 1A that uses a hardware processor such as a central processing unit (CPU).To this control unit 1A, a memory unit having a program memory unit 2A and a data memory unit 3A, an input / output interface (hereinafter, the interface will be abbreviated as I / F) unit 4A, and a communication I / F unit 5A are connected via a bus 6A.

[0018] The input / output I / F unit 4A is connected to the output device OU and the input device IN. The input / output I / F unit 4A outputs video data transmitted from the remote side B to the output device OU, and receives an operation signal representing the operation content of the operator US from the input device IN.

[0019] The program storage unit 2A is, for example, a combination of a non-volatile memory such as a HDD (Hard Disk Drive) or SSD (Solid State Drive) as a storage medium that can be written to and read from at any time, and a non-volatile memory such as a ROM (Read Only Memory), and stores application programs necessary to execute various processes related to one embodiment of the present invention, in addition to middleware such as an OS (Operating System).

[0020] The data storage unit 3A is, for example, a combination of a non-volatile memory such as an HDD or SSD as a storage medium that can be written to and read at any time, and a volatile memory such as a RAM (Random Access Memory), and has a storage area used to temporarily store data when displaying video data or transmitting various control signals for remote operation.

[0021] The control unit 1A includes a video data receiving and displaying processing unit 11A, a catheter insertion control signal transmission processing unit 12A, a suction control signal transmission processing unit 13A, and a catheter removal control signal transmission processing unit 14A as control functions necessary to realize one embodiment of the present invention.

[0022] Each of the processing units 11A to 14A is realized by causing the processor of the control unit 1A to execute an application program stored in the program storage unit 2A. Note that some or all of the processing units 11A to 14A may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0023] The video data receiving and display processing unit 11A receives video data of the patient's face transmitted from the robot control device RT on the remote side B via the communication I / F unit 5A, and outputs the received video data from the input / output I / F unit 4A to the output device OU to display it on the display device.

[0024] When the operator US performs remote operation on the input device IN to insert the catheter CT into the nasal cavity or oral cavity, the catheter insertion control signal transmission processing unit 12A generates a catheter insertion control signal representing the amount of operation and the operation speed, and transmits the generated catheter insertion control signal from the communication I / F unit 5A to the robot control device RT.

[0025] When the operator US performs a remote operation to suction phlegm using the input device IN while the catheter is inserted into the nasal cavity or oral cavity, the suction control signal transmission processing unit 13A generates a suction control signal to cause the suction device AS to perform a suction operation, and transmits the generated suction control signal from the communication I / F unit 5A to the robot control device RT.

[0026] When the catheter is removed from the nasal cavity or oral cavity after suction is completed and the operator US operates the input device IN to move the catheter outside the treatment area away from the patient PT, the catheter detachment control signal transmission processing unit 14A generates a catheter detachment control signal and transmits the generated catheter detachment control signal from the communication I / F unit 5A to the robot control device RT.

[0027] (3) Robot Controller RT FIGS. 4 and 5 are block diagrams showing an example of the hardware configuration and software configuration of the robot controller RT, respectively.

[0028] The robot control device RT is composed of, for example, a personal computer attached to the robot RB, and is configured by connecting a processor 1B constituting a central control unit (CPU) with a memory unit having a program memory unit 2B and a data memory unit 3B, a sensor I / F unit 4B, a communication I / F unit 5B, and a robot I / F unit 7B via a bus 6B.

[0029] The sensor I / F unit 4B captures video data output from the camera CM. The communication I / F unit 5B transmits and receives video data and various control signals to and from the remote control terminal UT via the network NW. The robot I / F unit 7B outputs control signals to the robot RB and the suction device AS.

[0030] The control unit 1B includes, as control functions for realizing one embodiment of the present invention, a video data acquisition and transmission processing unit 11B, an axial direction estimation processing unit 12B, a catheter posture setting processing unit 13B, a catheter insertion operation control unit 14B, a suction operation control unit 15B, and a catheter removal operation control unit 16B.

[0031] Each of the processing units 11B to 16B is realized by causing the processor of the control unit 1B to execute an application program stored in the program storage unit 2B. Note that some or all of the processing units 11B to 16B may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC.

[0032] The video data acquisition and transmission processing unit 11B acquires video data of the patient PT's face captured by the camera CM via the sensor I / F unit 4B, and transmits the video data from the communication I / F unit 5B to the remote control terminal UT of the operating side A.

[0033] The axis direction estimation processing unit 12B recognizes the position and internal shape of the nasal cavity or oral cavity of the patient PT based on the video data, and estimates the axis direction corresponding to the internal shape of the nasal cavity or oral cavity based on the recognition result.

[0034] The catheter attitude setting processing unit 13B sets the attitude including the position and direction of the catheter CT so that the insertion / removal direction of the catheter CT coincides with the axial direction estimated by the axial direction estimation processing unit 12B.

[0035] The catheter insertion operation control unit 14B sets the operation mode to a remote operation mode, and executes insertion control of the catheter CT in response to remote operation by the operator US.

[0036] The catheter insertion operation control unit 14B receives a catheter insertion control signal transmitted from the remote control terminal UT of the operating side A via the communication I / F unit 5B, and moves the arm of the robot RB in accordance with the received catheter insertion control signal, thereby inserting the catheter CT into the nasal cavity or oral cavity along the axial direction set by the catheter attitude setting processing unit 13B.

[0037] The suction operation control unit 15B receives a suction control signal transmitted from the remote control terminal UT on the operating side A via the communication I / F unit 5B, and operates the suction device AS in accordance with the received suction control signal to suction phlegm.

[0038] The catheter removal operation controller 16B sets the operation mode to the automatic operation mode and autonomously performs the operation of removing the catheter CT. That is, when the catheter removal operation controller 16B receives a catheter removal start signal from the remote control terminal UT of the operating side A, it operates the arm of the robot RB to remove the catheter CT at a constant speed along the axial direction set with respect to the nasal cavity or oral cavity.

[0039] In addition, when the catheter CT has been removed, the catheter removal operation control unit 16B cancels the automatic operation mode, resets the operation mode to the remote operation mode, and thereafter controls the catheter CT to move away and retreat outside the patient's treatment area in accordance with the remote operation of the operator US.

[0040] That is, the catheter removal operation control unit 16B stops the catheter CT once it has been removed from the nasal or oral cavity and the tip of the catheter CT is a certain distance away from the entrance of the nasal or oral cavity, and then, in accordance with a catheter removal control signal transmitted from the remote control terminal UT of the operating side A, moves the arm of the robot RB to remove and retract the catheter CT outside the treatment area for the patient PT.

[0041] (Example of Operation) Next, an example of operation of the remote control system configured as above will be described.

[0042] FIG. 6 is a flowchart showing an example of the processing procedure and processing contents of a series of processes by remote control, which are respectively executed by the control unit 1A of the remote control terminal UT and the control unit 1B of the robot control device RT.

[0043] (1) Transmission of video data of patient PT When the operator US performs a control start operation on the input device IN of the remote control terminal UT, the control unit 1A of the remote control terminal UT detects the control start operation in step S10, generates a control start signal, and transmits it from the communication I / F unit 5A to the robot control device RT.

[0044] In response to this, when the control unit 1B of the robot control device RT receives the control start signal from the remote control terminal UT in step S11, under the control of the video data acquisition and transmission processing unit 11B, in step S12 it acquires video data including the patient's face captured by the camera CM via the sensor I / F unit 4B, and then transmits the acquired video data from the communication I / F unit 5B to the remote control terminal UT in step S13.

[0045] In step S14, the control unit 1A of the remote control terminal UT receives the video data transmitted from the robot control device RT via the communication I / F unit 5A under the control of the video data receiving and display processing unit 11A. Then, in step S15, the received video data is output from the input / output I / F unit 4A to the output device OU and displayed on the display device.

[0046] Thus, the operator US can then perform sputum suction treatment by remote control while remaining at the operating side A and checking the state of the patient PT at the remote side B in real time using the above-mentioned video.

[0047] The above-described transmission process of the video data of the patient PT from the robot control device RT to the remote control terminal UT is continuously performed until the operator US performs a control end operation on the remote control terminal UT, the control end signal is detected in step S18 and sent to the robot control device RT, and the control unit 1B of the robot control device RT receives the control end signal in step S19.

[0048] (2) Remote control of robot RB As described above, while the image of the patient PT is being displayed in real time on the output device OU of the operating side A, the remote control terminal UT and the robot control device RT execute a series of processes related to the remote sputum suction treatment in steps S16 and S17 as follows.

[0049] FIG. 7 is a flowchart showing an example of the processing procedure and processing contents of the robot remote control processing and robot control processing executed by the control unit 1A of the remote control terminal UT and the control unit 1B of the robot control device RT, respectively.

[0050] (2-1) Setting the Posture of the Catheter CT First, in step S20, the control unit 1B of the robot control device RT, under the control of the axis direction estimation processing unit 12B, estimates the axis direction based on the internal shape of the nasal cavity or oral cavity of the patient PT as follows.

[0051] That is, the axis direction estimation processing unit 12B recognizes the position and internal shape of the nasal cavity or oral cavity of the patient PT from the video data acquired by the video data acquisition and transmission processing unit 11B. Specifically, an image of the nose or mouth is first recognized from the facial image using, for example, a pattern recognition technique, and then an image of the nasal cavity or oral cavity is identified from the recognized image of the nose or mouth and its position and internal shape are recognized. Then, the axis direction estimation processing unit 12B estimates the axis direction based on the recognized internal shape of the nasal cavity or oral cavity.

[0052] Next, in step S21, the control unit 1B of the robot control device RT sets the posture, including the position and direction, of the catheter CT relative to the nasal cavity or oral cavity of the patient PT under the control of the catheter posture setting processing unit 13B.

[0053] For example, the catheter attitude setting processing unit 13B sets an attitude representing the position and orientation of the catheter CT relative to the nasal cavity or oral cavity by operating the arm of the robot RB so that the insertion / removal direction of the catheter CT coincides with the axial direction of the nasal cavity or oral cavity estimated by the axial direction estimation processing unit 12B.

[0054] 8 shows an example of the orientation of the catheter CT set by the above process. This example shows a case where the orientation of the catheter CT is set with respect to the nasal cavity NT of the patient PT. In this figure, AX indicates the axial direction estimated based on the internal shape of the nasal cavity NT.

[0055] Furthermore, since the patient PT's face may move when awake, the above-mentioned axial direction estimation and the catheter CT posture setting process based on this are performed while following the facial movement based on the video data obtained in real time.

[0056] (2-2) Insertion Control of Catheter CT The insertion operation of the catheter CT into the nasal cavity or oral cavity is performed as follows under the remote control of the operator US.

[0057] That is, the operator US operates the input device IN while viewing the image of the patient PT in order to insert the catheter CT into the nasal cavity of the patient PT. The control unit 1A of the remote control terminal UT, under the control of the catheter insertion control signal transmission processing unit 12A, detects the speed and amount of the above operation at a predetermined sampling period in step S22.

[0058] Then, in step S23, the catheter insertion control signal transmission processing unit 12A generates a catheter insertion control signal representing the speed and amount of the detected operation, and transmits the generated catheter insertion control signal from the communication I / F unit 5A to the robot control device RT.

[0059] In response to this, upon receiving the catheter insertion control signal in step S24, the control unit 1B of the robot control device RT sets the remote operation mode. Then, under the control of the catheter insertion operation control unit 14B, the control unit 1B executes control to insert the catheter CT into the nasal cavity or oral cavity of the patient PT in accordance with the remote operation of the operator US in step S25 as follows.

[0060] FIG. 9 is a diagram showing a schematic structure of the nasal cavity NC to explain an example of the insertion operation of the catheter CT.

[0061] That is, the catheter insertion control unit 14B inserts the catheter CT from the initially set posture into the nasal cavity NC at the insertion speed and amount indicated by the catheter insertion control signal. At this time, the catheter insertion control unit 14B automatically controls the insertion direction Min of the catheter CT so that it always follows the axial direction AX.

[0062] Therefore, the operator US can insert the catheter CT by simply adjusting the insertion speed and amount, without manually adjusting the insertion direction of the catheter CT.

[0063] In controlling the insertion of the catheter CT, the catheter insertion control unit 14B may control the insertion speed of the catheter CT so that it slows as the tip of the catheter CT enters deeper into the nasal cavity or oral cavity. This speed control can be achieved, for example, by decreasing the operation scale of the input device IN, i.e., the ratio of the movement amount of the catheter CT to the unit operation amount of the operation stick of the input device IN, as the position of the tip of the catheter CT enters deeper into the nasal cavity or oral cavity. The operation scale control may also be performed by the catheter insertion control signal transmission processing unit 12A of the remote control terminal UT.

[0064] By performing the above-described control, it becomes possible to more accurately and stably align the tip of the catheter C with the phlegm accumulation site in the nasal cavity or oral cavity.

[0065] (2-3) Sputum Suction Control Suppose that the tip of the catheter CT reaches a sputum accumulation site and the operator US performs a suction operation on the input device IN. When the control unit 1A of the remote control terminal UT detects the suction operation in step S26, it generates a suction control signal instructing the execution of a suction operation under the control of the suction control signal transmission processing unit 13A in step S27 and transmits the generated suction control signal from the communication I / F unit 5A to the robot control device RT.

[0066] In response to this, upon receiving the suction control signal in step S28, the control unit 1B of the robot control device RT, under the control of the suction operation control unit 15B, outputs a suction start command to the suction device AS in accordance with the suction control signal in step S29. As a result, the suction device AS operates, and sputum remaining in the nasal cavity or oral cavity of the patient PT is sucked from the tip of the catheter CT and collected in the tank of the suction device AS.

[0067] It is also possible to provide a valve in the catheter CT or suction tube of the robot RB, and have the robot control device RT control the opening and closing of the valve in response to the suction control signal, thereby performing the sputum suction operation.

[0068] (2-4) Removal Control of Catheter CT Suppose that the suction of phlegm is completed and the operator US issues a command to remove the catheter CT using the input device IN. In this case, the control unit 1B of the remote control terminal UT detects the above-mentioned removal command operation in step S30 and transmits a catheter removal command signal from the communication I / F unit 5A to the robot control device RT.

[0069] On the other hand, when the control unit 1B of the robot control device RT receives the catheter removal instruction signal in step S32, it sets the automatic operation mode. Then, under the control of the catheter removal operation control unit 16B, in step S33, the control unit 1B executes automatic removal control of the catheter CT as follows.

[0070] That is, the catheter removal operation controller 16B operates the arm of the robot RB to retract the catheter CT within the nasal cavity or oral cavity at a constant speed and remove it from the nasal cavity or oral cavity. At this time, the retraction path of the catheter CT within the nasal cavity or oral cavity is set to follow the axial direction estimated from the internal shape of the nasal cavity NC, as shown by Mout1 in Fig. 10. Note that the retraction path of the catheter CT may also be set to follow the inner wall of the nasal cavity, as shown by Mout2 in Fig. 10.

[0071] In general, it is desirable to withdraw the inserted catheter CT along the same path as the insertion path. Focusing on this point, in one embodiment of the robot control device RT, the catheter CT is automatically withdrawn at a constant speed along the same path as the insertion path under the control of the catheter withdrawal operation control unit 16B. This allows the catheter CT to be withdrawn smoothly in a short time.

[0072] (2-5) Removal of the catheter CT from the treatment area When the tip of the catheter CT is removed from the nasal cavity or oral cavity of the patient PT by the above-mentioned catheter CT removal control, the catheter removal operation control unit 16B stops the catheter CT and cancels the automatic operation mode.

[0073] In this state, when the operator US operates the input device IN to detach the catheter CT from the patient PT, the control unit 1A of the remote control terminal UT detects the detachment operation in step S34. Then, under the control of the catheter detachment control signal transmission processing unit 14A, in step S35, a catheter detachment control signal is generated and transmitted from the communication I / F unit 5A to the robot control device RT.

[0074] In response to this, upon receiving the catheter detachment control signal in step S36, the control unit 1B of the robot control device RT sets the remote operation mode. Then, under the control of the catheter removal operation control unit 16B, the control unit 1B operates the arm of the robot RB in accordance with the received catheter detachment control signal in step S37, and removes the catheter CT to a position sufficiently away from the treatment area for the patient PT.

[0075] (Effects) As described above, in one embodiment, the robot control device RT on the remote side B recognizes the internal shape of the nasal cavity or oral cavity based on video data capturing an image of the patient PT's face, estimates the axial direction relative to the nasal cavity or oral cavity based on this internal shape, and sets the orientation of the catheter CT so that the insertion / removal direction coincides with this axial direction. In this state, the robot control device RT sets the remote operation mode, and when the operator US on the operating side A performs a remote insertion operation of the catheter CT, the robot control device RT moves the catheter CT along the axial direction while maintaining the orientation, according to the operation control signal, and inserts it into the nasal cavity or oral cavity. After the sputum suction treatment is completed, the robot control device RT sets the automatic operation mode, retracts the catheter CT along the axial direction, and removes it from the nasal cavity or oral cavity. Then, the operation mode is returned to the remote operation mode, and the catheter CT is retracted from the patient PT to outside the treatment area under the remote operation of the operator US.

[0076] Therefore, since the catheter CT is inserted into the nasal cavity or oral cavity by remote control by the operator US, even if the internal shape of the nasal cavity or oral cavity and the location where phlegm accumulates differ for each patient PT, the catheter CT can be inserted appropriately for each patient PT according to the manual operation of the operator US.

[0077] Meanwhile, the insertion direction of the catheter CT into the nasal or oral cavity is automatically controlled so that it follows the axial direction of the nasal or oral cavity. Therefore, the operator US can insert the catheter CT by simply adjusting the insertion speed and amount, without manually adjusting the insertion direction of the catheter CT. This prevents the tip of the catheter CT from strongly contacting or getting caught on the inner wall of the nasal or oral cavity during the insertion operation, allowing for a smooth insertion operation. This effect shortens the time required for the insertion operation and contributes to reducing the patient's pain and strain.

[0078] Furthermore, the catheter CT is automatically removed in the axial direction opposite to the insertion path under the control of the robot control device RT, eliminating the need for the operator US to perform any operation when removing the catheter CT, thereby reducing the operational burden on the operator US.

[0079] Furthermore, the catheter CT is removed after removal by remote control from the operator US, which prevents the catheter CT from colliding with the patient PT, nearby family members or assistants, or even objects, and allows the catheter CT to be removed safely.

[0080] [Other embodiments] (1) In both the case of insertion and removal, the process of setting the posture of the catheter CT may be set based on the axial direction estimated based on the internal shape of the nasal cavity or oral cavity, and then the operator US may be able to adjust it by turning it on and off at any timing.

[0081] (2) The speed at which the catheter CT is removed does not necessarily have to be constant. For example, the removal speed may be controlled to be slow at the beginning and to increase as the catheter is removed. In addition, the removal direction may be autonomously set by the robot control device RT, and the removal speed may be remotely controlled by the operator US.

[0082] (3) As a method for estimating the axial direction of the nasal cavity or oral cavity, for example, images of the nasal cavity or oral cavity may be taken from multiple directions using a camera CM, the internal shape of the nasal cavity or oral cavity may be recognized three-dimensionally based on the multiple images obtained, and the optimal axial direction may be estimated based on the recognition results.

[0083] (4) The automatic removal control of the catheter CT may be executed only when the operator US performs an authorization operation, for example, when a specific button is operated. Also, the operator US may perform a specific operation during the automatic removal control to cancel the automatic operation mode, and thereafter the removal control may be executed according to the operator's remote operation.

[0084] (5) The automatic removal control of the catheter CT may be performed by the robot control device RT monitoring the movements of the patient PT based on video data, and if it detects that the patient PT's face has moved more than a predetermined amount before the start of removal control or during the removal control period, the automatic removal control may be stopped and an alarm to that effect may be displayed to the operator US.

[0085] (6) In one embodiment, a case where a catheter CT is inserted into the nasal cavity or oral cavity of a patient PT to perform a procedure of suctioning phlegm has been described as an example. However, the present invention is not limited to this. For example, the present invention can be applied to cases where a catheter CT is inserted into the nasal cavity or oral cavity of a patient PT to inject nutrients, apply medicine, remove tissue, or remotely control the imaging of an affected area. Furthermore, the area to be treated is not limited to a human, but may also be an animal, a workpiece, or other object.

[0086] (7) All or part of the processing functions provided in the control unit 1A of the remote control terminal UT may be provided in the control unit 1B of the robot control device RT, and all or part of the processing functions provided in the control unit 1B of the robot control device RT may be provided in the control unit 1A of the remote control terminal UT.

[0087] Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

[0088] In short, this invention is not limited to the above-described embodiments, and in the implementation stage, the components can be modified and embodied without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

[0089] A...Operating side B...Remote side NW...Network UT...Remote control terminal IN...Input device OU...Output device US...Operator RT...Robot control device RB...Robot CT...Catheter AS...Suction device CM...Camera PT...Patient 1A, 1B...Controller 2A, 2B...Program memory 3A, 3B...Data memory 4A...Input / output I / F 4B...Sensor I / F 5A, 5B...Communication I / F 6A, 6B...Bus 7B...Robot I / F 11A...Video data receiving and display processing unit 12A...Catheter insertion control signal transmission processing unit 13A...Suction control signal transmission processing unit 14A...Catheter detachment control signal transmission processing unit 11B...Video data acquisition and transmission processing unit 12B...Axis direction estimation processing unit 13B...Catheter attitude setting processing unit 14B...Catheter insertion operation control unit 15B...Suction operation control unit 16B...Catheter removal operation control unit

Claims

1. A robot control device used in a remote operation system that connects a remote control terminal located on an operating side and a robot control device located on a remote side via a network, and in which the robot control device controls a robot's catheter to perform a predetermined treatment on a treatment target site in accordance with various control signals sent from the remote control terminal in response to operation by an operator, the robot control device comprising: a first processing unit that recognizes the internal shape of the treatment target site and estimates a treatment direction for the treatment target site based on the recognized internal shape of the treatment target site; a second processing unit that sets an attitude of the catheter including its position and orientation relative to the treatment target site based on the estimated treatment direction; a third processing unit that inserts the catheter into the treatment target site by moving the catheter from the set attitude along the treatment direction in accordance with an insertion control signal sent from the remote control terminal; a fourth processing unit that, with the catheter inserted inside the treatment target site, operates the catheter to perform the treatment on the treatment target site in accordance with a treatment control signal sent from the remote control terminal; and a fifth processing unit that, after the treatment is completed, autonomously retracts the catheter along the treatment direction to remove it from the inside of the treatment target site.

2. A robot control device as described in claim 1, further comprising a sixth processing unit that temporarily stops the catheter after the catheter has been removed, and thereafter removes the catheter from the spatial region where the treatment is performed in accordance with a removal control signal transmitted from the remote control terminal.

3. A robot control device as described in claim 1, wherein the first processing unit detects a change in the position of the treatment target area and causes the treatment direction relative to the treatment target area to follow the detected change in position.

4. A robot control device as described in claim 1, further comprising a seventh processing unit that acquires video data representing an area including the treatment target area and transmits the acquired video data to the remote control terminal for display, wherein the first processing unit extracts an image including the treatment target area from the video data, recognizes the internal shape of the treatment target area based on the extracted image, and estimates the treatment direction for the treatment target area based on the recognized internal shape of the treatment target area.

5. The robot control device according to claim 1, wherein the third processing unit controls the movement amount of the catheter relative to the unit operation amount in accordance with the insertion control signal so as to decrease as the catheter moves from the entrance to a deeper position within the treatment target area.

6. A robot control device as described in claim 1, wherein the fifth processing unit controls the speed at which the catheter is autonomously retracted along the treatment direction so as to increase as the catheter moves from deep inside the treatment target area to the entrance.

7. A robot control method in a remote operation system in which a remote control terminal located on an operating side and a robot control device located on a remote side are connected via a network, in which the robot control device controls a catheter of a robot in accordance with various control signals sent from the remote control terminal to perform a predetermined treatment on a treatment target site, the robot control method comprising the steps of: recognizing the internal shape of the treatment target site and estimating a treatment direction for the treatment target site based on the recognized internal shape of the treatment target site; setting an attitude including the position and orientation of the catheter with respect to the treatment target site based on the estimated treatment direction; inserting the catheter into the treatment target site by moving it from the set attitude along the treatment direction in accordance with an insertion control signal sent from the remote control terminal; operating the catheter to perform the treatment on the treatment target site in accordance with a treatment control signal sent from the remote control terminal; and, after the treatment is completed, autonomously retracting the catheter along the treatment direction to remove it from inside the treatment target site.

8. A program that causes a processor included in a robot control device according to any one of claims 1 to 6 to execute the processing executed by a processing unit included in said robot control device.

Images