Remote medical support system, remote medical support method, and program

Abstract

To more appropriately support diagnosis when performing diagnosis accompanied by a medical practice remotely.SOLUTION: A remote medical support system 1 includes a work mechanism 25, an operation mechanism 15, a movement control unit 313, and a presentation unit 314. The work mechanism 25 is a mechanism for performing a medical practice including palpation to a subject. The operation mechanism 15 is a mechanism for receiving a remote operation to the work mechanism 25 from an operator. The movement control unit 313 controls the movement of the work mechanism 25 in response to the remote operation received by the operation mechanism 15, and transmits reaction force given to the work mechanism 25 in response to the touch of the work mechanism 25 onto the subject to the operator as a force tactile sense through the operation mechanism 15. The presentation unit 314 presents support information for supporting diagnosis based on the medical practice acquired on a work mechanism side to the operator.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to a remote medical support system, a remote medical support method, and a program. [Background technology]

[0002] In the medical field, subjects suspected of having a disease usually go to a medical facility themselves to be diagnosed by a medical professional such as a doctor. In this case, the medical professional such as a doctor diagnoses the subject from various viewpoints such as palpation, percussion, visual inspection, and auscultation. However, depending on the congestion of medical facilities and other circumstances, it may not be possible to receive a diagnosis immediately.In addition, when receiving a diagnosis during emergency transport, when receiving a diagnosis during home medical care, or when performing triage in an emergency, it is difficult to receive a diagnosis from an experienced medical professional on the spot. For example, proper diagnosis using medical equipment that uses ultrasound (US: Ultrasonography) depends in part on the experience of medical personnel, and considering the human resources of medical personnel, it is not realistic to have such experienced personnel always on-site and able to make diagnoses at any time.

[0003] Considering such a situation, an example of a technology for supporting medical procedures by remote control is disclosed in Patent Document 1. The technology disclosed in Patent Document 1 is said to support the work of medical personnel by controlling the behavior of the remotely controlled robot arm so that it does not move at an excessive speed even if the operator operates the robot at an excessive speed. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2023-034696 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, medical procedures require high operational precision, and it is considered difficult to achieve the required operational precision by simply controlling the behavior of the robot arm so that it does not move at an excessive speed, as in the technology disclosed in Patent Document 1. In other words, there is still room for improvement in enabling medical professionals to remotely perform diagnoses from various perspectives, such as palpation, percussion, visual inspection, and auscultation.

[0006] The present invention has been made in view of the above circumstances, and an object of the present invention is to more appropriately support diagnosis when a diagnosis involving a medical procedure is performed remotely. [Means for solving the problem]

[0007] In order to solve the above problems, a remote medical support system according to an embodiment of the present invention comprises: A working mechanism for performing a medical procedure including palpation on a subject; an operation mechanism that receives remote operation of the working mechanism from an operator; an operation control means for controlling an operation of the working mechanism in response to a remote operation received by the operating mechanism, and for transmitting a reaction force applied to the working mechanism in response to the working mechanism coming into contact with the subject, to the operator via the operating mechanism, as a haptic sensation; an assistance information presenting means for presenting assistance information, acquired by the operating mechanism, for assisting a diagnosis based on the medical practice to the operator; The present invention is characterized by comprising: Effect of the Invention

[0008] According to the present invention, when a diagnosis involving a medical procedure is performed remotely, the diagnosis can be more appropriately supported. [Brief description of the drawings]

[0009] [Figure 1] 1 is a schematic diagram showing the overall configuration of a remote medical support system 1 according to the present embodiment. [Diagram 2] 1 is a block diagram showing the hardware configuration of an operation device 10 and a working device 20. FIG. [Diagram 3] 1 is a perspective view showing a configuration example of an operating device 10 and a working device 20. FIG. [Figure 4] A block diagram showing a hardware configuration 700, which is an example of the hardware configuration of the main control device 30 and the sub-control device 40. [Diagram 5] A block diagram showing the functional configuration of the main control device 30. [Figure 6] 1 is a table illustrating an example of a data structure of a communication control table. [Figure 7] 13 is a block diagram showing a control algorithm of an operation control unit 313. FIG. [Figure 8] 2 is a block diagram showing the functional configuration of the sub-control device 40. FIG. [Figure 9] 13 is a flowchart illustrating the flow of an operation control process executed by the remote medical support system 1. [Figure 10] 10 is a flowchart illustrating the flow of a remote medical support process executed by the remote medical support system 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings.

[0011] [System Configuration] Fig. 1 is a schematic diagram showing the overall configuration of a remote medical support system 1 according to this embodiment. As shown in Fig. 1, the remote medical support system 1 is installed separately in an "operation side" which is, for example, a medical facility such as a hospital, and an "operation side" which is, for example, the inside of an ambulance during emergency transport. Specifically, on the operation side, the remote medical support system 1 includes an operation device 10, a main control device 30, a display 30a, and a camera 30b. On the other hand, on the work side, the remote medical support system 1 includes a work device 20, a sub-control device 40, a display 40a, and a camera 40b.

[0012] Moreover, the devices on the operation side and the work side are connected to each other so that they can communicate with each other. This communication may be either wired or wireless, and there is no limitation on the communication method. In addition, the main control device 30 and the sub-control device 40 can also remotely communicate with each other via a network N. However, this network N is a public network such as the Internet, and is not a bandwidth guaranteed network in which a certain communication speed is guaranteed, but a best effort network in which a certain communication speed is not guaranteed.

[0013] FIG. 1 also illustrates an operator on the operating side, and an assistant and a subject on the working side. The operator performs a medical procedure on the subject by remotely operating the operating device 10 to operate the working device 20. The operator is, for example, a specialist or an ultrasound examiner certified by the Japan Society of Ultrasonics who has experience in this medical procedure. The assistant is, for example, a non-specialist medical staff such as a family doctor or an emergency team who has little experience in this medical procedure. The subject is a person who is suffering from a disease (or suspected of suffering from a disease) or is injured, and is a person who undergoes this medical procedure and is the subject for whom the operator makes a diagnosis. The subject may be, for example, a pregnant woman or a person suffering from a chronic disease, who requires regular checkups and follow-up.

[0014] In this embodiment, as shown in the "upward direction," an assistant requests an operator to perform a medical procedure on a subject. In response to this request, the operator remotely performs a medical procedure on the subject, as shown in the "downward direction."

[0015] In such a remote medical support system 1, the operating device 10 operates as a master device to drive a mechanism that accepts operations from an operator. Meanwhile, the working device 20 operates as a slave device to drive a mechanism that executes a medical procedure including palpation on a subject. Here, the medical procedure including palpation on a subject includes all medical procedures such as non-invasive palpation. For example, the working device 20 directly or indirectly contacts the subject via another attached member or the like to realize such a medical procedure including palpation on a subject. Note that when the working device 20 performs such a medical procedure including palpation, a medical procedure not including palpation may or may not be performed. In this case, the main control device 30 performs control (bilateral control) to transmit haptics between the mechanism driven by the operating device 10 and the mechanism driven by the working device 20. As a result, the operation (position and force input) of the operator on the operating device 10 is transmitted to the mechanism driven by the working device 20, and the reaction force (position and force response) from the working device 20 is transmitted to the mechanism driven by the operating device 10 as haptics.

[0016] The displays 30a and 40a each have a function of displaying an image and a function as a speaker, while the cameras 30b and 40b each have a function of capturing an image and a function as a microphone. Then, the images and sounds of the operating side captured and collected by the camera 40b, or the measurement results measured by the medical procedure, etc. are played on the display 30a on the operating side via the network N. Similarly, the images and sounds of the operating side captured and collected by the camera 30b, or the measurement results measured by the medical procedure, etc. are played on the display 40a on the operating side via the network N.

[0017] This allows the operator to perform medical procedures for diagnosis based on other senses (e.g., vision and hearing) in addition to the transmitted haptic sensation. In other words, the remote medical support system 1 enables medical professionals who perform diagnoses remotely to make diagnoses from various perspectives (e.g., palpation, percussion, visual inspection, and auscultation). Therefore, the remote medical support system 1 can solve the problem of more appropriately supporting a diagnosis when a diagnosis involving a medical procedure is performed remotely. The above is an outline of this embodiment.

[0018] [Device configuration] Next, the configuration of each device included in the remote medical support system 1 will be described. FIG. 2 is a block diagram showing the hardware configuration of the operation device 10 and the working device 20. As shown in FIG. 2, the operating device 10 includes an actuator 12 for driving an operating mechanism 15, a driver 11 for driving the actuator 12, and a position sensor 13 for detecting the position of a movable part of the operating mechanism 15 that is moved by being driven by the actuator 12. The operating device 10 also includes a communication unit 14 for communicating between the driver 11 and the position sensor 13 and the main control device 30, and the operating mechanism 15.

[0019] On the other hand, the working device 20 also includes an actuator 22 for driving a working mechanism 25, a driver 21 for driving the actuator 22, and a position sensor 23 for detecting the position of a movable part of the working mechanism 25 that is moved by being driven by the actuator 22. The working device 20 also includes a communication unit 24 for communication between the driver 21 and the position sensor 23 and the sub-controller 40, and the working mechanism 25.

[0020] In this case, the position of the movable part of the operation mechanism 15 detected by the position sensor 13 is, for example, the position of a predetermined part of the movable part of the operation mechanism 15. However, instead of the position of the movable part of the operation mechanism 15, the position of a predetermined part of the operator who operates the operation mechanism 15 may be used. Moreover, the position of the movable part of the working mechanism 25 detected by the position sensor 23 is, for example, the position of a predetermined part of the movable part of the working mechanism 25. However, instead of the position of the movable part of the working mechanism 25, the position of a predetermined part where the working mechanism 25 comes into contact with the subject (for example, the position of the tip of the working mechanism 25) may be used.

[0021] In this embodiment, instead of the position of the movable part of the operation mechanism 15 and the position of the movable part of the working mechanism 25, the rotation angle of the output shaft of each actuator may be detected by a rotary encoder built into each actuator. That is, in this embodiment, the concept of position includes an angle (for example, the rotation angle of the output shaft of the actuator, etc.), and the information on the position includes a position, an angle, a velocity, an angular velocity, an acceleration, and an angular acceleration. In addition, since the position and the velocity (or the acceleration) or the angle and the angular velocity (or the angular acceleration) are parameters that can be replaced by differential and integral calculations, when processing related to the position or the angle, it is possible to replace them with the velocity or the angular velocity, etc. as appropriate before processing. In addition, in the figure, only one system of each driver, each actuator, and each position sensor is illustrated in the operation device 10 and the working device 20, but multiple systems of these can be provided according to the number of the operation mechanisms 15 and the working mechanisms 25 and the number of the respective actuators.

[0022] In such a configuration, the main control device 30 realizes bilateral control that transmits haptic sensations between the operation mechanism 15 of the operating device 10, which is the master device, and the working mechanism 25 of the working device 20, which is the slave device, by outputting control commands to the driver 11 and the driver 21 based on the positions detected by the position sensors 13 and 23. In this case, the sub-control device 40 plays a role in relaying the positions detected by the position sensors 23 and the control commands output by the main control device 30 to the driver 21. A specific algorithm for realizing control (bilateral control) for transmitting this haptic sensation will be described later with reference to FIG.

[0023] Fig. 3 is a perspective view showing an example of the configuration of the operating device 10 and the working device 20. The operating device 10 and the working device 20 are not particularly limited in shape and structure, but in this embodiment, as an example, it is assumed that the operating device 10 and the working device 20 have the same shape and structure, and only the functions of the operating mechanism 15 and the working mechanism 25 are different. Note that in Fig. 3, the hardware configuration shown in Fig. 2 (here, drivers 11, 21, actuators 12, 22, position sensors 13, 23, and communication units 14, 24) is omitted from the illustration.

[0024] The operating device 10 and the working device 20 are realized as devices having a link mechanism including multiple (three in this case) link shafts that connect a base and an end effector in order to perform a predetermined task with the end effector. As shown in the figure, this link mechanism has a three-axis structure that is movable in the X direction on a horizontal plane, the Y direction perpendicular to the X direction on the horizontal plane, and the Z direction (i.e., the vertical direction) perpendicular to both the X direction and the Y direction, and can move the end effector three-dimensionally by driving actuators 12 and 22 realized by servo motors, air cylinders, etc.

[0025] In this case, the bases of the operating device 10 and the working device 20 are fixed to the ceiling or wall of the operating side or the working side, or to a workbench, etc. In addition, an operating mechanism 15 is attached to the tip of the operating device 10 as an end effector. In this embodiment, as an example, it is assumed that this operating mechanism 15 is a dummy device configured to have a shape that imitates an ultrasonic probe. This operating mechanism 15 does not need to have the function of an ultrasonic probe, and does not need to have components such as an acoustic lens or a transducer inside. However, it is desirable to make the surface texture and weight similar to those of an actual ultrasonic probe so that remote operation can be performed with the same feeling as when using an actual ultrasonic probe.

[0026] On the other hand, a working mechanism 25 is attached to the tip of the working device 20 as an end effector. In this embodiment, as an example, it is assumed that this working mechanism 25 is an ultrasonic probe that is normally used in medical settings. The working mechanism 25 receives reflected ultrasonic waves by being moved in a predetermined direction while in contact with the body of the subject, and can generate image data based on the reflected ultrasonic waves (hereinafter referred to as an "ultrasound image"). The ultrasonic image is played back on the working side display 30a, and the operator can refer to this ultrasonic image to observe the state of the subject's organs and fetus.

[0027] The configuration shown in FIG. 3 is merely an example. For example, the operation device 10 and the working device 20 may have different structures, may be a serial link mechanism instead of a parallel link mechanism, and the number of axes is not limited. For example, they may be 6-axis with a higher degree of freedom, or 2-axis with a simpler configuration. Also, assuming a case where palpation is performed by direct contact with the hand, a multi-fingered device worn by an operator may be the operation device 10, and a robot manipulator that contacts the subject may be the working device 20. Also, the operation device 10 and the working device 20 may be realized as a configuration specific to this embodiment, or may be realized as a general-purpose robot arm or the like. Furthermore, the operation mechanism 15 and the working device 20, which are end effectors, are not limited to an ultrasonic probe, and may be other medical devices and dummy devices configured to imitate the medical devices. In this case, various medical devices may be detachable, and different medical devices (and dummy devices configured to imitate the medical devices) may be replaced as end effectors depending on the situation.

[0028] FIG. 4 is a block diagram showing a hardware configuration 700, which is an example of the hardware configuration of the main control device 30 and the sub-control device 40. As shown in FIG. 4, this hardware configuration 700 includes a processor 701, a ROM (Read Only Memory) 702, a RAM (Random Access Memory) 703, a bus 704, an input unit 705, an output unit 706, a memory unit 707, a communication unit 708, and a drive 709.

[0029] The processor 701 executes various processes according to a program recorded in a ROM 702 or a program loaded from a storage unit 707 to a RAM 703 . The RAM 703 also stores data and the like necessary for the processor 701 to execute various processes.

[0030] The processor 701, the ROM 702, and the RAM 703 are connected to one another via a bus 704. To the bus 704, an input unit 705, an output unit 706, a storage unit 707, a communication unit 708, and a drive 709 are connected.

[0031] The input unit 705 includes input devices such as a mouse and a keyboard, and accepts input of various information to the main control device 30 and the sub-control device 40. In this case, the input unit 705 of the main control device 30 also accepts input of images and sounds of the operating side captured and collected by the camera 30b. On the other hand, the input unit 705 of the sub-control device 40 also accepts input of images and sounds of the working side captured and collected by the camera 40b, as well as measurement results of the working mechanism 25 (for example, an ultrasound image showing the measurement results).

[0032] The output unit 706 is composed of a display, a speaker, etc., and outputs images and sounds. In this case, the output unit 706 of the main control device 30 outputs images, sounds, measurement results, etc. of the working side received via the network N from the display 30a. On the other hand, the output unit 706 of the sub-control device 40 outputs images, sounds, measurement results, etc. of the operating side received via the network N from the display 40a. The storage unit 707 is composed of a hard disk or a dynamic random access memory (DRAM) or the like, and stores various data managed by each server. The communication unit 708 controls communications with other devices via the network.

[0033] Removable media 750, such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, is appropriately loaded into the drive 709. A program read from the removable media 750 by the drive 709 is installed in the storage unit 707 as necessary. Note that the above hardware configuration is the basic configuration of the main control unit 30 and the sub-control unit 40, and it is possible to have a configuration that does not include some of the hardware, to have additional hardware, or to change the hardware implementation form.

[0034] The main control device 30 and the sub-control device 40 having such a hardware configuration cooperate to perform "motion control processing" and "remote medical support processing." Here, the operation control process is a series of processes that control the operation of each mechanism by performing control (bilateral control) to transmit haptic sensation between the operation mechanism 15 driven by the operating device 10 and the working mechanism 25 driven by the working device 20. The remote medical support process is a series of processes that utilize the motion control process to support the operator in remote operation in remote medical care. The motion control process is realized as a subroutine of the remote medical support process.

[0035] FIG. 5 is a block diagram showing the functional configuration of main control device 30. As shown in FIG. When the above-mentioned operation control process or remote medical support process is performed, a monitor unit 311, a communication control unit 312, an operation control unit 313, and a presentation unit 314 function in the processor 701, as shown in Fig. 5. In addition, a communication control table storage unit 371 and a history data storage unit 372 are formed in the storage unit 707. Including cases not specifically mentioned below, data required to realize processing is transmitted and received between these functional blocks at appropriate times as appropriate.

[0036] The monitoring unit 311 monitors the communication quality of the communication path used by the main control unit 30 and the sub-control unit 40 to transmit and receive data. Here, the communication path includes the network N, but as described above, this network N is a best-effort type network in which a predetermined communication speed is not guaranteed. Therefore, the communication quality in the network N is not constant. Therefore, the monitoring unit 311 monitors a value indicating the communication quality based on various indices.

[0037] For example, the monitoring unit 311 performs segment transmission and ACK reception between the main control unit 30 and the sub-control unit 40, and monitors the value of the round trip time required for these transmissions and receptions. In addition, for example, when communication is performed between the main control unit 30 and the sub-control unit 40 by including the time of data transmission in the transmission parameters of the data to be transmitted for time synchronization, the monitoring unit 311 obtains and monitors the communication delay time from the difference between the time of transmission and the time of reception included in the received data. Alternatively, the monitor 311 measures and monitors a packet loss rate value indicating the rate at which packets are lost in the network, or a jitter value indicating fluctuations in network delay. Methods for measuring values ​​indicating communication quality based on these various indices are well known to those skilled in the art, so detailed explanations will be omitted here.

[0038] The monitoring unit 311 then outputs these monitored values ​​as the monitoring results of the communication quality to the communication control unit 312. Note that which value the communication control unit 312 outputs to the communication control unit 312 as the monitoring result can be appropriately set in consideration of the characteristics of the network N to be used, etc.

[0039] The communication control unit 312 changes the communication mode (so to speak, the communication mode) for transmitting and receiving data according to the communication quality monitored by the monitoring unit 311. As a result, even in an environment where the communication quality is not constant, it is possible to support medical practice by transmitting and receiving data in various modes according to the communication quality. In this way, the criteria for differentiating the communication modes and the specific contents of the communication modes are defined in advance, for example, as a communication control table, and are stored in the communication control table storage unit 371. That is, the communication control table storage unit 371 functions as a storage unit that stores the communication control table.

[0040] 6 is a table showing an example of the data structure of the communication control table. In this table, a row is provided for each communication mode. Also, each column stores setting contents corresponding to each communication mode. In this embodiment, as an example, it is assumed that there are three communication modes, namely, the first communication mode, the second communication mode, the third communication mode, and the like. However, this is merely an example, and more communication modes may be provided in stages, or there may be only two communication modes.

[0041] The communication control unit 312 selects a communication mode according to the communication quality monitored by the monitoring unit 311, and controls the communication between the operating device 10 and the maintenance device 20 via the network N in a method according to the selected communication mode. In this case, a criterion for selecting the communication mode may be, for example, a comparison between the communication quality monitored by the monitoring unit 311 and a preset threshold value. For example, the value indicating the communication quality monitored by the monitoring unit 311 indicates a higher value as the communication quality is better. In this case, for example, if the monitored value is equal to or greater than a first threshold value ("XX" in the figure), the first communication mode is selected. Also, for example, if the monitored value is less than the first threshold value ("XX" in the figure) and equal to or greater than a second threshold value ("YY" in the figure), the second communication mode is selected. Furthermore, for example, if the monitored value is less than the second threshold value ("YY" in the figure), the third communication mode is selected. In addition, if the value indicating the communication quality monitored by the monitoring unit 311 is a value that is lower as the communication quality is better, the magnitude relationship between the value indicating the communication quality and the threshold value will be the opposite of that described above.

[0042] In this embodiment, the first communication mode is assumed to satisfy a predetermined communication quality (i.e., the communication quality is good). In addition, in the first communication mode, the force haptics are transmitted bidirectionally in real time. Therefore, the first threshold value is set to a value corresponding to a communication quality sufficient for transmitting the force haptics bidirectionally in real time. For example, if the communication delay time is monitored, 20[ms] is set as the first threshold value, and if the delay time is less than this, the first communication mode is selected. Alternatively, if the random packet loss is monitored, 20[%] is set as the first threshold value, and if the packet loss is less than this, the first communication mode is selected. Note that such values ​​of 20[ms] and 20[%] are merely examples of values ​​that can be set as the first threshold value. An arbitrary value can be set as the first threshold value based on various criteria such as the environment in which this embodiment is implemented, the contents of the medical procedure actually performed, and the accuracy required by the operator.

[0043] The second communication mode is a state in which communication is possible but does not satisfy a predetermined communication quality. The third communication mode is a state in which communication is impossible or extremely difficult. The second threshold value can be determined based on this boundary. Note that these threshold values ​​are merely examples, and appropriate values ​​can be set depending on the type of medical procedure performed by remote control, the required accuracy, etc. In this way, when the communication control unit 312 selects the communication mode according to the communication quality monitored by the monitoring unit 311, it controls the communication between the operating device 10 and the working device 20 via the network N in a method defined in the communication control table. In this case, as shown in the figure, in the first communication mode, all of the force-tactile information, visual information, and audio information are transmitted and received instantly (i.e., in real time) and bidirectionally.

[0044] Here, the force-tactile information is a parameter value used for control (bilateral control) of transmitting force-tactile sensations between the operation mechanism 15 driven by the operating device 10 and the working mechanism 25 driven by the working device 20. For example, the force-tactile information is a value indicating the position detected by the position sensor 13 or the position sensor 23, or a control command value for the driver 11 or the driver 21 calculated by the main control device 30. In addition, for example, values ​​indicating speed, acceleration, force, impedance, etc. may be included in the force-tactile information. In addition, the visual information includes images of the facial expression of the operator on the operating side, the facial expression of the subject on the performing side, and the area targeted by the medical procedure, captured by camera 30b or camera 40b, as well as images of measurement results measured by the operating mechanism 25, etc. (e.g., ultrasound images showing the measurement results), and text indicating the patient's attributes (e.g., age, gender, and symptoms) entered by the assistant into the sub-control device 40. Furthermore, the auditory information includes the voice of the operator on the operating side, the voice of the subject on the working side, and the measurement results (for example, the subject's heartbeat indicating the measurement results) collected by the working mechanism 25, etc. At least a part of this visual information and auditory information corresponds to the "support information" of the present invention.

[0045] In the first communication mode, all of these force-tactile information, visual information, and audio information are transmitted and received in real time and in both directions in time series, so that, for example, an operator who receives these information can feel the reaction force from the actual patient based on the remote operation as force-tactile sensation through his / her sense of touch. In addition, in parallel with this, various visual and audio information can be felt visually and aurally. As a result, in this embodiment, it is possible for an operator who performs a diagnosis remotely to make a diagnosis from various viewpoints (for example, palpation, percussion, visual inspection, and auscultation).

[0046] For example, in the case of a medical procedure using an ultrasonic probe, it is important to make a diagnosis by visually checking an ultrasonic image measured by the ultrasonic probe. To do so, it is necessary to contact the ultrasonic probe with a target part of a subject with an appropriate force and move it appropriately (for example, reciprocating in a predetermined direction). However, it is not easy to make such appropriate contact and move it appropriately, and a technique based on the experience of an operator (for example, a doctor) is required. If an ultrasonic probe is used in an inappropriate manner, the ultrasonic image itself may be inaccurate in the first place, which may lead to a misdiagnosis. However, in the first communication mode, it is possible to support the execution of this procedure by transmitting haptic information, and to present an ultrasonic image, which is visual information, in parallel. Therefore, the operator can make a diagnosis by visually checking the ultrasonic image while confirming that the procedure is being performed appropriately by touch. In other words, even if there is only an inexperienced assistant on the operating side, the operator on the operating side can perform the medical procedure by remote control himself or herself, as if he or she were actually performing the medical procedure face-to-face with the subject.

[0047] In addition, the operator may intentionally press the ultrasound probe against the subject, regardless of the acquisition of an ultrasound image, and make a diagnosis based on the subject's reaction to the ultrasonic probe. For example, when the subject feels severe pain, a diagnosis may be made based on the possibility of peritonitis. In this embodiment, the pressure applied can be transmitted as a force haptic sensation, and visual information such as the subject's facial expression and audio information such as the subject's voice, which indicate the subject's reaction at that time, can also be presented, making it possible to make a diagnosis using this method.

[0048] In addition, for example, the facial expression and voice of the operator on the operating side and the facial expression and voice of the assistant on the working side may be transmitted and received as visual and audio information to allow communication between the operator and the assistant. Also, for example, the remote operation itself may be supported by presenting an image of the target part of the medical procedure on the subject or an image of the tip of the working mechanism 25.

[0049] In the second communication mode, all of this force-tactile information, visual information, and auditory information is accumulated in the working device 20 in chronological order. In other words, the acquired information itself (raw data) is not transmitted one by one, but is accumulated to form a data file. This data file is then transmitted collectively. Note that in the second communication mode, in consideration of communication quality, data files are transmitted from the working side to the operating side, but transmission from the operating side to the working side is canceled. In other words, it is a one-way communication.

[0050] In this case, for example, the operating device 20 is operated in a predetermined trajectory, and the haptic information, visual information, and audio information obtained thereby are transmitted to the operating side. As a result, the operator can accurately feel each piece of information obtained on the operating side, and can make a diagnosis based on this information, although the transmission of each piece of information, such as haptic information, is not in real time but asynchronous, and there is a slight delay in the transmission of each piece of information, such as haptic information. In addition to this, for example, the diagnosis may be further supported by the operator, the assistant, and the subject communicating with each other via a smartphone or the like using a mobile phone network. Operating the operating device 20 in a predetermined trajectory may be realized, for example, by storing haptic information for realizing the operation in the sub-control device 40 in advance and outputting this to the operating device 20. Alternatively, it may be realized by the assistant directly operating the operating device 20 in response to the operator's instructions, etc.

[0051] In the third communication mode, the transmission and reception of all of the haptic information, visual information, and audio information is stopped. In this case, the assistant may, for example, try to move to a place with a better communication environment or try to transport the patient to a nearby medical facility. In addition, the haptic information and visual information acquired by the working device 20 may be analyzed by the working sub-control device 40 using a statistical method, machine learning, or the like, so that analysis for diagnosis can be performed without relying on the assistant.

[0052] In this case, the statistical method is, for example, a rule-based diagnosis determined based on statistical information such as haptic information and visual information accumulated in the past for various subjects. In addition, in this case, machine learning is, for example, a learning model that uses the above-mentioned past statistical information as input and outputs the diagnosis result at that time as output, and constructs it in advance. A diagnosis can be made by inputting information on the subject currently being diagnosed into this learning model. In other words, even if communication is interrupted and the device is in a stand-alone state, it can support the operator and assistant.

[0053] The monitoring of communication quality by the monitoring unit 311 and the selection of the communication mode by the communication control unit 312 have been described above. These processes may be performed only at the beginning of the remote medical support process, or may be performed periodically thereafter in consideration of fluctuations in communication quality. In addition to automatically selecting the communication mode based on the communication quality, the communication mode may be manually selected according to the operation of the operator or assistant. For example, if the first communication mode is selected but the communication quality deteriorates and the operator feels uncomfortable with the remote operation, the operator may manually switch to the second communication mode, etc. Alternatively, if a compatible operator is not found, the third communication mode may be selected, and then, if a compatible operator is found, the assistant may manually switch to the first communication mode, etc. Furthermore, although the settings relating to the communication mode are managed in a table format called a communication control table, they may be managed in another data format.

[0054] In this way, according to this embodiment, even in various environments where the communication quality is not constant, it is possible to support diagnosis involving medical procedures in a manner appropriate to the communication quality.

[0055] Returning to FIG. 5, the operation control unit 313 controls the transmission of haptic sensation between the operation mechanism 15 driven by the operating device 10 and the working mechanism 25 driven by the working device 20, thereby controlling the operation of each mechanism. To achieve this control, in the first communication mode etc., the communication control unit 312 acquires haptic information for achieving the control. For example, the communication control unit 312 acquires the positions (specifically, positions or angles) of movable parts of each mechanism moved by driving the actuators 12, 22 from the position sensors 13, 23. The physical quantity data such as positions acquired by the communication control unit 312 is used as a reference value for the operation of each mechanism driven by the operating device 10 or the working device 20 in the haptic transmission algorithm.

[0056] FIG. 7 is a block diagram showing a control algorithm of the operation control section 313. As shown in FIG. As shown in Fig. 7, the algorithm implemented in the operation control unit 313 is expressed as a control law including a functional force-velocity allocation conversion block FT, an ideal force source block FC, an ideal velocity (position) source block PC, and an inverse conversion block IFT. The control algorithm shown in Fig. 7 is described in a patent publication (Japanese Patent No. 6382203) owned by Keio University, which constitutes the applicant of the present application, and various control algorithms described in the patent publication can be used appropriately in this embodiment. In this embodiment, the master device in the controlled system CS is configured by the operation device 10, and the slave device is configured by the working device 20.

[0057] The functional force-speed allocation transformation block FT is a block that defines the transformation of the control energy into the domain of speed (position) and force that are set according to the function of the controlled system CS. Specifically, the functional force-speed allocation transformation block FT defines a coordinate transformation that inputs a value (reference value) that is a reference for the function of the controlled system CS and the current position (or current angle) of the movable part of each mechanism that is moved by the actuators 12 and 22. This coordinate transformation generally transforms an input vector having elements of a reference value and a current position (current angle) into an output vector consisting of a position (angle) for calculating a control target value of the position (angle), and also transforms an input vector having elements of a reference value and a current force into an output vector consisting of a force for calculating a control target value of the force.

[0058] By setting the coordinate transformation in the functional force-velocity allocation transformation block FT to contents that represent a force-tactile transmission function, it is possible to realize a force-tactile transmission function between the operation device 10 and the working device 20, or to reproduce an operation of transmitting a force-tactile sensation in the working device 20 without using the operation device 10. Also, by setting coefficients in the elements of the transformation matrix in the coordinate transformation in the functional force-velocity allocation transformation block FT, it is possible to perform scaling of the position (angle) or force.

[0059] That is, in this embodiment, in the functional force / speed allocation conversion block FT, the variables (variables in real space) of the individual movable parts of each mechanism moved by the actuators 12 and 22 are "converted" into a group of variables (variables in space after coordinate conversion) of the entire system expressing the force haptic transmission function, and the control energy is assigned to the control energy of the position (angle) and the control energy of the force. That is, the coordinate conversion set in the functional force / speed allocation conversion block FT converts the coordinates (oblique coordinates) of the real space in which the position (angle) and the force are related to each other into the coordinates (orthogonal coordinates) of the virtual space in which the position (angle) and the force are independent from each other. Therefore, compared to the case where the control is performed as the variables (variables in real space) of the individual movable parts of each mechanism moved by the actuators 12 and 22, it is possible to independently provide the control energy of the position (angle) and the control energy of the force, that is, to independently control the position (angle) and the force.

[0060] In this embodiment, for example, when controlling the position (angle) and force output by the operating device 10, the state value in the space after the coordinate transformation can be calculated under the condition that the difference in position (angle) is zero and the sum of forces is zero (an equal force is output in the opposite direction) between the position (angle) of the movable part of each mechanism moved by driving the actuator 22 in the working device 20 and the input of the force calculated from these positions (angles), and the reference value serving as the reference for controlling the position (angle) and force. However, the reference value serving as the reference for controlling the position (angle) and force is the position (angle) of the movable part of each mechanism moved by driving the actuator 22 in the working device 20 and the force calculated from these positions (angles).

[0061] Similarly, in this embodiment, for example, when controlling the position (angle) and force output by the working device 20, a state value in the space after coordinate transformation can be calculated under the condition that the difference in position (angle) is zero and the sum of forces is zero (an equal force is output in the opposite direction) between the position (angle) of the movable part of each mechanism moved by driving the actuator 22 and the input of a force calculated from these positions (angles), and a reference value serving as a reference for controlling the position (angle) and force. However, the reference value serving as a reference for controlling the position (angle) and force is the position (angle) of the movable part of each mechanism moved by driving the actuator 12 in the operating device 10 and the force calculated from these positions (angles).

[0062] The ideal force source block FC is a block that performs calculations in the force domain according to the coordinate transformation defined by the functional force-velocity allocation transformation block FT. In the ideal force source block FC, a target value for the force when performing calculations based on the coordinate transformation defined by the functional force-velocity allocation transformation block FT is set. This target value is set as a fixed value or a variable value according to the function to be realized. For example, when a function similar to the function indicated by the reference value is to be realized, zero can be set as the target value, and when scaling is performed, a value obtained by enlarging or reducing the information indicating the function indicated by the reference value can be set. In addition, the ideal force source block FC can set an upper limit value for the energy of the force determined by calculations in the force domain. By setting an upper limit value for the energy of the force, for example, a contact force when the working device 20 contacts the subject is limited, and it is possible to prevent the working mechanism 25, etc. from being pressed excessively strongly against the body of the subject.

[0063] The ideal velocity (position) source block PC is a block that performs calculations in the area of ​​position (angle) according to the coordinate transformation defined by the functional force-velocity allocation transformation block FT. In the ideal velocity (position) source block PC, a target value for the position (angle) when performing calculations based on the coordinate transformation defined by the functional force-velocity allocation transformation block FT is set. This target value is set as a fixed value or a variable value according to the function to be realized. For example, when a function similar to the function indicated by the reference value is to be realized, zero can be set as the target value, and when scaling is performed, a value obtained by enlarging or reducing the information indicating the function to be reproduced can be set. In addition, the ideal velocity (position) source block PC can set an upper limit value for the energy of the force determined by calculations in the area of ​​position (angle). Setting an upper limit value for the energy of the position (angle) limits the distance that the working device 20 advances and retreats, and can prevent the working mechanism 25, etc. from being pressed excessively strongly against the body of the subject.

[0064] The inverse transformation block IFT is a block that inversely transforms values ​​in the position (angle) and force domains into values ​​in the input domain to the controlled system CS (e.g., voltage values ​​or current values, etc.) (i.e., determines command values ​​in the real space). Under such a control algorithm, the main control device 30 receives as input the time-series position (angle) detection values ​​detected by the position sensors 13, 23. These time-series position (angle) detection values ​​represent the operations of the actuators 12, 22, and the main control device 30 applies coordinate transformation to transmit haptics to the input positions (angles) and forces derived from these positions (angles).

[0065] Returning to Fig. 5, the presentation unit 314 presents to the operator various pieces of information acquired by the communication control unit 312 from the working side through communication via the network N. The presentation is realized, for example, by reproducing visual information and audio information on the display 30a via the output unit 706. By reproducing this information for the worker, it is possible to support the worker in making a diagnosis.

[0066] The history data storage unit 372 stores various information acquired by the communication control unit 312 as history data. The history data stored in the history data storage unit 372 can be used as a log accompanying medical procedures. For example, it can be used to analyze the success or failure of a medical procedure and the time required for the medical procedure. Alternatively, it can be used as teaching materials for teaching remote operation procedures to operators who have little experience in remote medical procedures.

[0067] FIG. 8 is a block diagram showing the functional configuration of the sub-control device 40. As shown in FIG. When the above-mentioned operation control process or remote medical support process is performed, a communication control unit 411, a data accumulation unit 412, an analysis unit 413, and a presentation unit 414 function in the processor 701, as shown in Fig. 8. In addition, a communication control table storage unit 471 and a history data storage unit 472 are formed in the storage unit 707. Including cases not specifically mentioned below, data required to realize processing is transmitted and received between these functional blocks at appropriate times as appropriate.

[0068] The communication control unit 411 corresponds to the communication control unit 312 of the main control unit 30. The communication control unit 411 is notified of the communication mode to be selected by the communication control unit 312 of the main control unit 30. This communication mode to be selected is the same communication mode as the communication mode selected by the communication control unit 312 of the main control unit 30. Furthermore, when the communication control unit 411 selects the communication mode in response to this, it transmits and receives haptic information, visual information, and auditory information to and from the communication control unit 312 of the main control unit 30 in accordance with the contents defined in the communication control table stored in the communication control table storage unit 471. However, as described above, depending on the communication mode, transmission and reception of these pieces of information is not performed.

[0069] The communication control table storage unit 471 functions as a storage unit that stores the communication control table. The communication control table stored in the communication control table storage unit 471 is the same as the communication control table stored in the communication control table storage unit 371.

[0070] The data accumulation unit 412 functions when the second communication mode is selected, and accumulates all of the haptic information, visual information, and audio information to be transmitted in chronological order in the operating device 20. That is, the acquired information itself (raw data) is accumulated to create a data file. In this case, the data accumulation unit 412 may accumulate the data by using a storage area of ​​the RAM 703, or may accumulate the data by using a storage area of ​​the storage unit 707. The communication control unit 411 then transmits the data file collectively at periodic timing or the like. This allows the operator to feel each piece of information obtained on the operating side, although not in real time and with a time error, and allows diagnosis to be made based on this information.

[0071] The analysis unit 413 functions when the third communication mode is selected, and analyzes the haptic information and visual information acquired by the working device 20 by a statistical method, machine learning, or the like. This allows the analysis for diagnosis to be performed without relying on an assistant. The specific contents of the statistical method, machine learning, or the like for performing the analysis are as described above.

[0072] The presentation unit 414 presents various pieces of information acquired by the data storage unit 412 from the operating side through communication via the network N to the assistant and the subject. The presentation unit 414 also presents the analysis results by the analysis unit 413 to the assistant and the subject. The presentation is realized, for example, by reproducing visual information and audio information from the display 40a via the output unit 706. By reproducing this information for the assistant and the subject, it is possible to support the assistant and the subject.

[0073] History data storage unit 472 stores various information acquired by data accumulation unit 412 as history data. The history data stored in history data storage unit 472 is similar to the history data stored in history data storage unit 372 of main control device 30 in that it can be utilized as a log associated with medical procedures. Next, the details of each process performed by the remote medical support system 1 will be described.

[0074] [Motion control processing] 9 is a flowchart illustrating the flow of the operation control processing executed by the remote medical support system 1. The operation control processing is executed as a subroutine in the remote medical support processing.

[0075] In step S1, the operation control unit 313 acquires the position (angle) of a movable part of a mechanism to be moved by driving the actuator. In this embodiment, the mechanisms to be moved are the operation mechanism 15 and the working mechanism 25. In this case, the operation control unit 313 acquires the positions corresponding to these mechanisms from the position sensor 13 and the position sensor 23 via the communication control unit 312.

[0076] In step S2, the motion control section 313 converts an input vector in the real space into a vector in the virtual space. In step S3, the motion control unit 313 executes calculations in the domain of velocity (position) and calculations in the domain of force.

[0077] In step S4, the motion control unit 313 inversely converts values ​​in the domains of velocity (position) and force into values ​​in the domain of input to the controlled system CS (vectors in real space). In step S5, the operation control unit 313 outputs command values ​​for the actuators 12 and 22. This ends the process. However, as long as the first communication mode is being performed, the process is performed continuously. That is, in practice, as long as the first communication mode is being performed, steps S1 to S5 of the process are repeated.

[0078] According to the operation control process described above, the operation of each mechanism can be controlled by performing control (bilateral control) to transmit haptic sensation between the operation mechanism 15 driven by the operating device 10 and the working mechanism 25 driven by the working device 20.

[0079] [Remote medical support processing] 10 is a flowchart explaining the flow of the remote medical support processing executed by the remote medical support system 1. The remote medical support processing is executed when the operation device 10 or the main control device 30 receives an instruction to start the remote medical support processing from a user such as an operator or an assistant.

[0080] In step S11, the monitoring unit 311 monitors the communication quality of the communication path used by the main control device 30 and the sub-control device 40 to transmit and receive data. In step S12, the communication control unit 312 selects a communication mode depending on the communication quality monitored by the monitoring unit 311.

[0081] In step S13, the communication control unit 312 determines which communication mode is the selected communication mode. If the selected communication mode is the first communication mode, it is determined to be the "first communication mode" in step S13, and the process proceeds to step S14. If the selected communication mode is the second communication mode, it is determined to be the "second communication mode" in step S13, and the process proceeds to step S17. If the selected communication mode is the third communication mode, it is determined to be the "third communication mode" in step S13, and the process proceeds to step S20.

[0082] In step S14, the communication control unit 312 and the communication control unit 411, which has been notified that the first communication mode has been selected, transmit and receive all of the haptic information, visual information, and auditory information in chronological order, instantly, and bidirectionally. In step S15, the operation control section 313 executes the above-mentioned operation control process as a subroutine. Note that this operation control process is repeatedly executed while the first communication mode is being performed. In step S16, the presentation unit 314 and the presentation unit 414 present various types of information such as visual information and audio information.

[0083] In step S17, the data storage unit 412 stores all of the force-tactile information, visual information, and audio information to be transmitted in chronological order in the operating device 20 to create a data file. In step S18, communication control unit 411 transmits the accumulated data files all at once. In step S19, the presentation unit 314 presents various types of information such as visual information and audio information. In this case, a haptic sensation may be transmitted to the operator via the operation mechanism 15 based on the haptic information received by the operation control unit 313.

[0084] In step S20, the analysis unit 413 analyzes the force-tactile information and visual information acquired by the working device 20 by a statistical method, machine learning, or the like. In step S21, the presentation unit 414 presents the analysis results obtained by the analysis unit 413 to an assistant or a subject.

[0085] In step S22, the communication control unit 312 determines whether or not a manual operation for switching the communication mode has been accepted. If a manual operation for switching the communication mode has been accepted, the result of step S22 is determined as Yes, and the process proceeds to step S23. On the other hand, if a manual operation for switching the communication mode has not been accepted, the result of step S22 is determined as No, and the process proceeds to step S24. In step S23, the communication control unit 312 selects a communication mode in response to the received switching operation. After that, the process returns to step S13, and the process based on the selected communication mode continues.

[0086] In step S24, the monitoring unit 311 judges whether or not to monitor the communication quality again. The monitoring unit 311 monitors the communication quality again, for example, when a predetermined time has passed since the previous monitoring, or when an operation to instruct monitoring again has been received from the operator or assistant. If the communication quality is to be monitored again, the result of the judgment in step S24 is Yes, and the process returns to step S11, and the process is repeated from monitoring the communication quality again. On the other hand, if the communication quality is not to be monitored again, the result of the judgment in step S24 is No, and the process proceeds to step S25.

[0087] In step S25, the communication control unit 312 determines whether or not to end this process. For example, this process ends on condition that an operation to instruct the end of this process is received from the operator or assistant. If this process ends, it is determined as Yes in step S25 and this process ends. On the other hand, if this process does not end, it is determined as No in step S13, and the process continues based on the selected communication mode.

[0088] According to the remote medical support process described above, it is possible to support the operator in remote operation of the remote medical support work by utilizing the motion control process.

[0089] In this way, in the remote medical support system 1, the operation mechanism 25 is controlled in response to the remote operation received by the operation mechanism 15, and the reaction force applied to the operation mechanism 25 in response to the contact of the operation mechanism 25 with the subject is transmitted to the operator via the operation mechanism 15 as a haptic sensation. That is, control for transmitting a haptic sensation between the operation mechanism 15 and the operation mechanism 25 (i.e., bilateral control) can be performed. This allows the operator to perform a medical procedure for diagnosis based on the haptic sensation as if the operator were performing a medical procedure by directly facing the subject. In addition, the presentation unit 314 presents the operator with support information acquired by the operation mechanism side for supporting a diagnosis based on a medical procedure. Therefore, the operator can perform a medical procedure for diagnosis based on other senses (e.g., vision and hearing) other than the haptic sensation. In other words, the remote medical support system 1 enables medical professionals who perform diagnoses remotely to make diagnoses from various perspectives (for example, palpation, percussion, visual inspection, and auscultation). Therefore, according to the remote medical support system 1, when a diagnosis involving a medical procedure is performed remotely, it is possible to more appropriately support the diagnosis.

[0090] [Variations] Although the embodiment of the present invention has been described above, this embodiment is merely an example and does not limit the technical scope of the present invention. The present invention can take various other embodiments and can be modified in various ways, such as omissions and substitutions, without departing from the gist of the present invention.

[0091] [Variation 1] In the above-described embodiment, a situation is assumed in which an operator is present on the operating side, which is a medical facility such as a hospital, and an assistant and a subject are present on the working side, which is the inside of an ambulance, etc. However, the scope of application of the present invention is not limited to this and can be applied to various situations. For example, when the operating side is a medical facility, the working side may be the subject's home or another medical facility. Or, when the operating side is a medical facility, the working side may be another room in the same medical facility. This makes it possible to apply the present invention even when the medical facility does not have experienced medical personnel for the medical procedure to be performed (for example, when the medical personnel specialize in different medical departments), when there is a shortage of experienced medical personnel, or when medical treatment is performed at home.

[0092] In addition, as long as communication via the network N is possible, there is no limit to the distance between the operating side and the working side, and the present invention can be applied, for example, to cases where the operating side and the working side are in different prefectures in the country, or where they are in the country and overseas. For example, when a disaster occurs in the country or overseas, triage can be performed from a remote location where the disaster does not occur. Furthermore, the operating side and the working side may correspond to each other in a one-to-one relationship, but may also correspond to each other in a one-to-many relationship, so that one operator can perform diagnoses involving medical procedures on multiple subjects in parallel or at different times.

[0093] [Variation 2] In the above-described embodiment, it is assumed that three communication modes are provided, but the present invention is not limited to this, and more communication modes may be provided in stages. In this case, for example, the first communication mode is further divided into stages, and when the communication quality is good, the force haptics is transmitted so that the upper limit of performance that can be realized by the control algorithm of the operation control unit 313 can be exhibited. After that, when the communication quality deteriorates somewhat, the operation of the operating device 20 is suppressed (i.e., the transmission of the force haptics is suppressed) to stabilize the operation. For example, by lowering the control gain in the control of the operation control unit 313, the sensitivity of transmitting the force haptics is reduced, but the operation can be stabilized. Alternatively, for example, the force or speed of the operating device 20 may be limited or the operation of the operating device 20 may be forcibly stopped so that the operating device 20 does not perform an unexpected operation.

[0094] In addition, for example, the first communication mode and the second communication mode may be further divided, and when the communication quality deteriorates, the force-tactile information may be transmitted in real time to reduce the amount of communication, but the visual information and auditory information may be transmitted with a delay as a data file.

[0095] Furthermore, even in the first communication mode or the second communication mode, the analysis unit 413 may perform analysis and present the analysis result to the operator to assist the operator in making a diagnosis. Alternatively, a functional block similar to the analysis unit 413 may be provided in the main control device 30 on the operating side so that the analysis may also be performed on the operating side.

[0096] [Variation 3] In the above-described embodiment, the operation device 10 and the main control device 30 are realized as separate devices. However, the present invention is not limited to this, and for example, the operation device 10 and the main control device 30 may be realized as an integrated device and installed on the operation side. Alternatively, the working device 20 and the main control device 30 may be realized as an integrated device and installed on the working side.

[0097] Alternatively, the control of the transmission of haptics, which was previously performed by the operation control section 313 of the main control device 30, may be distributed between the operating device 10 and the working device 20. In this case, a functional block for motion control is added to each device. Then, the functional block for motion control of each device acquires detection values ​​from its own sensor and detection values ​​from the sensors of the other devices, and outputs command values ​​to the driver of its own device based on the same control algorithm as the main control device 30 described above. In this way, the transmission of haptics between the other devices may be realized. It should be noted that the above-mentioned modified examples 1 to 3 can also be appropriately combined to form separate modified examples.

[0098] [Configuration example] As described above, the remote medical support system 1 according to this embodiment includes the working mechanism 25, the operation mechanism 15, the operation control unit 313, and the presentation unit 314. The working mechanism 25 is a mechanism for performing medical procedures, including palpation, on a subject. The operation mechanism 15 is a mechanism that receives remote control of the working mechanism 25 from an operator. The operation control unit 313 controls the operation of the working mechanism 25 in response to the remote operation received by the operating mechanism 15, and transmits the reaction force applied to the working mechanism 25 in response to the working mechanism 25 coming into contact with the subject to the operator via the operating mechanism 15 as a force haptic sensation. The presentation unit 314 presents to the operator assistance information that is acquired by the operating mechanism and that supports diagnosis based on medical practice. In this way, in the remote medical support system 1, the operation mechanism 25 is controlled in response to the remote operation received by the operation mechanism 15, and the reaction force applied to the operation mechanism 25 in response to the contact of the operation mechanism 25 with the subject is transmitted to the operator via the operation mechanism 15 as a haptic sensation. That is, control for transmitting a haptic sensation between the operation mechanism 15 and the operation mechanism 25 (i.e., bilateral control) can be performed. This allows the operator to perform a medical procedure for diagnosis based on the haptic sensation as if the operator were performing a medical procedure by directly facing the subject. In addition, the presentation unit 314 presents the operator with support information acquired by the operation mechanism side for supporting a diagnosis based on a medical procedure. Therefore, the operator can perform a medical procedure for diagnosis based on other senses (e.g., vision and hearing) other than the haptic sensation. In other words, the remote medical support system 1 enables medical professionals who perform diagnoses remotely to make diagnoses from various perspectives (for example, palpation, percussion, visual inspection, and auscultation). Therefore, according to the remote medical support system 1, when a diagnosis involving a medical procedure is performed remotely, it is possible to more appropriately support the diagnosis.

[0099] The remote medical support system 1 further includes a monitor unit 311 and a communication control unit 312. The monitor 311 monitors the communication quality of the communication path used by the operation controller 313 and the presenter 314 to transmit and receive data. The communication control unit 312 changes the manner in which data is transmitted and received depending on the communication quality monitored by the monitoring unit 311 . This makes it possible to transmit and receive data in a manner appropriate to the communication quality even in various environments where the communication quality is not constant.

[0100] The communication control unit 312 When the communication quality monitored by the monitoring unit 311 satisfies a predetermined communication quality, data is transmitted and received in real time in a time series manner; If the communication quality monitored by the monitor 311 does not satisfy a predetermined communication quality, data is accumulated in chronological order, and the accumulated data is then transmitted and received collectively. This allows data to be transmitted and received using the most appropriate means depending on the communication quality.

[0101] The remote medical support system 1 further includes a presentation unit 414 . When the communication quality monitored by the monitoring unit 311 is such that it is difficult to execute communication, the presentation unit 414 performs an analysis regarding a diagnosis based on a medical procedure based on either or both of the reaction force and the support information applied to the working mechanism 25, and presents the analysis result to the working mechanism side. This makes it possible to support a user (for example, an assistant) on the operation mechanism side even in a situation where it is difficult to execute communication.

[0102] The operation control unit 313 is When the communication quality monitored by the monitoring unit 311 does not satisfy a predetermined communication quality, the operation of the working mechanism 25 corresponding to the remote operation accepted by the operating mechanism 15 is suppressed. This makes it possible to suppress the influence of communication quality and prevent the operating mechanism 25 from performing unexpected operations.

[0103] The working mechanism 25 is equipped with measuring equipment for measuring the condition of the subject, The support information includes at least information indicating the measurement results obtained by the measuring device. This allows the operator performing remote operation to not only obtain the measurement results of the measuring device, but also to recognize whether the measuring device is being remotely operated so as to perform measurements appropriately.

[0104] The operation control unit 313 is Instead of remote operation received by the operating mechanism 15 from an operator, the operation of the working mechanism 25 is controlled in response to operation received directly from another operator, and the reaction force given to the working mechanism 25 in response to the working mechanism 25 coming into contact with the subject is transmitted to the operator via the operating mechanism 15 as a force haptic sensation. This allows the operator to make a diagnosis based on the haptic sensation felt when actually touching the subject, even in cases where remote operation is difficult.

[0105] [Realization of functions through hardware and software] The function of executing the series of processes according to the above-mentioned embodiment can be realized by hardware, software, or a combination of these. In other words, it is sufficient that the function of executing the series of processes described above is realized in any one of the remote medical support systems 1, and there is no particular limitation on how this function is realized.

[0106] For example, when the function of executing the above-mentioned series of processes is realized by a processor that executes arithmetic processing, the processor that executes this arithmetic processing includes not only processors that are composed of various types of processing devices alone, such as single processors, multiprocessors, and multicore processors, but also processors that combine these various types of processing devices with processing circuits such as ASICs (Application Specific Integrated Circuits) or FPGAs (Field-Programmable Gate Arrays).

[0107] Furthermore, for example, when the function of executing the above-mentioned series of processes is realized by software, the program constituting the software is installed in a computer via a network or a recording medium. In this case, the computer may be a computer in which dedicated hardware is built in, or may be a general-purpose computer (e.g., general electronic devices such as general-purpose personal computers) capable of executing a predetermined function by installing a program. Furthermore, the step of writing the program may include only processes that are performed chronologically according to the order, but may also include processes that are performed in parallel or individually. Furthermore, the step of writing the program may be performed in any order within the scope of the gist of the present invention.

[0108] The recording medium on which such a program is recorded may be provided to the user by being distributed separately from the computer main body, or may be provided to the user in a state in which it is pre-installed in the computer main body. In this case, the storage medium distributed separately from the computer main body is, for example, a removable medium 750, which is composed of a magnetic disk (including a floppy disk), an optical disk, or a magneto-optical disk. The optical disk is, for example, a CD-ROM (Compact Disc-Read Only Memory), a DVD (Digital Versatile Disc), or a Blu-ray (registered trademark) Disc. The magneto-optical disk is, for example, an MD (Mini Disc). The recording medium provided to the user in a state in which it is pre-installed in the computer main body is, for example, a storage unit 707 on which the program is recorded, which is composed of an HDD (hard disk drive) or an SSD (Solid State Drive). [Explanation of symbols]

[0109] 1 Remote medical support system, 10 Operation device, 11, 21 Driver, 12, 22 Actuator, 13, 23 Position sensor, 14, 24, 708 Communication unit, 15 Operation mechanism, 20 Working device, 25 Working mechanism, 311 Monitoring unit, 312, 411 Communication control unit, 313 Motion control unit, 314, 414 Presentation unit, 371, 471 Communication control table storage unit, 372, 472 History data storage unit, 412 Data accumulation unit, 413 Analysis unit, 700 Hardware configuration, 700 Control device, 701 Processor, 702 ROM, 703 RAM, 704 Bus, 705 Input unit, 706 Output unit, 707 Storage unit, 709 Drive, 750 Removable media, CS Control target system, FT Force-speed allocation conversion block, FC Ideal force source block, PC Ideal velocity (position) source block, IFT inverse transformation block

Claims

1. A working mechanism for performing a medical procedure including palpation on a subject; an operation mechanism that receives remote operation of the working mechanism from an operator; an operation control means for controlling an operation of the working mechanism in response to a remote operation received by the operating mechanism, and for transmitting a reaction force applied to the working mechanism in response to the working mechanism coming into contact with the subject, to the operator via the operating mechanism, as a haptic sensation; an assistance information presenting means for presenting assistance information, acquired by the operating mechanism, for assisting a diagnosis based on the medical practice to the operator; A remote medical support system comprising:

2. a monitoring means for monitoring a communication quality of a communication path used for transmitting and receiving data by the operation control means and the support information providing means; a communication control means for changing a mode of transmitting and receiving the data depending on the communication quality monitored by the monitoring means; The remote medical support system according to claim 1, further comprising:

3. The communication control means When the communication quality monitored by the monitoring means satisfies a predetermined communication quality, the data is transmitted and received in a time series manner in real time; When the communication quality monitored by the monitoring means does not satisfy a predetermined communication quality, the data is accumulated in a time series, and then the accumulated data is transmitted and received collectively.

3. The remote medical support system according to claim 2.

4. an analysis result presenting means for performing an analysis regarding a diagnosis based on the medical practice based on either or both of the reaction force applied to the working mechanism and the support information when the communication quality monitored by the monitoring means is such that it is difficult to execute communication, and presenting the analysis result to the working mechanism; The remote medical support system according to claim 2 or 3, further comprising:

5. The operation control means includes: When the communication quality monitored by the monitoring means does not satisfy a predetermined communication quality, the operation of the working mechanism corresponding to the remote operation accepted by the operating mechanism is suppressed.

4. The remote medical support system according to claim 2 or 3.

6. a measuring device for measuring a condition of the subject is attached to the working mechanism; The support information includes at least information indicating a measurement result by the measuring device.

4. The remote medical support system according to claim 1, wherein the remote medical support system comprises:

7. The operation control means includes: The operation of the working mechanism is controlled in response to an operation directly received by the working mechanism from another operator, instead of a remote operation received by the operating mechanism from the operator, and a reaction force applied to the working mechanism in response to the working mechanism coming into contact with the subject is transmitted to the operator via the operating mechanism as a haptic sensation.

4. The remote medical support system according to claim 1, wherein the remote medical support system comprises:

8. A remote medical support method performed by a system including a working mechanism for performing a medical procedure including palpation on a subject, and an operation mechanism for receiving remote operation of the working mechanism from an operator, comprising: an operation control step of controlling an operation of the working mechanism in response to a remote operation received by the operating mechanism, and transmitting a reaction force applied to the working mechanism in response to the working mechanism contacting the subject to the operator via the operating mechanism as a haptic sensation; an assistance information presenting step of presenting assistance information, acquired by the operating mechanism, for assisting a diagnosis based on the medical practice to the operator; A remote medical support method comprising:

9. A computer controls a system including a working mechanism for performing a medical procedure including palpation on a subject, and an operation mechanism for receiving remote operation of the working mechanism from an operator, an operation control function that controls the operation of the working mechanism in response to a remote operation received by the operating mechanism, and transmits a reaction force applied to the working mechanism in response to the working mechanism coming into contact with the subject to the operator via the operating mechanism as a haptic sensation; an assistance information presentation function that presents assistance information acquired by the operating mechanism for assisting a diagnosis based on the medical practice to the operator; A program for realizing the above.

Images