Force display device and force display method for medical robot system

Abstract

In order to provide a force display device and a force display method for a medical robot system that can easily improve the accuracy and safety of operations in the medical robot system, a force display device (50) of a medical robot system (1) displays to an operator in a manner that is perceptible to the operator the external force acting on a surgical tool (14) and at least one of a slave device (10A, 10B) supporting the surgical tool, and the force display device includes an input unit (51), an image generating unit (53), and an output unit (54), wherein a measurement signal of the external force measured at two or more locations in the surgical tool and at least one of the slave devices is input to the input unit, and the image generating unit creates image information that is associated with the location where the external force is measured and is used to display the information of the external force as an image in a second display area (63) different from the first display area (61), based on the measurement signal, wherein information related to the operation of the surgical tool is displayed in the first display area (61), and the output unit is configured to output the created image information to a display unit (60) that displays the image information.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This international application claims the benefit of Japanese Patent Application No. 2019-048547 filed in the Japan Patent Office on March 15, 2019, the entire contents of which are hereby incorporated by reference. Technical Field

[0003] The present disclosure relates to a force display device and a force display method for a medical robot system. Background Art

[0004] In recent years, endoscopic surgery using a medical robot for surgical assistance has become increasingly popular. In endoscopic surgery, surgical tools such as a laparoscope, an endoscope, and forceps mounted on a medical robot are used.

[0005] Doctors can remotely operate surgical tools of medical robots. In addition, a medical robot capable of intuitive operation has been proposed. Moreover, in order to improve accuracy and safety, a medical robot capable of force presentation has been proposed, which can transmit external forces acting on surgical tools to doctors operating the medical robot (for example, refer to patent documents 1 and 2).

[0006] Patent Document 1 discloses receiving tactile feedback from a tactile controller based on measured stress received by a medical device, and displays the measured stress on a display so as to provide a tactile sensation of stress received by a needle.

[0007] Patent document 2 discloses updating the content of a three-dimensional tactile representation based on information generated by tactile feedback. In addition, the three-dimensional representation is also denoted as 3D in the following text. Specifically, it is recorded that when it is determined that a surgical instrument contacts a patient's bone, a 3D representation of the patient's state, such as a 3D graphical representation and / or a 3D tactile representation, is updated.

[0008] Prior art literature

[0009] Patent Literature

[0010] Patent Document 1: Japanese Patent Application No. 2005-510289

[0011] Patent Document 2: Japanese Patent Application Publication No. 2018-158104 Summary of the invention

[0012] Problem that the invention aims to solve

[0013] In the case where the force sense is presented by giving the doctor who is performing the operation a sense of touch, there is an advantage that the doctor who is performing the operation can intuitively grasp the force sense. However, on the other hand, even if the intensity presented to the sense of touch is the same, the intensity perception may be different if the receiving doctor is different, which may lack accuracy. In addition, since the force sense is only presented to the doctor who is performing the operation and not to other doctors, it is difficult to ensure real-time inspection and objectivity by a third party.

[0014] As disclosed in Patent Documents 1 and 2, the method of displaying the measured stress, etc. on a display, in other words, the method of displaying force sense, has the following advantages: it is relatively easy to ensure the real-time inspection and objectivity implemented by a third party as described above. However, since the force sense is displayed on a display with a limited area together with the image and / or picture taken by an endoscope, etc., it may be difficult to grasp the force sense. In particular, if multiple different force senses are displayed, the difficulty in grasping the force sense becomes significant.

[0015] One aspect of the present disclosure preferably provides a force display device and a force display method for a medical robot system, which can easily improve the accuracy and safety of operations in the medical robot system.

[0016] Technical solutions to the problem

[0017] The first aspect of the present disclosure relates to a force display device of a medical robot system, which displays to an operator in a manner perceptible to the operator the external force acting on a surgical tool and at least one of a slave device supporting the surgical tool, and the force display device of the medical robot system is provided with: an input unit, to which measurement signals of external forces measured at two or more locations of the surgical tool and at least one of the slave devices are input; an image generating unit, which creates image information based on the measurement signals, the image information is associated with the location where the external force is measured and is used to display the information on the external force as an image in a second display area different from the first display area, wherein information related to the operation of the surgical tool is displayed in the first display area; and an output unit, which outputs the created image information to a display unit that displays the image information.

[0018] The second aspect of the present disclosure relates to a force display method for a medical robot system, which displays to an operator in a manner perceptible to the operator the external force acting on a surgical tool and at least one of a slave device supporting the surgical tool, and the force display method for the medical robot system comprises the following steps: an input step, in which a measurement signal of the external force measured at two or more locations of the surgical tool and at least one of the slave devices is input; an image generation step, in which image information is created based on the measurement signal, the image information is associated with the location where the external force is measured and is used to display the information on the external force as an image in a second display area different from the first display area, wherein information related to the operation of the surgical tool is displayed in the first display area; and an output step, in which the created image information is output to a display unit that displays the image information.

[0019] According to the force display device of the first aspect and the force display method of the second aspect of the present disclosure, it is possible to create image information that is associated with the location where the external force is measured and displays the information of the external force as an image, and the image information can be displayed on the display unit. In other words, the displayed external force information can be displayed in a manner that allows identification of where the measurement was performed.

[0020] Furthermore, since the information on the external force is displayed as an image in the second display area different from the first display area, it is easier to grasp the displayed information on the external force than when the information on the external force is displayed in a superimposed manner in the first display area, and it does not hinder visual recognition of the information related to the surgical tool displayed in the first display area.

[0021] In the first aspect of the present disclosure, it is preferred that the image generation unit creates the image information that expresses the magnitude of the value of the external force through at least one of color, shape, and area of ​​the image displayed as the external force.

[0022] As described above, by expressing the magnitude of the external force value by at least one of color, shape, and area, it is easier to intuitively grasp the magnitude of the external force value than when expressing the magnitude of the external force value by a numerical value.

[0023] In the first aspect of the present disclosure, the slave device preferably has a plurality of slave devices, and the image generating unit preferably sets a sub-area for each slave device within the second display area, and displays the image displayed as the external force inside the sub-area.

[0024] As described above, by displaying the external force measured by the corresponding slave device inside the sub-area set for each slave device, it is easy to understand which slave device the displayed external force information is measured from even when multiple slave devices are set.

[0025] In the first aspect of the present disclosure, it is preferred that the image generation section creates the image information capable of identifying the plurality of slave devices by the shape of the sub-region and the color of at least a part of the sub-region.

[0026] As described above, by making the shapes of the sub-regions provided for each slave device different in at least a part of color, it is easy to understand in which slave device the displayed external force information is the external force information measured.

[0027] In the first aspect of the present disclosure, type information is preferably input into the input unit, the type information indicating the type of the surgical tool held by the slave device, and the image generating unit preferably creates the image information based on the type information, the image information including an image representing the shape of the surgical tool in the sub-area.

[0028] As described above, by displaying an image representing the shape of the surgical tool held by the slave device in the sub-area, it is easy to understand which surgical tool the displayed external force information is the external force information measured by the slave device holding the surgical tool.

[0029] In the first aspect of the present disclosure, the surgical tool is preferably configured at the end of the slave device extending in a rod shape, and a joint portion capable of changing the posture of the surgical tool is provided near the end of the slave device. The measurement signal input to the input portion preferably includes: a joint measurement signal related to the force acting on the joint portion, and an arm measurement signal related to the external force acting on a portion of the slave device other than the joint portion. The image generating portion preferably creates the image information based on the joint measurement signal and the arm measurement signal, and the image information displays information related to the force acting on the joint portion and information related to the external force acting on a portion of the slave device other than the joint portion.

[0030] As described above, by displaying information related to the force acting on the joint and the external force acting on the parts of the slave device other than the joint, it is easier to understand the status of the slave device being operated, compared with the case of displaying the external force acting on one part. For example, when the slave devices are in contact with each other, the external force generated by the contact will act on the parts of the slave device other than the joint (for example, the shaft). Since the information related to the external force is displayed, the operator can understand the situation that the slave devices are in contact with each other.

[0031] In the first aspect of the present disclosure, the measurement signal input to the input unit preferably includes a surgical tool measurement signal related to the force acting on the surgical tool, and the image generating unit preferably creates the image information based on the surgical tool measurement signal, and the image information displays information related to the force acting on the surgical tool.

[0032] By displaying information related to the force acting on the surgical tool in the above manner, it is easy to understand the operating status of the surgical tool. For example, when the surgical tool is a forceps, the gripping force when the forceps are closed to clamp the object can be displayed. In addition, the peeling force when the object is peeled off by opening the forceps can also be displayed.

[0033] Effects of the Invention

[0034] According to the force display device involved in the first aspect and the force display method involved in the second aspect of the present disclosure, by creating image information that is associated with the location where the external force is measured and displays the information of the external force as an image, and displaying the image information on the display unit, it is possible to easily improve the accuracy and safety of operations in a medical robot system. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 This is a block diagram illustrating the configuration of the medical robot system according to the first embodiment of the present disclosure.

[0036] Figure 2 Is the description set in Figure 1 A three-dimensional diagram of the structure of the cassette of the slave unit.

[0037] Figure 3 Is the description and Figure 1 Flowchart of control related to force sense display in a control device.

[0038] Figure 4A Yes Description Figure 1 A schematic diagram of a display method in a display device of Figure 4B is a schematic diagram illustrating another method. Figure 4C This is a schematic diagram for explaining still another method.

[0039] Figure 5 Yes Description Figure 1 Schematic diagram of other display modes in a display device.

[0040] Figure 6 It is a schematic diagram for explaining the display method in the force sense display unit.

[0041] Fig. 7A is a diagram illustrating a peeling display method, Figure 7B is a diagram illustrating how to hold the display. Figure 7C This is a diagram for explaining the display method when no force is measured.

[0042] Figure 8 This is a block diagram illustrating the configuration of a medical robot system according to a first modification example of the first embodiment of the present disclosure.

[0043] Fig. 9 Yes Description Figure 8 A schematic diagram of a display method in a display device.

[0044] Fig.10 This is a block diagram illustrating the configuration of a medical robot system according to a second modification example of the first embodiment of the present disclosure.

[0045] Fig.11A is a schematic diagram for explaining a display method in a first display device, Fig. 11B It is a schematic diagram for explaining the display method in the second display device.

[0046] Fig.12 This is a block diagram illustrating the configuration of a medical robot system according to a second embodiment of the present disclosure.

[0047] Fig.13 Yes Description Fig.12 A block diagram of the composition of the slave unit.

[0048] Fig.14 Yes Description Fig.12 A schematic diagram of a display method in a display device.

[0049] Fig.15 Yes Description Fig.14 Schematic diagram of display methods in the first force sense display unit, the second force sense display unit, and the third force sense display unit.

[0050] Fig.16 This is a block diagram illustrating the configuration of a medical robot system according to a first modified example of the second embodiment of the present disclosure.

[0051] Fig.17 Yes Description Fig.16 Schematic diagram of display methods in the first force sense display unit, the second force sense display unit, and the third force sense display unit in the display device.

[0052] Fig.18 This is a block diagram illustrating the configuration of a medical robot system according to a second modification example of the second embodiment of the present disclosure.

[0053] Fig.19 Yes Description Fig.18Schematic diagram of display methods in the first force sense display unit, the second force sense display unit, and the third force sense display unit in the display device.

[0054] Fig. 20 This is a block diagram illustrating the configuration of a medical robot system according to a third modified example of the second embodiment of the present disclosure.

[0055] Fig.21 Yes Description Fig. 20 Schematic diagram of display methods in the first force sense display unit, the second force sense display unit, and the third force sense display unit in the display device.

[0056] Fig. 22 This is a block diagram illustrating the configuration of a medical robot system according to a fourth modification example of the second embodiment of the present disclosure.

[0057] Fig.23 Yes Description Fig. 22 Schematic diagram of display methods in the first force sense display unit, the second force sense display unit, and the third force sense display unit in the display device.

[0058] Description of Reference Numerals

[0059] 1. 101, 201, 301, 401, 501, 601, 701… medical robot systems;

[0060] 10, 310 ... slave unit (slave device); 14 ... surgical tool;

[0061] 10A...a first slave device (slave device); 10B...a second slave device (slave device);

[0062] 10C…third slave device (slave device); 13…joint portion; 50, 150, 250, 350, 450, 550, 650, 750…control device (force sense display device);

[0063] 51 ... an input unit; 53, 153, 253, 353, 453, 553, 653, 753 ... an image generating unit;

[0064] 54 ... output unit; 60 ... display device (display unit);

[0065] 61, 361 ... endoscope image display unit (first display area);

[0066] 62, 162 ... first force sense display unit (second display area);

[0067] 63, 163 ... second force sense display unit (second display area);

[0068] 260A: a first display device (display unit); 260B: a second display device (display unit);

[0069] 261 ...endoscopic image display unit (first display area);

[0070] 262 ...a first force sense display unit (a second display area);

[0071] 263 ... second force sense display unit (second display area); 362 ... first force sense display unit (sub-area);

[0072] 363 ...second force sense display portion (sub-region); 364 ...third force sense display portion (sub-region);

[0073] 365 ...force display unit (second display area); S12, S15, S18 ...input step;

[0074] S14, S17, S20...image generation step, output step DETAILED DESCRIPTION

[0075] [First embodiment]

[0076] Below, we will refer to Figure 1 A medical robot system 1 according to a first embodiment of the present disclosure will be described with reference to Fig. 7. The medical robot system 1 according to this embodiment is used to assist surgery. The medical robot system 1 according to this embodiment is suitable for assisting, for example, endoscopic surgery.

[0077] like Figure 1 As shown, the medical robot system 1 includes a slave unit 10, an endoscope unit 20, an operation unit 30, a sound generator 40, a control device 50, and a display device 60. The slave unit 10 is equivalent to an example of a slave device, the control device 50 is equivalent to an example of a force display device, and the display device 60 is equivalent to an example of a display unit.

[0078] like Figure 1 as well as Figure 2 As shown, the slave unit 10 supports a surgical tool 14 such as forceps used in endoscopic surgery. In addition, the slave unit 10 controls at least one of the position and posture of the surgical tool 14 according to the operation input to the operation unit 30. In addition, when the surgical tool 14 is a forceps, the slave unit 10 drives the forceps to open and close.

[0079] In this embodiment, the slave unit 10 includes a first slave device 10A and a second slave device 10B. The number of slave devices included in the slave unit 10 may be greater than or less than two. The first slave device 10A and the second slave device 10B each correspond to an example of a slave device.

[0080] In the present embodiment, the first slave device 10A and the second slave device 10B are robot arms, and the first slave device 10A and the second slave device 10B have the same configuration and support the same type of surgical tool 14 .

[0081] The first slave device 10A and the second slave device 10B only need to be able to control at least one of the position and posture of the surgical tool 14 or to drive the forceps as the surgical tool 14 to open and close, and do not need to have an arm-shaped structure.

[0082] like Figure 2 As shown, the first slave device 10A and the second slave device 10B are provided with at least a cassette 11, a shaft 12, and a joint portion 13. In addition, when it is not necessary to distinguish whether the cassette 11, the shaft 12, and the joint portion 13 are provided in the first slave device 10A or the second slave device 10B, they are marked as the cassette 11, the shaft 12, and the joint portion 13. In the case where it is necessary to distinguish whether they are provided in the first slave device 10A or the second slave device 10B, they are marked as the cassette 11A, the shaft 12A, and the joint portion 13A, and the cassette 11B, the shaft 12B, and the joint portion 13B, respectively.

[0083] The cassette 11 supports the shaft 12, the joint 13, and the surgical tool 14. In the present embodiment, the cassette 11 has a cubic shape.

[0084] The cassette 11 may contain a driving source such as a pneumatic actuator for driving at least one of the shaft 12, the joint 13, and the surgical tool 14. In addition, the cassette 11 may contain at least one of a shaft sensor 15 for measuring an external force applied to the shaft 12, a joint sensor 16 for measuring an external force applied to the joint 13, and a surgical tool sensor 17 for measuring an opening and closing force applied to the outside by forceps as the surgical tool 14.

[0085] In addition, when it is not necessary to distinguish whether the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17 are provided in the first slave device 10A or the second slave device 10B, they are marked as the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17. When it is necessary to distinguish whether they are provided in the first slave device 10A or the second slave device 10B, they are marked as the axis sensor 15A, the joint sensor 16A, and the surgical tool sensor 17A, and the axis sensor 15B, the joint sensor 16B, and the surgical tool sensor 17B, respectively.

[0086] The shaft 12 is a cylindrical or columnar member extending from the cassette 11. The shaft 12 is configured to be rotatable about its own central axis relative to the cassette 11. The shaft 12 may be cylindrical or columnar, or may be prismatic.

[0087] The joint portion 13 supports the surgical tool 14 and controls the position and posture of the surgical tool 14 by bending. The joint portion 13 is located between the shaft 12 and the surgical tool 14. In other words, one end of the joint portion 13 is adjacent to the end of the shaft 12 on the side away from the cassette 11, and the other end is adjacent to the surgical tool 14. In addition, as long as the structure of the joint portion 13 can be bent, its specific structure is not particularly limited.

[0088] like Figure 1 As shown, the endoscope unit 20 is used to obtain images or pictures used in endoscopic surgery. Specifically, the image or picture of the area in the patient's body cavity that is the surgical object is obtained. The information of the obtained image or picture is output to the control device 50. In addition, the endoscope unit 20 is not limited to its structure as long as it can output the information of the image or picture.

[0089] The operation unit 30 is used to control the slave unit 10. Specifically, the operation unit 30 is used when an endoscopic surgical operation is performed by operating the first slave device 10A and the second slave device 10B.

[0090] The operation unit 30 includes a hand operation unit 31 and a foot operation unit 35. The hand operation unit 31 is used when an operator of the medical robot system 1 (for example, an operator of endoscopic surgery) manually inputs an operation of the slave unit 10.

[0091] The hand operation unit 31 includes a right-hand operation unit 32 corresponding to the right hand of the operator and a left-hand operation unit 33 corresponding to the left hand of the operator. The right-hand operation unit 32 and the left-hand operation unit 33 convert information input by the operator into a signal and output it to the control unit 52. In addition, the right-hand operation unit 32 and the left-hand operation unit 33 present a force sense to the operator based on the control signal input from the control unit 52.

[0092] The configuration and form of the right-hand operation unit 32 and the left-hand operation unit 33 are not limited. The hand operation unit 31 may include both the right-hand operation unit 32 and the left-hand operation unit 33 as described above, or may include only one of the right-hand operation unit 32 and the left-hand operation unit 33.

[0093] The operator uses the foot operation unit 35 to operate the operation and function of the slave unit 10. The foot operation unit 35 includes a first foot switch 36, a second foot switch 37, and a third foot switch 38. The first foot switch 36, the second foot switch 37, and the third foot switch 38 are switches operated by the operator by stepping on them.

[0094] For example, supplying and stopping the high-frequency current to the surgical tool 14, switching the control mode of the slave unit 10, etc. can be cited. In addition, the type and content of the operation performed by the foot operating unit 35 are not particularly limited. In addition, the foot operating unit 35 may have three foot switches as described above, may have less than three foot switches, or may have more than three foot switches.

[0095] The sound generating device 40 generates sound to at least some of the persons involved in the endoscopic surgery including the operator based on the control signal input from the control device 50. The configuration for generating the sound is not particularly limited.

[0096] The control device 50 controls the operation of the slave unit 10 and the sound generation in the sound generation device 40. The control device 50 controls the content displayed on the display device 60 and the display method.

[0097] like Figure 1 As shown, the control device 50 is an information processing device such as a computer having a CPU, ROM, RAM, an input / output interface, etc. The program stored in the storage device such as the ROM causes the CPU, ROM, RAM, and the input / output interface to work together and function as at least an input unit 51, a control unit 52, an image generation unit 53, and an output unit 54. CPU is an abbreviation for a central processing unit.

[0098] Signals are input from the outside to the input unit 51. Specifically, the input unit 51 is connected to the operation unit 30 so that signals related to operations can be input to the input unit 51, and the input unit 51 is connected to the endoscope unit 20 so that signals of images and / or pictures can be input to the input unit 51.

[0099] Moreover, the input unit 51 is connected to an axis sensor 15 for measuring an external force applied to the axis 12, a joint sensor 16 for measuring an external force applied to the joint 13, and a surgical tool sensor 17 for measuring an opening and closing force applied to the outside by a forceps serving as a surgical tool 14, so that a measurement signal can be input to the input unit 51.

[0100] In addition, the positions of the axis sensor 15, joint sensor 16, and surgical tool sensor 17 are not limited as long as they can measure the external force of their respective measurement objects. In addition, the method of measuring external force by the axis sensor 15, joint sensor 16, and surgical tool sensor 17 is not particularly limited.

[0101] The control unit 52 generates a signal for controlling the operation and function of the slave unit 10 and the surgical tool 14. The control signal is generated based on a signal related to the operation input from the operation unit 30.

[0102] The image generating unit 53 generates image information, which is information of a video and / or image displayed on the display device 60. Specifically, the image information is generated based on a signal of a video and / or image input from the endoscope unit 20, and a signal input from the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17. The content of the image information will be described later.

[0103] The output unit 54 outputs control signals generated by the control unit 52 and image information generated by the image generating unit 53. The output unit 54 is connected to the slave unit 10 and the surgical tool 14 to output control signals and is connected to the display device 60 to output image information.

[0104] The display device 60 displays images and / or pictures based on the image information output from the control device 50. The display device 60 displays images and / or pictures to the surgical participants including the operator of the endoscopic surgical operation. The display device 60 may be any device as long as it can display images and / or pictures, and its form or structure is not particularly limited.

[0105] Next, the control in the medical robot system 1 having the above-mentioned configuration, specifically, the control related to the display of force sense, will be described. Figure 3 The flow of the calculation process in the control device 50 will be described, and then the displayed content will be described.

[0106] like Figure 3 As shown, if power is supplied to the control device 50, the control device 50 performs a process of determining whether the time counter has reached a predetermined time in S11. In this embodiment, the predetermined time is 200 ms. In addition, the predetermined time may be 200 ms as described above, or may be greater or less than 200 ms.

[0107] When it is determined that the time counter has not reached the predetermined time, that is, when the determination is “NO”, the control device 50 executes the determination process of S11 again.

[0108] When it is determined that the time counter has reached the predetermined time, that is, when it is determined as "yes", the control device 50 transfers the processing to the input step of S12. In S12, the control device 50 performs processing to determine whether an external force acting on the shaft 12 has been detected. In other words, the control device 50 performs processing to determine whether an arm measurement signal as a force measurement signal is input from the shaft sensor 15 to the input unit 51.

[0109] In the case where it is determined that the external force acting on the shaft 12 has been detected, that is, in the case where the determination is "yes", the control device 50 transfers the processing to S13. In S13, the control device 50 performs processing to generate a control signal that presents the detected external force as a force sense. The detected external force is the magnitude of the force measured by the shaft sensor 15. The detected external force is estimated based on the measurement signal output from the shaft sensor 15.

[0110] Specifically, the following processing is performed: the hand operating unit 31 corresponding to the slave device provided with the shaft sensor 15 that outputs the measurement signal presents the detected external force as a force sense. For example, the following processing is performed: when the shaft sensor 15 that outputs the measurement signal measures the force acting on the shaft 12 of the first slave device 10A, the hand operating unit 31 (i.e., the right-hand operating unit 32 or the left-hand operating unit 33) used to operate the first slave device 10A presents the detected external force as a force sense.

[0111] The corresponding hand operation unit 31 provides the operator with a force sense corresponding to the force acting on the axis sensor 15 based on the control signal output from the control device 50 .

[0112] If the processing of S13 is completed, the control device 50 transfers the processing to the image generation step and output step of S14. In S14, the control device 50 performs the following processing: presenting the information about the external force corresponding to the detected external force as information that can be grasped visually. Specifically, the processing of generating image information is performed, and the image information is used to display an image and / or image corresponding not only to the slave device of the shaft sensor 15 that outputs the measurement signal, but also to the magnitude of the force estimated based on the input measurement signal. The content of the processing of generating image information will be described later. The generated image information is output to the display device 60. The display device 60 displays the image and / or image based on the input image information.

[0113] If the processing of S14 is completed, or if it is determined in S12 that no external force acting on the shaft 12 is detected, that is, if the determination is "No", the control device 50 transfers the processing to the input step of S15. In S15, the control device 50 performs a process of determining whether an external force acting on the joint part 13 has been detected. In other words, a process of determining whether a joint measurement signal as a force measurement signal is input from the joint sensor 16 to the input unit 51 is performed.

[0114] In the case where it is determined that an external force acting on the joint part 13 has been detected, that is, in the case where the determination is "yes", the control device 50 transfers the processing to S16. In S16, the control device 50 performs a process of generating a control signal, wherein the control signal is used to present the detected external force as a force sense. The detected external force is the magnitude of the force measured by the joint sensor 16. The detected external force is estimated based on the measurement signal output from the joint sensor 16.

[0115] Specifically, the following processing is performed: the hand operating unit 31 corresponding to the slave device provided with the joint sensor 16 that outputs the measurement signal is made to present the detected external force as a force sense. In addition, the processing of setting the intensity of the force sense to be presented is performed according to the magnitude of the force estimated based on the input measurement signal. The corresponding hand operating unit 31 presents the force sense corresponding to the force acting on the joint sensor 16 to the operator based on the control signal output from the control device 50.

[0116] If the processing of S16 is completed, the control device 50 transfers the processing to the image generation step and output step of S17. In S17, the control device 50 performs the following processing: presenting the information about the force corresponding to the detected external force as information that can be grasped visually. Specifically, the processing of generating image information is performed, and the image information is used to display an image and / or image corresponding not only to the slave device provided with the joint sensor 16 that outputs the measurement signal, but also to the magnitude of the force estimated based on the input measurement signal. The content of the processing of generating image information will be described later. The generated image information is output to the display device 60. The display device 60 displays the image and / or image based on the input image information.

[0117] If the processing of S17 is completed, or when it is determined in S15 that no external force acting on the joint part 13 is detected, that is, in the case of "No", the control device 50 transfers the processing to the input step of S18. In S18, the control device 50 performs a process of determining whether an external force acting on the surgical tool 14 has been detected. In other words, a process of determining whether a surgical tool measurement signal as a force measurement signal is input from the surgical tool sensor 17 to the input unit 51 is performed.

[0118] In the case where it is determined that an external force acting on the surgical tool 14 has been detected, that is, in the case where the determination is "yes", the control device 50 transfers the processing to S19. In S19, the control device 50 performs processing to generate a control signal that presents the detected external force as a force sense. The detected external force is the magnitude of the force measured by the surgical tool sensor 17. Specifically, the magnitude of the force measured by the surgical tool sensor 17 corresponds to the magnitude of the force of the forceps for gripping or the force for peeling. The detected external force is inferred based on the measurement signal output from the surgical tool sensor 17.

[0119] Specifically, the following processing is performed: the hand operating unit 31 corresponding to the slave device provided with the surgical tool sensor 17 that outputs the measurement signal presents the detected external force as a force sense. In addition, the processing of setting the intensity of the force sense to be presented is performed according to the magnitude of the force estimated based on the input measurement signal. The corresponding hand operating unit 31 presents the force sense corresponding to the force acting on the surgical tool sensor 17 to the operator based on the control signal output from the control device 50.

[0120] If the processing of S19 is completed, the control device 50 transfers the processing to the image generation step and output step of S20. In S20, the control device 50 performs the following processing: presenting the information about the force acting on the surgical tool corresponding to the detected external force as information that can be grasped visually. Specifically, the control device 50 performs the processing of generating image information, which is used to display images and / or images corresponding not only to the slave device of the surgical tool sensor 17 that outputs the measurement signal, but also to the magnitude of the force inferred based on the input measurement signal. The content of the processing of generating image information will be described later. The generated image information is output to the display device 60. The display device 60 displays the image and / or image based on the input image information.

[0121] When the process of S20 is completed, the control device 50 transfers the process to S21. In S21, the control device 50 performs a process of resetting the time counter to 0. When the reset process is completed, the control device 50 returns to S11 and repeatedly executes the above process. For example, the above process is repeatedly executed at least during the period when the medical robot system 1 assists the endoscopic surgery.

[0122] Next, the image information generated by the image generating unit 53 will be described. Specifically, the display method on the display device 60 will be described. Figure 4A It is a schematic diagram for explaining the manner in which the display device 60 performs display based on the image information generated by the image generating unit 53 of the present embodiment.

[0123] like Figure 4AAs shown in FIG. 1 , the image generating unit 53 sets the endoscopic image display unit 61 at the center in the left-right direction on the display surface of the display device 60, sets the first force sense display unit 62 at the lower left of the endoscopic image display unit 61, and sets the second force sense display unit 63 at the upper right of the endoscopic image display unit 61. In addition, the endoscopic image display unit 61 corresponds to an example of the first display area, and the first force sense display unit 62 and the second force sense display unit 63 each correspond to an example of the second display area.

[0124] The endoscope image display unit 61 is a region having a larger area than the first force sense display unit 62 and the second force sense display unit 63. The image generation unit 53 generates image information for displaying video or image information input from the endoscope unit 20 on the endoscope image display unit 61.

[0125] The first force sense display unit 62 and the second force sense display unit 63 are regions having a smaller area than the endoscopic image display unit 61. The image generation unit 53 generates image information based on signals input from the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17.

[0126] The first force sense display unit 62 and the second force sense display unit 63 are divided from other areas by a frame. In addition, the first force sense display unit 62 and the second force sense display unit 63 display based on the signals input from the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17 in the divided interior. The first force sense display unit 62 and the second force sense display unit 63 of this embodiment have frames formed in the same rectangular shape and in the same color. The display method in the first force sense display unit 62 and the second force sense display unit 63 will be described later.

[0127] In this embodiment, the first force sense display unit 62 is an area that displays based on signals input from the axis sensor 15, joint sensor 16 and surgical tool sensor 17 corresponding to the first slave device 10A, and the second force sense display unit 63 is an area that displays based on signals input from the axis sensor 15, joint sensor 16 and surgical tool sensor 17 corresponding to the second slave device 10B.

[0128] In addition, the first force sense display unit 62 may be an area that displays based on signals input from the axis sensor 15, joint sensor 16 and surgical tool sensor 17 corresponding to the second slave device 10B, and the second force sense display unit 63 may be an area that displays based on signals input from the axis sensor 15, joint sensor 16 and surgical tool sensor 17 corresponding to the first slave device 10A.

[0129] In the present embodiment, the first force sense display unit 62 is located at the lower left of the display surface of the display device 60, and the second force sense display unit 63 is located at the upper right of the display surface of the display device 60. However, the second force sense display unit 63 may be located at the upper right of the display surface of the display device 60. Figure 4B As shown, the first force sense display unit 62 is located at the upper left of the display surface, and the second force sense display unit 63 is located at the lower right of the display surface. Figure 4C As shown, the first force sense display portion 62 is located slightly above the left center of the display surface, and the second force sense display portion 63 is located slightly below the right center of the display surface.

[0130] In addition, in the present embodiment, the first force display unit 62 and the second force display unit 63 have frames formed in the same rectangular shape and in the same color as an example, but the first force display unit 62 and the second force display unit 63 may have frames formed in different shapes. Figure 5 As shown, the first force sense display portion 62 may have a frame formed in a rectangular shape, and the second force sense display portion 63 may have a frame formed in an elliptical shape.

[0131] Furthermore, in the present embodiment, the first force sense display unit 62 and the second force sense display unit 63 have frames of the same color. However, the first force sense display unit 62 and the second force sense display unit 63 may have frames of different colors.

[0132] Next, the display methods in the first force sense display unit 62 and the second force sense display unit 63 are described. In addition, since the display methods in the first force sense display unit 62 and the second force sense display unit 63 are the same, the display method in the first force sense display unit 62 is used as a representative for description, and the display method in the second force sense display unit 63 is omitted. In addition, the first force sense display unit 62 and the second force sense display unit 63 may display the names of the first slave device 10A and the second slave device 10B in order to easily distinguish between them.

[0133] exist Figure 6 , the display method in the first force sense display unit 62 is shown. That is, the three-dimensional model of the shaft 12, the joint 13, and the surgical tool 14 of the first slave device 10A is displayed in the first force sense display unit 62. That is, the three-dimensional model of the shaft 12m, the three-dimensional model of the joint 13m, and the three-dimensional model of the surgical tool 14m are displayed. The three-dimensional model is displayed in a manner reflecting the actual posture of the shaft 12 and the bending state of the joint 13.

[0134] In addition, the shape of the surgical tool 14m in the three-dimensional model is determined based on the type information indicating the type of the surgical tool 14 inputted via the input unit 51. The type information indicating the type of the surgical tool 14 is associated with the shape of the surgical tool 14m in the three-dimensional model to be displayed and stored in the control device 50. The image generating unit 53 acquires the shape information of the surgical tool 14m associated with the input type information, and generates image information for displaying the three-dimensional model.

[0135] The type information input via the input unit 51 may be information input by a worker who sets up the medical robot system 1 or an operator who performs endoscopic surgery, or may be a detection signal output from a sensor for detecting the type of the installed surgical tool 14 .

[0136] In the part of the shaft 12m in the three-dimensional model, a conical figure 15f is displayed, which indicates the magnitude and direction of the force based on the signal input from the shaft sensor 15. In addition, in the part of the joint 13m in the three-dimensional model, a conical figure 16f is displayed, which indicates the magnitude and direction of the force based on the signal input from the joint sensor 16.

[0137] In this embodiment, the distance from the bottom surface of the cone-shaped figure to the vertex corresponds to the magnitude of the force. In addition, the color displayed by the cone-shaped figure changes according to the magnitude of the force. For example, as the force increases, the color changes to green, yellow, and red. Moreover, in this embodiment, the direction from the bottom surface of the cone-shaped figure to the vertex represents the direction of the force.

[0138] In an area adjacent to the surgical tool 14 in the three-dimensional model, for example, in an area adjacent to the front end side of the surgical tool 14 in the three-dimensional model, a strip graph is displayed based on a signal input from the surgical tool sensor 17, and the strip graph shows the magnitude of the force of closing or opening the force of the forceps serving as the surgical tool 14.

[0139] In this embodiment, if Fig. 7A As shown, the force of opening the forceps as the surgical tool 14 is expressed by displaying in a peeling display mode, wherein the peeling display mode is a display mode in which the length of the strip-shaped graphic 14f is greater than the predetermined reference index 14s. Figure 7B As shown, the closing force of the surgical tool 14 is expressed by displaying it in a holding display mode, wherein the holding display mode is a display mode in which the length of the strip graphic 14f is smaller than the reference indicator 14s.

[0140] In addition, the difference in length between the strip graph 14f and the reference indicator 14s changes according to the magnitude of the force, and the magnitude of the force is indicated by the difference in length. For example, if the force increases, the difference in length is displayed larger, and if the force decreases, the difference in length is displayed smaller.

[0141] In addition, the color of the strip graphic 14f can change according to the magnitude of the force. For example, as the force increases, the color will change to green, yellow, and red. In the case where the surgical tool sensor 17 does not measure the force, such as Figure 7C As shown, the strip-shaped graphic 14f is displayed to have the same length as the reference indicator 14s and is displayed in white.

[0142] According to the control device 50 configured as above, it is possible to create image information, and display the image information on the display device 60, wherein the image information is associated with the location where the external force is measured and the information of the external force is displayed as an image. In other words, the displayed external force information can be displayed in a manner that allows identification of where the measurement was performed.

[0143] Furthermore, since the information on the external force is displayed as an image on the first force sense display unit 62 and the second force sense display unit 63 which are different from the endoscopic image display unit 61, it is easier to grasp the displayed information on the external force than when the information on the external force is displayed superimposed on the endoscopic image display unit 61. Furthermore, it does not hinder visual recognition of the force acting on the shaft 12, the joint 13, and the surgical tool 14 displayed on the endoscopic image display unit 61.

[0144] Compared with the case where the magnitude of the external force value is expressed by a numerical value, it is easier to intuitively grasp the magnitude of the external force value by expressing the magnitude of the external force value by at least one of color, shape, and area.

[0145] Compared to displaying the external force acting on one part (for example, only on the joint part 13), it is easier to grasp the conditions of the operated first slave device 10A and the second slave device 10B by displaying information related to the force acting on the joint part 13 and the external force acting on the shaft 12 which is a part other than the joint part 13 in the first slave device 10A and the second slave device 10B.

[0146] For example, when the first slave device 10A and the second slave device 10B are in contact, the external force generated by the contact acts on the first slave device 10A and the second slave device 10B except for the joint 13 (for example, the shaft 12). Since the information related to the external force is displayed, the operator can understand the situation that the first slave device 10A and the second slave device 10B are in contact.

[0147] By displaying the information of the force of forceps opening and the force of forceps closing as information on the force acting on the surgical tool 14, it is easy to understand the operation status of the surgical tool 14. In addition, the force of forceps opening is also expressed as peeling force, and the force of forceps closing is expressed as gripping force.

[0148] By displaying an image representing the shape of the surgical tool 14 held by the first slave device 10A and the second slave device 10B, it is easy to understand whether the displayed external force information is the external force information measured by the first slave device 10A or the external force information measured by the second slave device 10B.

[0149] As described above, by changing the length of the strip-shaped pattern 14f in comparison with the reference indicator 14s, whether the information related to the holding force or the information related to the peeling force is displayed, it is easy to visually recognize the information. In addition, by displaying the difference between the length of the strip-shaped pattern 14f and the reference indicator 14s as the size of the holding force or the size of the peeling force, it is easy to visually recognize the size of the holding force or the size of the peeling force.

[0150] As described above, the magnitude of the force of the surgical tool 14 is easily grasped by changing the color or hue of the strip pattern 14f based on the magnitude of the force measured by the surgical tool sensor 17. Therefore, the magnitude of the displayed gripping force or peeling force is easily grasped.

[0151] [First Modification of the First Embodiment]

[0152] Next, we will refer to Figure 8 as well as Fig. 9 The medical robot system involved in the first variant of the first embodiment of the present disclosure is described. The basic structure of the medical robot system of this variant is the same as that of the first embodiment, but the difference from the first embodiment is that no endoscope unit is provided. Therefore, in this variant, the endoscope unit is used. Figure 8 as well as Fig. 9 The contents related to the display of force sense will be described, and other descriptions will be omitted.

[0153] like Figure 8 As shown, the medical robot system 101 of this modification includes a slave unit 10, an operation unit 30, a sound generating device 40, a control device 150, and a display device 60. The control device 150 corresponds to an example of a force sense display device.

[0154] The control device 150 controls the operation of the slave unit 10 and the sound generation in the sound generation device 40 , and controls the content and display method displayed in the display device 60 .

[0155] like Figure 8As shown, the control device 150 is an information processing device such as a computer having a CPU, ROM, RAM, an input / output interface, etc. The program stored in the storage device such as the ROM causes the CPU, ROM, RAM, and the input / output interface to cooperate and function as at least an input unit 51, a control unit 52, an image generating unit 153, and an output unit 54.

[0156] The image generating unit 153 generates image information, which is information of a video and / or image displayed on the display device 60. Specifically, the image information is generated based on the signals input from the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17. The difference from the image generating unit 53 of the first embodiment is that the image information is not generated based on the signal of the video and / or image input from the endoscope unit 20.

[0157] Next, the control in the medical robot system 101 having the above-mentioned configuration will be described. In addition, since the control related to the display of force sense in the control device 150 is the same as that in the first embodiment, the description thereof will be omitted.

[0158] Next, the image information generated by the image generating unit 153 will be described. Specifically, the display method on the display device 60 will be described. Fig. 9 1 is a schematic diagram for explaining the manner in which the display device 60 performs display based on the image signal generated by the image generating unit 153 of the present modification.

[0159] like Fig. 9 As shown in the figure, the display surface of the display device 60 is divided into a left side and a right side, and the image generation unit 153 sets the first force sense display unit 162 on the left side and the second force sense display unit 163 on the right side. In other words, compared with the first embodiment, the difference is that the endoscope image display unit 61 is not set in the display surface of the display device 60. In addition, the first force sense display unit 162 and the second force sense display unit 163 are equivalent to an example of the second display area.

[0160] In addition, in this modification, the display surface is divided into left and right as an example, but the display surface may also be divided into upper and lower parts, and the division method is not limited. In addition, in this modification, the display area of ​​the first force sense display unit 162 is equal to the display area of ​​the second force sense display unit 163, but the display area of ​​the first force sense display unit 162 may be smaller or larger than the display area of ​​the second force sense display unit 163.

[0161] Moreover, in this modification, the first force sense display unit 162 and the second force sense display unit 163 are both surrounded by the same rectangular frame, but the frames of the first force sense display unit 162 and the second force sense display unit 163 may have different shapes or shapes other than rectangular shapes, and the shapes are not particularly limited. In addition, the colors of the frames of the first force sense display unit 162 and the second force sense display unit 163 may be the same color or different colors.

[0162] Since the display method in the first force sense display unit 162 and the second force sense display unit 163 of this modification is the same as the display method in the first force sense display unit 62 and the second force sense display unit 63 of the first embodiment, the description thereof will be omitted.

[0163] According to the control device 150 , since the endoscope unit 20 is not an essential configuration, the medical robot system 101 can be flexibly used even in surgeries that do not use the endoscope unit 20 , compared to the medical robot system 1 of the first embodiment.

[0164] [Second Modification of First Embodiment]

[0165] Next, we will refer to Fig.10 FIG. 11 and FIG. 12 illustrate a medical robot system according to a second variant of the first embodiment of the present disclosure. The basic structure of the medical robot system and the force display device of this variant is the same as that of the first embodiment, but the number of display devices provided is different from that of the first embodiment. Therefore, in this variant, Fig.10 11 illustrates the contents related to the display of force sense, and other explanations are omitted.

[0166] like Fig.10 As shown, the medical robot system 201 of the present modification includes a slave unit 10, an endoscope unit 20, an operation unit 30, a sound generating device 40, a control device 250, a first display device 260A, and a second display device 260B.

[0167] The control device 250 corresponds to a force sense display device, and the first display device 260A and the second display device 260B correspond to an example of a display unit.

[0168] The control device 250 controls the operation of the slave unit 10 and the sound generation in the sound generation device 40 , and controls the content and display method displayed on the first display device 260A and the second display device 260B.

[0169] like Fig.10As shown, the control device 250 is an information processing device such as a computer having a CPU, ROM, RAM, an input / output interface, etc. The program stored in the storage device such as the ROM causes the CPU, ROM, RAM, and the input / output interface to cooperate and function as at least the input unit 51, the control unit 52, the image generation unit 253, and the output unit 54.

[0170] The control device 250 controls the operation of the slave unit 10 and the sound generation in the sound generation device 40 , and controls the content and display method displayed in the display device 60 .

[0171] The image generation unit 253 generates image information, which is information of images and / or pictures displayed on the first display device 260A and the second display device 260B. Specifically, the image information output to the first display device 260A is generated based on the signal of the image and / or picture input from the endoscope unit 20, and the image information output to the second display device 260B is generated based on the signal input from the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17. The content of the image information will be described later.

[0172] The first display device 260A and the second display device 260B display images and / or pictures based on the image information output from the control device 250. The first display device 260A and the second display device 260B display images and / or pictures to the participants of the operation including the operator of the endoscopic surgery. The first display device 260A and the second display device 260B may be any device as long as they can display images and / or pictures, and their form or configuration is not limited.

[0173] In addition, the first display device 260A and the second display device 260B may have display surfaces of the same size, or one of them may have a larger display surface than the other. In addition, the method or structure of displaying images and / or pictures may be the same or different.

[0174] Next, the control in the medical robot system 201 having the above-mentioned configuration will be described. In addition, since the control related to the display of force sense in the control device 250 is the same as that in the first embodiment, the description thereof will be omitted.

[0175] Next, the image information generated by the image generating unit 253 will be described. Specifically, the display method in the first display device 260A and the second display device 260B will be described. FIG11 is a schematic diagram illustrating the display method of the first display device 260A and the second display device 260B based on the image signal generated by the image generating unit 253 of this modification.

[0176] like Fig.10 As shown, the image generation unit 253 sets the endoscope image display unit 261 on the display surface of the first display device 260A. In addition, the second display device 260B is divided into a left side and a right side, and the first force display unit 262 is set on the left side and the second force display unit 263 is set on the right side.

[0177] The endoscopic image display unit 261 corresponds to an example of the first display region. In addition, the first force sense display unit 262 and the second force sense display unit 263 correspond to an example of the second display region.

[0178] In addition, in this variation, the endoscopic image display unit 261 is set in the first display device 260A, and the first force sense display unit 262 and the second force sense display unit 263 are set in the second display device 260B. However, the first force sense display unit 262 and the second force sense display unit 263 may be set in the first display device 260A, and the endoscopic image display unit 261 may be set in the second display device 260B.

[0179] According to the control device 250, since the display device provided with the endoscopic image display unit 261 and the display device provided with the first force display unit 262 and the second force display unit 263 are different, it is easy to improve the visibility of the endoscopic image and the visibility of the force display.

[0180] [Second embodiment]

[0181] Next, we will refer to Figures 12 to 15 The medical robot system and force display device according to the second embodiment of the present disclosure are described. The basic structure of the medical robot system and force display device according to this embodiment is the same as that of the first embodiment, but the structure of the slave unit and the force display method are different from those of the first embodiment. Therefore, in this embodiment, the force display device is used. Figures 12 to 15 The configuration of the slave unit and the force sense display method are described, and other descriptions are omitted.

[0182] like Fig.12 As shown, the medical robot system 301 of this embodiment includes a slave unit 310, an endoscope unit 20, an operation unit 30, a sound generator 40, a control device 350, and a display device 60. The slave unit 310 is an example of a slave device, and the control device 350 is an example of a force display device.

[0183] Similar to the slave unit 10 of the first embodiment, the slave unit 310 supports a surgical tool 14 such as forceps used in endoscopic surgery, and controls at least one of the position and posture of the surgical tool 14 according to an operation input to the operation unit 30. In addition, when the surgical tool 14 is a forceps, the slave unit 310 drives the forceps to open and close.

[0184] In the present embodiment, the slave unit 310 includes a first slave device 10A, a second slave device 10B, and a third slave device 10C.

[0185] In this embodiment, the first slave device 10A, the second slave device 10B, and the third slave device 10C are robot arms. The third slave device 10C has the same configuration as the first slave device 10A and the second slave device 10B, and supports the same type of surgical tool 14 .

[0186] In addition, similar to the first slave device 10A and the second slave device 10B, the third slave device 10C only needs to control at least one of the position and posture of the surgical tool 14 or drive the forceps as the surgical tool 14 to open and close. In addition, the third slave device 10C does not need to have an arm-shaped structure.

[0187] and Figure 2 The first slave device 10A and the second slave device 10B are shown to be the same, and the third slave device 10C is provided with at least a cassette 11, a shaft 12, and a joint 13. In addition, when it is not necessary to distinguish in which of the first slave device 10A, the second slave device 10B, and the third slave device 10C the cassette 11, the shaft 12, and the joint 13 are provided, they are labeled as the cassette 11, the shaft 12, and the joint 13.

[0188] When it is necessary to distinguish whether it is set in the first slave device 10A, the second slave device 10B or the third slave device 10C, they are respectively marked as cassette 11A, shaft 12A and joint 13A; cassette 11B, shaft 12B and joint 13B; and cassette 11C, shaft 12C and joint 13C.

[0189] Moreover, if Fig.13 As shown, similar to the first slave device 10A and the second slave device 10B, the third slave device 10C is provided with an axis sensor 15 for measuring the external force applied to the axis 12, a joint sensor 16 for measuring the external force applied to the joint 13, and a surgical tool sensor 17 for measuring the opening and closing force applied to the outside by the forceps as the surgical tool 14.

[0190] In addition, when it is necessary to distinguish the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17 provided in the third slave device 10C, they are labeled as the axis sensor 15C, the joint sensor 16C, and the surgical tool sensor 17C.

[0191] The control device 350 controls the operation of the slave unit 310 and the sound generation in the sound generation device 40 , and controls the content and display method displayed in the display device 60 .

[0192] like Fig.12 As shown, the control device 350 is an information processing device such as a computer having a CPU, ROM, RAM, an input / output interface, etc. The program stored in the storage device such as the ROM causes the CPU, ROM, RAM, and the input / output interface to work together and function as at least the input unit 51, the control unit 52, the image generation unit 353, and the output unit 54.

[0193] The image generation unit 353 generates image information, which is information of a video and / or image displayed on the display device 60. Specifically, the image information is generated based on a signal of a video and / or image input from the endoscope unit 20, and a signal input from the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17. The content of the image information will be described later.

[0194] Next, the control in the medical robot system 301 having the above-mentioned configuration will be described. In addition, since the control related to the display of force sense in the control device 350 is the same as that in the first embodiment, the description thereof will be omitted.

[0195] Next, the image information generated by the image generating unit 353 will be described. Specifically, the display method on the display device 60 will be described. Fig.14 1 is a schematic diagram for explaining the manner in which the display device 60 performs display based on the image signal generated by the image generating unit 353 of this modification.

[0196] like Fig.14 As shown, the image generation unit 353 sets the endoscope image display unit 361 on the left side in the left-right direction and sets the force sense display unit 365 on the right side in the left-right direction on the display surface of the display device 60. Inside the force sense display unit 365, a first force sense display unit 362, a second force sense display unit 363, and a third force sense display unit 364 are sequentially set from the left side to the right side.

[0197] In addition, the endoscopic image display unit 361 corresponds to an example of the first display area, the force display unit 365 corresponds to an example of the second display area, and the first force display unit 362, the second force display unit 363, and the third force display unit 364 each correspond to an example of a sub-area.

[0198] The endoscope image display unit 361 has a larger area than the first force sense display unit 362 , the second force sense display unit 363 , and the third force sense display unit 364 . The image generation unit 353 generates image information for displaying video or image information input from the endoscope unit 20 on the endoscope image display unit 361 .

[0199] The first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are regions set inside the force sense display unit 365. In addition, the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are regions having a smaller area than the endoscopic image display unit 361. The image generation unit 353 generates image information based on the signals input from the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17.

[0200] The first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are divided from other areas by a frame. In addition, the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 display based on the signals input from the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17 in the divided interior. In the present embodiment, the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 have frames formed in the same rectangular shape. In addition, the first force sense display unit 362 and the second force sense display unit 363 have frames of the same color, and the third force sense display unit 364 has a frame of a different color from the first force sense display unit 362 and the second force sense display unit 363. The display method in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 will be described later.

[0201] In this embodiment, the first force sense display unit 362 is an area that displays based on signals input from the axis sensor 15, joint sensor 16 and surgical tool sensor 17 corresponding to the first slave device 10A, the second force sense display unit 363 is an area that displays based on signals input from the axis sensor 15, joint sensor 16 and surgical tool sensor 17 corresponding to the second slave device 10B, and the third force sense display unit 364 is an area that displays based on signals input from the axis sensor 15, joint sensor 16 and surgical tool sensor 17 corresponding to the third slave device 10C.

[0202] Next, the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are described. In addition, since the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are the same, the display method in the first force sense display unit 362 is described as a representative, and the description of the display methods in the second force sense display unit 363 and the third force sense display unit 364 is omitted.

[0203] In this embodiment, in order to easily distinguish the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364, the name "Left Arm" is displayed on the first force sense display unit 362, the name "Right Arm" is displayed on the second force sense display unit 363, and the name "Third Arm" is displayed on the third force sense display unit 364.

[0204] exist Fig.15 3 shows the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364. In the first force sense display unit 362, the three-dimensional model of the shaft 12, the joint 13, and the surgical tool 14 of the first slave device 10A is displayed. That is, the three-dimensional model of the shaft 12m, the three-dimensional model of the joint 13m, and the three-dimensional model of the surgical tool 14m are displayed. The three-dimensional model is displayed in a manner that reflects the actual posture of the shaft 12 and the bending state of the joint 13.

[0205] Similarly, the three-dimensional models of the shaft 12 , the joint 13 , and the surgical tool 14 of the second slave device 10B and the third slave device 10C are displayed on the second force sense display unit 363 and the third force sense display unit 364 , respectively.

[0206] In addition, the shape of the surgical tool 14m in the three-dimensional model is determined based on the type information indicating the type of the surgical tool 14 input via the input unit 51. The type information indicating the type of the surgical tool 14 is associated with the shape of the surgical tool 14m in the three-dimensional model to be displayed and stored in the control device 350. The image generation unit 353 acquires the shape information of the surgical tool 14m associated with the input type information, and generates image information for displaying the three-dimensional model.

[0207] The type information input via the input unit 51 may be information input by a worker who sets up the medical robot system 301 or an operator who performs endoscopic surgery, or may be a detection signal output from a sensor for detecting the type of the installed surgical tool 14 .

[0208] At a position adjacent to the shaft 12m in the three-dimensional model, a bar graph 315f is displayed that indicates the magnitude of the force based on the signal input from the shaft sensor 15. In the present embodiment, "Force (Shaft)" is displayed at the position where the bar graph 315f is displayed, and "Base" is displayed at the position of the base line of the bar graph 315f.

[0209] In addition, at a position adjacent to the joint part 13m in the three-dimensional model, a bar graph 316f is displayed that indicates the magnitude of the force based on the signal input from the joint sensor 16. In the present embodiment, "Force (Wrist)" is displayed at the position where the bar graph 316f is displayed, and "Base" is displayed at the position of the base line of the bar graph 316f.

[0210] In this embodiment, the column graph 315f and the column graph 316f are graphs extending in the horizontal direction, and the length of the column graph 315f and the column graph 316f corresponds to the magnitude of the force. In addition, the color of the column graph 315f and the column graph 316f changes according to the magnitude of the force. For example, as the force increases, the color changes to green, yellow, and red.

[0211] At a position adjacent to the surgical tool 14 in the three-dimensional model, a sector graphic 314 f is displayed that indicates the magnitude of the closing force or the opening force of the forceps as the surgical tool 14 based on the signal input from the surgical tool sensor 17 .

[0212] In this embodiment, the fan-shaped figure 314f is displayed in a semicircular frame. The semicircular frame is composed of a frame extending linearly in the horizontal direction and an arc-shaped frame protruding upward. In addition, a reference line extending upward from the center of the frame extending in the horizontal direction is provided.

[0213] In this embodiment, the force of closing the surgical tool 14 is expressed in a holding display mode, wherein the holding display mode is a display mode in which a fan-shaped figure 314f is displayed on the left side of the reference line. In addition, the force of opening the forceps as the surgical tool 14 is expressed in a peeling display mode, wherein the peeling display mode is a display mode in which a fan-shaped figure 314f is displayed on the right side of the reference line. In addition, the force of closing the surgical tool 14 can also be expressed by displaying a fan-shaped figure 314f on the right side of the reference line, and the force of opening the forceps as the surgical tool 14 can be expressed by displaying a fan-shaped figure 314f on the left side of the reference line.

[0214] The magnitude of the force is expressed by the area of ​​the fan-shaped figure 314f. In other words, the magnitude of the force is expressed by the size of the angle from the reference line. For example, if the force increases, the area of ​​the fan-shaped figure 314f will increase, and if the force decreases, the area of ​​the fan-shaped figure 314f will decrease.

[0215] Alternatively, the color of the sector graphic 314f may be changed according to the magnitude of the force. For example, as the force increases, the color may change to green, yellow, and red. When the surgical tool sensor 17 does not measure the force, only the semicircular frame and the reference line are displayed, and the sector graphic 314f is not displayed.

[0216] According to the above-mentioned control device 350, by displaying the external force measured in the corresponding slave device on the inner side of the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 respectively arranged corresponding to the first slave device 10A, the second slave device 10B, and the third slave device 10C, it is easy to grasp which slave device the displayed external force information is measured from even when a plurality of slave devices are arranged.

[0217] By making the shapes and at least a part of the colors of the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 different corresponding to the first slave device 10A, the second slave device 10B, and the third slave device 10C, respectively, it is easy to understand in which slave device the displayed external force information is measured.

[0218] By displaying images representing the shapes of the surgical tools 14 held by the first slave device 10A, the second slave device 10B, and the third slave device 10C respectively in the corresponding first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364, it is easy to grasp that the displayed external force information is the external force information measured by the slave device holding which surgical tool 14.

[0219] [First Modification of Second Embodiment]

[0220] Next, we will refer to Fig.16 as well as Fig.17 The medical robot system involved in the first variant of the second embodiment of the present disclosure is described. The basic structure of the medical robot system and the force display device of this variant is the same as that of the second embodiment, but the display method in the force display unit is different from that of the second embodiment. Therefore, in this variant, the force display unit is used. Fig.16 as well as Fig.17 The display method in the force sense display unit is described, and other descriptions are omitted.

[0221] like Fig.16 As shown, the medical robot system 401 of this embodiment includes the slave unit 310, the endoscope unit 20, the operation unit 30, the sound generating device 40, the control device 450, and the display device 60. The control device 450 corresponds to an example of a force display device.

[0222] The control device 450 controls the operation of the slave unit 310 and the sound generation in the sound generation device 40 , and controls the content and display method displayed in the display device 60 .

[0223] like Fig.16 As shown, the control device 450 is an information processing device such as a computer having a CPU, ROM, RAM, an input / output interface, etc. The program stored in the storage device such as the ROM causes the CPU, ROM, RAM, and the input / output interface to work together and function as at least the input unit 51, the control unit 52, the image generation unit 453, and the output unit 54.

[0224] The image generation unit 453 generates image information, which is information of a video and / or image displayed on the display device 60. Specifically, the image information is generated based on a signal of a video and / or image input from the endoscope unit 20, and a signal input from the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17. The content of the image information will be described later.

[0225] Next, the control in the medical robot system 401 having the above-mentioned structure is described. In addition, since the control related to the display of force sense in the control device 450 is the same as that in the first embodiment, its description is omitted. In addition, since the display method in the display device 60 is the same as that in the second embodiment, its description is omitted.

[0226] Next, the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are described. In addition, since the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are the same, the display method in the first force sense display unit 362 is described as a representative, and the description of the display methods in the second force sense display unit 363 and the third force sense display unit 364 is omitted.

[0227] exist Fig.173 shows the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364. In the first force sense display unit 362, the three-dimensional model of the shaft 12, the joint 13, and the surgical tool 14 of the first slave device 10A is displayed. That is, the three-dimensional model of the shaft 12m, the three-dimensional model of the joint 13m, and the three-dimensional model of the surgical tool 14m are displayed. The three-dimensional model is displayed in a manner that reflects the actual posture of the shaft 12 and the bending state of the joint 13.

[0228] Similarly, the three-dimensional models of the shaft 12 , the joint 13 , and the surgical tool 14 of the second slave device 10B and the third slave device 10C are displayed on the second force sense display unit 363 and the third force sense display unit 364 , respectively.

[0229] At the axis 12 m in the three-dimensional model, a cylindrical graphic 415 f having a conical front end indicating the magnitude and direction of force based on the signal input from the axis sensor 15 is displayed.

[0230] Furthermore, at the joint portion 13 m in the three-dimensional model, a cylindrical graphic 416 f having a conical tip indicating the magnitude and direction of force based on the signal input from the joint sensor 16 is displayed.

[0231] In this embodiment, the length of the cylindrical figure 415f and the cylindrical figure 416f corresponds to the magnitude of the force, and the extension direction of the cylindrical figure 415f and the cylindrical figure 416f corresponds to the direction of the force. In addition, the color of the cylindrical figure 415f and the cylindrical figure 416f can be changed according to the magnitude of the force. For example, as the force increases, the color can change to green, yellow and red.

[0232] At a position adjacent to the surgical tool 14 in the three-dimensional model, a sector graphic 414 f is displayed that indicates the magnitude of the closing force or the opening force of the forceps as the surgical tool 14 based on the signal input from the surgical tool sensor 17 .

[0233] In this embodiment, the closing force of the surgical tool 14 is expressed by a holding display mode in which one sector-shaped figure 414f is displayed. Also, the opening force of the forceps as the surgical tool 14 is expressed by a peeling display mode in which two sector-shaped figures 414f are displayed.

[0234] Furthermore, when one sector graphic 414f is displayed, the sector graphic 414f may be displayed including the reference line, and when two sector graphics 414f are displayed, the sector graphics 414f may be displayed with the reference line sandwiched therebetween and spaced apart.

[0235] The magnitude of the force is expressed by the area of ​​the sector-shaped figure 414f. For example, if the force increases, the area of ​​the sector-shaped figure 414f increases, and if the force decreases, the area of ​​the sector-shaped figure 414f decreases.

[0236] Alternatively, the color of the sector graphic 414f may be changed according to the magnitude of the force. For example, as the force increases, the color may change to green, yellow, and red. When the surgical tool sensor 17 does not measure the force, the sector graphic 414f is not displayed.

[0237] According to the control device 450, one or two sector graphics 414f are displayed to indicate information related to the holding force or the peeling force, thereby making it easy to visually identify the information. In addition, by displaying the area of ​​the sector graphics 414f as the size of the holding force or the size of the peeling force, it is easy to visually identify the size of the holding force or the size of the peeling force.

[0238] Moreover, when displaying one sector graphic 414f, the sector graphic 414f is displayed in a manner including a baseline, and when displaying two sector graphics 414f, the sector graphics 414f are displayed in a manner sandwiched between the baseline and separated by a gap, thereby making it easier to visually identify information.

[0239] [Second Modification of Second Embodiment]

[0240] Next, we will refer to Fig.18 as well as Fig.19 The medical robot system involved in the second variant of the second embodiment of the present disclosure is described. The basic structure of the medical robot system and the force display device of this variant is the same as that of the second embodiment, but the display method in the force display unit is different from that of the second embodiment. Therefore, in this variant, the force display unit is used. Fig.18 as well as Fig.19 The display method in the force sense display unit is described, and other descriptions are omitted.

[0241] like Fig.18 As shown, the medical robot system 501 of the present embodiment includes a slave unit 310, an endoscope unit 20, an operation unit 30, a sound generating device 40, a control device 550, and a display device 60. The control device 550 corresponds to an example of a force display device.

[0242] The control device 550 controls the operation of the slave unit 310 and the sound generation in the sound generation device 40. In addition, the control device 550 controls the content displayed on the display device 60 and the display method.

[0243] like Fig.18As shown, the control device 550 is an information processing device such as a computer having a CPU, ROM, RAM, an input / output interface, etc. The program stored in the storage device such as the ROM causes the CPU, ROM, RAM, and the input / output interface to work together and function as at least the input unit 51, the control unit 52, the image generation unit 553, and the output unit 54.

[0244] The image generating unit 553 generates image information, which is information of a video and / or image displayed on the display device 60. Specifically, the image information is generated based on a signal of a video and / or image input from the endoscope unit 20, and a signal input from the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17. The content of the image information will be described later.

[0245] Next, the control in the medical robot system 501 having the above-mentioned structure is described. In addition, since the control related to the display of force sense in the control device 550 is the same as that in the first embodiment, its description is omitted. In addition, since the display method in the display device 60 is the same as that in the second embodiment, its description is omitted.

[0246] Next, the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are described. In addition, since the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are the same, the display method in the first force sense display unit 362 is described as a representative, and the description of the display methods in the second force sense display unit 363 and the third force sense display unit 364 is omitted.

[0247] exist Fig.19 3 shows the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364. In the first force sense display unit 362, the three-dimensional model of the shaft 12, the joint 13, and the surgical tool 14 of the first slave device 10A is displayed. That is, the three-dimensional model of the shaft 12m, the three-dimensional model of the joint 13m, and the three-dimensional model of the surgical tool 14m are displayed. The three-dimensional model is displayed in a manner that reflects the actual posture of the shaft 12 and the bending state of the joint 13.

[0248] Similarly, the three-dimensional models of the shaft 12 , the joint 13 , and the surgical tool 14 of the second slave device 10B and the third slave device 10C are displayed on the second force sense display unit 363 and the third force sense display unit 364 , respectively.

[0249] At the axis 12 m in the three-dimensional model, a ring-shaped graphic 515 f indicating the magnitude and direction of force based on the signal input from the axis sensor 15 is displayed.

[0250] In addition, at the joint 13m in the three-dimensional model, a circular ring-shaped figure 516f is displayed, which indicates the magnitude and direction of the force based on the signal input from the joint sensor 16. In the present embodiment, the circular ring-shaped figures 515f and 516f are partially notched, but may be notched.

[0251] In addition, in this embodiment, the color displayed in the annular graphic 515f and the annular graphic 516f corresponds to the magnitude of the force and the direction in which the force acts. Specifically, the direction in which the force acts is displayed by changing the color of the area in the annular graphic 515f and the annular graphic 516f corresponding to the direction in which the force acts. And as the force increases, the color of the area corresponding to the direction in which the force acts changes to green, yellow, and red.

[0252] At a position adjacent to the surgical tool 14 in the three-dimensional model, a sector graphic 514 f is displayed that indicates the magnitude of the closing force or the opening force of the forceps as the surgical tool 14 based on the signal input from the surgical tool sensor 17 .

[0253] In this embodiment, the force of closing the surgical tool 14 is expressed by the gripping display mode in which the fan-shaped graphic 514f is displayed in red. Also, the force of opening the forceps as the surgical tool 14 is expressed by the peeling display mode in which the fan-shaped graphic 514f is displayed in blue.

[0254] The magnitude of the force is expressed by the area of ​​the sector-shaped figure 514f. For example, if the force increases, the area of ​​the sector-shaped figure 514f increases, and if the force decreases, the area of ​​the sector-shaped figure 514f decreases.

[0255] Alternatively, the depth of the color of the sector graphic 514f may be changed according to the magnitude of the force. For example, as the force increases, the color may become darker, and as the force decreases, the color may become lighter. When the surgical tool sensor 17 does not measure the force, the sector graphic 514f is not displayed.

[0256] According to the control device 550, the information can be easily visually recognized by displaying the sector graphics 514f in different colors or hues. For example, the sector graphics 514f displayed in the grip display mode and the peeling display mode can be identified by the displayed colors or hues.

[0257] The color or depth of the fan-shaped pattern 514f is changed according to the magnitude of the force, so that the magnitude of the force in the fan-shaped pattern 514f can be easily understood. Therefore, the magnitude of the displayed holding force or peeling force can be easily understood.

[0258] [Third Modification of Second Embodiment]

[0259] Next, we will refer to Fig. 20 as well as Fig.21 The medical robot system according to the third variant of the second embodiment of the present disclosure is described. The basic structure of the medical robot system and the force display device of this variant is the same as that of the second embodiment, but the display method in the force display unit is different from that of the second embodiment. Therefore, in this variant, the force display unit is used. Fig. 20 as well as Fig.21 The display method in the force sense display unit is described, and other descriptions are omitted.

[0260] like Fig. 20 As shown, the medical robot system 601 of the present embodiment includes the slave unit 310, the endoscope unit 20, the operation unit 30, the sound generating device 40, the control device 650, and the display device 60. The control device 650 corresponds to an example of a force sense display device.

[0261] The control device 650 controls the operation of the slave unit 310 and the sound generation in the sound generation device 40. In addition, the control device 650 controls the content displayed on the display device 60 and the display method.

[0262] like Fig. 20 As shown, the control device 650 is an information processing device such as a computer having a CPU, ROM, RAM, an input / output interface, etc. The program stored in the storage device such as the ROM causes the CPU, ROM, RAM, and the input / output interface to work together and function as at least the input unit 51, the control unit 52, the image generation unit 653, and the output unit 54.

[0263] The image generation unit 653 generates image information, which is information of a video and / or image displayed on the display device 60. Specifically, the image information is generated based on a signal of a video and / or image input from the endoscope unit 20, and a signal input from the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17. The content of the image information will be described later.

[0264] Next, the control in the medical robot system 601 having the above-mentioned structure is described. In addition, since the control related to the display of force sense in the control device 650 is the same as that in the first embodiment, its description is omitted. In addition, since the display method in the display device 60 is the same as that in the second embodiment, its description is omitted.

[0265] Next, the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are described. In addition, since the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are the same, the display method in the first force sense display unit 362 is described as a representative, and the description of the display methods in the second force sense display unit 363 and the third force sense display unit 364 is omitted.

[0266] exist Fig.21 , the display method in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 is shown. A circular graphic 615f indicating the magnitude of force based on the signal input from the shaft sensor 15 is displayed at the bottom of the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364. In the present embodiment, "Force (Shaft)" is displayed at the position where the circular graphic 615f is displayed.

[0267] A circular graphic 616f indicating the magnitude of force based on the signal input from the joint sensor 16 is displayed in the middle of the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364. In the present embodiment, "Force (Wrist)" is displayed at the position where the circular graphic 616f is displayed.

[0268] A circular graphic 614f indicating the magnitude of force based on the signal input from the surgical tool sensor 17 is displayed above the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364. In the present embodiment, "Grip" is displayed at the position where the circular graphic 614f is displayed.

[0269] In this embodiment, the colors of the circular graphics 615f, the circular graphics 616f, and the circular graphics 614f change according to the magnitude of the force. For example, as the force increases, the colors change to green, yellow, and red (or orange).

[0270] According to the control device 650, by expressing the magnitude of the external force value by color, it is easier to intuitively grasp the magnitude of the external force value than when expressing the magnitude of the external force value by numerical value.

[0271] [Fourth Modification of Second Embodiment]

[0272] Next, we will refer to Fig. 22 as well as Fig.23The medical robot system according to the fourth variant of the second embodiment of the present disclosure is described. The basic structure of the medical robot system and the force display device of this variant is the same as that of the second embodiment, but the display method in the force display unit is different from that of the second embodiment. Therefore, in this variant, the force display unit is used. Fig. 22 as well as Fig.23 The display method in the force sense display unit is described, and other descriptions are omitted.

[0273] like Fig. 22 As shown, the medical robot system 701 of this embodiment includes the slave unit 310, the endoscope unit 20, the operation unit 30, the sound generating device 40, the control device 750, and the display device 60. The control device 750 corresponds to an example of a force display device.

[0274] The control device 750 controls the operation of the slave unit 310 and the sound generation in the sound generation device 40. In addition, the control device 750 controls the content displayed on the display device 60 and the display method.

[0275] like Fig. 22 As shown, the control device 750 is an information processing device such as a computer having a CPU, ROM, RAM, an input / output interface, etc. The program stored in the storage device such as the ROM causes the CPU, ROM, RAM, and the input / output interface to work together and function as at least the input unit 51, the control unit 52, the image generation unit 753, and the output unit 54.

[0276] The image generation unit 753 generates image information, which is information of a video and / or image displayed on the display device 60. Specifically, the image information is generated based on a signal of a video and / or image input from the endoscope unit 20, and a signal input from the axis sensor 15, the joint sensor 16, and the surgical tool sensor 17. The content of the image information will be described later.

[0277] Next, the control in the medical robot system 701 having the above-mentioned structure is described. In addition, since the control related to the display of force sense in the control device 750 is the same as that in the first embodiment, its description is omitted. In addition, since the display method in the display device 60 is the same as that in the second embodiment, its description is omitted.

[0278] Next, the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are described. In addition, since the display methods in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 are the same, the display method in the first force sense display unit 362 is described as a representative, and the description of the display methods in the second force sense display unit 363 and the third force sense display unit 364 is omitted.

[0279] exist Fig.23 , the display method in the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364 is shown. A circular graphic 715f indicating the magnitude of force based on the signal input from the shaft sensor 15 is displayed at the bottom of the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364. In the present embodiment, "Force (Shaft)" is displayed at the position where the circular graphic 715f is displayed.

[0280] A circular graphic 716f indicating the magnitude of force based on the signal input from the joint sensor 16 is displayed in the middle of the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364. In the present embodiment, "Force (Wrist)" is displayed at the position where the circular graphic 716f is displayed.

[0281] A circular graphic 714f indicating the magnitude of force based on the signal input from the surgical tool sensor 17 is displayed above the first force sense display unit 362, the second force sense display unit 363, and the third force sense display unit 364. In the present embodiment, "Grip" is displayed at the position where the circular graphic 714f is displayed.

[0282] In this embodiment, the areas of the circular graphics 715f, 716f, and 714f change according to the magnitude of the force, and the colors of the circular graphics 715f, 716f, and 714f also change according to the magnitude of the force. For example, as the force increases, the areas of the circular graphics increase, and the colors of the circular graphics change to green, yellow, and red (or orange).

[0283] According to the control device 750, by expressing the magnitude of the external force value by color, shape, and area, it is easier to intuitively grasp the magnitude of the external force value than when expressing the magnitude of the external force value by numerical values.

[0284] In addition, the technical scope of the present disclosure is not limited to the above-mentioned embodiments, and various changes can be applied within the scope of the main purpose of the present disclosure. For example, the present disclosure is not limited to the application in the above-mentioned embodiments or modified examples, and can also be applied to embodiments formed by appropriate combinations of these embodiments or modified examples, without special limitation.

Claims

1. A force display device for a medical robot system, which displays external forces acting on at least one of a surgical tool and a slave device supporting the surgical tool to an operator in a manner perceptible to the operator, wherein the force display device for the medical robot system is characterized in that: an input unit to which measurement signals of external forces measured at two or more locations of at least one of the surgical tool and the slave device are input; An image generating unit, which creates image information based on the measurement signal, the image information is associated with the location where the external force is measured and is used to display the information of the external force as an image in a second display area different from the first display area, wherein displaying information related to the operation of the surgical tool in the first display area; as well as an output unit that outputs the created image information to a display unit that displays the image information, In the force display device of the medical robot system, The slave device has multiple, The image generating unit sets a sub-area for each of the slave devices in the second display area, and displays the image displayed as the external force inside the sub-area. The image generation unit creates the image information capable of identifying the plurality of slave devices by the shape of the sub-region and the color of at least a part of the sub-region.

2. The force display device for a medical robot system according to claim 1, characterized in that: The image generation unit creates the image information that expresses the magnitude of the value of the external force through at least one of a color, a shape, and an area of ​​the image displayed as the external force.

3. The force display device of the medical robot system according to claim 1, characterized in that: inputting type information into the input unit, the type information indicating the type of the surgical tool held by the slave device, The image generation unit creates the image information including an image representing the shape of the surgical tool in the sub-region based on the type information.

4. The force display device for a medical robot system according to any one of claims 1 to 3, characterized in that: The surgical tool is arranged at an end of the slave device extending in a rod shape, and a joint portion capable of changing the posture of the surgical tool is provided near the end of the slave device. The measurement signal input to the input unit includes a joint measurement signal related to the force acting on the joint, and an arm measurement signal related to the external force acting on a portion of the slave device other than the joint. The image generation unit creates the image information based on the joint measurement signal and the arm measurement signal, the image information displaying information related to the force acting on the joint and information related to the external force acting on a portion of the slave device other than the joint.

5. The force display device for a medical robot system according to claim 4, characterized in that: The measurement signal input to the input unit includes a surgical tool measurement signal related to the force acting on the surgical tool, The image generation section creates the image information based on the surgical tool measurement signal, the image information displaying information related to the force acting on the surgical tool.

6. A force display method for a medical robot system, which displays to an operator in a manner perceptible to the operator an external force acting on at least one of a surgical tool and a slave device supporting the surgical tool, the force display method for a medical robot system comprising the following steps: an input step in which measurement signals of external forces measured at two or more locations of at least one of the surgical tool and the slave device are input; an image generating step, in which image information is created based on the measurement signal, the image information is associated with the location where the external force is measured and is used to display the information of the external force as an image in a second display area different from the first display area, wherein information related to the operation of the surgical tool is displayed in the first display area; as well as an output step of outputting the created image information to a display unit that displays the image information, In the force sense display method of the medical robot system, The slave device has multiple, In the image generating step, a sub-area is provided for each of the slave devices in the second display area, and the image displayed as the external force is displayed inside the sub-area. In the image generating step, the image information capable of identifying the plurality of slave devices by the shape of the sub-region and the color of at least a part of the sub-region is created.

7. The force sense display method of the medical robot system according to claim 6, characterized in that: In the image generating step, the image information is created, the image information expressing the magnitude of the value of the external force by at least one of the color, shape, and area of ​​the image displayed as the external force.

8. The force sense display method of the medical robot system according to claim 7, characterized in that: In the input step, type information is input, wherein the type information indicates the type of the surgical tool held by the slave device. In the image generation step, the image information including an image representing the shape of the surgical tool in the sub-region is created based on the category information.

9. The force sense display method for a medical robot system according to any one of claims 6 to 8, characterized in that: The surgical tool is arranged at an end of the slave device extending in a rod shape, and a joint portion capable of changing the posture of the surgical tool is provided near the end of the slave device. The measurement signals input in the input step include: a joint measurement signal related to the force acting on the joint part, and an arm measurement signal related to the external force acting on a part of the slave device other than the joint part, In the image generating step, the image information is created based on the joint measurement signal and the arm measurement signal, the image information showing information related to the force acting on the joint part and information related to the external force acting on a part of the slave device other than the joint part.

10. The force sense display method of the medical robot system according to claim 9, characterized in that: The measurement signal input in the input step includes a surgical tool measurement signal related to the force acting on the surgical tool, In the image generating step, the image information is created based on the surgical tool measurement signal, the image information showing information related to the force acting on the surgical tool.

Images