Medical support arm and medical system

Abstract

A medical support arm includes a support arm that supports an endoscope, an arm control unit configured to cause the support arm to perform a plurality of different interference avoidance operations to avoid interference between the endoscope and a surgical tool while maintaining a state in which an objective lens of the endoscope is directed toward an observation target, and a determination unit that determines a combination of operation amounts of the plurality of interference avoidance operations.

Description

Technical Field

[0001] The present disclosure relates to a medical support arm and a medical system. Background Art

[0002] In endoscopic surgery, an image of the patient's abdominal cavity is captured using an endoscope (eg, an oblique endoscope), and surgery is performed while displaying the captured image captured by the endoscope on a display.

[0003] For example, Patent Document 1 discloses a technique related to controlling the degree of insertion of an oblique-viewing endoscope into a human body and the posture of the oblique-viewing endoscope.

[0004] Reference List

[0005] Patent Literature

[0006] Patent Document 1: JP 2016-219521 A Summary of the Invention

[0007] Technical issues

[0008] During laparoscopic surgery, surgical tools are inserted into the body separately from the endoscope. In this situation, it is desirable for the support arm supporting the endoscope to move the endoscope to avoid interfering with the surgical tools so that the operator can properly perform the surgery. On the other hand, it is also necessary to move the endoscope so that the operator can easily see the observation target (e.g., the area to be treated). Therefore, it is not easy to control the support arm to keep the endoscope in a position suitable for surgery.

[0009] Therefore, the present disclosure proposes a medical support arm and a medical system capable of properly controlling the movement of the support arm.

[0010] Solution to the problem

[0011] In order to solve the above problems, the medical support arm according to the present disclosure includes: a support arm that supports an endoscope; an arm control unit that is configured to cause the support arm to perform multiple different interference avoidance operations to avoid interference between the endoscope and the surgical tool while keeping the objective lens of the endoscope pointing to the observation target; and a determination unit that determines the combination of the operation amounts of the multiple interference avoidance operations. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 is a diagram showing the configuration of a robot arm that supports an endoscope.

[0013] Figure 2 1 is a diagram showing the appearance of an oblique viewing endoscope.

[0014] Figure 3 is a schematic diagram showing a three-dimensional surface that expands conically relative to the observation point.

[0015] Figure 4 is a diagram for describing an interference avoidance zone.

[0016] Figure 5 : is a diagram showing a three-dimensional surface that expands conically with respect to an observation point and a cylindrical interference avoidance area in an overlapping manner.

[0017] Figure 6 This is an enlarged view of the area near the current position of the oblique endoscope.

[0018] Figure 7 is a diagram showing an example of a pre-designed program diagram.

[0019] Figure 8 is a diagram illustrating an example of a schematic configuration of an endoscopic surgery system to which the technology according to the present disclosure can be applied.

[0020] Figure 9 It shows Figure 8 A block diagram showing an example of the functional configuration of a camera head and a camera control unit (CCU) is shown.

[0021] Figure 10 is a schematic diagram showing the appearance of the support arm device according to the present embodiment.

[0022] Figure 11 is a schematic diagram showing the configuration of an oblique looking endoscope according to an embodiment of the present disclosure.

[0023] Figure 12 is a schematic diagram showing a comparison of an oblique-viewing endoscope and a forward-viewing endoscope.

[0024] Figure 13 is a block diagram illustrating a configuration example of a medical observation system according to an embodiment of the present disclosure.

[0025] Figure 14 is a diagram illustrating a specific configuration example of a robot arm device according to an embodiment of the present disclosure.

[0026] Figure 15 is a flowchart illustrating an example of an interference avoidance process for avoiding interference between an oblique looking endoscope and a surgical tool.

[0027] Figure 16 is a diagram showing a modification of the oblique looking endoscope. DETAILED DESCRIPTION

[0028] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. Note that in each of the following embodiments, the same reference numerals denote the same parts, and overlapping descriptions will be omitted.

[0029] Furthermore, the present disclosure will be described in the following order.

[0030] 1. INTRODUCTION

[0031] 1-1. Objectives of the Present Embodiment, etc.

[0032] 1-2. Overview of the Present Embodiment

[0033] 2. Configuration of the Medical System

[0034] 2-1. First Configuration Example (Endoscope System)

[0035] 2-2. Specific Configuration Example of the Support Arm Device

[0036] 2-3. Specific Configuration Example of the Endoscope

[0037] 2-4. Second Configuration Example (Medical Observation System)

[0038] 3. Operation of the Medical System

[0039] 4. Modifications

[0040] 5. CONCLUSION

[0041] 1. INTRODUCTION

[0042] 1-1. Objectives of the Present Embodiment, etc.

[0043] In a minimally invasive surgery (e.g., laparoscopic surgery), an assistant called a scope doctor usually holds and operates an endoscope with his / her hands according to the surgeon's instructions or a surgical procedure. The skill of the scope doctor allows the surgeon to see what he / she wants to see through the images captured by the endoscope.

[0044] In recent years, in surgeries using an endoscope, a method of replacing the scope doctor with an endoscope holding arm has been proposed. However, this method has a problem of a complicated operation method. To solve the problem of operability, the holding arm (hereinafter referred to as a support arm) itself can autonomously move the endoscope.

[0045] Note that in a minimally invasive surgery, a oblique-viewing endoscope, a side-viewing endoscope, or the like is used as an endoscope, and there also exists a rigid endoscope having a variable oblique angle. Further, there also exists a rigid endoscope having a configuration in which a distal end portion can be bent. These rigid endoscopes have various advantages, such as being able to observe a lesion from different directions or being able to observe a lesion without interfering with other surgical instruments in the body.

[0046] Traditionally, scopists have avoided interference between the strabismus endoscope and surgical instruments by adjusting the rotation amount and the insertion / removal degree of the strabismus endoscope based on experience. Note that interference avoidance using adjustment of the rotation amount has the disadvantage of changing the viewing direction. On the other hand, interference avoidance using adjustment of the insertion / removal degree has the disadvantage of losing detail in the observed target. For this reason, scopists instinctively combine the two processing variables (rotation amount and insertion / removal degree) to achieve optimal image capture desired by the surgeon while avoiding interference between the strabismus endoscope and the instrument.

[0047] In order for the support arm of an endoscope to perform such an operation, a control device (e.g., a processor) that controls the support arm needs to autonomously determine each of the two processing quantities (rotation amount and insertion / removal degree) without relying on human senses. However, such a determination method has not been implemented to date.

[0048] For example, Patent Document 1 (JP 2016-219521 A) discloses a technique related to controlling the insertion degree and posture of an oblique looking endoscope, but the technique described in Patent Document 1 is not a model that considers the rotation of the oblique looking endoscope.

[0049] Therefore, in this embodiment, a benchmark called the rotation-insertion ratio (R / I ratio) is defined, allowing designers to design the amount of rotation and the degree of insertion of the oblique endoscope according to the situation. Then, in this embodiment, the control device of the support arm uses the design results to operate the support arm according to the situation. As a result, the surgeon can capture the optimal image desired while avoiding interference between the oblique endoscope and the instrument.

[0050] Note that in the following description, "insertion" can be used as a broad insertion, including removal (pulling operation). The term "insertion" appearing in the following description can be appropriately replaced with "removal" or "insertion / removal". In addition, the term "insertion / removal" appearing in the following description can be appropriately replaced with "insertion" or "removal". Similarly, the term "removal" appearing in the following description can be appropriately replaced with "insertion" or "insertion / removal".

[0051] <1-2. Overview of this embodiment>

[0052] The operation for avoiding interference between the oblique endoscope and the surgical tool (hereinafter referred to as the interference avoidance operation) is determined by combining an operation of pulling the oblique endoscope (removal operation) and an operation of rotating the oblique endoscope (rotation operation). However, as described above, the rotation operation causes a change in the viewing direction, and the removal operation causes a loss of detail. Therefore, the control device of the support arm does not simply move the oblique endoscope in a predetermined constant direction (e.g., the direction in which the oblique endoscope is pulled) to avoid interference.

[0053] In the present embodiment, the control device of the support arm calculates a ratio between a minimum operation amount of the support arm to pull the oblique endoscope until the interference is eliminated and a minimum operation amount of the support arm to rotate the oblique endoscope until the interference is eliminated. Then, the control device determines a combined operation amount of the two operations (the removal operation and the rotation operation) based on the ratio and information of a pre-designed program chart. The ratio and the program chart will be described in detail later.

[0054] Note that the operation amount can also be referred to as a processing amount. The term "operation amount" appearing in the following description can be appropriately replaced with the term "processing amount".

[0055] The method of determining the operation amount according to the present embodiment is a method of determining the operation amount according to a program chart. Therefore, the designer of the control device can pre-design a plurality of program charts so that the control device of the support arm can change the adjustment method of the rotation operation and the removal operation according to the stage of the surgery. The control device of the support arm can perform appropriate interference avoidance operation according to the stage of the surgery with the use of the information of the pre-designed program chart.

[0056] For easy understanding, an outline of the present embodiment will be described below with reference to the drawings.

[0057] (Outline of device configuration)

[0058] Figure 1 is a view showing the configuration of a mechanical arm A (one aspect of a computer-assisted surgery system) that supports an oblique endoscope E. The mechanical arm A is an example of the medical support arm of the present embodiment. The oblique endoscope E is connected to the mechanical arm A. As described above, the oblique endoscope is a kind of endoscope. Note that, in the present embodiment, the endoscope includes a scope (a scope tube) and a camera, but the endoscope does not necessarily include the camera. For example, only the portion corresponding to the scope (the scope tube) can be regarded as the endoscope. The mechanical arm of the present embodiment supports, for example, the camera attached to the scope (the scope tube).

[0059] Motors for controlling each joint are provided inside the mechanical arm A. The oblique endoscope E is inserted into the patient's body through a trocar T1, and captures an image of an object or a point (hereinafter referred to as an observation target or an observation point) of interest to the operator and its surroundings. Here, the trocar T3 is an instrument called a medical puncture instrument. Note that surgical instruments (for example, the instruments S1 and S2 shown) are also inserted into the patient's body through a trocar (for example, the trocars T1 and T2 shown). Figure 1 Figure 1 The operator (for example, a surgeon) performs a laparoscopic surgery while watching the image captured by the endoscope E

[0060] (Relationship between oblique endoscope and cone surface)

[0061] ​ Figure 2 is a diagram illustrating the appearance of an oblique endoscope E. The oblique endoscope E is located on an axis and includes an objective lens F located at the distal end of the axis. The orientation of the objective lens F toward the observation point is tilted at an angle t1 relative to the axial direction of the oblique endoscope E. As an example, the angle t1 is 30° to 40°. In the following description, the angle t1 may be referred to as the oblique angle.

[0062] As long as the three-dimensional surface expands conically relative to the observation point, the oblique endoscope E can observe around the same point. Figure 3 This diagram is a schematic diagram showing a three-dimensional surface that expands conically relative to an observation point. The control device of robotic arm A can maintain the position of objective lens F of oblique endoscope E on the conical surface, so that objective lens F faces the observation point. The angle t2 of the cone's apex is determined based on the oblique angle t1.

[0063] (Setting of interference avoidance area)

[0064] Note that in this embodiment, in order to avoid interference between the oblique endoscope E and the surgical instrument, the control device of the robot arm A is operated so that the oblique endoscope E does not enter the column portion (column) predetermined according to the observation point. In the following description, this area for avoiding interference is referred to as the interference avoidance area.

[0065] Figure 4 is a diagram for describing an interference avoidance area. Figure 4 In the example shown in FIG, a cylindrical region with a predetermined radius centered on surgical tool S1 is the interference avoidance region. The diameter of the cylindrical region can be arbitrarily set according to the surgical tool. Note that the interference avoidance region does not necessarily have to be cylindrical. For example, the interference avoidance region can have a shape that combines multiple cylindrical regions with different diameters. In this case, the shape of the cylindrical region can change depending on the distance from the observation point.

[0066] (Definition of R / I Ratio)

[0067] Figure 5 is a diagram showing a three-dimensional surface that expands conically relative to the observation point and the cylindrical interference avoidance area in an overlapping manner. Figure 5 , direction R represents the direction of the rotation operation of the oblique endoscope E (rotation direction), and direction I represents the direction of the insertion / removal operation (removal operation and insertion operation) of the oblique endoscope E (insertion / removal direction). In addition, point P0 represents the current position of the objective lens F of the oblique endoscope E. The rotation direction R, the insertion / removal direction I, and the current position P0 are all located on the conical surface.

[0068] Note that in this embodiment, the rotation operation means that the objective lens F of the oblique endoscope E moves in the rotation direction R along the conical surface, and the insertion / removal operation (removal operation and insertion operation) means that the objective lens F of the oblique endoscope E moves in the insertion / removal direction I along the conical surface.

[0069] Figure 6 It is an enlarged view of the area near the current position P0 of the oblique looking endoscope E. Figure 6 The oblique line in is the intersection of the surfaces of the two solid bodies (the cone and the cylindrical portion) near the current position P0. Here, the rotation-to-insertion ratio (R / I ratio) is defined as shown in the following equation (1) or the following equation (2). The R / I ratio can be either equation (1) or equation (2).

[0070] R / I ratio = rθ / L...(1)

[0071] R / I ratio = θ / L...(2)

[0072] Here, θ is the minimum amount of rotation that can avoid interference by only rotating from the current position P0. Furthermore, r is the radius of the circle formed by cutting the cone through the current position P in the direction of rotation. Furthermore, L is the minimum degree of insertion / removal that can avoid interference by only removing (pulling) from the current position P0. Note that the degree of insertion / removal may also be referred to as a removal degree, an insertion degree (negative insertion degree), etc.

[0073] A large R / I ratio indicates that interference cannot be avoided unless the amount of rotation is large, and a small R / I ratio indicates that interference cannot be avoided unless the degree of insertion / removal is high.

[0074] Since equation (1) is an equation that takes into account the rotation angle and radius r, both the denominator and the numerator have the same distance unit. Therefore, when equation (1) is used to define the R / I ratio, a high-precision calculation result can be expected. However, the radius r needs to be calculated accordingly, which increases the processing load of the control device. On the other hand, equation (2) is a simplified expression that omits the radius r. Therefore, when equation (2) is used to define the R / I ratio, although the accuracy is slightly sacrificed, the calculation load of the control device can be reduced. Taking these advantages and disadvantages into consideration, the control device (or the designer of the control device) can choose whether to use equation (1) or equation (2) to define the R / I ratio.

[0075] (Procedure diagram)

[0076] The control device determines the combined operation amount of the two operations (the removal operation and the rotation operation) based on the R / I ratio and information of a pre-designed program map.

[0077] Figure 7 is a diagram showing an example of a pre-designed program diagram. Figure 7 The program diagram shown is a graph with R as the horizontal axis and I as the vertical axis. Note that in the following description, R may be used as a variable indicating the amount of rotation, rather than a symbol indicating the direction of rotation. Also, in the following description, I may be used as a variable indicating the degree of insertion / removal (degree of insertion or degree of removal), rather than a symbol indicating the direction of insertion / removal (direction of insertion or direction of removal). Figure 7 In the program diagram shown, the degree of removal increases upward and the amount of rotation increases to the right. Note that the amount of rotation R on the horizontal axis can be expressed in units of radius × rotation angle or rotation angle.

[0078] The control device of the robot arm A determines the insertion / removal degree and the rotation amount indicated by the intersection of the line indicated by the calculated R / I ratio (hereinafter also referred to as the oblique line) and the pre-designed line (hereinafter also referred to as the design line) as the combined operation amount of the oblique endoscope E. Here, the design line is Figure 7 The lines indicated by "suction" or "clamping" in the examples.

[0079] At any point on the oblique line, the R / I ratio has the same value. The control device of the robot arm A can achieve interference avoidance by setting the values ​​of R and I indicated by any point on the oblique line as the combined operation amount (insertion / removal degree and rotation amount). Note that the designer of the control device can design multiple design lines according to the situation of the operation, for example, Figure 7 The lines indicated by "Aspiration" and "Clamping" are shown. Here, aspiration is the process of removing fluid from the body using a suction instrument, while clamping is the process of clamping a blood vessel. Because clamping is delicate work, it requires high-quality images, whereas aspiration does not necessarily require high-quality images.

[0080] The designer of the control device designs the program diagram in consideration of these circumstances. For example, the designer performs a design so that the degree of insertion / removal does not vary as much as possible, thereby maintaining image quality when performing clamping that requires high precision. Figure 7 The design line of clamping shown is an example in which the design is performed so that the variation of the insertion / removal degree does not occur as much as possible when clamping is performed. On the other hand, the design is performed so that a relatively large variation of the insertion degree is allowed when suction is performed. Figure 7 The suction design lines shown are examples that allow for relatively large variations in the degree of insertion / removal when performing suction.

[0081] Note that the program diagram can be designed by a computer rather than a person (designer). In this case, the computer can be the control device of the robot arm A, or a computer (for example, a server device or a personal computer) used to design the program diagram independently of the robot arm A. The term "designer" that appears in the following description can be replaced by a computer (control device or design device).

[0082] The control device of the robot arm A determines the combined operation amount (insertion / removal degree and rotation amount) based on such a program diagram. For example, in a case where the treatment currently performed by the operator is "suction", the control device sets the values ​​of the rotation amount (R) and the insertion / removal degree (I) indicated by the intersection CP1 of the oblique line indicating the R / I ratio and the design line indicating suction as the combined operation amount. On the other hand, in a case where the treatment currently performed by the operator is "clamping", the control device sets the values ​​of R and I indicated by the intersection CP2 of the oblique line indicating the R / I ratio and the design line indicating clamping as the combined operation amount. By determining the combined operation amount based on the program diagram, the robot arm A can perform appropriate interference avoidance operations according to the situation of the operation.

[0083] Although the overview of the present embodiment has been described above, a medical system (computer-assisted surgery system) including the medical support arm (eg, robot arm A) of the present embodiment will be described in detail below.

[0084] <<2. Configuration of medical system>>

[0085] Before describing the operation of the medical system of the present embodiment, the configuration (device configuration and functional configuration) of the medical system will be described. For the medical system of the present embodiment, several configuration examples can be considered.

[0086] <2-1. First Configuration Example (Endoscope System)>

[0087] First, the configuration of an endoscope system will be described as an example of the medical system of the present embodiment.

[0088] Figure 8 is a diagram showing an example of a schematic configuration of an endoscopic surgery system 5000 to which the technology according to the present disclosure can be applied. Figure 8 , an operator (e.g., doctor) 5067 is shown performing surgery on a patient 5071 on a bed 5069 using an endoscopic surgery system 5000. As shown, the endoscopic surgery system 5000 includes an endoscope 5001, other surgical tools 5017, a support arm device 5027 for supporting the endoscope 5001, and a cart 5037 on which various devices used for endoscopic surgery are mounted.

[0089] Endoscope 5001 corresponds to, for example, Figures 1 to 3 and Figure 5 The endoscope E shown, the support arm device 5027 corresponds to, for example Figure 1 Robotic arm A is shown.

[0090] In endoscopic surgery, instead of cutting and opening the abdominal wall, a plurality of cylindrical puncture instruments called trocars 5025a to 5025d puncture the abdominal wall. Then, the lens barrel 5003 of the endoscope 5001 and other surgical tools 5017 are inserted into the body cavity of the patient 5071 through the trocars 5025a to 5025d. In the example shown, as other surgical tools 5017, a pneumoperitoneum tube 5019, an energy treatment tool 5021 and forceps 5023 are inserted into the body cavity of the patient 5071. In addition, the energy treatment tool 5021 is a therapeutic tool for cutting and peeling tissue, closing blood vessels, etc. by using high-frequency electric current or ultrasonic vibration. However, the surgical tool 5017 shown is merely an example, and various surgical tools (e.g., forceps and retractors) commonly used in endoscopic surgery can be used as the surgical tool 5017.

[0091] An image of the surgical site in the body cavity of a patient 5071, captured by the endoscope 5001, is displayed on the display device 5041. An operator 5067 performs treatment, for example, excision of an affected part, by using the energy treatment tool 5021 or the forceps 5023, while viewing the image of the surgical site displayed on the display device 5041 in real time. Note that, although not shown, the pneumoperitoneum tube 5019, the energy treatment tool 5021, and the forceps 5023 are supported by the operator 5067, an assistant, or the like during surgery.

[0092] [Support arm device]

[0093] The support arm device 5027 includes an arm 5031 extending from a base 5029. In the illustrated example, the arm 5031 includes joint portions 5033a, 5033b, and 5033c and links 5035a and 5035b, and is driven under the control of an arm control device 5045. The arm 5031 supports the endoscope 5001 and controls the position and posture of the endoscope 5001. As a result, the position of the endoscope 5001 can be stably fixed.

[0094] [Endoscope]

[0095] The endoscope 5001 includes a lens barrel 5003 that is inserted into a body cavity of a patient 5071 corresponding to an area of ​​a predetermined length starting from a distal end, and a camera 5005 connected to the proximal end of the lens barrel 5003. In the illustrated example, the endoscope 5001 is configured as a so-called rigid endoscope including the rigid lens barrel 5003, but the endoscope 5001 may be configured as a so-called flexible endoscope including a flexible lens barrel 5003.

[0096] An opening portion in which the objective lens is mounted is provided at the distal end of the lens barrel 5003. A light source device 5043 is connected to the endoscope 5001, and light generated by the light source device 5043 is guided to the distal end of the lens barrel by a light guide extending into the interior of the lens barrel 5003 and emitted toward an observation target in the body cavity of the patient 5071 via the objective lens. Note that the endoscope 5001 may be a forward-viewing endoscope, an oblique-viewing endoscope, or a side-viewing endoscope.

[0097] The optical system and imaging element are arranged inside the camera 5005, and the reflected light (observation light) from the observation target is collected by the optical system on the imaging element. The observation light is photoelectrically converted by the imaging element, and an electrical signal corresponding to the observation light is generated, that is, an image signal corresponding to the observed image. The image signal is transmitted as raw data to the camera control unit (CCU) 5039. Note that the camera 5005 has the function of adjusting the magnification and focal length by appropriately driving the optical system.

[0098] Note that, for example, to support stereoscopic viewing (3D display), a plurality of imaging elements may be provided in the camera 5005. In this case, a plurality of relay optical systems are provided inside the lens barrel 5003 to guide observation light to each of the plurality of imaging elements.

[0099] [Various devices installed on the cart]

[0100] The CCU 5039 is implemented by a central processing unit (CPU), a graphics processing unit (GPU), and the like, and controls the overall operation of the endoscope 5001 and the display device 5041. Specifically, the CCU 5039 performs various types of image processing on the image signals received from the camera 5005, for displaying an image based on the image signals, such as development processing (demosaicing). The CCU 5039 provides the image signals that have undergone image processing to the display device 5041. In addition, the CCU 5039 sends control signals to the camera 5005 to control its operation. The control signals may include information about imaging conditions, such as magnification and focal length.

[0101] Under the control of the CCU 5039, the display device 5041 displays images based on image signals processed by the CCU 5039. If the endoscope 5001 supports high-resolution imaging, such as 4K (3840 horizontal pixels × 2160 vertical pixels) or 8K (7680 horizontal pixels × 4320 vertical pixels), and / or if the endoscope supports 3D display, a display device capable of high-resolution display and / or a display device capable of 3D display can be used as the display device 5041 in each case. When a display device supports high-resolution imaging, such as 4K or 8K, a further sense of immersion can be achieved by using a display device of 55 inches or larger as the display device 5041. Furthermore, multiple display devices 5041 having different resolutions and sizes can be provided, depending on the application.

[0102] The light source device 5043 is realized by a light source such as a light emitting diode (LED), for example, and provides the endoscope 5001 with irradiation light for capturing an image of a surgical site.

[0103] The arm control device 5045 is implemented by a processor such as a CPU, for example, and operates according to a predetermined program to control the drive of the arm portion 5031 of the support arm device 5027 according to a predetermined control method. The arm control device 5045 corresponds to a control device for controlling the support arm of this embodiment (for example, a control device for the robot arm A). Note that the CCU 5039 can also be regarded as a control device of this embodiment.

[0104] The input device 5047 is an input interface of the endoscopic surgery system 5000. The user can input various types of information or instructions to the endoscopic surgery system 5000 via the input device 5047. For example, the user can input various types of information related to the surgery, such as the patient's physical information and information about the surgical procedure, via the input device 5047. In addition, for example, the user can input instructions for driving the arm unit 5031, instructions for changing the imaging conditions of the endoscope 5001 (type of irradiation light, magnification, focal length, etc.), instructions for driving the energy treatment tool 5021, etc. via the input device 5047.

[0105] The type of input device 5047 is not limited, and the input device 5047 may be various known input devices. As the input device 5047, for example, a mouse, a keyboard, a touch panel, a switch, a foot switch 5057, a lever, etc. can be applied. In the case where a touch panel is used as the input device 5047, the touch panel can be provided on the display surface of the display device 5041.

[0106] Optionally, the input device 5047 is a device worn by the user, such as a glasses-type wearable device or a head-mounted display (HMD), and various inputs are performed based on gestures or gazes of the user detected by these devices. In addition, the input device 5047 includes a camera capable of detecting the movement of the user, and various inputs are performed based on gestures or gazes of the user detected from the video captured by the camera. In addition, the input device 5047 includes a microphone capable of collecting the user's voice, and various inputs are performed by voice via the microphone. As described above, the input device 5047 is configured to be able to input various types of information in a non-contact manner, so that, in particular, a user belonging to a clean area (e.g., operator 5067) can operate a device belonging to an unclean area in a non-contact manner. In addition, since the user can operate the device without releasing his / her hands from the handheld surgical tool, the convenience of the user is improved.

[0107] The treatment tool control device 5049 controls the driving of the energy treatment tool 5021 for purposes such as cauterization and cutting of tissue, blood vessel sealing, etc. The pneumoperitoneum device 5051 supplies gas to the body cavity of the patient 5071 via the pneumoperitoneum tube 5019 to insulate the cavity, thereby ensuring a clear field of view for the endoscope 5001 and ensuring a working space for the operator. The recorder 5053 is a device capable of recording various types of information related to the surgery. The printer 5055 is a device capable of printing various types of information related to the surgery in various formats, such as text, images, or graphics.

[0108] Hereinafter, the specific characteristic configuration of the endoscopic surgery system 5000 will be described in more detail.

[0109] [Support arm device]

[0110] The support arm device 5027 includes a base 5029 serving as a foundation and an arm 5031 extending from the base 5029. The support arm device 5027 may include a control device serving as an arm control device 5045 and / or a CCU 5039. The support arm device 5027 corresponds to the support arm (e.g., robot arm A) of this embodiment. The arm 5031 may be considered the support arm of this embodiment.

[0111] In the illustrated example, the arm portion 5031 includes a plurality of joint portions 5033a, 5033b, and 5033c and a plurality of links 5035a and 5035b connected by the joint portion 5033b, but in Figure 8, for simplicity, the configuration of the arm 5031 is shown in a simplified manner. In actual implementation, the shape, number, and arrangement of the joint portions 5033a to 5033c and the connecting rods 5035a and 5035b, the rotation axis directions of the joint portions 5033a to 5033c, etc. can be appropriately set so that the arm 5031 has a desired degree of freedom. For example, the arm 5031 can be appropriately configured to have six or more degrees of freedom. As a result, since the endoscope 5001 can move freely within the movable range of the arm 5031, the lens barrel 5003 of the endoscope 5001 can be inserted into the body cavity of the patient 5071 from a desired direction.

[0112] Actuators are provided in the joint portions 5033a to 5033c, and the joint portions 5033a to 5033c are configured to rotate about a predetermined rotation axis by driving the actuators. The drive of the actuators is controlled by the arm control device 5045, thereby controlling the rotation angle of each joint portion 5033a to 5033c and controlling the drive of the arm portion 5031. As a result, the position and posture of the endoscope 5001 can be controlled. At this time, the arm control device 5045 can control the drive of the arm portion 5031 using various known control methods such as power control or position control.

[0113] For example, the operator 5067 can appropriately perform operation input via the input device 5047 (including the foot switch 5057) so that the arm control device 5045 appropriately controls the drive of the arm 5031 according to the operation input, thereby controlling the position and posture of the endoscope 5001. Through this control, the endoscope 5001 located at the distal end of the arm 5031 can be moved from any position to any position and then fixedly supported in the position after the move. Note that the arm 5031 can be operated by a so-called master-slave method. In this case, the user can remotely operate the arm 5031 (slave device) via the input device 5047 (master console) installed in a place away from the operating room or in the operating room.

[0114] In addition, when power control is applied, the arm control device 5045 can perform so-called power-assisted control, which receives external force from the user and drives the actuator of each joint portion 5033a to 5033c, so that the arm 5031 moves smoothly in accordance with the external force. As a result, when the user moves the arm 5031 while directly touching the arm 5031, the arm 5031 can be moved with relatively small force. Therefore, the endoscope 5001 can be moved more intuitively with simpler operations, and user convenience can be improved.

[0115] Here, generally speaking, in endoscopic surgery, the endoscope 5001 is supported by a doctor called a scopist. However, the use of the support arm device 5027 makes it possible to more reliably fix the position of the endoscope 5001 without manual operation, and thus, it is possible to stably obtain an image of a surgical site and smoothly perform surgery.

[0116] Note that the arm control device 5045 does not necessarily have to be provided in the cart 5037. Furthermore, the arm control device 5045 does not necessarily have to be one device. For example, the arm control device 5045 can be provided in each joint portion 5033a to 5033c of the arm portion 5031 of the support arm device 5027, and drive control for the arm portion 5031 can be achieved by a plurality of arm control devices 5045 that cooperate with each other.

[0117] [Light source device]

[0118] The light source device 5043 provides irradiation light for capturing an image of a surgical site to the endoscope 5001. The light source device 5043 includes a white light source realized by, for example, an LED, a laser light source, or a combination thereof. At this time, in the case where the white light source is realized by a combination of RGB laser light sources, the output intensity and the output time of each color (each wavelength) can be controlled with high precision, and thus, white balance adjustment of a captured image can be performed in the light source device 5043. Furthermore, in this case, the observation target is irradiated with laser light from each RGB laser source in a time-division manner, and the drive of the imaging element of the camera head 5005 is controlled in synchronization with the time of irradiation, so that an image corresponding to each RGB can also be captured in a time-division manner. With this method, a color image can be obtained without providing a color filter in the imaging element.

[0119] Furthermore, the drive of the light source device 5043 can be controlled so as to change the intensity of light to be output every predetermined time. The drive of the imaging element of the camera head 5005 is controlled in synchronization with the time of the change in light intensity to acquire images in a time-division manner, and the images are combined, so that a high dynamic range image can be generated without so-called underexposure and overexposure.

[0120] In addition, the light source device 5043 can be configured to provide light of a predetermined wavelength band corresponding to special light observation. In special light observation, for example, so-called narrow-band imaging is performed, in which, by utilizing the wavelength dependence of light absorption in body tissue, an image of a predetermined tissue (e.g., blood vessels in the mucosal epithelial layer) is captured with high contrast by radiating light in a narrower frequency band than the illumination light (i.e., white light) used in normal observation. Alternatively, in special light observation, fluorescence observation can be performed to obtain an image by fluorescence generated by irradiation with excitation light. In fluorescence observation, for example, fluorescence from body tissue can be observed by irradiating body tissue with excitation light (autofluorescence observation), or a fluorescence image can be obtained by locally injecting a reagent such as indocyanine green (ICG) into body tissue and irradiating the body tissue with excitation light corresponding to the fluorescence wavelength of the reagent. The light source device 5043 can be configured to provide narrow-band light and / or excitation light corresponding to such special light observation.

[0121] [Camera and CCU]

[0122] Will refer to Figure 9 The functions of the camera 5005 and CCU 5039 of the endoscope 5001 are described in more detail. Figure 9 It shows Figure 8 A block diagram of an example of the functional configuration of the camera 5005 and CCU 5039 is shown.

[0123] refer to Figure 9 The camera 5005 includes a lens unit 5007, an imaging unit 5009, a drive unit 5011, a communication unit 5013, and a camera control unit 5015. Furthermore, the CCU 5039 includes a communication unit 5059, an image processing unit 5061, and a control unit 5063. The camera 5005 and the CCU 5039 are connected via a transmission cable 5065 for bidirectional communication.

[0124] First, the functional configuration of the camera 5005 will be described. The lens unit 5007 is an optical system provided at the portion of the camera 5005 that connects to the lens barrel 5003. Observation light captured from the distal end of the lens barrel 5003 is guided to the camera 5005 and incident on the lens unit 5007. The lens unit 5007 is implemented by combining multiple lenses, including a zoom lens and a focus lens. The optical characteristics of the lens unit 5007 are adjusted to focus the observation light on the light-receiving surface of the imaging element of the imaging unit 5009. Furthermore, the zoom lens and the focus lens are configured to be movable along their optical axes to adjust the magnification and focus of the captured image.

[0125] The imaging unit 5009 includes an imaging element and is provided at a stage subsequent to the lens unit 5007. Observation light that has passed through the lens unit 5007 is collected on a light-receiving surface of the imaging element, and an image signal corresponding to the observed image is generated through photoelectric conversion. The image signal generated by the imaging unit 5009 is provided to the communication unit 5013.

[0126] For example, a complementary metal oxide semiconductor (CMOS) image sensor having a Bayer array and capable of color capture is used as the imaging element included in the imaging unit 5009. Note that as the imaging element, for example, an imaging element capable of 4K or higher resolution imaging can be used. Since a high-resolution image of the surgical site is obtained, the operator 5067 can grasp the state of the surgical site in more detail and can perform the surgery more smoothly.

[0127] Furthermore, the imaging elements included in imaging unit 5009 include a pair of imaging elements for acquiring image signals for the right and left eyes, respectively, corresponding to 3D display. When performing 3D display, operator 5067 can more accurately grasp the depth of living tissue within the surgical site. Note that when imaging unit 5009 is configured as a multi-lens type, multiple lens units 5007 are provided corresponding to the respective imaging elements.

[0128] Furthermore, the imaging unit 5009 does not have to be provided in the camera head 5005. For example, the imaging unit 5009 may be provided just behind the objective lens in the lens barrel 5003.

[0129] The driving unit 5011 is implemented by an actuator and moves the zoom lens and the focus lens of the lens unit 5007 by a predetermined distance along the optical axis under the control of the camera control unit 5015. As a result, the magnification and focus of the image captured by the imaging unit 5009 can be appropriately adjusted.

[0130] The communication unit 5013 is implemented by a communication device for sending and receiving various types of information to and from the CCU 5039. The communication unit 5013 transmits the image signal obtained from the imaging unit 5009 as raw data to the CCU 5039 via the transmission cable 5065. At this time, in order to display the captured image of the surgical site with low latency, the image signal is preferably transmitted via optical communication. This is because during surgery, the operator 5067 performs the surgery while observing the condition of the affected part in the captured image. Therefore, for safer and more reliable surgery, it is necessary to display a moving image of the surgical site as real-time as possible. When performing optical communication, a photoelectric conversion module is provided in the communication unit 5013 to convert electrical signals into optical signals. The image signal is converted into an optical signal by the photoelectric conversion module and then transmitted to the CCU 5039 via the transmission cable 5065.

[0131] In addition, the communication unit 5013 receives a control signal for controlling the driving of the camera 5005 from the CCU 5039. The control signal includes, for example, information about imaging conditions, such as information for specifying the frame rate of the captured image, information for specifying the exposure value during imaging, and / or information for specifying the magnification and focus of the captured image. The communication unit 5013 provides the received control signal to the camera control unit 5015. Note that the control signal from the CCU 5039 can also be transmitted via optical communication. In this case, a photoelectric conversion module that converts an optical signal into an electrical signal is provided in the communication unit 5013, and the control signal is converted into an electrical signal by the photoelectric conversion module and then provided to the camera control unit 5015.

[0132] Note that imaging conditions such as the frame rate, exposure value, magnification, and focus are automatically set based on the acquired image signal by the control unit 5063 of the CCU 5039. That is, the endoscope 5001 has a so-called automatic exposure (AE) function, an automatic focus (AF) function, and an automatic white balance (AWB) function.

[0133] The camera control unit 5015 controls the driving of the camera 5005 based on a control signal received from the CCU 5039 via the communication unit 5013. For example, the camera control unit 5015 controls the driving of the imaging element of the imaging unit 5009 based on information for specifying the frame rate of the captured image and / or information for specifying the exposure during imaging. In addition, for example, the camera control unit 5015 appropriately moves the zoom lens and focus lens of the lens unit 5007 via the drive unit 5011 based on information for specifying the magnification and focus of the captured image. The camera control unit 5015 may also have a function of storing information for identifying the lens barrel 5003 or the camera 5005.

[0134] Note that since the lens unit 5007, the imaging unit 5009, and the like are provided in a sealed structure having high airtightness and waterproofness, the camera head 5005 is resistant to high-pressure sterilization.

[0135] Next, the functional configuration of the CCU 5039 will be described. The communication unit 5059 is implemented by a communication device for transmitting and receiving various types of information to and from the camera 5005. The communication unit 5059 receives image signals transmitted from the camera 5005 via the transmission cable 5065. In this case, as described above, the image signals can be appropriately transmitted via optical communication. In this case, for optical communication, a photoelectric conversion module that converts optical signals into electrical signals is provided in the communication unit 5059. The communication unit 5059 provides the image signals converted into electrical signals to the image processing unit 5061.

[0136] Further, the communication unit 5059 transmits a control signal for controlling driving of the camera head 5005 to the camera head 5005. The control signal can also be transmitted through optical communication.

[0137] The image processing unit 5061 performs various types of image processing on the image signal that is the raw data transmitted from the camera head 5005. Examples of the image processing include various types of known signal processing, such as development processing, image quality enhancement processing (band emphasis processing, super-resolution processing, noise reduction (NR) processing, image stabilization processing, and the like), and / or enlargement processing (electronic zoom processing). Further, the image processing unit 5061 performs wave detection processing on the image signal for performing AE, AF, and AWB.

[0138] The image processing unit 5061 is implemented by a processor such as a CPU or a GPU, and the processor operates in accordance with a predetermined program, whereby the above-described image processing and wave detection processing can be performed. Note that in a case where the image processing unit 5061 is implemented by a plurality of GPUs, the image processing unit 5061 appropriately divides information related to the image signal, and the plurality of GPUs perform the image processing in parallel.

[0139] The control unit 5063 performs various types of control related to capturing of an image of a surgical site performed by the endoscope 5001 and display of the captured image. For example, the control unit 5063 generates a control signal for controlling driving of the camera head 5005. At this time, in a case where an imaging condition is input by a user, the control unit 5063 generates the control signal based on the input from the user. Alternatively, in a case where the endoscope 5001 has AE functionality, AF functionality, and AWB functionality, the control unit 5063 appropriately calculates an optimum exposure value, a focal distance, and a white balance in accordance with a result of the wave detection processing performed by the image processing unit 5061, and generates the control signal.

[0140] Further, the control unit 5063 causes the display device 5041 to display an image of the surgical site based on the image signal that has undergone the image processing by the image processing unit 5061. At this time, the control unit 5063 recognizes various objects in the image of the surgical site by using various image recognition techniques. For example, the control unit 5063 can recognize a surgical tool, such as a forceps, a specific site in a living body, bleeding, mist when the energy treatment tool 5021 is used, and the like, by detecting an edge shape, a color, and the like of the object included in the image of the surgical site. When the image of the surgical site is displayed on the display device 5041, the control unit 5063 superimposes various types of surgical support information on the image of the surgical site by using the recognition result. The surgical support information is superimposed and presented to the operator 5067, so that the surgery can be performed more safely and reliably.

[0141] The transmission cable 5065 connecting the camera head 5005 and the CCU 5039 is an electrical signal cable supporting electrical signal communication, an optical fiber supporting optical communication, or a composite cable thereof.

[0142] Here, in the example shown, wired communication is performed using the transmission cable 5065, but wireless communication can be performed between the camera 5005 and the CCU 5039. In the case where wireless communication is performed between the camera 5005 and the CCU 5039, there is no need to install the transmission cable 5065 in the operating room, and therefore, it is possible to eliminate the situation where the transmission cable 5065 hinders the movement of medical staff in the operating room.

[0143] In the above, an example of an endoscopic surgery system 5000 to which the technology according to the present disclosure can be applied has been described. Note that, while the endoscopic surgery system 5000 has been described here as an example, the system to which the technology according to the present disclosure can be applied is not limited to such an example. For example, the technology according to the present disclosure can be applied to a flexible endoscope system for inspection or a microsurgery system.

[0144] <2-2. Specific Configuration Example of Support Arm Device>

[0145] The medical system of this embodiment includes a support arm device. Hereinafter, a specific configuration example of the support arm device according to the embodiment of the present disclosure will be described in detail. Note that the use of the support arm device described below is not limited to medical use.

[0146] The support arm device described below is an example of a support arm device configured to support an endoscope at the distal end of the arm portion, but the present embodiment is not limited to such an example. In addition, when the support arm device according to the embodiment of the present disclosure is applied to the medical field, the support arm device according to the embodiment of the present disclosure can be used as a medical support arm device.

[0147] Note that the support arm device described below can be applied not only to the endoscopic surgery system 5000 but also to other medical systems. Of course, the support arm device described below can also be applied to systems other than medical systems. Furthermore, since the control unit (control device) that executes the processing of this embodiment is installed in the support arm device, the support arm device itself can be regarded as the medical system of this embodiment.

[0148] Figure 10 : is a schematic diagram showing the appearance of the support arm device 400 according to the present embodiment. The support arm device 400 corresponds to, for example Figures 1 to 3 and Figure 5 The robot arm A shown. In the following, reference will be made to Figure 10 A schematic configuration of the support arm device 400 according to the present embodiment is described.

[0149] The support arm device 400 according to the present embodiment includes a base 410 and an arm 420. The base 410 is the base of the support arm device 400, and the arm 420 extends from the base 410. Figure 10 Although not shown, a control unit that comprehensively controls the support arm device 400 may be provided in the base 410, and the drive of the arm 420 may be controlled by the control unit. The control unit is implemented by various signal processing circuits such as a CPU and a digital signal processor (DSP), for example.

[0150] The arm portion 420 includes a plurality of active joint portions 421 a to 421 f , a plurality of links 422 a to 422 f , and an endoscope device 423 as a distal end unit provided at a distal end of the arm portion 420 .

[0151] The links 422a to 422f are essentially rod-shaped members. One end of link 422a is connected to the base 410 via an active joint 421a, the other end of link 422a is connected to one end of link 422b via an active joint 421b, and the other end of link 422b is connected to one end of link 422c via an active joint 421c. The other end of link 422c is connected to link 422d via a passive slide mechanism 431, and the other end of link 422d is connected to one end of link 422e via a passive joint 433. The other end of link 422e is connected to one end of link 422f via active joints 421d and 421e. The endoscopic device 423 is connected to the distal end of the arm 420, i.e., the other end of link 422f, via the active joint 421f. In this manner, ends of the plurality of links 422 a to 422 f are connected to one another through active joint portions 421 a to 421 f with the base 410 as a fulcrum, the passive sliding mechanism 431 and the passive joint portion 433 , thereby forming an arm shape extending from the base 410 .

[0152] The position and posture of the endoscope device 423 are controlled by controlling the drive actuators provided in the active joint portions 421a to 421f of the arm 420. In this embodiment, the distal end of the endoscope device 423 enters the patient's body cavity serving as the surgical site and captures images of a portion of the surgical site. However, the distal unit provided at the distal end of the arm 420 is not limited to the endoscope device 423; various medical devices may be connected to the distal end of the arm 420 as the distal unit. As described above, the support arm device 400 according to this embodiment is configured as a medical support arm device that includes medical devices.

[0153] In the following, Figure 104. The coordinate axes are defined as shown to describe the support arm device 400. In addition, the up-down direction, the front-back direction, and the left-right direction are defined according to the coordinate axes. That is, the up-down direction relative to the base 410 installed on the floor surface is defined as the z-axis direction and the up-down direction. In addition, the direction perpendicular to the z-axis and in which the arm 420 extends from the base 410 (that is, the direction in which the endoscope device 423 is positioned relative to the base 410) is defined as the y-axis direction and the front-back direction. In addition, the direction orthogonal to the y-axis and the z-axis is defined as the x-axis direction and the left-right direction.

[0154] Active joint parts 421a to 421f are rotatably connected to each other by connecting rods. Active joint parts 421a to 421f all have actuators, and have a rotary mechanism that rotates relative to a predetermined rotation axis by driving the actuator. The drive of arm 420 can be controlled by controlling the rotation of each active joint part 421a to 421f, for example, the extension or contraction (folding) of arm 420. Here, the drive of active joint parts 421a to 421f can be controlled by, for example, known whole-body collaborative control and ideal joint control. As mentioned above, since active joint parts 421a to 421f all have a rotary mechanism, therefore in the following description, the drive control of active joint parts 421a to 421f specifically means controlling the rotation angle of active joint parts 421a to 421f and / or the torque (torque generated by active joint parts 421a to 421f) generated.

[0155] The passive sliding mechanism 431 is one aspect of the passive shape-changing mechanism and connects the connecting rod 422c and the connecting rod 422d to each other so that they can move forward and backward along a predetermined direction. For example, the passive sliding mechanism 431 can connect the connecting rod 422c and the connecting rod 422d to each other so that they can move linearly. However, the forward and backward movement of the connecting rod 422c and the connecting rod 422d is not limited to linear movement and can be forward and backward movement in the direction of forming an arc. For example, the user moves the passive sliding mechanism 431 forward and backward so that the distance between the active joint part 421c and the passive joint part 433 on one end side of the connecting rod 422c changes. As a result, the overall shape of the arm 420 can be changed.

[0156] Passive joint portion 433 is one aspect of the passive shape-changing mechanism and rotatably connects link 422d and link 422e. For example, when a user rotates passive joint portion 433, the angle formed by link 422d and link 422e changes. As a result, the overall shape of arm portion 420 can be changed.

[0157] The support arm device 400 according to the present embodiment includes six active joint sections 421a to 421f, and realizes six degrees of freedom when the arm section 420 is driven. That is, although the drive control of the support arm device 400 is realized by the drive control of the six active joint sections 421a to 421f by the control unit, the passive sliding mechanism 431 and the passive joint section 433 are not targets of the drive control performed by the control unit.

[0158] Specifically, as shown in Figure 10 the active joint sections 421a, 421d, and 421f are provided to have a long axis direction of each of the connected links 422a and 422e and an imaging direction of the endoscope device 423 as a rotation axis direction. The active joint sections 421b, 421c, and 421e are provided to have an x-axis direction as a rotation axis direction, the x-axis direction being a direction in which a connection angle of each of the connected links 422a to 422c, 422e, and 422f and the endoscope device 423 is changed in a y-z plane (a plane defined by a y-axis and a z-axis). As described above, in the present embodiment, the active joint sections 421a, 421d, and 421f have a function of performing so-called yaw, and the active joint sections 421b, 421c, and 421e have a function of performing so-called pitch.

[0159] With this configuration of the arm section 420, in the support arm device 400 according to the present embodiment, six degrees of freedom are realized when the arm section 420 is driven, and thus the endoscope device 423 can be freely moved within a movable range of the arm section 420. In Figure 10 , a hemisphere is shown as an example of the movable range of the endoscope device 423. Assuming that a center point of the hemisphere, that is, a remote center of motion (RCM) is a center of an image of a surgical site captured by the endoscope device 423, in a state in which the center of the image captured by the endoscope device 423 is fixed to the center point of the hemisphere, by moving the endoscope device 423 on a spherical surface of the hemisphere, the image of the surgical site can be captured at various angles.

[0160] The schematic configuration of the support arm device 400 according to the present embodiment has been described above. Next, the whole-body cooperative control and ideal joint control, that is, the drive of the active joint sections 421a to 421f, for controlling the drive of the arm section 420 in the support arm device 400 according to the present embodiment will be described.

[0161] Note that while the arm portion 420 of the support arm assembly 400 has been described as having multiple joints and six degrees of freedom, the present disclosure is not limited thereto. Specifically, the arm portion 420 may have a configuration in which an endoscope device 423 or an exoscope is provided at its distal end. For example, the arm portion 420 may have a configuration with only one degree of freedom, driving the endoscope device 423 to move in both the direction of entry into the patient's body cavity and the direction of rearward movement.

[0162] <2-3. Specific Configuration Example of Endoscope>

[0163] An endoscope can be mounted on the support arm device of this embodiment. Hereinafter, the basic configuration of an oblique endoscope will be described as an example of the endoscope of this embodiment. Note that the endoscope of this embodiment is not limited to the oblique endoscope described below, as long as the direction of the objective lens is inclined (or can be inclined) relative to the axial direction of the endoscope body.

[0164] Figure 11 Schematic diagram showing the configuration of an oblique viewing endoscope 4100 according to an embodiment of the present disclosure. Figure 11 As shown, the oblique endoscope 4100 is attached to the distal end of the camera head 4200. The oblique endoscope 4100 corresponds to the reference Figure 8 The lens barrel 5003 described, and the camera 4200 corresponds to the reference Figure 8 and 9 Description of camera 5005. Note that Figure 8 The endoscope 5001 shown can be considered as an oblique endoscope 4100.

[0165] The oblique viewing endoscope 4100 and the camera head 4200 can rotate independently of each other. Similar to each of the joint portions 5033a, 5033b, and 5033c, an actuator is provided between the oblique viewing endoscope 4100 and the camera head 4200, and the oblique viewing endoscope 4100 rotates relative to the camera head 4200 by the drive of the actuator.

[0166] The oblique viewing endoscope 4100 is supported by a support arm device 5027. The support arm device 5027 has a function of holding the oblique viewing endoscope 4100 rather than the scopist holding the oblique viewing endoscope 4100 and moving the oblique viewing endoscope 4100 so that a desired site can be observed according to an operation performed by an operator or an assistant.

[0167] Figure 12Schematic diagram comparing an oblique-viewing endoscope 4100 and a forward-viewing endoscope 4150. In the forward-viewing endoscope 4150, the direction (C1) of the objective lens toward the subject coincides with the longitudinal direction (C2) of the forward-viewing endoscope 4150. On the other hand, in the oblique-viewing endoscope 4100, a predetermined angle φ is formed between the direction (C1) of the objective lens toward the subject and the longitudinal direction (C2) of the oblique-viewing endoscope 4100. Note that when the angle φ is 90°, the oblique-viewing endoscope 4100 is referred to as a side-viewing endoscope.

[0168] <2-4. Second Configuration Example (Medical Observation System)>

[0169] Next, the configuration of a medical observation system 1 will be described as another configuration example of the medical system of this embodiment. Note that the support arm device 400 and the oblique viewing endoscope 4100 described above can also be applied to the medical observation system described below. Furthermore, the medical observation system described below can be considered a functional configuration example or modification of the endoscopic surgery system 5000 described above.

[0170] Figure 13 1 is a block diagram showing a configuration example of a medical observation system 1 according to an embodiment of the present disclosure. Figure 13 The configuration of a medical observation system according to an embodiment of the present disclosure is described.

[0171] like Figure 13 As shown, the medical observation system 1 includes a robot arm device 10 , a control unit 20 , an operation unit 30 , and a display unit 40 .

[0172] Figure 14 1 is a diagram showing a specific configuration example of the robot arm device 10 according to an embodiment of the present disclosure. The robot arm device 10 includes, for example, an arm portion 11 (articulated arm) having a multi-link structure including a plurality of joint portions and a plurality of links. The robot arm device 10 corresponds to, for example, Figures 1 to 3 and Figure 5 The robot arm A shown or Figure 10 The support arm device 400 is shown. The robot arm device 10 is operated under the control of the control unit 20. The robot arm device 10 controls the position and posture of the distal unit (e.g., endoscope) provided at the distal end of the arm 11 by driving the arm 11 within the movable range. The arm 11 corresponds to, for example, Figure 10 Arm 420 is shown.

[0173] The arm 11 includes a plurality of joint portions 111 . Figure 13 The configuration of one joint portion 111 is shown as a representative of the plurality of joint portions.

[0174] The joint portion 111 rotatably connects the link in the arm 11 and controls its rotation under the control of the control unit 20, thereby driving the arm 11. The joint portion 111 corresponds to, for example, Figure 10 Active joint parts 421a to 421f are shown. In addition, the joint part 111 can have an actuator.

[0175] like Figure 13 As shown, the joint part 111 includes one or more joint driving units 111 a and one or more joint state detection units 111 b .

[0176] The joint driving unit 111a is a driving mechanism in the actuator of the joint part 111, and the joint driving unit 111a performs driving to rotate the joint part 111. The joint driving unit 111a corresponds to Figure 14 The motor 5011 shown in FIG. 1 is driven by the arm control unit 23. For example, the joint drive unit 111a corresponds to a motor and a motor driver. The drive performed by the joint drive unit 111a corresponds to, for example, the motor driver driving the motor with a current according to a command from the control unit 20.

[0177] The joint state detection unit 111b is, for example, a sensor that detects the state of the joint portion 111. Here, the state of the joint portion 111 may mean the motion state of the joint portion 111. For example, the state of the joint portion 111 includes information such as the rotation angle, rotation angular velocity, rotation angular acceleration, and generated torque of the joint portion 111. The joint state detection unit 111b corresponds to Figure 14 The encoder 5021 shown in the figure is shown. In this embodiment, the joint state detection unit 111b functions as, for example, a rotation angle detection unit that detects the rotation angle of the joint portion 111 and a torque detection unit that detects the torque generated by the joint portion 111 and the external torque. Note that the rotation angle detection unit and the torque detection unit may be an encoder and a torque sensor of the actuator, respectively. The joint state detection unit 111b transmits the detected state of the joint portion 111 to the control unit 20.

[0178] Back to Figure 13 In addition to the arm 11, the robot arm device 10 also includes an endoscope 12. The endoscope 12 is, for example, an oblique endoscope. The endoscope 12 corresponds to, for example Figures 1 to 3 and Figure 5 The oblique endoscope E shown, Figure 8 The endoscope 5001 shown or Figure 11 The oblique endoscope 4100 shown in FIG. For example, the endoscope 12 is detachably provided at the distal end of the arm 11. Figure 13 As shown, the endoscope 12 includes an imaging unit 12a and a light source unit 12b.

[0179] The imaging unit 12a captures images of various imaging targets. The imaging unit 12a captures images of a surgical area including various medical instruments, organs, etc. in the abdominal cavity of a patient, for example. Specifically, the imaging unit 12a is a camera or the like capable of capturing images of imaging targets in the form of moving images or still images. More specifically, the imaging unit 12a is a wide-angle camera including a wide-angle optical system. That is, the surgical field image is a surgical field image captured by a wide-angle camera. For example, although the viewing angle of a normal endoscope is approximately 80°, the viewing angle of the imaging unit 12a according to the present embodiment may be 140°. Note that the viewing angle of the imaging unit 12a may be greater than 80° and less than 140°, or may be equal to or greater than 140°. The imaging unit 12a sends an electrical signal (image signal) corresponding to the captured image to the control unit 20. Note that in Figure 13 In the embodiment, the imaging unit 12 a does not need to be included in the robot arm device, and this aspect is not limited as long as the imaging unit 12 a is supported by the arm portion 11 .

[0180] In the light source unit 12b, the imaging unit 12a illuminates the imaging target with light. The light source unit 12b can be implemented by, for example, a wide-angle lens LED. For example, the light source unit 12b can be implemented by combining a normal LED and a lens to diffuse light. In addition, the light source unit 12b can be configured to diffuse the light transmitted through the optical fiber using a lens (increase its angle). In addition, the light source unit 12b can expand the irradiation range by irradiating the optical fiber itself with light in multiple directions. Note that in Figure 13 In the embodiment, the light source unit 12 b does not need to be included in the robot arm device 10 , and this aspect is not limited as long as the irradiation light can be guided to the imaging unit 12 a supported by the arm portion 11 .

[0181] Next, we will refer to Figure 14 A specific configuration example of the robot arm device 10 according to an embodiment of the present disclosure is described.

[0182] For example, Figure 14 As shown, the arm portion 11 of the robot arm device 10 includes a first joint portion 1111 , a second joint portion 1112 , a third joint portion 1113 and a fourth joint portion 1114 .

[0183] The first joint portion 1111 includes a motor 5011, an encoder 5021, a motor controller 5031, and a motor driver 5041. Since the second to fourth joint portions 1112 to 1114 also have the same configuration as the first joint portion 1111, the first joint portion 1111 will be described below as an example.

[0184] Note that each joint portion, including the first joint portion 1111, may include a brake for the motor 5011. In this case, the brake may be a mechanical brake. The joint portion may be configured to maintain the current state of the arm 11 by using the brake, for example, when the motor is not operating. Even if power to the motor is stopped for some reason, the arm 11 is secured by the mechanical brake, preventing the endoscope from moving to an unintended position.

[0185] The motor 5011 is driven under the control of the motor driver 5041 to drive the first joint portion 1111. The motor 5011 and / or the motor driver 5041 corresponds to, for example Figure 11 The joint driving unit 111a is shown. For example, the motor 5011 drives the first joint portion 1111 in the direction of the arrow attached to the first joint portion 1111. The motor 5011 controls the position and posture of the arm 11 or the position and posture of the lens barrel and the camera by driving the first joint portion 1111. Note that in this embodiment, as a form of endoscope, a camera (e.g., imaging unit 12a) can be provided at the distal end of the lens barrel.

[0186] The encoder 5021 detects information about the rotation angle of the first joint portion 1111 under the control of the motor controller 5031. That is, the encoder 5021 acquires information about the posture of the first joint portion 1111. The encoder 5021 detects information about the motor torque under the control of the motor controller 5031.

[0187] The control unit 20 controls the position and posture of the arm 11. Specifically, the control unit 20 controls the motor controllers 5031 to 5034, the motor drivers 5041 to 5044, etc. to control the first joint portion 1111 to the fourth joint portion 1114. By doing so, the control unit 20 controls the position and posture of the arm 11. The control unit 20 may be included in the robot arm device 10, or may be a device separate from the robot arm device 10. The control unit 20 corresponds to, for example, controlling Figures 1 to 3 and Figure 5 The control device of the robot arm A shown. Alternatively, the control unit 20 corresponds to, for example Figure 8 The CCU 5039 or arm control unit 5045 is shown.

[0188] The control unit 20 is implemented by, for example, a central processing unit (CPU), a microprocessor unit (MPU), etc., executing a program (for example, a program according to the present invention) stored in a storage unit (not shown) having a random access memory (RAM) or the like as a work area. Furthermore, the control unit 20 is a controller and can be implemented by, for example, an integrated circuit such as an application-specific integrated circuit (ASIC) or a field-programmable gate array (FPGA).

[0189] like Figure 13 As shown, the control unit 20 includes an acquisition unit 21, a determination unit 22, an arm control unit 23 and a display control unit 24. Each block included in the control unit 20 (acquisition unit 21, display control unit 24, etc.) is a functional block representing the function of the control unit 20. These functional blocks can be software blocks or hardware blocks. For example, each of the above-mentioned functional blocks can be a software module implemented by software (including microprograms), or can be a circuit block on a semiconductor chip (die). Of course, each functional block can be a processor or an integrated circuit. The method of configuring the functional blocks is arbitrary. Note that the control unit 20 can be configured with functional units different from the above-mentioned functional blocks.

[0190] For example, the acquisition unit 21 acquires instructions from a user (eg, an operator or a person assisting the operator) who operates the operation unit 30. For example, the acquisition unit 21 acquires information about surgical conditions (eg, information about currently performed treatment).

[0191] The determination unit 22 determines a combination of multiple operation amounts of the interference avoidance operation. For example, the determination unit 22 determines a combination of the operation amount of the first interference avoidance operation and the operation amount of the second interference avoidance operation. Here, the first interference avoidance operation is, for example, a removal operation to move the oblique endoscope so that the objective lens of the oblique endoscope moves away from the observation point. In addition, the second interference avoidance operation is, for example, a rotation operation to move the oblique endoscope to change the observation direction of the observation point.

[0192] The determination unit 22 may be configured to determine a combination of the amount of operation for the removal operation and the amount of operation for the rotation operation. For example, the determination unit 22 may determine the combination of the amount of operation for the removal operation and the amount of operation for the rotation operation based on a ratio between a minimum amount of operation for the removal operation when interference with the surgical tool is avoided only by the removal operation and a minimum amount of operation for the rotation operation when interference with the surgical tool is avoided only by the rotation operation. More specifically, the determination unit 22 may determine the combination of the amount of operation for the removal operation and the amount of operation for the rotation operation by calculating a ratio in a predetermined interference avoidance operation and applying the calculated ratio to design information, wherein the relationship between an arbitrary ratio and a combination that can avoid interference at that arbitrary ratio is recorded.

[0193] Here, the design information may be information of a program diagram (e.g., Figure 7 The determination unit 22 can determine the combination of the operation amount of the removal operation and the operation amount of the rotation operation by using different design information for each process performed by the operator.

[0194] Note that the treatment performed by the operator can include at least a first treatment and a second treatment that requires more precision than the first treatment. The design information can include first design information and second design information designed so that the operation amount of the removal operation is smaller than the operation amount of the first design information at least in some cases. At this time, in a case where the current treatment is the first treatment, the determination unit 22 can determine the combination of the operation amount of the removal operation and the operation amount of the rotation operation based on the first design information. Further, in a case where the current treatment is the second treatment, the determination unit 22 can determine the combination of the operation amount of the removal operation and the operation amount of the rotation operation based on the second design information.

[0195] Note that the treatment performed by the operator can include at least one of a treatment of sucking a liquid in a body, a treatment of pinching a blood vessel, a treatment of suturing, dissection processing, and discission processing. For example, the above-described first treatment can be the treatment of sucking a liquid in a body. The above-described second treatment can also be the treatment of pinching a blood vessel.

[0196] Further, the treatment performed by the operator can include at least discission processing. Then, the determination unit 22 can determine a different combination for each of a time at which the operator pinches tissue with the surgical tool for discission and a time at which discission is performed.

[0197] Note that the information used to select the design information is not limited to information about the treatment. For example, the determination unit 22 can determine the combination of the operation amount of the removal operation and the operation amount of the rotation operation by using design information selected based on information about the size of the workspace (for example, information about the size of an area around a site to be treated by the operator).

[0198] The arm control unit 23 comprehensively controls the robot arm device 10 and controls the driving of the arm portion 11. Specifically, the arm control unit 23 controls the driving of the arm portion 11 by controlling the driving of the joint portion 111. More specifically, the arm control unit 23 controls the rotation angle and the generated torque of the joint portion 111 by controlling the amount of current supplied to the motor in the actuator provided to the joint portion 111, thereby controlling the rotation speed of the motor.

[0199] The arm control unit 23 can cause the support arm to perform a plurality of different interference avoidance operations for avoiding interference between the strabismus endoscope and the surgical tool while maintaining a state in which the objective lens of the strabismus endoscope is directed toward the observation point. For example, the arm control unit 23 can cause the support arm to perform a first interference avoidance operation and a second interference avoidance operation different from the first interference avoidance operation as the interference avoidance operation. Here, the first interference avoidance operation is, for example, a removal operation of moving the strabismus endoscope so that the objective lens of the strabismus endoscope is distanced from the observation point. Further, the second interference avoidance operation is, for example, a rotation operation of moving the strabismus endoscope so as to change the observation direction of the observation point.

[0200] The display control unit 24 causes the display unit 40 to display various images (including not only still images but also videos). For example, the display control unit 24 causes the display unit 40 to display an image captured by the imaging unit 12a.

[0201] The operation unit 30 receives various types of operation information from the user. The operation unit 30 is implemented by, for example, a microphone that detects a sound, a gaze sensor that detects a gaze, a switch or a touch panel that receives a physical operation. The operation unit 30 can be implemented by other physical mechanisms.

[0202] The display unit 40 displays various images. The display unit 40 is, for example, a display. For example, the display unit 40 can be a display such as a liquid crystal display (LCD) or an organic electroluminescence (EL) display. For example, the display unit 40 displays an image captured by the imaging unit 12a.

[0203] The storage unit 50 is a data readable / writable storage device such as a dynamic random access memory (DRAM), a static random access memory (SRAM), a flash memory, or a hard disk. The storage unit 50 stores information of a program graph. Here, for example, as illustrated in Figure 7 the information of the program graph can be design information designed so that a first axis (for example, a vertical axis) represents an operation amount of a removal operation and a second axis (for example, a horizontal axis) orthogonal to the first axis represents an operation amount of a rotation operation.

[0204] A plurality of pieces of design information can be recorded in the storage unit 50. For example, different design information can be recorded in the storage unit 50 for each treatment performed by an operator. At this time, the storage unit 50 can include first design information (for example, Figure 7 design information of “suction” illustrated in Figure 7 design information of “pinching”) and second design information (for example,

[0205] The treatment targeted by the design information is not limited to aspiration and clipping, and may be suturing treatment, stripping treatment, or excision treatment.

[0206] In the case of suturing treatment, it is desirable to be able to observe at a certain magnification to a certain extent so that the position of needle insertion can be finely adjusted even if the direction of observing the treated area is slightly changed. Therefore, it is desirable for the designer to design the design information of the suturing treatment so that the operation amount of the removal operation is less than the operation amount of the predetermined design information (for example, the design information of the suction treatment) in at least some cases.

[0207] The peeling process has similar observation requirements as the stitching process. However, in the peeling process, magnification is relatively less important than in the stitching process. Therefore, the designer can design the design information for the peeling process so that, at least in some cases, the operation volume of the removal operation is greater than the operation volume of the removal operation in the design information for the stitching process.

[0208] In the resection process, two times can be assumed, including the time when the operator grasps the tissue with the surgical tool for resection and the time when the resection is performed. The designer can design different design information for each of the time when the operator grasps the tissue with the surgical tool for resection and the time when the resection is performed.

[0209] Generally speaking, it is assumed that when the operator is grasping tissue with a surgical tool for resection, the operator is focused on magnified observation. Therefore, it is desirable that the designer design information so that the avoidance operation using the rotation of the oblique endoscope is actively selected rather than the removal operation when the operator is grasping tissue with a surgical tool for resection.

[0210] On the other hand, assuming that the operator wants to significantly reduce the screen size when performing a cutout to perform work, it is desirable that the designer design the design information so that the operator can actively select the removal operation instead of the rotation operation when performing a cutout to avoid interference.

[0211] Note that the design information is not limited to information for each treatment. For example, the storage unit 50 may store design information for each dimension of the workspace. For example, the storage unit 50 may store design information for each dimension (e.g., each specific dimension level) of the area surrounding the site to be treated by the operator.

[0212] For example, in cases where there are many organs around (e.g., the stomach and liver) and the working space is small, such as in pancreatic treatment, it is difficult to avoid using rotational surgery. Therefore, designers design information so that the degree of insertion / removal is relatively high. On the other hand, in cases where a relatively large space can be secured around, such as in gallbladder treatment, designers design information so that the amount of rotational surgery operation is greater than in treatments in a small space (e.g., pancreatic treatment).

[0213] Note that in this example, the design information is divided based on the organ to be treated. The design information can be divided only based on the size of the space, without considering the organ to be treated. In this case, the acquisition unit 21 of the control unit 20 can obtain the distance to the surrounding organs or tissues or the image information processing results from a time-of-flight (ToF) sensor or a stereo image sensor. Then, the determination unit 22 of the control unit 20 can select the design information for determining the combined surgical volume based on the size of the space rather than the organ to be treated.

[0214] <<3. Operation of medical system>>

[0215] The configuration of the medical system has been described above, and the operation of the medical system will be described below. In the following description, an example of controlling a support arm that supports an oblique looking endoscope will be described.

[0216] Note that, although the following description assumes that the medical system of the present embodiment is the medical observation system 1 , the operations described below can be applied not only to the medical observation system 1 but also to other medical systems.

[0217] The medical observation system 1 autonomously performs interference avoidance operations between the oblique viewing endoscope and the surgical tool. As described above, the interference avoidance operation is determined based on the combination of a removal operation (pulling the oblique viewing endoscope) and a rotation operation (rotating the oblique viewing endoscope). The control unit 20 included in the medical observation system 1 determines the combined operation amount of the removal and rotation operations based on the R / I ratio and information from a pre-designed sequence diagram.

[0218] The R / I ratio is the ratio between the minimum amount of removal operation in case of avoiding interference with the surgical tool by only the removal operation and the minimum amount of rotation operation in case of avoiding interference with the surgical tool by only the rotation operation. In the following description, it is assumed that information of a plurality of pre-designed program diagrams (e.g., Figure 7 Design information for the "Suction" shown and Figure 7 The design information of “clamping” shown in FIG. 1 is recorded in the storage unit 50 of the medical observation system 1 .

[0219] Figure 15 is a flowchart showing an example of an interference avoidance process for avoiding interference between an oblique looking endoscope and a surgical tool. Figure 15 A control process according to an embodiment of the present invention is described.

[0220] First, the control unit 20 detects the position of the surgical tool and the posture of the endoscope 12 based on an image captured by the endoscope 12 (step S101). As described above, the endoscope 12 is an oblique viewing endoscope.

[0221] Then, the control unit 20 determines whether the endoscope 12 and the surgical tool interfere with each other (step S102). Figure 4 As shown, the control unit 20 determines that the endoscope 12 ( Figure 5 Is the distal end portion of the oblique endoscope E) located around the surgical tool ( Figure 4 The surgical tool S1 in the example is arranged in a columnar interference avoidance area, for example, as Figure 5 If there is no interference (step S102: No), the control unit 20 ends the process.

[0222] If there is interference (step S102: Yes), the control unit 20 calculates the minimum operation amount (rotation amount) that can avoid interference with the rotation operation of the surgical tool by only the rotation operation (step S103). The operation amount is, for example, Figure 6 The amount of rotation in the example of . rθ can be the amount of operation calculated in step S103. Here, r is obtained by passing through the rotation direction R. Figure 5 In the example, the current position P is the radius of the circle formed by cutting the cone.

[0223] Then, the control unit 20 calculates the minimum operation amount (insertion / removal degree) that can avoid interference with the rotation operation of the surgical tool by only the removal operation (step S104). This operation amount is, for example, Figure 6 The insertion / removal degree L in the example.

[0224] Subsequently, the control unit 20 calculates the R / I ratio based on the rotation amount calculated in step S103 and the insertion / removal degree calculated in step S104 (step S105). As described above, the R / I ratio is the ratio between the minimum amount of removal operation required to avoid interference with the surgical tool by only removing the surgical tool and the minimum amount of rotation operation required to avoid interference with the surgical tool by only rotating the surgical tool. For example, the control unit 20 calculates the R / I ratio based on equation (1) or equation (2) described in <1-1. Purpose of the Present Embodiment, etc.>.

[0225] Then, the control unit 20 acquires the information of the program map from the storage unit 50 (step S106). The information of the program map is used to determine, for example, Figure 7 At this time, the control unit 20 may select design information for determining the combined operation amount from among a plurality of pieces of design information based on information on the treatment performed by the operator.

[0226] Subsequently, the control unit 20 determines the combined operation amount of the rotation operation and the removal operation based on the R / I ratio calculated in step S105 and the information of the program map acquired in step S106 (step S107). For example, assuming that the R / I ratio calculated in step S105 is Figure 7 The oblique lines shown in , and the information of the program diagram acquired in step S106 is Figure 7 , or the design information of "suction" or "clamping" shown in . At this time, in a case where the treatment currently performed by the operator is "suction", the control unit 20 sets the values ​​of R and I indicated by the intersection CP1 of the oblique line indicating the R / I ratio and the design line indicating suction as the combined operation amount. On the other hand, in a case where the treatment currently performed by the operator is "clamping", the control unit 20 sets the values ​​of R and I indicated by the intersection CP2 of the oblique line indicating the R / I ratio and the design line indicating clamping as the combined operation amount. Note that information about the treatment currently performed by the operator can be input to the control unit 20 by the operator or his assistant via the operation unit 30, or can be discerned by the control unit 20 from, for example, the shape of the surgical tool based on the image captured by the endoscope 12.

[0227] Then, the control unit 20 controls the arm 11 based on the combined operation amount determined in step S107 (step S108). Once the control of the arm 11 is completed, the control unit 20 ends the interference avoidance process.

[0228] As a result, the medical observation system 1 can perform an appropriate interference avoidance operation according to the situation of the operation. For example, the medical observation system 1 can perform an interference avoidance operation in which the loss of detail and the change of rotation direction are balanced according to the treatment performed by the operator or the size of the workspace in which the treatment is performed.

[0229] <<4. Modification>>

[0230] The above-described embodiments are merely examples, and various modifications and applications are possible.

[0231] For example, in the above embodiment, if Figure 2 and Figure 11 As shown in FIG, the oblique viewing endoscope in which the distal end portion of the shaft-like body is cut obliquely with respect to the axial direction has been exemplified as the oblique viewing endoscope. However, the oblique viewing endoscope is not limited to this shape. Figure 16 1 is a diagram showing a modification of the oblique endoscope. For example, the oblique endoscope may have a shape in which the distal end portion is bent relative to the axial direction. At this time, in the oblique endoscope, the bending angle t3 may be changed according to the operation performed by the operator.

[0232] For example, in the above-described embodiment, two operations, namely, a rotation operation and an insertion / removal operation (a removal operation or an insertion operation), are exemplified as interference avoidance operations, but interference avoidance operations are not limited to these two operations. For example, the interference avoidance operation does not have to be an operation of moving the distal end of the oblique endoscope on a conical surface. For example, as long as the target observation point is included in the image, the control device of the support arm can move the oblique endoscope out of the conical surface. Therefore, it is easier for the control device to achieve a balance between the loss of detail and the change in the rotation direction. For example, although the observation point is not located at the center of the image, the control device can perform operations such as maintaining detail.

[0233] Furthermore, the interference avoidance operation is not limited to the two operations of a rotation operation and an insertion / removal operation (removal operation or insertion operation). Three or more interference avoidance operations are possible. The three or more interference avoidance operations may or may not include a rotation operation and an insertion / removal operation. As the number of options for the interference avoidance operation increases, it becomes easier for the control device to strike a balance between loss of detail and change of rotation direction.

[0234] The control device for controlling the support arm of this embodiment (for example, the control device of the robot arm A, the CCU 5039, the arm control device 5045 or the control unit 20) can be implemented by a dedicated computer system or a general-purpose computer system.

[0235] For example, a program for executing the above-mentioned control processing is stored in a computer-readable recording medium such as an optical disc, a semiconductor memory, a magnetic tape, or a floppy disk, and distributed. Then, for example, the control device is implemented by installing the program in a computer and executing the above-mentioned processing. In this case, the control device can be a device (e.g., a personal computer) external to the support arm (e.g., a medical support arm such as the robot arm A, the support arm device 5027, the support arm device 400, or the robot arm device 10). In addition, the control device can be a device inside the support arm (e.g., a processor installed on the support arm).

[0236] In addition, the communication program can be stored in a disk device included in a server device on a network (e.g., the Internet) and can be downloaded to the computer. In addition, the above functions can be realized by cooperation between an operating system (OS) and application software. In this case, the parts other than the OS can be stored in a medium and distributed, or the parts other than the OS can be stored in a server device and downloaded to the computer.

[0237] Furthermore, in the various processes described in the above embodiments, all or some of the processes described as being automatically performed may be performed manually. Alternatively, all or some of the processes described as being manually performed may be performed automatically using known methods. Furthermore, unless otherwise specified, the processes, specific names, and information including various data and parameters shown in the specification and drawings may be arbitrarily changed. For example, the various information shown in each figure is not limited to the information shown.

[0238] Furthermore, each illustrated component of each device is conceptual in function and does not necessarily need to be physically configured as shown in the drawings. That is, the specific modes of distribution / integration of the various devices are not limited to those shown in the drawings. Depending on various loads or usage conditions, all or some of the devices may be functionally or physically distributed / integrated in any arbitrary unit.

[0239] Furthermore, the above-described embodiments can be appropriately combined as long as the processing contents do not contradict each other. Furthermore, the order of each step shown in the flowcharts of the above-described embodiments can be appropriately changed.

[0240] In addition, for example, the present embodiment can be implemented as any component included in a device or system, such as a processor as a system large-scale integration (LSI) or the like, a module using multiple processors or the like, a unit using multiple modules or the like, a collection obtained by further adding other functions to the unit, etc. (i.e., some components of the device).

[0241] Note that in this embodiment, a system refers to a collection of multiple components (devices, modules (components), etc.), and it does not matter whether all components are in the same housing. Therefore, multiple devices housed in separate housings and connected via a network, and a device in which multiple modules are housed in a single housing, are both systems.

[0242] Furthermore, for example, the present embodiment may adopt a configuration of cloud computing in which one function is shared and processed by a plurality of devices in collaboration via a network.

[0243] 5. Conclusion

[0244] The medical support arm of this embodiment includes: a support arm that supports an endoscope; an arm control unit that is configured to cause the support arm to perform multiple different interference avoidance operations to avoid interference between the endoscope and surgical tools while keeping the objective lens of the endoscope pointing to the observation target; and a determination unit that determines a combination of operation amounts of the multiple interference avoidance operations.

[0245] As a result, interference can be avoided by a combination of multiple operations rather than simply avoiding interference by one operation, so that the medical support arm can perform an interference avoidance operation suitable for the surgery.

[0246] Note that the effects described in this specification are merely examples, and the effects of the present disclosure are not limited thereto, and other effects may be obtained.

[0247] Note that the present technology can also have the following configurations.

[0248] (1) A medical support arm comprising:

[0249] a support arm, supporting the endoscope;

[0250] an arm control unit configured to cause the support arm to perform a plurality of different interference avoidance operations to avoid interference between the endoscope and the surgical tool while maintaining a state in which the objective lens of the endoscope is directed toward the observation target; and

[0251] The determining unit determines a combination of operation amounts of a plurality of interference avoiding operations.

[0252] (2) The medical support arm according to (1), wherein the arm control unit is configured to cause the support arm to perform a first interference avoidance operation as the interference avoidance operation and a second interference avoidance operation different from the first interference avoidance operation, and

[0253] The determination unit determines a combination of an operation amount of the first interference avoiding operation and an operation amount of the second interference avoiding operation.

[0254] (3) The medical support arm according to (2), wherein the first interference avoidance operation is a removal operation of moving the endoscope so that the objective lens of the endoscope is away from the observation target,

[0255] The second interference avoidance operation is a rotation operation of moving the endoscope to change the observation direction of the observation target, and

[0256] The determination unit determines a combination of an operation amount of the removal operation and an operation amount of the rotation operation.

[0257] (4) The medical support arm according to (3), wherein the determination unit determines a combination of an operation amount of the removal operation and an operation amount of the rotation operation based on a ratio between a minimum operation amount of the removal operation in a case where interference with the surgical tool is avoided only by the removal operation and a minimum operation amount of the rotation operation in a case where interference with the surgical tool is avoided only by the rotation operation.

[0258] (5) The medical support arm according to (4), wherein the determination unit determines the combination of the operation amount of the removal operation and the operation amount of the rotation operation by calculating a ratio in a predetermined interference avoidance operation and applying the calculated ratio to design information, wherein the relationship between an arbitrary ratio and a combination capable of avoiding interference at the arbitrary ratio is recorded in the design information.

[0259] (6) The medical support arm according to (5), wherein the design information is information of a program diagram in which a first axis represents an operation amount of a removal operation and a second axis orthogonal to the first axis represents an operation amount of a rotation operation.

[0260] (7) The medical support arm according to (5) or (6), wherein the determination unit determines a combination of an operation amount of the removal operation and an operation amount of the rotation operation by using different design information for each treatment performed by the operator.

[0261] (8) The medical support arm according to (7), wherein the treatment performed by the operator includes at least a first treatment and a second treatment requiring more precision than the first treatment,

[0262] The design information includes first design information and second design information, the second design information being designed so that an operation amount of a removal operation is smaller than an operation amount of the first design information in at least some cases, and

[0263] The determination unit determines a combination of an operation amount of the removal operation and an operation amount of the rotation operation based on first design information when performing the first treatment, and determines a combination of an operation amount of the removal operation and an operation amount of the rotation operation based on second design information when performing the second treatment.

[0264] (9) The medical support arm according to (8), wherein the first treatment is a treatment for aspirating body fluid, and

[0265] The second treatment is clamping of the blood vessels.

[0266] (10) The medical support arm according to any one of (7) to (9), wherein the treatment performed by the operator includes at least one of aspiration of body fluid, clamping of blood vessels, suturing, stripping, and excision.

[0267] (11) The medical support arm according to (10), wherein the treatment performed by the operator includes at least a resection process, and

[0268] The determination unit determines a different combination for each of the time when the operator grasps the tissue with the surgical tool for resection and the time when the resection is performed.

[0269] (12) The medical support arm according to (5), wherein the determination unit determines a combination of the operation amount of the removal operation and the operation amount of the rotation operation by using design information selected based on information about a size of an area around a site to be treated by the operator.

[0270] (13) A medical system comprising:

[0271] a support arm that supports the endoscope; and

[0272] Control device, control support arm,

[0273] The control device includes:

[0274] an arm control unit configured to cause the support arm to perform a plurality of different interference avoidance operations to avoid interference between the endoscope and the surgical tool while maintaining a state in which the objective lens of the endoscope is directed toward the observation target; and

[0275] The determining unit determines a combination of operation amounts of a plurality of interference avoiding operations.

[0276] (14) A control device for controlling a support arm supporting an endoscope, the control device comprising:

[0277] an arm control unit configured to cause the support arm to perform a plurality of different interference avoidance operations to avoid interference between the endoscope and the surgical tool while maintaining a state in which the objective lens of the endoscope is directed toward the observation target; and

[0278] The determining unit determines a combination of operation amounts of a plurality of interference avoiding operations.

[0279] (15) A method for controlling a support arm for supporting an endoscope, the method comprising:

[0280] determining a combination of operation amounts of a plurality of different interference avoiding operations for avoiding interference between the endoscope and the surgical tool while maintaining a state in which the objective lens of the endoscope is directed toward the observation target; and

[0281] The support arm is controlled based on the combination of the operation amounts.

[0282] (16) A program for causing a computer that controls a support arm for supporting an endoscope to:

[0283] an arm control unit configured to cause the support arm to perform a plurality of different interference avoidance operations to avoid interference between the endoscope and the surgical tool while maintaining a state in which the objective lens of the endoscope is directed toward the observation target; and

[0284] The determining unit determines a combination of operation amounts of a plurality of interference avoiding operations.

[0285] Reference Signs List

[0286] 1 Medical Observation System

[0287] 10. Robotic Arm Device

[0288] 11 Arm

[0289] 111 joints

[0290] 111a joint drive unit

[0291] 111b joint state detection unit

[0292] 12 Endoscope

[0293] 12a Imaging unit

[0294] 12b Light source unit

[0295] 20 Control Unit

[0296] 21 Get Unit

[0297] 22 Determine unit

[0298] 23 Arm control unit

[0299] 24 Display control unit

[0300] 30 operating units

[0301] 40 display units.

Claims

1. A medical support arm, comprising: a support arm, supporting the endoscope; an arm control unit configured to cause the support arm to perform a plurality of different interference avoidance operations to avoid interference between the endoscope and a surgical tool while maintaining a state in which the objective lens of the endoscope is directed toward an observation target; as well as a determination unit that determines a combination of operation amounts of a plurality of interference avoidance operations, wherein the arm control unit is configured to cause the support arm to perform a first interference avoidance operation as an interference avoidance operation and a second interference avoidance operation different from the first interference avoidance operation, and The determining unit determines a combination of an operation amount of the first interference avoiding operation and an operation amount of the second interference avoiding operation, wherein the first interference avoiding operation is a removal operation of moving the endoscope to move the objective lens of the endoscope away from the observation target, The second interference avoiding operation is a rotation operation of moving the endoscope to change the observation direction of the observation target, and The determining unit determines a combination of an operation amount of a removal operation and an operation amount of a rotation operation, In which, the determination unit determines the combination of the operation amount of the removal operation and the operation amount of the rotation operation based on the ratio between the minimum operation amount of the removal operation when interference with the surgical tool is avoided only by the removal operation and the minimum operation amount of the rotation operation when interference with the surgical tool is avoided only by the rotation operation.

2. The medical support arm according to claim 1, wherein: The determination unit determines a combination of an operation amount of a removal operation and an operation amount of a rotation operation by calculating the ratio in a predetermined interference avoiding operation and applying the calculated ratio to design information in which a relationship between an arbitrary ratio and a combination capable of avoiding interference at the arbitrary ratio is recorded.

3. The medical support arm according to claim 2, wherein: The design information is information of a program diagram in which a first axis represents an operation amount of a removal operation and a second axis orthogonal to the first axis represents an operation amount of a rotation operation.

4. The medical support arm according to claim 2, wherein: The determination unit determines a combination of an operation amount of the removal operation and an operation amount of the rotation operation by using different design information for each treatment performed by an operator.

5. The medical support arm according to claim 4, wherein: The treatment performed by the operator includes at least a first treatment and a second treatment requiring more precision than the first treatment, The design information includes first design information and second design information, the second design information being designed so that an operation amount of a removal operation is smaller than an operation amount of the first design information in at least some cases, and The determination unit determines a combination of an operation amount of a removal operation and an operation amount of a rotation operation based on the first design information when performing a first treatment, and determines a combination of an operation amount of a removal operation and an operation amount of a rotation operation based on the second design information when performing a second treatment.

6. The medical support arm according to claim 5, wherein: The first treatment is a treatment to aspirate body fluid, and The second treatment is a blood vessel clamping treatment.

7. The medical support arm according to claim 4, wherein: The treatment performed by the operator includes at least one of a treatment of aspirating body fluid, a treatment of clamping a blood vessel, a treatment of suturing, a stripping treatment, and a resection treatment.

8. The medical support arm according to claim 7, wherein: The treatment performed by the operator includes at least a resection procedure, and The determination unit determines a different combination for each of the time when the operator grasps the tissue with the surgical tool for resection and the time when the resection is performed.

9. The medical support arm according to claim 2, wherein: The determination unit determines a combination of an operation amount of the removal operation and an operation amount of the rotation operation by using design information selected based on information about a size of an area around a site to be treated by an operator.

10. A medical system comprising: a support arm, supporting the endoscope; as well as Control device, control support arm, Wherein, the control device includes: an arm control unit configured to cause a support arm to perform a plurality of different interference avoidance operations to avoid interference between the endoscope and a surgical tool while maintaining a state in which the objective lens of the endoscope is directed toward an observation target; and a determining unit that determines a combination of operation amounts of a plurality of interference avoiding operations, wherein the arm control unit is configured to cause the support arm to perform a first interference avoidance operation as an interference avoidance operation and a second interference avoidance operation different from the first interference avoidance operation, and The determining unit determines a combination of an operation amount of the first interference avoiding operation and an operation amount of the second interference avoiding operation, wherein the first interference avoiding operation is a removal operation of moving the endoscope to move the objective lens of the endoscope away from the observation target, The second interference avoiding operation is a rotation operation of moving the endoscope to change the observation direction of the observation target, and The determining unit determines a combination of an operation amount of a removal operation and an operation amount of a rotation operation, In which, the determination unit determines the combination of the operation amount of the removal operation and the operation amount of the rotation operation based on the ratio between the minimum operation amount of the removal operation when interference with the surgical tool is avoided only by the removal operation and the minimum operation amount of the rotation operation when interference with the surgical tool is avoided only by the rotation operation.

Images