Control device, robotic arrangement, surgical microscope, and method for controlling movement of a surgical instrument

Abstract

The present invention relates to a control device (1) for controlling the movement of a surgical instrument (2), wherein the control device (1) includes a sensor device (6), an evaluation device (7) and a human-machine interface (8), wherein the sensor device (6) is designed to optically capture image data of a region of tissue (4), the evaluation device (7) is designed to determine at least one feature of the tissue (4) in the captured region based on the image data captured by means of the sensor device (6), wherein a change in the feature characterizes the contact between the surgical instrument (2) and the tissue (4), and the human-machine interface (8) is designed to output a feedback signal based on the determined at least one feature.

Description

Technical Field

[0001] This invention relates to control devices, robotic arrangements, and surgical microscopes for use in microsurgery, particularly ophthalmic and neurosurgical procedures. The invention also relates to a method for controlling the movement of surgical instruments. Background Technology

[0002] Currently, surgical procedures are typically performed manually. In certain areas, such as laparoscopy, remotely operated surgical robots are gradually gaining popularity. In the future, systems with task-level automation or higher levels of automation will become increasingly important. Corresponding robotic systems will require further development of sensor systems to adapt to various needs and functions, tailored to the measures to be implemented in each situation.

[0003] In manual interventions, surgeons rely on visual feedback to estimate depth, and thus instrument-tissue interactions. Surgeons acquire this ability through years of experience. Therefore, robotic systems require appropriate feedback to ensure safe and reliable contact with tissue.

[0004] In many ophthalmic surgical procedures, surgical instruments must come into contact with sensitive tissue, such as to hold the capsular bag during cataract surgery or to remove a membrane from the retina during membrane peeling in retinal surgery. The optimal force for this is in the range of a few millinewtons (mN). These optimal forces between the tip of the surgical instrument and the tissue to be treated are typically below the range perceptible to humans. If the force is too high, there is a risk of the surgical instrument damaging surrounding tissues, such as damaging the retina during retinal surgery or the small band fibrosis during cataract surgery. If the force is too low, the surgical procedure may not be possible, for example, because the tissue cannot be reliably held. Furthermore, capturing or sensing the force is complex because forces and torques are absorbed at the point of entry into the eye during many ophthalmic surgical procedures, such as by the cannula in retinal surgery or by the incision in the cornea during cataract surgery. Therefore, surgeons cannot rely on tactile feedback during many ophthalmic surgical procedures and instead rely on sensors other than their own bodily perception.

[0005] Ophthalmic surgical microscopes are well-suited for capturing the relative lateral position of surgical instruments and tissue, and for roughly estimating depth, such as the depth to which an instrument penetrates the tissue. To compensate for the limitations of force and depth capture, surgeons often rely on additional indicators, such as observed shadows, color changes, or tissue folds under load. In other cases, surgical steps must be repeated at different depths until, for example, the instrument arrives and the procedure can be successfully performed. This is often only identifiable by tissue response, such as the extension of the tear point. Many visual indicators typically perceived by surgeons are difficult to detect using camera systems or computer algorithms.

[0006] For example, femtosecond lasers can be used to support cataract surgery. These systems typically integrate optical coherence tomography (OCT) scanners to improve depth sensor technology. However, this results in additional costs; contact lenses to be placed on the eye are necessary, and generally, in OCT systems, contact between surgical instruments and tissue may be only partially captured in some cases.

[0007] Document US 2011 / 0 106 102 A1 describes a robotic surgical system for ocular surgery in which optical sensors are used in the region of the tip of the surgical instrument. Document US 2017 / 0 312 431 A1 describes an ocular surgical system capable of pressure measurement during perfusion. Document US 2016 / 0 074 212 A1 discloses a device for introducing a drug into the eye, the device including a detection and visualization system for detecting and visualizing the injection's puncture into the choroid as feedback for the surgeon and / or for automated control.

[0008] Document US 2007 / 0 151 390 A1 describes a robotic surgical instrument with force and torque sensors at its tip. Documents US 2012 / 0 265 102 A1 and WO 2017 / 118949 A1 disclose cardiac catheters with force sensors at their tips. Summary of the Invention

[0009] In light of the described background, an object of the present invention is to provide an advantageous control device, for example for microsurgery, for controlling the movement of surgical instruments. A further object is to provide an advantageous robot arrangement, a surgical microscope, and a method for controlling the movement of surgical instruments, which, in their respective cases, are used, for example, in microsurgery.

[0010] The objective is achieved by a control device as described in claim 1, a robot arrangement as described in claim 9, a surgical microscope as described in claim 17, and a method for controlling the movement of surgical instruments as described in claim 18. The dependent claims contain further advantageous configurations of the invention.

[0011] The control device for controlling the movement of surgical instruments, particularly surgical tools, according to the present invention, preferably used in microsurgery, includes a sensor device, an evaluation device, and a human-machine interface (HMI). In other words, the sensor device can be understood as a device for capturing physically measurable variables, such as a camera, detector, signal acquisition device, or measuring device. In other words, the evaluation device can be understood as a device for receiving, processing, and outputting data and / or signals, such as a computer or data processing system. In other words, the human-machine interface (HMI) can be understood as a user interface for user input and / or output to the user.

[0012] The sensor device is designed to optically capture, particularly visually, image data relating to a region of tissue (i.e., the area of ​​tissue to be manipulated). The evaluation device is designed to determine at least one feature of the tissue in the captured region (i.e., an optically captured feature) based on the image data captured by the sensor device, wherein changes in this feature indicate contact or interaction between the surgical instrument and the tissue. Based on the determined feature, the human-machine interface is designed to output feedback signals, for example, to the surgeon and / or robotic manipulator.

[0013] In other words, the control device according to the invention is therefore designed to capture and assess tissue changes caused by contact with surgical instruments. Determining at least one characteristic of the tissue may include capturing and / or detecting and / or qualitatively and / or quantitatively analyzing that characteristic. For example, changes in the tissue's color and / or geometry may be captured and assessed.

[0014] In the current context, surgical instruments should be understood as physical tools or devices that can physically act on tissues within the scope of surgical procedures.

[0015] At least one feature determined based on image data may be a parameter or variable indicating a tissue response caused or resulting from interaction or contact with surgical instruments, such as color or color change, or distance, or size, or shape of shadow, or type of tissue response, or another parameter.

[0016] The control device according to the invention, particularly for microsurgery, is preferably designed to control the movement of surgical instruments during surgical procedures or interventions. This control device can be used in ophthalmic or neurosurgical procedures. Specifically, the control device according to the invention can be designed for individual steps during cataract surgery, such as capsulorhexis or capsular bag polishing. Other examples of application are: membrane detachment in retinal surgery, such as brushing to remove the detached membrane; corneal surgery; or generally avoiding excessive force during phacoemulsification or lens injection. Additionally, the control device according to the invention can be designed for assisting or automating suturing in microsurgery (e.g., ophthalmic or neurosurgical procedures).

[0017] The control device according to the invention has the advantage that, for microsurgical applications, it provides improved feedback when controlling the movement of surgical instruments to optimally execute planned interventions or methods. Therefore, the control is highly reliable. Simultaneously, the control device is cost-effective because it eliminates the need for cost-intensive sensor systems.

[0018] In advantageous variations, the evaluation device is designed to define the trajectory of surgical instruments based on image data captured by a sensor device, and / or based on at least one determined feature, and / or based on an output feedback signal. In this case, defining the trajectory may further include determining and / or adapting the trajectory. In particular, the trajectory can be continuously adapted based on the behavior of the determined feature, especially changes in the determined feature. This is advantageous for assistive functions for surgeons to manually guide surgical instruments or for at least partially automated robotic applications. The action of the surgical instruments on tissue can be optimized using the corresponding trajectory.

[0019] In advantageous variations, the control device can be designed to output a trajectory for the movement (e.g., at least partially automated movement) of a surgical instrument based on image data captured by means of a sensor device, and / or based on at least one determined feature, and / or based on an output feedback signal. In this case, the trajectory may include at least one repetitive movement pattern. In a preferred variation, the control device is designed to control at least partially automated movement of the surgical instrument along a defined trajectory including repetitive movement patterns, particularly the movement of the tip of the surgical instrument.

[0020] The trajectory may include, for example, a continuous path along an imaginary surface, wherein the surface may have a conical or cylindrical shape, and / or the path may have circular segments, and / or elliptical segments, and / or spiral segments, and / or parabolic segments, and / or hyperbolic segments, and / or linear segments. The movement may be at least partially configured as a clamping movement.

[0021] In another variant, the trajectory may lie at least partially in a plane. In this case, the plane may extend perpendicular to the transverse plane and / or perpendicular to the object plane of the sensor device. The object plane should be understood to refer, for example, to the plane in which tissue is captured optically. With the aid of the aforementioned variants, for example, the clamping or cutting movement of surgical instruments can be optimized.

[0022] The sensor device may include at least one camera for simultaneously capturing multiple laterally arranged pixels and / or a scanning device for capturing laterally arranged pixels in a time sequence. The sensor device may include a stereo camera, and / or an optical coherence tomography (OCT) scanner, and / or a surgical microscope, and / or a confocal system. In this case, low-resolution axial information can be fused with, combined with, or supplemented by more precise information (e.g., information obtained from tissue response). In this way, reliable data, particularly axial data, for controlling surgical instruments can be obtained in a simple and cost-effective manner.

[0023] Human-machine interfaces can also be designed to input the lateral path of movement of surgical instruments. For example, remote manipulation of the lateral position of surgical instruments can be provided, while maintaining a specific (e.g., predefined) axial position of the instruments relative to tissue is achieved through robotic control. Control devices can be designed to plan and / or define the trajectory of surgical instruments for partially or fully automated control of the instruments via robotic deployment.

[0024] The robotic arrangement according to the invention, particularly for microsurgery, comprises: at least one robot-guided surgical instrument, in other words, a surgical instrument whose movement can be controlled by a robot (i.e., can be moved or manipulated by means of a robot); and at least one robotic manipulator, in other words, a robot-movable device for physical interaction with the environment, the robotic manipulator having a fastening device for securing the surgical instrument to the robotic manipulator, in other words, having a device for establishing a reversible physical connection. The robotic arrangement includes the control device according to the invention described above, which is designed to control the movement of the surgical instrument by means of the robotic manipulator. The robotic arrangement can be designed for partially automated or fully automated operation. The robotic arrangement according to the invention has the same features and advantages as the control device according to the invention described above.

[0025] The robotic manipulator preferably has at least three (e.g., six) degrees of freedom for manipulation, and the control device is preferably designed to control the movement of surgical instruments by means of at least three (e.g., six) degrees of freedom of the robotic manipulator. Degrees of freedom can be translational and / or rotational. For example, three translational degrees of freedom relative to a Cartesian coordinate system and / or, for example, three rotational degrees of freedom about corresponding axes of the Cartesian coordinate system can be provided. Depending on the requirements of the application and the reference system used in this case (e.g., Cartesian coordinate system, cylindrical coordinate system, or spherical coordinate system, etc.), a certain number of translational and / or rotational degrees of freedom can be provided. A manipulator with six degrees of freedom (including three translational and three rotational degrees of freedom) has the advantage that movement can be manipulated unrestricted in all directions. For some applications, a smaller number of degrees of freedom may be sufficient. For example, a manipulator with only three or four degrees of freedom has the advantage that it is more cost-effective and potentially less prone to error compared to a manipulator with a higher number of degrees of freedom.

[0026] The robot is advantageously positioned such that the entry point into a specific tissue (e.g., a defined tissue of a specific organ) is defined, or can be defined, as a reference point (e.g., in the form of a coordinate origin) for the movement of surgical instruments. The lateral position of the surgical instrument corresponds to the angular orientation of the instrument or manipulator relative to the reference point (i.e., at least one rotational angle of the instrument or manipulator relative to the reference point), and the axial position of the surgical instrument corresponds to the depth of penetration of the instrument or manipulator through the entry point. The entry point can be defined by a cannula or a puncture into the tissue (e.g., the cornea). Defining the entry point as a reference point allows for intuitive and computationally efficient control of the movement of surgical instruments, in a manner relevant to movement patterns in the case of manual intervention.

[0027] Robotic placement can be designed to interrupt the movement of surgical instruments, particularly repetitive movement patterns, if contact between tissue and surgical instruments is determined, for example by means of an evaluation device. This has the advantage of avoiding potentially undesirable effects on the tissue. In this case, the robotic placement can be designed to switch to a mode with a predefined (e.g., user-defined) lateral movement path. Additionally or alternatively, the robotic placement can be designed to maintain the movement of surgical instruments, such as defined axial movement and / or repetitive movement patterns, while the instruments are simultaneously moving along a lateral path (e.g., a lateral path that may be defined by the surgeon or user, or defined by the surgeon or user, or defined by the user).

[0028] Robotic placement can be designed for partially or fully automated operations, where the lateral component of the surgical instrument's trajectory can be controlled based on data captured by sensor devices, maintaining contact between the instruments and tissue during repetitive movement patterns (e.g., axial movement). Partial automation offers the advantage that some movements of the instruments can be manually controlled, meaning the surgeon can act independently in that area, while simultaneously utilizing the high accuracy and precision of automation for other movements. Fully automated operation ensures high accuracy and precision in surgical interventions.

[0029] The term "control" also includes closed-loop control in the sense of open-loop and closed-loop control technologies. Specifically, the contact between tissue and surgical instruments can be controlled in a partially or fully automated manner. In this case, the robot arrangement can be designed to maintain contact between tissue and surgical instruments, for example, for a defined period of time, using a closed-loop controller for closed-loop control.

[0030] In another variation, the robot arrangement may include a human-machine interface by which a surgical instrument can be controlled by a user to move laterally (i.e., the lateral movement component) relative to tissue or tissue surface, while the axial or inward movement (i.e., the axial or inward movement component) of the surgical instrument relative to tissue or tissue surface can be controlled automatically. In this case, control can be implemented in a fully or partially automated manner and / or in the form of closed-loop control. For example, an input device (e.g., a joystick) can be provided by which the surgeon or user laterally controls the surgical instrument, and the contact or axial relationship between the surgical instrument and the tissue is maintained based on data captured by a sensor device. The described variation has the advantage of avoiding excessive or insufficient force exerted by the surgical instrument on the tissue, while simultaneously enabling manual lateral control of the surgical instrument.

[0031] Furthermore, the robotic arrangement may include a human-machine interface with a planning function (i.e., a corresponding component), by means of which the area of ​​tissue to be treated can be defined by the user, for example, based on lateral position data captured by means of sensor devices, and the robotic arrangement is designed to perform the defined surgical steps, at least in a partially automated manner (i.e., partially or fully automated), based on data captured by means of sensor devices and related to the contact between surgical instruments and the tissue surface (e.g., based on movement with repetitive movement patterns). For example, with the help of the planning function, the surgeon or user can define the spatial area to be treated based on lateral position data captured by means of sensor devices, and the robotic arrangement can be designed to perform surgical steps (e.g., grasping the capsule membrane) at the user-defined lateral position while bringing the surgical instruments to a specific (e.g., defined) axial position for the procedure. This is preferably performed based on at least one determined feature and / or a determined feedback signal. In this case, a closed-loop control function may be set or present, which is designed to automatically bring the surgical instruments to the correct lateral position and the correct distance from the tissue.

[0032] The robot deployment advantageously includes a human-machine interface (HMI) designed to output assistive information, particularly warnings or suggestions related to the trajectory of surgical instruments. This improves the quality of intervention and allows for consideration of potential safety protocols. Closed-loop control can also be implemented within this context.

[0033] In another variant, the robot deployment can switch between two operating modes during the execution of surgical procedures or tasks. In a first operating mode, the surgical instruments and tissue are in contact, and in a second operating mode, the surgical instruments and tissue are not in contact. The second operating mode can be designed for or used to reposition the surgical instruments. The operating modes can be selected based on data captured by sensor devices. The operating modes can be switched manually and / or automatically. Closed-loop control can be set within the range of operating modes. For example, in the first operating mode, a membrane can be held by the surgical instruments for tearing, and in the second operating mode, the membrane can be released for repositioning the surgical instruments. In both operating modes, sensor devices or multiple sensor devices provide the necessary information to maintain defined or definable contact between the surgical instruments and tissue. For example, switching can be achieved manually based on lateral position information or automatically based on the progress of the surgical steps and information from the sensor devices.

[0034] The control device according to the invention and / or the robot arrangement according to the invention can be designed to distinguish different types (e.g., classified) of contact between tissue and surgical instruments based on at least one determined characteristic (e.g., observed specific tissue response). The output of feedback signals and / or the control of the surgical instruments can be provided based on distinguishing the determined contact types. In one example, good contact, moderate contact, and no contact between tissue and surgical instruments can be differentiated or distinguished. In the case of capsulorhexis surgery, good contact can be identified by the extension of the tear point. Moderate contact can be identified by the fact that relatively large movement of the surgical instrument results in relatively small extension of the tear point. No contact can be identified by the absence of extension of the tear point.

[0035] The robotic arrangement can be designed to capture tissue movement and, upon capturing tissue movement (e.g., movement caused by heartbeat or respiration), to at least cause surgical instruments (and possibly other components) to follow the captured movement. Therefore, unplanned effects of surgical instruments on the tissue caused by tissue movement can be reduced or avoided. Following can be designed or implemented in an automated manner (e.g., partially or fully automated). Following can be subject to finite or limited constraints. Tissue movement can be captured by means of sensor devices or additional sensor devices (e.g., head trackers or eye trackers). The robotic arrangement can be designed to capture tissue movement in the lateral and / or axial directions. Additionally, the robotic arrangement can be designed to cause surgical instruments and optionally additional components to follow in the lateral and / or axial directions. In this case, specific limitations on movement can be set.

[0036] For example, a sensor system can be configured to capture lateral position data, providing low-resolution axial position data, such as a stereo camera system or an OCT system. The data captured by this sensor system can be provided for overlaying, supplementing, or fusing with axial data obtained from at least one determined feature. Axial data or information can be used specifically to reposition surgical instruments to points intended for performing repetitive movement patterns.

[0037] In the case of capsulorhexis, the algorithm provided for assessing tissue response can take into account the fact that the movement length of the surgical instrument is twice the extension length of the tear point.

[0038] Robotic placement can be designed to provide axial control of surgical instruments (e.g., in the case of remotely operated applications) to output tactile feedback to the surgeon. This can be achieved via a human-machine interface (e.g., a joystick) in the form of vibration or through active resistance in a specific direction of movement. Furthermore, in exemplary configurations for ophthalmic surgery, axial movement toward the underside of the capsular pouch can be restricted if the surgical instruments achieve adequate gripping of the tissue.

[0039] In general, the tissue's response to contact with surgical instruments can be determined based on optical flow or images of the tissue captured sequentially over time. The optical flow of the image sequence should be understood as a vector field representing the velocity of visible points in object space, projected onto the image plane in the reference frame of the imaging optics unit. Furthermore, when capturing differences between images captured sequentially over time, the area or region where the surgical instrument is located can be excluded. In specific design variations, the algorithm for capturing differences between sequentially captured images can be designed to distinguish between cases where the entire image moves (e.g., due to microscope vibration) and cases where different segments of the image (e.g., the torn edge of the captured tissue) move at different speeds.

[0040] A surgical microscope according to the invention, for example for microsurgery, includes the control device described above according to the invention or the robotic arrangement described above according to the invention. The surgical microscope according to the invention has the features and advantages already described in combination with the control device and the robotic arrangement according to the invention.

[0041] A method for controlling the movement of a surgical instrument according to the present invention comprises the following steps: optically capturing image data relating to a region of tissue using a sensor device; determining, using an evaluation device, at least one feature of the tissue in the captured region based on the image data captured by the sensor device, wherein changes in the feature indicate contact between the surgical instrument and the tissue; and outputting a feedback signal based on the determined at least one feature using a human-machine interface. The movement of the surgical instrument is preferably controlled based on the feedback signal. In this case, the control can be implemented in a manner that is at least partially automated. The movement may include repetitive movement patterns, as described in detail above. The method according to the present invention has features and advantages described in combination with the control device according to the present invention and the robot arrangement according to the present invention. The method can be implemented in variations described in combination with the control device according to the present invention and the robot arrangement according to the present invention. The control device according to the present invention, the robot arrangement according to the present invention, and the surgical microscope according to the present invention can be designed to perform the method according to the present invention. Attached Figure Description

[0042] The invention is explained in more detail below with reference to the accompanying drawings and exemplary embodiments. Although the invention has been illustrated and described in more detail by way of preferred exemplary embodiments, the invention is not limited to the disclosed examples, and other variations can be derived from the invention by those skilled in the art without departing from the scope of protection of the invention.

[0043] These figures are not necessarily drawn to scale in every detail, and may be presented in enlarged or reduced form for clarity. Therefore, the functional details disclosed herein should not be construed as limiting, but merely as an illustrative basis to provide guidance for those skilled in the art to use the invention in various ways.

[0044] When used in a series of two or more elements, the expression “and / or” as used herein means that any of the listed elements may be used alone, or any combination of two or more of the listed elements may be used. For example, if a structure is described as containing parts A, B, and / or C, then the structure may contain: A alone; B alone; C alone; a combination of A and B; a combination of A and C; a combination of B and C; or a combination of A, B, and C.

[0045] Figure 1 The control device according to the present invention is schematically shown in the form of a block diagram.

[0046] Figure 2 The robot arrangement according to the invention is schematically illustrated in block diagram form.

[0047] Figure 3 The surgical microscope according to the present invention is schematically shown in the form of a block diagram.

[0048] Figure 4 The control method according to the present invention is schematically illustrated in the form of a flowchart.

[0049] Figure 5 A schematic three-dimensional view of the surface of the cystic tissue is shown.

[0050] Figure 6 The movement of the tip of the surgical instrument is illustrated schematically during capsulotomy (capsulotomy).

[0051] Figure 7 and Figure 8 An example of the movement of a surgical instrument designed for brushing is illustrated schematically.

[0052] Figure 9 An example of surgical instruments being close to the tissue to be manipulated is illustrated.

[0053] Figure 10 and Figure 11Each schematically illustrates a handheld mechatronic controllable surgical instrument for brushing tissue. Detailed Implementation

[0054] Figure 1 The control device according to the invention is schematically illustrated in block diagram form. The control device 1 is designed to control the movement of surgical instruments 2, preferably for microsurgery. The control device 1 includes a sensor device 6, an evaluation device 7, and a human-machine interface (HMI) 8. The sensor device 6 is designed to optically capture image data relating to a region of tissue 4, particularly a region of the surface 5 of tissue 4. Preferably, tissue 4 is biological tissue intended to be acted upon by means of surgical instruments 2, such as organ tissue, for example, ocular tissue. The sensor device 6 can be a camera, such as the camera of a surgical microscope, or a scanning device for capturing laterally arranged pixels in a time sequence, or an OCT (Optical Characteristic Telescope).

[0055] The evaluation device 7 is designed to determine at least one feature of the tissue 4 in the captured area based on image data captured by means of the sensor device 6. In this case, the feature is one whose changes indicate contact between the surgical instrument 2 and the tissue 4. In other words, the feature thus represents tissue changes caused by the action of the surgical instrument 2. Advantageously, the evaluation device 7 can be designed to define the trajectory of the surgical instrument 2 based on image data captured by means of the sensor device 6. The trajectory may include a repetitive movement pattern. The repetitive movement pattern may include circular segments, and / or elliptical segments, and / or spiral segments, and / or parabolic segments, and / or hyperbolic segments, and / or linear segments. In a preferred variant, the control device 1 is designed to control at least partially automated movement of the surgical instrument 2 along the defined trajectory, for example, at least partially automated movement along a trajectory including a repetitive movement pattern.

[0056] HMI 8 is designed to output a feedback signal based on at least one identified feature. The feedback signal can be output to the surgeon, for example, in acoustic and / or visual and / or audiovisual and / or tactile form. HMI 8 can be further designed to input the lateral movement path of surgical instrument 2. In other words, the surgeon can thus be configured to manually control the lateral movement of surgical instrument 2 using HMI 8. However, the feedback signal can also be output to a corresponding robotic device for at least partial automated control of surgical instrument 2.

[0057] The signal transmission between sensor device 6, evaluation device 7 and HMI 8 is indicated by arrows with reference numeral 9 in their respective cases.

[0058] Figure 2A robot arrangement according to the invention is schematically illustrated in block diagram form. The robot arrangement 10, preferably designed for microsurgery, includes at least one robot-guided surgical instrument 2 and at least one robot manipulator 12 having a fastening device 11 for securing the surgical instrument 2 to the robot manipulator 12. The robot arrangement 10 further includes a control device 1 designed to control the surgical instrument 2 by means of the robot manipulator 12. The control device 1 may be pre-integrated... Figure 1 The control device 1 is described.

[0059] The robotic manipulator 12 may have at least three degrees of freedom for manipulation, and the control device 1 may be designed to control the movement of the surgical instrument 2 by means of the at least three degrees of freedom of the robotic manipulator 12. The robotic manipulator 12 may also be designed to interrupt and / or maintain the movement of the surgical instrument based on characteristics of a determined tissue, by means of which the contact between the surgical instrument 2 and the tissue 4 can be determined. Variations of this configuration have been described in detail above. The robotic arrangement 10 may be designed for partially or fully automated operation.

[0060] Figure 3 A surgical microscope according to the invention is schematically illustrated in block diagram form. The surgical microscope 13 includes a robotic arrangement 10 according to the invention, for example, combined with... Figure 3 The robot arrangement described is 10.

[0061] Figure 4 The control method according to the invention is schematically illustrated in the form of a flowchart. The method for controlling the surgical instrument 2 shown includes the following steps: In step 21, image data relating to a region of tissue 4 is optically captured by means of a sensor device 6 (e.g., by means of a camera or scanning device). In step 22, at least one feature of the tissue in the captured region is determined based on the image data captured by means of the sensor device 6, wherein a change in this feature indicates contact between the surgical instrument 2 and the tissue 4. This is performed using an evaluation device (e.g., evaluation device 7). In step 23, a feedback signal is output by means of an HMI 8 based on the determined at least one feature. In an optional step 24, a trajectory for movement or movement control of the surgical instrument 2 may be determined and output based on the image data captured by means of the sensor device 6 and / or based on the determined at least one feature and / or based on the output feedback signal, wherein the trajectory may have at least one repetitive movement pattern. Also optionally, movement of the surgical instrument 2 along the trajectory may be controlled.

[0062] The following explanation of embodiments of the invention is based on ophthalmic surgical applications. First, the individual aspects of the invention are described using the example of capsular tearing (capsulotomy). Figure 5 A perspective view of the surface 5 of the capsule tissue 4 is schematically shown. During capsule tearing, a segment of the capsule 31 is torn open, and the underlying lens is exposed through the formed opening 30. The movement of the surgical instrument used for this purpose is indicated by an arrow with reference numeral 32, which represents the trajectory of the tip of the surgical instrument. During the tearing process, the tear point of the tissue 4 moves away from the first location indicated by reference numeral 33. This is indicated by an arrow with reference numeral 34. An exemplary second location of the tear point is indicated by reference numeral 35.

[0063] Figure 6 The described process is illustrated in two steps. In this case, the movement of the tip of the surgical instrument 2 used is... Figure 6 The two xy-z curves at the top are shown. Here, the xy plane represents the transverse plane, which in this case is the plane extending from the surface 5 of the sac tissue 4. The z-direction represents the axial direction perpendicular to the extension of the xy plane. Figure 6 The bottom shows a three-dimensional view of the pouch tissue 4, which corresponds to the curves arranged above each of them. Figure 5 The view shown is similar.

[0064] exist Figure 6 In the steps shown at the top, the surgical instrument or its tip moves along the trajectory indicated by reference numeral 36 until it reaches the surface 5 of the tissue region 31 of the pouch 4 to be displaced. Then, that is, once contact is established between the surgical instrument 2 and the pouch 4, the surgical instrument 2 moves laterally in the xy plane. This is caused by the trajectory... Figure 6 The area indicated by reference numeral 37 at the bottom of the figure is shown. The displacement of the pouch tissue 31 caused by the action of the surgical instrument 2 is indicated by the arrow with reference numeral 38.

[0065] Contact between the tip of the surgical instrument and the pouch 4 is indirectly detected by changes (dynamic effects) that can be optically observed from the geometry of the torn tissue region 31, the geometry of the opening 30, or the tear point. Based on the captured image data, the tip of the surgical instrument can be controlled so that the tear extends or continues in a calculated direction in the xy-plane. Once movement or a change in geometry of the torn tissue region 31 is detected or captured, for example by means of a camera and / or microscope, the surgical instrument is held at its current position in the z-direction (i.e., relative to the current height of the tissue region 31), and movement of the surgical instrument in the xy-plane continues until a defined target point is reached. Subsequently, the surgical instrument 2 is lifted away from the tissue region 31. If no change in the geometry of the tissue region 31 is observed or captured, the same movement is repeated with a smaller z-value (i.e., a smaller distance from the pouch 4 or the tissue region 31). This operation is repeated until a change in the geometry of the tissue region 31 is observed or captured.

[0066] To determine or identify changes in tissue 4 caused by the action of surgical instrument 2, the geometry of the formed opening 30, and / or the geometry of the torn tissue region 31, and / or the geometry and / or progression of the tear point 33 can be captured and evaluated. Known image processing methods, as well as machine learning image processing methods or methods for image processing or image enhancement, can be used to determine, for example, changes in the geometry of the tear. For example, image processing or image enhancement can be achieved by increasing phase contrast or by using dark-field microscopy.

[0067] Figure 7 and Figure 8 An example of the movement of a surgical instrument designed for brushing is schematically illustrated. In this case, the membrane or tissue to be removed 4 is indicated by reference numeral 39 in its respective case. Reference numerals 40 and 41 are respectively used to indicate the trajectory of the tip of the surgical instrument in the axial plane (that is, a plane extending perpendicular to the transverse plane). Here, the trajectories 40 and 41 shown each have areas indicated by reference numerals 42 and 43 and have a repeating movement pattern. Figure 7 In the variant shown, the repetitive movement pattern 42 consists of parabolic segments. Figure 8 In the variants shown, the repetitive movement pattern 43 has a continuous circular or elliptical form. The size of the repetitive movement pattern is preferably in the micrometer range. The surgical instrument moves along... Figure 7 and Figure 8 The movement shown in the example trajectory can also be, for example, in... Figure 5 and Figure 6 Continue within the range of the application shown until the movement at the tear point is captured.

[0068] Figure 9This illustration schematically shows an example of surgical instruments close to tissue to be manipulated. This could be, for example... Figure 6 The steps are shown at the top. In the illustrated variant, the surgical instrument 2 is repeatedly moved along an elliptical or parabolic trajectory indicated by reference numeral 44, wherein, in each case, an examination is performed to determine whether the movement of the surgical instrument 2 affects the geometry of the torn area 31 of the pouch 4. If no change in the tissue area 31 is observed, an offset is added to the movement in the z-direction toward the surface 5 of the pouch 4, that is, the distance from the tissue area 31 or the surface 5 is reduced, and the movement is repeated at this distance. The multiple movements performed in sequence are indicated by trajectory 44.

[0069] If a change in the geometry of tissue region 31 is detected or determined (as is the case with trajectory 45 in the current situation), the z-position is defined as the corresponding height or a defined lower limit, and movement continues at that height until the planned area for tearing in the lateral direction is reached. Subsequently, surgical instrument 2 is lifted away from tissue 4. If the specific, defined, or desired displacement of tissue region 31 is achieved, subsequent steps can be planned or performed.

[0070] exist Figure 9 In the figure, reference numeral 46 is used to indicate the desired point of action of the surgical instrument 2 on the tissue 4. Reference numeral 45 is used to indicate the trajectory of movement of the determined tissue region 31. When trajectory 45 is reached, movement along the z-direction is stopped, and the method continues only along the xy-direction (that is, along the lateral direction).

[0071] The following text is based on Figure 10 and Figure 11 Describe a variant of surgical instrument 2 that is controlled in a partially manual and partially automated manner. Figure 10 and Figure 11 Each is schematically shown as a handheld mechatronic controllable surgical instrument 2, which includes, for example, a cystotome 51 for brushing the membrane with its tip 47. In this case, the surgical instrument 2 can be manually controlled in the lateral direction. This can be achieved by means of a control device 1 according to the invention or a robotic manipulator 12 according to the invention, such as... Figure 10 The axial position indicated by arrow 48 is automatically set, for example, on a micrometer scale. Figure 11 As shown, the arrangement is designed to automatically perform brushing movements. For this purpose, three robotic devices, with dimensions in the micrometer range, can be configured to control the movement of the tip 47 of the surgical instrument 2 in three-dimensional space using three degrees of freedom of motion. In this case, rotational movement of the capsulorhexis about point 50 can be provided.

[0072] The feedback signal regarding the axial position of the tip of the surgical instrument, output by the control device 1 according to the invention, can also be used by the surgeon as AR feedback (AR: Augmented Reality). Once the tear point of the tissue moves, for example, the area around the tip of the surgical instrument 2 can be displayed in a set color (e.g., green). Alternatively, the tear point itself can also be displayed in color. As an additional feature, the displayed color can change continuously according to the determined characteristics indicating the contact between the surgical instrument 2 and the tissue 4. For example, a first color (e.g., red) can be displayed when there is no contact, and a second color (e.g., green) can be displayed when good grip or contact is achieved. Whether the contact or grip is good can be determined by comparing the movement of the tip of the surgical instrument 2 with the movement of the tear point. It should be considered that, due to the dual position of the capsule tissue (i.e., the torn tissue area 31 and the intact capsule tissue 5), the movement distance of the surgical instrument 2 is greater than the movement distance of the tear point.

[0073] List of reference numerals in the attached diagram:

[0074] 1. Control equipment

[0075] 2 Surgical Instruments

[0076] 3 Control

[0077] 4 organizations

[0078] 5. Tissue surface

[0079] 6 sensor devices

[0080] 7 Evaluation Devices

[0081] 8HMI

[0082] 9. Signal transmission

[0083] 10 robots deployed

[0084] 11 Fastening devices

[0085] 12 robot manipulators

[0086] 13 Surgical Microscopes

[0087] 21. Optically capture image data related to the tissue region.

[0088] 22. Determining at least one feature of tissue in the captured area based on image data captured by means of a sensor device.

[0089] 23. Output a feedback signal based on at least one determined feature.

[0090] 24. Determine and output the trajectory used to control the movement of surgical instruments.

[0091] 30 openings / cornea

[0092] 31 torn tissue areas

[0093] 32. Movement of surgical instruments

[0094] 33 tear points

[0095] 34. Continuation of tear points

[0096] 35 tear points

[0097] The first region of 36 trajectories

[0098] The second region of trajectory 37

[0099] 38. Direction of movement of the torn tissue area

[0100] 39 membranes / tissue

[0101] 40 tracks

[0102] 41 trajectories

[0103] 42 areas with repetitive movement patterns

[0104] 43 Areas with repetitive movement patterns

[0105] 44 tracks

[0106] 45 tracks

[0107] 46 points of action

[0108] 47 end

[0109] 48 Controllable movement directions

[0110] 49 Robotic Devices

[0111] 50 pivot points

[0112] 51. Bag-cutting knife.

Claims

1. A control device (1) for controlling the movement of surgical instruments (2), Its features are, The control device (1) includes a sensor device (6), an evaluation device (7), and a human-machine interface (8), wherein, The sensor device (6) is designed to optically capture image data related to the region of the tissue (4). The evaluation device (7) is designed to determine at least one feature of the tissue (4) in the captured area based on image data captured by the sensor device (6), wherein changes in the feature indicate contact between the surgical instrument (2) and the tissue (4). Furthermore, the human-machine interface (8) is designed to output a feedback signal based on the determined at least one feature.

2. The control device (1) as described in claim 1. Its features are, The evaluation device (7) is designed to define the trajectory (32, 40, 41, 44) of the surgical instrument (2) based on image data captured by means of the sensor device (6), and / or based on the determined at least one feature of the tissue (4), and / or based on the output feedback signal.

3. The control device (1) as described in claim 1 or claim 2. Its features are, The control device (1) is designed to output a trajectory (32, 40, 41, 44) for the movement of the surgical instrument (2) based on image data captured by means of the sensor device (6), and / or based on the determined at least one feature of the tissue (4), and / or based on the output feedback signal, wherein the trajectory (32, 40, 41, 44) includes at least one repetitive movement pattern (42, 43). And / or the control device (1) is designed to control the surgical instrument (2) to move at least partially automatically along the determined trajectory (32, 40, 41, 44).

4. The control device (1) as described in any one of claims 2 to 3. Its features are, The trajectory (32, 40, 41, 44) comprises a continuous path along an imaginary surface having a conical or cylindrical shape, and / or the path having circular, elliptical, spiral, parabolic, hyperbolic, or linear segments.

5. The control device (1) as described in any one of claims 2 to 4. Its features are, The trajectory (32, 40, 41, 44) lies at least partially in a plane that extends perpendicularly to the transverse plane and / or to the object plane of the sensor device (6).

6. The control device (1) as described in any one of claims 1 to 5. Its features are, The sensor device (6) includes at least one camera for simultaneously capturing multiple horizontally arranged pixels and / or a scanning device for capturing horizontally arranged pixels in chronological order.

7. The control device (1) as described in any one of claims 1 to 6. Its features are, The human-machine interface (8) is designed to input the lateral path of movement of the surgical instrument (2).

8. The control device (1) as described in any one of claims 1 to 7. Its features are, The control device (1) is designed to distinguish different types of contact between the tissue (4) and the surgical instrument (2) based on the determined at least one feature.

9. A robot arrangement (10) comprising at least one robot-guided surgical instrument (2) and at least one robot manipulator (12) having a fastening device (11) for fastening the surgical instrument (2) to the robot manipulator (12). Its features are, The robot arrangement (10) includes a control device (1) as claimed in any one of claims 1 to 8, the control device being designed to control the movement of the surgical instrument (2) by means of the robot manipulator (12).

10. The robot arrangement as described in claim 9 (10). Its features are, The robotic manipulator (12) has at least three degrees of freedom for manipulation, and the control device (1) is designed to control the movement of the surgical instrument (2) by means of the at least three degrees of freedom of the robotic manipulator (12).

11. The robot arrangement (10) as described in claim 9 or 10. Its features are, The robot arrangement (10) is designed such that the entry point into a specific tissue is defined or can be defined as a reference point for the movement of the surgical instrument (2), wherein the lateral position of the surgical instrument (2) corresponds to the angular orientation of the surgical instrument (2) or the robot manipulator (12), and the axial position of the surgical instrument (2) corresponds to the depth of entry of the surgical instrument (2) or the robot manipulator (12) through the entry point.

12. The robot arrangement (10) as described in any one of claims 9 to 11. Its features are, The robot deployment (10) is designed to interrupt the movement of the surgical instrument (2) if it is determined that contact has occurred between the tissue (4) and the surgical instrument (2), and / or The robot arrangement (10) is designed to maintain the axial movement of the surgical instrument (2) while it moves laterally along a lateral path that can be defined by the surgeon.

13. The robot arrangement (10) as described in any one of claims 9 to 12. Its features are, The robot arrangement (10) is designed for partially or fully automated operation, wherein the lateral component of the trajectory (32, 40, 41, 44) of the surgical instrument (2) can be controlled based on data captured by the sensor device (6), wherein the contact between the surgical instrument (2) and the tissue (4) is maintained during the movement of the surgical instrument (2) in a repetitive movement pattern (42, 43).

14. The robot arrangement (10) as described in any one of claims 9 to 13. Its features are, The surgical instrument (2) can be controlled by the user via the human-machine interface (8) to move laterally relative to the tissue (4) or tissue surface (5), and the axial or inward movement of the surgical instrument (2) relative to the tissue (4) or tissue surface (5) can be controlled automatically, and / or The human-machine interface (8) includes a planning function, by means of which the area of ​​tissue to be treated can be defined based on lateral position data captured by means of the sensor device (6), and the robot arrangement (10) is designed to perform defined surgical steps, at least in a partially automated manner, based on data captured by means of the sensor device (6) and related to the contact between the surgical instrument (2) and the tissue (4), and / or The human-machine interface (8) is designed to output information for assistance.

15. The robot arrangement (10) as described in any one of claims 9 to 14. Its features are, The robot arrangement (10) is able to switch between two operating modes during the execution of surgical procedures, wherein in the first operating mode the surgical instrument (2) is in contact with the tissue (4) and in the second operating mode the surgical instrument (2) is not in contact with the tissue (4).

16. The robot arrangement (10) as described in any one of claims 9 to 15. Its features are, The robot arrangement (10) is designed to capture the movement of the tissue (4) and, in the event of capturing the movement of the tissue (4), to at least cause the surgical instrument (2) to follow the captured movement.

17. A surgical microscope (13), the surgical microscope Includes the control device (1) as claimed in any one of claims 1 to 8 or the robot arrangement (10) as claimed in any one of claims 9 to 16.

18. A method for controlling the movement of a surgical instrument (2), Its features are, The method includes the following steps: - Image data (21) related to the region of the tissue (4) is captured optically by means of sensor device (6). - Using the evaluation device (7), at least one feature (22) of the tissue (4) in the captured area is determined, wherein the at least one feature of the tissue (4) is determined based on image data captured by means of the sensor device (6), and wherein changes in the feature indicate contact between the surgical instrument (2) and the tissue (4). - By means of the human-machine interface (8), a feedback signal (23) is output based on the determined at least one feature.

Images