Control apparatus, robotic arrangement, surgical microscope and method for controlling the movement of a surgical instrument

Abstract

The invention relates to a control apparatus (1) for controlling the movement of a surgical instrument (2), wherein the control apparatus (1) comprises a sensor device (6), an evaluation device (7) and a human-machine interface (8), and wherein the sensor device (6) is designed to optically capture image data of a region of a tissue (4), the evaluation device (7) is designed to determine at least one feature of the tissue (4) in the captured region on the basis of image data captured by means of the sensor device (6), with a change in the feature characterizing a contact between the surgical instrument (2) and the tissue (4), and the human-machine interface (8) is designed to output feedback signals on the basis of the at least one determined feature.

Description

[0001] Control device, robotic assembly, surgical microscope and method for controlling the movement of a surgical instrument

[0002] The present invention relates to a control device, a robotic assembly, and a surgical microscope, for example for microsurgery, in particular for ophthalmic surgery and neurosurgery. The invention also relates to a method for controlling the movement of a surgical instrument.

[0003] Currently, surgical procedures are typically performed manually. In certain areas, such as laparoscopy, telemanipulable surgical robots are gaining popularity. In the future, systems with task-level or higher levels of automation will become increasingly important. Such robotic systems will require advanced sensor technology tailored to the specific requirements and functions adapted to the specific procedure being performed.

[0004] During manual procedures, operators, such as surgeons, rely on their visual feedback to estimate depth and thus instrument-tissue interaction. Surgeons acquire this skill through years of experience. A robotic system therefore requires appropriate feedback to ensure safe and reliable contact with the tissue.

[0005] In many ophthalmic surgical procedures, surgical instruments must come into contact with delicate tissue, for example to grasp a capsular bag during cataract surgery or to remove a membrane on the retina during membrane peeling in retinal surgery. The optimal forces are in the range of a few millinewtons (mN). These optimal forces between the tip of the surgical instrument and the tissue to be treated are often below the range perceptible to humans. If the forces are too high, there is a risk that the surgical instrument will damage surrounding tissue, e.g. the retina during retinal surgery or the zonular fibers during cataract surgery. If the forces are too low, the surgical step may not be possible, e.g. because it is not possible to reliably grasp a tissue.Capturing or sensing force is also complicated because during many ophthalmic surgical steps, forces and torques are absorbed at the point of entry into the eye, e.g., through a trocar for retinal surgery or a corneal incision during cataract surgery. As a result, the surgeon (hereafter referred to as the operating surgeon) cannot rely on haptic feedback during many ophthalmic surgical steps and must rely on sensors outside of their own body perception.

[0006] Ophthalmic surgical microscopes are suitable for recording the relative lateral position between a surgical instrument and the tissue and for a rough estimation of depth, e.g. the depth of penetration of the instrument into the tissue. To compensate for inadequate force and depth detection, surgeons often rely on additional cues such as observed shadows, color changes or wrinkling of the tissue under load. In other cases, surgical steps must be repeated at different depths until, for example, the instrument takes hold and the surgical step can be successfully performed. This is often only recognizable by tissue reactions, such as the propagation of a tear point. Many visual cues that a surgeon typically captures are difficult to detect with a camera system or computer algorithms. Femtosecond lasers, for example, are available to assist cataract surgery.These systems typically feature an integrated optical coherence tomography (OCT) scanner for enhanced depth sensing. However, this incurs additional costs, requires contact lenses to be placed on the eye, and, in general, OTC systems may not be able to adequately detect contact between the surgical instrument and the tissue.

[0007] Document US 2011 / 0 106 102 A1 describes a robotic surgical system for eye surgery, using an optical sensor in the region of the tip of a surgical instrument. Document US 2017 / 0 312 431 A1 describes an eye surgery system that enables pressure measurement during an infusion. Document US 2016 / 0 074 212 A1 discloses a device for introducing a medication into an eye, which includes a detection and visualization system for detecting and visualizing penetration of the choroid by an injection needle as feedback for a surgeon and / or for automated control.

[0008] Document US 2007 / 0 151 390 A1 describes a robotic surgical instrument with a force and torque sensor at the tip. Documents US 2012 / 0 265 102 A1 and WO 2017 / 118949 A1 disclose cardiac catheters with force sensors at the tip.

[0009] Against the background described, it is an object of the present invention to provide an advantageous control device, for example for microsurgery, for controlling the movement of a surgical instrument. Further objects consist in providing an advantageous robotic arrangement, a surgical microscope and a method for controlling the movement of a surgical instrument, each for example for microsurgery. The stated objects are achieved by a control device according to patent claim 1, a robotic arrangement according to patent claim 9, a surgical microscope according to patent claim 17 and a method for controlling the movement of a surgical instrument according to patent claim 18. The dependent claims contain further advantageous embodiments of the invention.

[0010] The control device according to the invention, preferably for microsurgery, for controlling the movement of a surgical instrument, in particular a surgical tool, comprises a sensor device, an evaluation device, and a human-machine interface. In other words, a sensor device can be understood as a means for detecting physically measurable quantities, such as a camera, a detector, a signal detection device, or a measuring device, etc. In other words, an evaluation device can be understood as a means for receiving, processing, and outputting data and / or signals, e.g., a computer or a data processing system. In other words, a human-machine interface (HMI) can be understood as a user interface for inputs by a user and / or outputs to the user.

[0011] The sensor device is designed for the optical detection, in particular the visual detection, of image data of a region of tissue, i.e. the tissue region to be manipulated. The evaluation device is designed, based on image data detected by the sensor device, to determine at least one feature, i.e. an optically detected feature, of the tissue in the detected region, wherein a change in the feature indicates contact or interaction between the surgical instrument and the tissue. The human-machine interface is designed, based on the at least one detected feature, to output feedback signals, e.g., to a surgeon and / or a robotic manipulator. In other words, the control device according to the invention is designed to detect and evaluate tissue changes caused by contact with the surgical instrument.Determining the at least one feature of the tissue may include capturing and / or detecting and / or qualifying and / or quantifying the feature. For example, changes in the tissue with respect to its color and / or geometric shape may be detected and evaluated.

[0012] A surgical instrument is understood here to be a physical tool or device with which tissue can be physically influenced as part of a surgical procedure.

[0013] The at least one feature determined based on image data may be a parameter or quantity, e.g. a color or color change or a distance or a dimension or a shape of a shadow or a type of tissue reaction or another parameter that characterizes a tissue reaction caused or induced by an interaction or contact with a surgical instrument.

[0014] The control device according to the invention, in particular for microsurgery, is preferably designed to control the movement of a surgical instrument during a surgical procedure or intervention. It can be a control device for ophthalmic surgery or neurosurgery. In particular, the control device according to the invention can be designed for individual steps, e.g., capsulorhexis or capsular bag polishing, during cataract surgery. Further application examples include membrane peeling in retinal surgery, e.g., the brushing step for detaching the torn membrane, corneal surgery, or generally avoiding the application of excessive force during phacoemulsification or lens injections. Furthermore, the control device according to the invention can be designed for assisted or automated suturing in microsurgery, e.g., in ophthalmic surgery or neurosurgery.

[0015] The control device according to the invention has the advantage that it provides improved feedback for microsurgical applications when controlling the movement of the surgical instrument, ensuring optimal execution of the planned intervention or procedure. The control system is thus highly reliable. It is also cost-effective, as no expensive sensor technology is required.

[0016] In an advantageous variant, the evaluation device is designed to define a trajectory of the surgical instrument based on image data acquired by the sensor device and / or based on the at least one determined feature and / or based on the output feedback signal. Defining the trajectory can also include determining and / or adapting the trajectory. In particular, the trajectory can be continuously adapted depending on the behavior of the determined feature, in particular a change in the determined feature. This is advantageous for an assistance function for manual guidance of the surgical instrument by a surgeon or for an at least partially automated robotic application. The effect of the surgical instrument on the tissue can be optimized by means of a corresponding trajectory.

[0017] In an advantageous variant, the control device can be designed to output a trajectory for a movement, e.g., an at least partially automated movement, of the surgical instrument based on image data acquired by the sensor device and / or based on the at least one determined feature and / or based on the output feedback signal. The trajectory can comprise at least one repeating movement pattern. In a preferred variant, the control device is designed to control an at least partially automated movement of the surgical instrument, in particular the movement of a tip of the surgical instrument, along a defined trajectory comprising a repeating movement pattern.

[0018] The trajectory can, for example, comprise a continuous path along an imaginary surface, wherein the surface can be conical or cylindrical in shape and / or the path can have circular and / or elliptical and / or helical and / or parabolic and / or hyperbolic and / or linear sections. The movement can be configured at least partially as a grasping movement.

[0019] In a further variant, the trajectory can lie at least partially in a plane. This plane can extend perpendicular to a lateral plane and / or perpendicular to an object plane of the sensor device. The object plane is understood to be a plane in which, for example, the tissue is optically detected. Using these variants, gripping movements or cutting movements of the surgical instrument can be optimized, for example.

[0020] The sensor device can comprise at least one camera for simultaneously capturing a plurality of laterally arranged pixels and / or a scanning device for time-sequentially capturing laterally arranged pixels. The sensor device can comprise a stereo camera and / or an optical coherence tomograph (OCT) and / or a surgical microscope and / or a confocal system. Low-resolution axial information can be fused, combined, or supplemented with more precise information, e.g., information derived from a tissue reaction. In this way, reliable data, particularly axial data, for controlling the surgical instrument can be obtained in a simple and cost-effective manner.

[0021] The human-machine interface can further be configured to input a lateral path of movement of the surgical instrument. For example, telemanipulation of the lateral position of the surgical instrument can be provided, while a specific, e.g., predetermined, axial position of the surgical instrument relative to the tissue is maintained by means of a robotic controller. The control device can be configured to plan and / or define a trajectory of the surgical instrument for partially or fully automated control of the surgical instrument by means of a robotic arrangement.

[0022] The robotic assembly according to the invention, in particular for microsurgery, comprises at least one robotically guided surgical instrument, in other words a surgical instrument whose movement can be controlled by a robot, i.e., a surgical instrument that can be moved or operated by a robot, and at least one robotic manipulator, in other words a robotically movable device for physical interaction with the environment, having a fastening device, in other words, a means for establishing a reversible physical connection, for fastening the surgical instrument to the robotic manipulator. The robotic assembly comprises a previously described control device according to the invention, which is designed to control the movement of the surgical instrument by means of the robotic manipulator. The robotic assembly can be designed for partially or fully automated operation.The robotic arrangement according to the invention has the same features and advantages as the previously described control device according to the invention.

[0023] The robotic manipulator preferably has at least 3, e.g., 6, degrees of freedom for manipulation, and the control device is preferably designed to control the movement of the surgical instrument by means of the at least 3 degrees of freedom, e.g., the 6 degrees of freedom, of the robotic manipulator. The degrees of freedom can be translational degrees of freedom and / or rotational degrees of freedom. For example, 3 translational degrees of freedom relative to a Cartesian coordinate system and / or 3 rotational degrees of freedom, e.g., around the respective axes of the Cartesian coordinate system, can be provided. Depending on the requirements of the application and the reference system used, e.g., a Cartesian coordinate system, a cylindrical coordinate system, a spherical coordinate system, etc., a specific number of translational degrees of freedom and / or rotational degrees of freedom can be provided.A manipulator with 6 degrees of freedom, comprising 3 translational degrees of freedom and 3 rotational degrees of freedom, has the advantage that it allows unrestricted manipulation of movement in all directions. For certain applications, a smaller number of degrees of freedom may be sufficient. A manipulator with, for example, only 3 or 4 degrees of freedom has the advantage of being more cost-effective and potentially less error-prone than a manipulator with a higher number of degrees of freedom.

[0024] Advantageously, the robotic arrangement is designed such that an entry point into a specific tissue, e.g. a defined tissue of a specific organ, is defined or can be defined as a reference point, e.g. in the form of a coordinate origin, for a movement of the surgical instrument, wherein the lateral position of the surgical instrument corresponds to an angular orientation of the surgical instrument or the manipulator with respect to the reference point, i.e. at least one rotation angle of the surgical instrument or the manipulator with respect to the reference point, and the axial position of the surgical instrument corresponds to an entry depth of the surgical instrument or the manipulator through the entry point. The entry point can be defined by a trocar or a puncture into a tissue, e.g. the cornea.Defining the entry point as a reference point enables intuitive control of the movement of the surgical instrument, modeled on the movement pattern during a manual procedure and also computationally efficient.

[0025] The robotic arrangement can be designed to interrupt a movement, in particular the repetitive movement pattern, of the surgical instrument if contact between the tissue and the surgical instrument is detected, e.g. by means of the evaluation device. This has the advantage that a potentially undesired effect on the tissue can be avoided. The robotic arrangement can be designed to switch to a mode with a predetermined, e.g. user-defined, lateral movement path. Additionally or alternatively, the robotic arrangement can be designed to maintain a movement, e.g. a fixed axial movement and / or the repetitive movement pattern, of the surgical instrument while simultaneously moving the surgical instrument along a lateral path, e.g. a lateral path that can be determined or is fixed or user-defined by a surgeon or user.

[0026] The robotic arrangement can be designed for partially or fully automated operation, wherein the lateral components of a trajectory of the surgical instrument are controllable based on data acquired by the sensor device, wherein contact between the surgical instrument and the tissue is maintained during the execution of a movement, e.g., an axial movement, of the surgical instrument with a repetitive movement pattern. Partially automated operation has the advantage that certain movements of the surgical instrument are manually controllable, thus allowing a surgeon to act independently in this regard, while at the same time the high accuracy and precision of automated operation is utilized for other movements. Fully automated operation has the advantage that a high accuracy and precision of the surgical procedure is guaranteed.

[0027] The term control also includes regulation in the sense of control and regulation technology. In particular, contact between the tissue and the surgical instrument can be controlled partially or fully automatically. In this case, the robotic arrangement can be designed for regulation by means of a closed-loop controller to maintain contact between the tissue and the surgical instrument, e.g., for a definable period of time. In a further variant, the robotic arrangement can comprise a human-machine interface by means of which the surgical instrument can be controlled by a user with regard to its lateral movement, i.e. with regard to the lateral movement components, in relation to the tissue or the tissue surface, while an axial or medial movement, i.e. the axial or medial movement component, of the surgical instrument in relation to the tissue or the tissue surface can be controlled automatically.The control can be fully or partially automated and / or in the form of a closed-loop control. For example, an input device, e.g., a joystick or similar, can be provided, by means of which a surgeon or user can control the surgical instrument laterally while maintaining contact or the axial relationship between the surgical instrument and the tissue based on data acquired by the sensor device. The described variant has the advantage of preventing excessive or insufficient force from the surgical instrument on the tissue, while simultaneously enabling manual lateral control of the surgical instrument.

[0028] Furthermore, the robotic arrangement can comprise a human-machine interface with a planning function, i.e. a corresponding component, by means of which a tissue area to be treated can be defined, e.g. by a user, on the basis of lateral position data detected by the sensor device, and the robotic arrangement is designed to carry out a defined surgical step at least partially automatically, i.e. partially or fully automatically, based on data detected by the sensor device regarding contact between the surgical instrument and the tissue surface, e.g. based on the movement with a repetitive movement pattern. For example, by means of the planning functions, a surgeon orUsers can define a spatial area to be treated based on lateral position data acquired by the sensor device, and the robotic arrangement can be configured to perform a surgical step, for example, gripping a capsular bag membrane, at the user-defined lateral position while the surgical instrument is brought into a specific, e.g., fixed, axial position for the procedure. This is preferably done based on the at least one determined feature and / or the determined feedback signals. A closed-loop control function can be provided or present, which is configured to automatically bring the surgical instrument into the correct lateral position and the correct distance from the tissue.

[0029] Advantageously, the robotic arrangement comprises a human-machine interface designed to output information for assistance, in particular warnings and recommendations, e.g., regarding the trajectory of the surgical instrument. This can improve the quality of the procedure and allow for compliance with possible safety requirements. Closed-loop control can also be provided in this context.

[0030] In a further variant, the robotic arrangement can be switchable between two operating modes during the execution of a surgical procedure or task, wherein in a first operating mode the surgical instrument and the tissue are in contact and in a second operating mode the surgical instrument and the tissue are not in contact. The second operating mode can be designed or used to reposition the surgical instrument. The operating mode can be selected based on data acquired by the sensor device. The operating modes can be switchable manually and / or automatically. Closed-loop controls can be provided within the operating modes. For example, in the first operating mode a membrane can be grasped by the surgical instrument to tear it off and in the second operating mode the membrane can be released to reposition the surgical instrument.In both operating modes, the sensor device or multiple sensor devices provide the information required to maintain a fixed or definable contact between the surgical instrument and the tissue. Switching can be performed manually, for example, based on lateral position information, or automated based on the progress of a surgical step and information from the sensor device.

[0031] The control device according to the invention and / or the robotic arrangement according to the invention can be designed to differentiate between different, e.g. classified, types of contact between the tissue and the surgical instrument based on the at least one determined feature, e.g. a specific observed tissue reaction. The output of feedback signals and / or the control of the surgical instrument can be provided based on a type of contact determined by means of the differentiation. In one example, a distinction can be made or differentiated between good contact, moderate contact, and no contact between the tissue and the surgical instrument. In the case of a capsulorhexis operation, good contact can be recognized by an expansion of the tear point.Moderate contact can be identified by the fact that a relatively large movement of the surgical instrument results in a relatively small extension of the tear point. No contact can be identified by the lack of extension of the tear point.

[0032] The robotic arrangement can be designed to detect a movement of the tissue and, in the event of a detected movement of the tissue, for example as a result of heartbeat or respiration, to track at least the surgical instrument and, if necessary, other components as well, according to the detected movement. In this way, unplanned effects of the surgical instrument on the tissue caused by the movement of the tissue can be reduced or avoided. The tracking can be designed or carried out in an automated manner, e.g., semi-automated or fully automated. The tracking can be limited by definable or fixed limits. The detection of a movement of the tissue can take place using the sensor device or other sensor devices, e.g., a head tracker or an eye tracker. The robotic arrangement can be designed to detect a movement of the tissue in the lateral and / or axial direction.In addition, the robotic assembly can be designed to track the surgical instrument and, if necessary, other components in a lateral and / or axial direction. Specific movement restrictions can be provided.

[0033] For example, a sensor system for acquiring lateral position data can be provided, which provides axial position data with low resolution, e.g., a stereo camera system or OCT system. The data acquired with this sensor system can be provided for superimposing, supplementing, or merging with axial data derived from the at least one determined feature. The axial data or information can be used, in particular, to reposition the surgical instrument to a point at which a repetitive movement pattern is to be executed.

[0034] A proposed algorithm for evaluating tissue response in the case of capsulorhexis may take into account that the length of movement of the surgical instrument is twice the extension of the tear point.

[0035] The robotic assembly can be designed for axial control of the surgical instrument, e.g., in the case of teleoperative applications, to provide haptic feedback to a surgeon. This can be implemented via the human-machine interface, e.g., a joystick, in the form of vibrations or through active resistance in a specific direction of movement. Furthermore, in an exemplary embodiment for ophthalmic surgery, axial movement below a capsular bag can be restricted if sufficient grip of the tissue is achieved by the surgical instrument.

[0036] In general, a tissue reaction to contact with a surgical instrument can be determined from an optical flow or sequentially acquired images of the tissue. The optical flow of an image sequence is the vector field of the velocity of visible points of the object space projected into the image plane in the reference system of the imaging optics. Furthermore, an area or region in which the surgical instrument is located can be excluded when detecting differences between sequentially acquired images. In a special embodiment, an algorithm for detecting differences between sequentially acquired images can be designed to distinguish between cases in which the entire image is moving (e.g., due to a vibration of the microscope) and cases in which different partial areas of the image (e.g.,detected tear edges of the fabric) move at different speeds.

[0037] The surgical microscope according to the invention, e.g., for microsurgery, comprises a control device according to the invention as described above or a previously described robotic arrangement according to the invention. The surgical microscope according to the invention has the features and advantages already described in connection with the control device according to the invention and the robotic arrangement according to the invention.

[0038] The method according to the invention for controlling the movement of a surgical instrument comprises the following steps: optically capturing image data of a region of tissue using a sensor device; determining at least one feature of the tissue in the captured region based on the image data captured by the sensor device, wherein a change in the feature indicates contact between the surgical instrument and the tissue, using an evaluation device; and outputting feedback signals based on the at least one determined feature using a human-machine interface. The movement of the surgical instrument is preferably controlled based on the feedback signals. The control can be at least partially automated. The movement can comprise a repeating movement pattern, as already described in detail above.The method according to the invention has the features and advantages described in connection with the control device according to the invention and the robotic arrangement according to the invention. It can be implemented in the variants described in connection with the control device according to the invention and the robotic arrangement according to the invention. The control device according to the invention, the robotic arrangement according to the invention, and the surgical microscope according to the invention can be designed to carry out the method according to the invention.

[0039] The invention will be explained in more detail below using exemplary embodiments with reference to the accompanying figures. Although the invention is illustrated and described in more detail by the preferred embodiments, the invention is not limited to the disclosed examples, and other variations may be derived therefrom by a person skilled in the art without departing from the scope of the invention.

[0040] The figures are not necessarily detailed or to scale and may be enlarged or reduced to provide a better overview. Therefore, the functional details disclosed herein are not to be considered limiting, but merely as an illustrative basis for providing guidance to one skilled in the art for variously employing the present invention.

[0041] As used herein, the term "and / or," when used in a series of two or more elements, means that any of the listed elements may be used alone, or any combination of two or more of the listed elements may be used. For example, if a composition is described which contains components A, B, and / or C, the composition may contain A alone; B alone; C alone; A and B in combination; A and C in combination; B and C in combination; or A, B, and C in combination. Fig. 1 schematically shows a control device according to the invention in the form of a block diagram.

[0042] Fig. 2 shows schematically a robotic arrangement according to the invention in the form of a block diagram.

[0043] Fig. 3 shows schematically a surgical microscope according to the invention in the form of a block diagram.

[0044] Fig. 4 shows schematically a control method according to the invention in the form of a flow chart.

[0045] Fig. 5 shows schematically a perspective view of a surface of a capsular bag tissue.

[0046] Fig. 6 shows schematically the movement of a tip of a surgical instrument during capsular bag opening (capsulorhexis).

[0047] Figs. 7 and 8 show schematic examples of the movement of a surgical instrument designed for brushing.

[0048] Fig. 9 shows schematically an example of an approach of a surgical instrument to a tissue to be manipulated.

[0049] Fig. 10 and 11 each schematically show a hand-held mechatronically controllable surgical instrument for brushing tissue.

[0050] Figure 1 schematically shows a control device according to the invention in the form of a block diagram. The control device 1 is designed to control 3 the movement of a surgical instrument 2, preferably for microsurgery. The control device 1 comprises a sensor device 6, an evaluation device 7, and a human-machine interface (HMI) 8. The sensor device 6 is designed to optically capture image data of a region of a tissue 4, in particular a region of a surface 5 of the tissue 4. The tissue 4 is preferably a biological tissue, for example the tissue of an organ, e.g., eye tissue, which is to be acted upon by means of the surgical instrument 2. The sensor device 6 can be a camera, for example a camera of a surgical microscope, or a scanning device for the time-sequential capture of laterally arranged pixels, or an OCT.

[0051] The evaluation device 7 is designed to determine at least one feature of the tissue 4 in the detected region based on image data acquired by the sensor device 6. This feature is a feature whose change characterizes contact between the surgical instrument 2 and the tissue 4. In other words, the feature depicts a tissue change resulting from an action by the surgical instrument 2. Advantageously, the evaluation device 7 can be designed to define a trajectory of the surgical instrument 2 based on image data acquired by the sensor device 6. The trajectory can comprise a repeating movement pattern. The repeating movement pattern can comprise circular and / or elliptical and / or helical and / or parabolic and / or hyperbolic and / or linear subregions.In a preferred variant, the control device 1 is designed to control an at least partially automated movement of the surgical instrument 2 along a defined trajectory, for example along a trajectory which comprises a repeating movement pattern.

[0052] The HMI 8 is designed to output feedback signals based on the at least one determined feature. Feedback signals can be output to a surgeon, for example, acoustically and / or visually and / or audiovisually and / or haptically. The HMI 8 can also be designed to input a lateral movement path of the surgical instrument 2. In other words, it can be provided that a surgeon manually controls the lateral movement of the surgical instrument 2 using the HMI 8. However, the feedback signals can also be output to a corresponding robotic device for at least partially automated control of the surgical instrument 2.

[0053] The signal transmission between the sensor device 6, the evaluation device 7 and the HMI 8 is each marked by an arrow with the reference number 9.

[0054] Figure 2 schematically shows a robotic arrangement according to the invention in the form of a block diagram. The robotic arrangement 10, which is preferably designed for microsurgery, comprises at least one robotically guided surgical instrument 2 and at least one robotic manipulator 12 with a fastening device 11 for fastening the surgical instrument 2 to the robotic manipulator 12. The robotic arrangement 10 also comprises a control device 1, which is designed to control the surgical instrument 2 by means of the robotic manipulator 12. The control device 1 can be a control device 1 already described in connection with Figure 1.

[0055] The robotic manipulator 12 can have at least three degrees of freedom of movement for manipulation, and the control device 1 can be designed to control the movement of the surgical instrument 2 using the at least three degrees of freedom of movement of the robotic manipulator 12. The robotic manipulator 12 can further be designed to interrupt a movement of the surgical instrument and / or to maintain a movement of the surgical instrument depending on the determined feature of the tissue by means of which contact between the surgical instrument 2 and the tissue 4 can be determined. Variants of this have already been described in detail above. The robotic arrangement 10 can be designed for partially or fully automated operation. Figure 3 schematically shows a surgical microscope according to the invention in the form of a block diagram.The surgical microscope 13 comprises a robotic arrangement 10 according to the invention, for example a robotic arrangement 10 described in connection with Figure 3.

[0056] Figure 4 schematically shows a control method according to the invention in the form of a flow chart. The method shown for controlling a surgical instrument 2 comprises the following steps: In step 21, image data of a region of tissue 4 are optically acquired by means of a sensor device 6, e.g., by means of a camera or a scanning device. In step 22, based on the image data acquired by means of the sensor device 6, at least one feature of the tissue in the acquired region is determined, wherein a change in the feature indicates contact between the surgical instrument 2 and the tissue 4. This is done using an evaluation device, e.g., the evaluation device 7. In step 23, based on the at least one determined feature, feedback signals are output by means of an HMI 8.In an optional step 24, a trajectory for a movement or the control of a movement of the surgical instrument 2 can be determined and output based on image data acquired by the sensor device 6 and / or based on the at least one determined feature and / or based on the output feedback signal. The trajectory can have at least one repeating movement pattern. Also optionally, the movement of the surgical instrument 2 can be controlled along the trajectory.

[0057] In the following, embodiments of the invention are explained using ophthalmic surgical applications. First, individual aspects of the invention are described using the example of a capsular bag opening (capsulorhexis). Figure 5 shows a schematic perspective view of a surface 5 of a capsular bag tissue 4. During a capsular bag opening, a portion of the capsular bag 31 is opened up, and the underlying lens is exposed through the resulting opening 30. The movement of a surgical instrument used for this purpose is indicated by an arrow with the reference number 32, which depicts a trajectory of the tip of the surgical instrument. During the opening process, a tear point of the tissue 4 moves away from a first position indicated by the reference number 33. This is indicated by an arrow with the reference number 34.An exemplary second position of the tear point is indicated by reference number 35.

[0058] Figure 6 illustrates the described process in two steps. The top of Figure 6 shows the movement of the tip of the surgical instrument 2 used in two xy-z diagrams. The xy plane denotes a lateral plane, in this case the plane in which the surface 5 of the capsular bag tissue 4 extends. The z direction denotes the axial direction, which runs perpendicular to the xy plane. The bottom of Figure 6 shows perspective views of the capsular bag tissue 4 corresponding to the diagrams arranged above, analogous to the view shown in Figure 5.

[0059] In the step shown at the top of Figure 6, the surgical instrument or the tip of the surgical instrument is moved along the part of the trajectory designated by reference number 36 to the surface 5 of the tissue region 31 of the capsular bag 4 to be displaced. Subsequently, i.e. as soon as contact is established between the surgical instrument 2 and the capsular bag 4, a lateral movement of the surgical instrument 2 occurs in the xy plane. This is indicated by the region of the trajectory designated by reference number 37 at the bottom of Figure 6. The movement of the displacement of the capsular bag tissue 31 caused by the action of the surgical instrument 2 is indicated by an arrow designated by reference number 38.

[0060] The contact between the tip of the surgical instrument and the capsular bag 4 is detected indirectly through the changes (dynamic effects) optically observable in the geometry of the unfolded tissue region 31 or the geometry of the opening 30 or the tear point. Based on acquired image data, the tip of the surgical instrument can be controlled such that the tear propagates or continues in a calculated direction in the xy plane. As soon as a movement or change in the geometry of the unfolded tissue region 31 is detected or recorded, for example by means of a camera and / or a microscope, the surgical instrument is held at the current position in the z direction, i.e. the current height in relation to the tissue region 31, and the movement of the surgical instrument in the xy plane is continued until a specified target point is reached.Subsequently, the surgical instrument 2 is lifted from the tissue region 31. If no change in the geometry of the tissue region 31 is observed or detected, the same movement is repeated with a lower z-value, i.e., a smaller distance from the capsular bag 4 or the tissue region 31. This is repeated until a change in the geometry of the tissue region 31 is observed or detected.

[0061] To detect or identify a change resulting from the action of the surgical instrument 2 on the tissue 4, the geometry of the resulting opening 30 and / or the geometry of the unfolded tissue region 31 and / or the geometry and / or the course of the tear point 33 can be recorded and evaluated. To detect a change, for example, the geometry of the tear, known image processing methods can be used, as can machine-learning image processing methods or methods for image processing or image enhancement. Image processing or image enhancement can be achieved, for example, by increasing the phase contrast or by using dark-field microscopy.

[0062] Figures 7 and 8 schematically show examples of the movement of a surgical instrument designed for brushing. In each case, the reference numeral 39 denotes a membrane or tissue 4 to be removed. The reference numerals 40 and 41 denote the trajectories of the tip of the surgical instrument in an axial plane, i.e., a plane running perpendicular to the lateral plane. The trajectories 40 and 41 shown each have a region, denoted by the reference numerals 42 and 43, with a repeating movement pattern. In the variant shown in Figure 7, the repeating movement pattern 42 is composed of parabolically shaped sub-regions. In the variant shown in Figure 8, the repeating movement pattern 43 is loop-shaped or continues elliptically.The dimensions of the repetitive movement patterns are preferably in the micrometer range. The movements of the surgical instrument along the trajectories shown as examples in Figures 7 and 8 can be continued, for example, also within the scope of the application shown in Figures 5 and 6, until a movement of the tear point is detected.

[0063] Figure 9 schematically shows an example of an approach of a surgical instrument to a tissue to be manipulated. This can, for example, be the step shown at the top of Figure 6. In the variant shown, the surgical instrument 2 is repeatedly moved along an elliptical or parabolic trajectory designated by the reference numeral 44, whereby it is checked in each case whether the movement of the surgical instrument 2 has an effect on the geometry of the unfolded region 31 of the capsular bag 4. If no change in the tissue region 31 is observed, an offset of the movement in the direction of the surface 5 of the capsular bag 4 is added in the z-direction, i.e. the distance to the tissue region 31 or the surface 5 is reduced, and the movement is repeated at this distance. The various successively executed movements are identified by the trajectories 44.

[0064] If a change in the geometry of tissue region 31 is detected or determined, which is the case here with trajectory 45, the z-position is limited to the corresponding height or a specified lower value, and the movement continues at this height until the planned lateral extension of the tear is reached. Subsequently, the surgical instrument 2 is lifted from the tissue 4. Once a specific, specified, or desired displacement of tissue region 31 is achieved, a subsequent step can be planned or performed.

[0065] In Figure 9, reference numeral 46 indicates a desired point of application of the surgical instrument 2 on the tissue 4. Reference numeral 45 indicates the trajectory at which a movement of the tissue region 31 was determined. Upon reaching trajectory 45, the movement in the z-direction is simultaneously stopped, and the process continues with only a movement in the xy-direction, i.e., in the lateral direction.

[0066] In the following, a variant is described with reference to Figures 10 and 11 in which the surgical instrument 2 is controlled partly manually and partly automatically. Figures 10 and 11 each schematically show a hand-held, mechatronically controllable surgical instrument 2, which comprises, for example, a cystotome 51 for brushing membranes with a tip 47. In this case, the surgical instrument 2 is manually controllable in the lateral direction. By means of a control device 1 according to the invention or a robotic manipulator 12 according to the invention, the axial position, as indicated by the arrow 48 in Figure 10, is automatically set, for example on a micrometer scale. The arrangement shown is designed to automatically execute a brushing movement, as shown in Figure 11.For this purpose, three robotic devices with dimensions in the micrometer range can be provided for controlling the movement of the tip 47 of the surgical instrument 2 in a three-dimensional space using three degrees of freedom. A rotational movement of the cystotome around the point 50 can be provided.

[0067] A feedback signal output by the control device 1 according to the invention with regard to an axial position of a tip of a surgical instrument can also be made available to a surgeon as AR feedback (AR - Augmented Reality). As soon as the tearing point of the tissue moves, for example, an area around the tip of the surgical instrument 2 can be displayed in a specified color, for example green. Alternatively, the tearing point itself can also be displayed in color. As an additional feature, the displayed color can change continuously depending on the determined feature that indicates contact between the surgical instrument 2 and the tissue 4. For example, a first color, for example red, can be displayed when there is no contact and a second color, for example green, when a good grip or contact is achieved.Whether a contact or grip is good or not can be determined by comparing the movement of the tip of the surgical instrument 2 with the movement of the tear point. It should be noted that the distance of movement of the surgical instrument 2 is greater than the distance of movement of the tear point due to the double layer of capsular tissue, i.e., the unfolded tissue area 31 and the intact capsular tissue 5.

[0068] List of reference symbols:

[0069] 1 control device

[0070] 2 surgical instruments

[0071] 3 Control

[0072] 4 fabrics

[0073] 5 Tissue surface

[0074] 6 Sensor device

[0075] 7 Evaluation device

[0076] 8 HMI

[0077] 9 Signal transmission

[0078] 10 robotic arrangement

[0079] 11 Fastening device

[0080] 12 robotic manipulator

[0081] 13 Operating microscope

[0082] 21 optical acquisition of image data of a region of a tissue

[0083] 22 based on image data acquired by the sensor device

[0084] Determining at least one feature of the tissue in the detected area

[0085] 23 Output of feedback signals based on at least one determined feature

[0086] 24 Determining and outputting a trajectory to control the movement of the surgical instrument

[0087] 30 Opening / Cornea

[0088] 31 unfolded tissue area

[0089] 32 Movement of the surgical instrument

[0090] 33 Tear point

[0091] 34 Continuation of the tear point

[0092] 35 tear point

[0093] 36 first area of ​​the trajectory

[0094] 37 second area of ​​the trajectory

[0095] 38 Direction of movement of the unfolded tissue area

[0096] 39 Membrane / Tissue

[0097] 40 Trajectory 41 Trajectory

[0098] 42 Area with repetitive movement pattern

[0099] 43 Area with repetitive movement pattern

[0100] 44 Trajectory 45 Trajectory

[0101] 46 Point of attack

[0102] 47 lace

[0103] 48 controllable movement directions

[0104] 49 robotic device 50 pivot point

[0105] 51 cystotomes

Claims

Patent claims 1. Control device (1) for controlling the movement of a surgical instrument (2), characterized in that the control device (1) comprises a sensor device (6), an evaluation device (7) and a human-machine interface (8), wherein the sensor device (6) is designed to optically capture image data of a region of a tissue (4), the evaluation device (7) is designed to determine at least one feature of the tissue (4) in the detected region based on image data captured by means of the sensor device (6), wherein a change in the feature indicates contact between the surgical instrument (2) and the tissue (4), and the human-machine interface (8) is designed to output feedback signals based on the at least one determined feature.

2. Control device (1) according to claim 1, characterized in that the evaluation device (7) is designed to define a trajectory (32, 40, 41, 44) of the surgical instrument (2) based on image data acquired by means of the sensor device (6) and / or based on the at least one determined feature of the tissue (4) and / or based on the output feedback signal.

3. Control device (1) according to claim 1 or claim 2, characterized in that the control device (1) is designed to output a trajectory (32, 40, 41, 44) for a movement of the surgical instrument (2) based on image data acquired by means of the sensor device (6) and / or based on the at least one determined feature of the tissue (4) and / or based on the output feedback signal, wherein the Trajectory (32, 40, 41, 44) comprises at least one repeating movement pattern (42, 43), and / or the control device (1) is designed to control an at least partially automated movement of the surgical instrument (2) along a fixed trajectory (32, 40, 41, 44).

4. Control device (1) according to one of claims 2 to 3, characterized in that the trajectory (32, 40, 41, 44) comprises a continuous path along an imaginary surface, wherein the surface is conical or cylindrical in shape and / or the path has circular and / or elliptical and / or helical and / or parabolic and / or hyperbolic and / or linear subregions.

5. Control device (1) according to one of claims 2 to 4, characterized in that the trajectory (32, 40, 41, 44) lies at least partially in a plane, wherein the plane extends perpendicular to a lateral plane and / or perpendicular to an object plane of the sensor device (6).

6. Control device (1) according to one of claims 1 to 5, characterized in that the sensor device (6) comprises at least one camera for simultaneously detecting a plurality of laterally arranged pixels and / or a scanning device for time-sequentially detecting laterally arranged pixels.

7. Control device (1) according to one of claims 1 to 6, characterized in that the human-machine interface (8) is designed to input a lateral path of movement of the surgical instrument (2).

8. Control device (1) according to one of claims 1 to 7, characterized in that the control device (1) is designed to differentiate between different types of contact between the tissue (4) and the surgical instrument (2) based on the at least one determined feature.

9. Robotic arrangement (10) which comprises at least one robotically guided surgical instrument (2) and at least one robotic manipulator (12) with a fastening device (11) for fastening the surgical instrument (2) to the robotic manipulator (12), characterized in that the robotic arrangement (10) comprises a control device (1) according to one of claims 1 to 8, which is designed to control the movement of the surgical instrument (2) by means of the robotic manipulator (12).

10. Robotic arrangement (10) according to claim 9, characterized in that the robotic manipulator (12) has at least 3 degrees of freedom of movement for manipulation and the control device (1) is designed to control the movement of the surgical instrument (2) by means of the at least 3 degrees of freedom of movement of the robotic manipulator (12).

11. Robotic arrangement (10) according to claim 9 or 10, characterized in that the robotic arrangement (10) is designed such that an entry point into a specific tissue is defined or can be defined as a reference point for a movement of the surgical instrument (2), wherein the lateral position of the surgical instrument (2) corresponds to an angular orientation of the surgical instrument (2) or the robotic manipulator (12) and the axial position of the surgical instrument (2) corresponds to an entry depth of the surgical instrument (2) or the robotic manipulator (12) through the entry point.

12. Robotic arrangement (10) according to one of claims 9 to 11, characterized in that the robotic arrangement (10) is designed to interrupt a movement of the surgical instrument (2) if contact between the tissue (4) and the surgical instrument (2) is detected, and / or the robotic arrangement (10) is designed to maintain an axial movement of the surgical instrument (2) with a simultaneous lateral movement of the surgical instrument (2) along a lateral path that can be defined by a surgeon.

13. Robotic arrangement (10) according to one of claims 9 to 12, characterized in that the robotic arrangement (10) is designed for partially automated or fully automated operation, wherein the lateral parts of a trajectory (32, 40, 41, 44) of the surgical instrument (2) are controllable based on data acquired by means of the sensor device (6), wherein contact between the surgical instrument (2) and the tissue (4) is maintained during execution of a movement of the surgical instrument (2) with a repeating movement pattern (42, 43).

14. Robotic arrangement (10) according to one of claims 9 to 13, characterized in that the surgical instrument (2) can be controlled by a user by means of the human-machine interface (8) with respect to its lateral movement with respect to the tissue (4) or the tissue surface (5), while an axial or medial movement of the surgical instrument (2) with respect to the tissue (4) or the tissue surface (5) can be controlled automatically, and / or the human-machine interface (8) comprises a planning function by means of which a tissue region to be treated can be determined based on lateral position data acquired by means of the sensor device (6), and the robotic arrangement (10) is designed to carry out a specified surgical step at least partially automatically based on data acquired by means of the sensor device (6) regarding contact between the surgical instrument (2) and the tissue (4), and / or the human-machine interface (8) is designed to output information for assistance.

15. Robotic arrangement (10) according to one of claims 9 to 14, characterized in that the robotic arrangement (10) can be switched between two operating modes during the execution of a surgical procedure, wherein in a first operating mode the surgical instrument (2) and the tissue (4) are in contact and in a second operating mode the surgical instrument (2) and the tissue (4) are not in contact.

16. Robotic arrangement (10) according to one of claims 9 to 15, characterized in that the robotic arrangement (10) is designed to detect a movement of the tissue (4) and, in the event of a detected movement of the tissue (4), to guide at least the surgical instrument (2) according to the detected movement.

17. Surgical microscope (13) comprising a control device (1) according to one of claims 1 to 8 or a robotic arrangement (10) according to one of claims 9 to 16.

18. A method for controlling the movement of a surgical instrument (2), characterized in that the method comprises the following steps: - optically capturing image data (21) of a region of a tissue (4) by means of a sensor device (6), - Determining at least one feature (22) of the tissue (4) in the detected area using an evaluation device (7), wherein the at least one feature of the tissue (4) is determined based on image data detected by the sensor device (6), and wherein a change in the feature indicates contact between the surgical instrument (2) and the tissue (4), -Outputting feedback signals (23) based on the at least one determined feature by means of a human-machine interface (8).

Images