Method for positioning a robot tip on a trocar, surgical robot system

Abstract

The invention relates to a method for positioning a robot tip (43) of a surgical robot (40) on a trocar (50), with a trocar adapter (45) formed on the robot tip (43). In the method, sensor data (70, 70a) of the trocar (50) are recorded by means of a sensor device (14), wherein, in the event of a movement of the robot tip (43) relative to the trocar (50) starting from a starting position, the sensor device (14) is arranged stationarily relative to the starting position. Position data of the trocar (50) are determined, with a position of the trocar (50) being derived from the sensor data (70, 70a). The position data of the trocar (50) are processed in a control unit (60) in order to generate control commands for the surgical robot (40). Actuators (47) of the surgical robot (40) are controlled by the control commands such that the trocar adapter (45) is made to approach the trocar (50). The invention also relates to a surgical robot system.

Description

CZMEO O IPWO 01.10.2025 / PH / US / LE Method for positioning a robot tip on a trocar, surgical robot system

[0001] The invention relates to a method for positioning a robot tip on a trocar and a surgical robot system.

[0002] In minimally invasive surgery, surgical instruments are inserted into the patient's body through perforated openings held open by funnel-shaped trocars. The trocars restrict the movement of the instruments during the operation. This reduces tissue damage that can be caused by inserting surgical instruments into the patient's body and by accessing the surgical site.

[0003] The structures to be operated on are particularly small, especially in eye surgery, for example in vitreoretinal surgery. Operating microscopes are used to provide a magnified view of the surgical area. Particularly when used with a light source inserted into the eye, the operating microscope can visualize not only the surface of the eye and its external surroundings, but also the surgical area inside the eye.

[0004] To improve precision and miniaturize the operation, surgical robots can be used to guide and manipulate surgical instruments. The surgical instruments are arranged at a robotic tip, which can be attached to the trocar inserted into the eye. This allows for fixation of the eye and the insertion point for the surgical instrument. Manually guiding the robot tip to the trocar using a robot tip controller requires specialized staff training due to the trocar's small size and the complexity of the controller. Furthermore, manually guiding the robot tip to the trocar using a robot tip controller is time-consuming, occurring at the beginning of each operation and after each interruption of the surgical robot's instrument use during the procedure.

[0005] From US 2022 / 0280238 Al it is known to move a robotic arm of a surgical robotic system along a planned trajectory to a predefined intervention position.

[0006] From US 2021 / 0128260 Al, US 11,529,734 B2, a tool drive coupled to a robot arm of a surgical robot system is known, to which one or more cameras are attached.

[0007] From S. Dehghani et al: “ColibriDoc: An Eye-in-Hand Autonomous Trocar Docking System”, arXiv: 2111.15373vl [cs.RO] it is also known to mount an RGB camera on the end effector of an ophthalmic robot.

[0008] The invention is based on the objective of presenting a method for positioning a robot tip on a trocar and a surgical robot system that reduces the aforementioned disadvantages. This objective is achieved by the features of the independent claims. Advantageous embodiments are specified in the dependent claims.

[0009] In an inventive method for positioning a robot tip of a surgical robot on a trocar A trocar adapter is integrated at the robot tip. Sensor data from the trocar is acquired by a sensor device. When the robot tip moves relative to the trocar from a starting position, the sensor device remains stationary. Position data of the trocar is determined, and its position is derived from this sensor data. The trocar's position data is processed in a control unit to generate control commands for the surgical robot. Actuators of the surgical robot are then controlled with these commands, bringing the trocar adapter closer to the trocar.

[0010] The invention proposes determining trocar position data using a sensor device that does not move with the robot tip when it moves relative to the trocar. This position data is then used to guide the robot tip towards the trocar, controlled by the surgical robot. This eliminates the need for a camera mounted on the robot tip. The complexity of the robot tip is thus reduced, and the space required in the operating environment is also minimized.

[0011] The robot tip can be positioned as an end stop on the surgical robot and may have a proximal and a distal end. For example, the proximal end of the robot tip may be attached to a robot arm segment and / or a joint of the surgical robot. Actuators of the surgical robot may be configured to move the robot tip. Instruments of the surgical robot may be located at the robot tip. At its distal end, the trocar adapter may be located, which is designed to dock the robot tip to a trocar. The trocar adapter may The trocar must have a channel through which surgical instruments can be guided. The channel may, for example, be in the form of a concentric hole. The channel may have a cross-section that remains constant along its axis. The channel may taper towards its distal end. The smallest diameter of the channel may correspond to the inner diameter of the trocar at its thinnest point.

[0012] The trocar can be designed to keep a puncture opening open into the interior of the eye. In this way, surgical instruments can be inserted through the trocar into the patient's eye, particularly into the vitreous humor. The trocar can have a trocar opening. The trocar opening can face outwards and be visible from outside the eye. The trocar can have a hollow cylindrical shape and / or a hollow cone shape. Preferably, the trocar can include a trocar funnel that protrudes from the eye when the puncture opening is kept open. The trocar opening can include the outer, wider rim of the trocar funnel. The trocar can have a trocar neck that projects into the interior of the eye, particularly into the vitreous humor, when the puncture opening is kept open. The trocar neck can be shaped as a hollow cylinder and / or a hollow cone.

[0013] The trocar adapter of the robot tip can be shaped to match the interior of the trocar, with the outer shape of the adapter corresponding to the inner shape of the trocar, in particular the inner shape of the trocar opening and / or the trocar funnel. The trocar adapter can be designed to engage with the trocar funnel, preferably in a positive-locking manner. In this way, a force can be transferred from the surgical robot to the trocar funnel. This allows for fixation of the eye and the instrument entry point (remote center of motion). When the trocar adapter engages the trocar funnel with a positive fit, the channel of the trocar adapter can be axially aligned with the axis of the trocar neck. The opening of the trocar adapter channel can rest at the transition between the trocar funnel and the trocar neck.

[0014] The sensor device can be configured to acquire images and / or spatial data and / or distance data. Preferably, the sensor device can be a surgical microscope, a camera, in particular an optical camera, an imaging device, a depth imaging device, an optical coherence tomography (OCT) device, and / or a LiDAR device. The sensor device can include an image sensor to acquire the sensor data, especially if the sensor device is a camera and / or a surgical microscope, wherein the sensor data can be in the form of an image, for example, a camera image and / or a microscope image. The sensor device can preferably be arranged separately from the robot tip so that it can be independent of any movement of the robot tip.When the robot tip moves relative to the trocar, the sensor device cannot be moved along with the robot tip, so it remains fixed in position, particularly relative to the trocar, the eye, the patient, a mounting point for the surgical robot, and / or the operating room. The initial position of the robot tip's movement can be defined, in particular, in relation to the operating room, a mounting point for the surgical robot, the patient, the eye, and / or the trocar. The initial position can be a position of the robot tip within a movement of the robot tip. The sensor device can be movably mounted, for example, in the operating room, with its mobility being independent. The sensor device is stationary relative to the trocar and is subject to movement of the robot tip relative to the trocar. In a sensor device with individual moving elements, for example, a LiDAR device with movable LiDAR sensors, the moving elements can be independent of the robot tip's movement, allowing individual moving elements to move relative to their initial position.

[0015] The sensor data can include, for example, image data, a microscope image, a camera image, volume data, depth data, OCT image data, distance data, and / or point cloud data. The sensor data can include coordinates of a coordinate system of the sensor device. The coordinate system can be defined by the sensor device. To derive the position of the trocar, a data position of the trocar can be identified in the sensor data through data processing. The data position can, in particular, include coordinates of the coordinate system of the sensor device. The orientation of the trocar can be determined from the trocar's sensor data.

[0016] The sensor device may include a camera. In particular, two cameras may be used to capture stereo data. The camera may electronically capture image data of objects using an image sensor. The trocar may have an identification mark. The identification mark may include a flashing light source and / or a computer vision mark. The computer vision mark may, for example, include a two-dimensional standardized pattern. The identification mark with a flashing light source may have a specific luminous color of the light source and / or a temporal flashing pattern of the light source. To deduce the position of the trocar, The trocar's position in the sensor data can be identified through digital image processing. To determine the trocar's position and / or orientation, the distance and / or alignment of the identification mark to the sensor device can be detected by processing the sensor data. If the trocar has an identification mark, this offers the advantage of enabling fast and reliable trocar detection.

[0017] The sensor device can include an OCT device. In optical coherence tomography (OCT) imaging, light from a light source with a short coherence length can be split so that a beam directed at a sample and backscattered by it interferes with a beam passing through a reference path. An interference spectrum can be generated in time-domain OCT (TD-OCT) by changing the reference arm length, and in frequency-domain OCT (FD-OCT) by using the spectral information and recorded with a detection unit. In FD-OCT, spectral information can be obtained by spectral decomposition of the interfered light (spectrometric OCT, SD-OCT) or by changing the wavelength of the light source (swept source OCT, SS-OCT).Through reconstruction, which in particular includes a Fourier transform, the interference spectrum can be converted into a depth profile known as an A-scan. By laterally scanning the sample in a first direction and combining the reconstructed A-scans, a cross-sectional plane can be imaged (B-scan), which can be extended to a volumetric image by additional lateral scanning in a direction perpendicular to the first direction (C-scan). OCT image data can be used as A-scans. A scan, B-scan, or C-scan may be present. When using multiple scans... A four-dimensional scan can be obtained from OCT images acquired at different times. The OCT device can be integrated into the operating microscope and / or be designed as a separate OCT device. In particular, the OCT device can be integrated into a surgical instrument.

[0018] The sensor device can employ an optoelectronic measuring method, in particular LiDAR. A measuring beam, especially a pulsed laser beam, can be emitted by the sensor device and reflected by objects. Measuring beams can be emitted in different directions by optical elements of the sensor device. The distance to the objects can be determined from the travel time of the reflected portions of the measuring beam. The reflected portions of the measuring beam can be analyzed to determine the coordinates of the objects. These coordinates can be coordinates of a coordinate system of the sensor device. By emitting measuring beams in different directions, a point cloud can be obtained that includes the coordinates of a plurality of points on the objects. In particular, the sensor device can be a LiDAR device with a measuring accuracy of 0.3 to 0.5 mm.

[0019] As described below, the invention can be implemented, in particular, using an operating microscope as a sensor device. Acquiring sensor data of the trocar with the sensor device can include capturing a microscope image of the trocar with an image sensor arranged in an image plane of the operating microscope. Determining the trocar's position data can include deriving the trocar's position in a plane perpendicular to the optical axis of the operating microscope from the microscope image. An advantage of using an operating microscope is the high precision of the image data. Furthermore, to take advantage of the use of an operating microscope as a sensor device in eye surgery, that an operating microscope may be present as standard.

[0020] In a preferred embodiment, the sensor device can acquire the sensor data of the trocar while the trocar is located within a field of view of the sensor device. The field of view can define an area in space, wherein points of the area can be at least partially mapped or captured in the sensor data by the sensor device. In particular, the field of view can comprise a plane and / or a volume in space.

[0021] When capturing the microscopic image of the trocar, the operating microscope can be positioned separately from the robot tip and / or directed towards a surgical area and / or its surroundings. Imaging optics of the operating microscope can optically map an object plane onto the image plane of the operating microscope, in which the image sensor is located. The imaging optics can generate an image in the image plane. The image sensor can capture the microscopic image of the trocar while the trocar is located within an image field of the operating microscope. The image field can be the portion of the object plane that the imaging optics map onto the image plane. For the trocar to be located within the image field of the operating microscope, at least a portion of the trocar must be within the field of view. Preferably, the microscopic image can include an image of the trocar opening.The trocar opening can be located completely or partially within the field of view of the operating microscope.

[0022] The imaging optics of the surgical microscope may include a zoom system that allows for increasing the magnification of the image. This can be varied. To capture the microscopic image of the trocar, it may be necessary to reduce the magnification of the image and thus enlarge the field of view of the operating microscope, so that the trocar and preferably also the surrounding area of ​​the surgical field are captured by the field of view of the operating microscope.

[0023] To capture the trocar's microscope image, the image can be focused. This can be done, for example, by manually adjusting the focus and / or by using an autofocus system.

[0024] To focus the image, the distance between the imaging optics and the trocar can be varied. This variation can be done manually and / or automatically. The image is considered to be in focus when a specific contrast level is achieved. This occurs when the image contrast is at its maximum and / or reaches a threshold value. The image contrast can be determined for the entire image or for a portion of the image that is in focus. The image contrast can be derived from an image of the trocar captured by the image sensor. The portion of the image that is in focus can preferably be the entire trocar or a part of it.

[0025] Preferably, the focusing portion of the image can depict the outer boundary of the trocar opening or a part thereof. This has the advantage that the trocar opening stands out well from its surroundings in the image. If the trocar opening is inclined relative to the object plane, the focusing portion of the image can lie between the point of the trocar opening closest to and furthest from the object. The distance of the focusing part of the image from the point of the trocar opening furthest from the object can be greater than 10%, preferably greater than 20% and further preferably greater than 30% of the distance in the direction of the optical axis of the operating microscope between the nearest and furthest points of the trocar opening.

[0026] The trocar microscope image can preferably be digitally captured by the image sensor. This can be achieved by the image sensor capturing the image in the image plane. Preferably, the image is captured by the image sensor at the moment when the image has been brought into focus.

[0027] In the captured microscope image of the trocar, the trocar's position can be identified using digital image processing. This digital image processing can be performed manually and / or using a trocar recognition algorithm. Such algorithms can be based, for example, on a machine learning-trained model and / or on pattern recognition of a predefined trocar shape projected onto the microscope image, and / or they can utilize contrast between the surroundings and the trocar in the image. The trocar's position can be represented by a single point on the trocar, a portion of the trocar, or the entire trocar. Preferably, the identification of the trocar's position can be based on the trocar opening. This has the advantage that the trocar opening, due to its typically larger diameter compared to the rest of the trocar, is imaged larger and thus easily identifiable.Furthermore, their typically circular geometry can be exploited for detection. The image position of the trocar can be used to map a single point onto the trocar opening. Describe part of the trocar opening, or the entire trocar opening.

[0028] The image position of the trocar can, for example, include image coordinates corresponding to the two dimensions of the image.

[0029] The trocar position data determined according to the invention can include information about the trocar's position in space. In particular, they can include data obtained from the microscope image by deriving the trocar's position in the plane perpendicular to the optical axis of the operating microscope.

[0030] To derive the trocar's position in the plane perpendicular to the optical axis of the operating microscope, the identified image position of the trocar in the acquired microscope image can be used. The trocar's image position can be projected back from the image plane to the object plane by inverting an optical beam path in the operating microscope. The optical beam path can be determined by the operating microscope's imaging optics. The inversion can, for example, take into account a known magnification of the image produced by the operating microscope's imaging optics and / or a calibration. The inversion can be performed in the plane perpendicular to the operating microscope's optical axis, preferably in the object plane. The determined trocar position data can include the trocar's position obtained by inversion in the plane perpendicular to the operating microscope's optical axis.

[0031] A coordinate system can be defined using the operating microscope. The coordinate system can also be defined by... Other reference points can be defined. The origin of the coordinate system can be defined by the mounting of the operating microscope, for example, by mounting it on a wall and / or on a floor and / or on a patient's bed and / or on the surgical robot. Alternatively or additionally, the origin of the coordinate system can be defined by calibration to a point, for example, a point in an operating room and / or a point at the patient's bed and / or a point on the patient and / or a point on the surgical robot.

[0032] For example, a Cartesian coordinate system can be defined by the operating microscope, in which, without loss of generality, the x and y coordinates can be defined perpendicular to each other and parallel to the object plane, and the z coordinates along the optical axis of the operating microscope perpendicular to the object plane. The x and y coordinates in the image plane can be projected from the image plane to the object plane by inverting the optical beam path within the operating microscope. The position of the trocar in a plane perpendicular to the optical axis of the operating microscope can be specified by x and y coordinates in the Cartesian coordinate system defined by the operating microscope. The trocar's position data can include these x and y coordinates.

[0033] In one implementation form, in addition to the position of the trocar in a plane perpendicular to the optical axis of the operating microscope, a position of the trocar parallel to the optical axis of the operating microscope can also be determined.

[0034] The position of the trocar parallel to the optical axis of the operating microscope can be adjusted by setting a distance. The distance between the operating microscope and the trocar can be determined. This distance can be adjusted so that the trocar lies in a focal plane of the operating microscope. The position of the trocar parallel to the optical axis of the operating microscope can then be determined from the working distance of the operating microscope.

[0035] To adjust the distance between the operating microscope and the trocar, the magnification of the image can be varied. The magnification can be increased ("zoom in") or decreased ("zoom out"). This can be achieved using a zoom system in the operating microscope's imaging optics. The magnification can be varied manually and / or automatically. When varying the magnification of the operating microscope's image, the depth of field—that is, the area, particularly parallel to the optical axis of the operating microscope, that appears sharply focused—is also typically varied: When the magnification is decreased, the depth of field increases; when the magnification is increased, the depth of field decreases.

[0036] Increasing the magnification of the image therefore reduces the depth of field. If the distance between the operating microscope and the trocar is adjusted so that the trocar is in sharp focus, the trocar's position can be limited to the depth of field. To adjust the distance between the operating microscope and the trocar, it is therefore advantageous to narrow down the position of the trocar in sharp focus by increasing the magnification of the image and consequently reducing the depth of field. The depth of field can be, for example, between 0.4 mm and 2 mm, preferably between 0.5 mm and 1 mm.

[0037] A sharp image of the trocar can be obtained as described above for focusing the image. A specific contrast level for the trocar image can, in any case, include a portion of the imaged trocar. Preferably, the contrast level can include the entire trocar opening or a portion thereof. A specific contrast level can be achieved for the sharp image of the trocar.

[0038] To adjust the distance between the operating microscope and the trocar so that the trocar lies within the focal plane of the operating microscope, the depth of field can be determined. The operating microscope can then be moved until the trocar is positioned within this depth of field. This movement of the operating microscope can be performed manually and / or automatically.

[0039] The depth of field can be determined by calculating and comparing the contrast of the trocar image to images acquired while shifting the focal plane parallel to the optical axis. Shifting the focal plane relative to the trocar can be achieved by moving the operating microscope incrementally along the optical axis by a certain step length. At each step, an image of the trocar image is captured, and the resulting contrast of the trocar image is used to calculate the contrast of that step. Moving the operating microscope, capturing the image, and determining the contrast can each be performed manually and / or automatically. The step length is preferably less than 1 mm, and more preferably less than 0.1 mm. The contrast of each step can be... The contrast level can be determined in the same way as when focusing the image as described above. In particular, the contrast level of the displacement step can be determined over the entire image or over a focusing portion of the image. The focusing portion of the image can preferably depict the outer boundary of the trocar and / or the trocar opening, or a part thereof.

[0040] As long as the trocar, and in particular the part of the trocar corresponding to the focusing portion of the image, is at least partially within the depth of field at a given displacement step of the operating microscope along the optical axis, the contrast value at that displacement step can be essentially equal to a sharpness value that may correspond to the contrast value of the trocar's sharp image. If the trocar, and in particular the part of the trocar corresponding to the focusing portion of the image, is outside the depth of field at a given displacement step, the contrast value at that displacement step may differ from the sharpness value.

[0041] By shifting the focal plane, an upper and / or a lower limit of the sharp image can be determined. The upper and / or lower limits of the sharp image can each correspond to a boundary shift whose contrast measure of the shift step is essentially equal to the sharpness value, while the shift step adjacent to it above and below has a contrast measure that differs from the sharpness value. The deviation can preferably be more than 10%, more preferably more than 20%, and even more preferably more than 30% of the sharpness value. The upper and / or lower The limit of sharp imaging can also be determined by the limiting stage. and lie on the displacement step adjacent to the top or bottom, or on the displacement step adjacent to the top or bottom.

[0042] To adjust the distance between the operating microscope and the trocar so that the trocar lies in a focal plane of the operating microscope, the operating microscope can be moved along its optical axis. This movement can be performed manually and / or automatically. The operating microscope can be moved to a point between the upper and lower limits of the sharp image. Preferably, the operating microscope can be moved to the midpoint between the upper and lower limits of the sharp image. Then the trocar can be located in the focal plane assumed to be at the center of the depth of field.

[0043] The operating microscope can have a known working distance. The working distance can specify the distance of an objective of the imaging optics to the focal plane of the operating microscope. The known working distance can be a fixed working distance, preferably 100 to 300 mm, more preferably 150 to 250 mm. Alternatively, it can be a variable working distance.

[0044] The position of the trocar parallel to the optical axis of the operating microscope can then be determined from the known working distance of the microscope. Relative to the operating microscope, the position of the trocar can be defined as parallel to the optical axis of the operating microscope, spaced a distance from the objective lens of the operating microscope by the working distance, if the trocar lies in a focal plane of the operating microscope.

[0045] The position of the trocar parallel to the optical axis of the operating microscope can be described as its z-coordinate in the Cartesian coordinate system defined by the operating microscope. The trocar's position data can include this z-coordinate.

[0046] The position of the trocar thus determined, parallel to the optical axis of the operating microscope, preferably corresponds to the position of the trocar opening parallel to the optical axis of the operating microscope. To achieve this, the distance between the operating microscope and the trocar opening described above can be set when determining the position of the trocar parallel to the optical axis of the operating microscope. This has the advantage that the focusing of the image and the determination of the depth of field are based on the clearly defined and comparatively large trocar opening.

[0047] Alternatively or additionally, the position of the trocar parallel and / or perpendicular to the optical axis of the operating microscope can be determined from another type of distance measurement, for example using LiDAR.

[0048] In another embodiment, the orientation of the trocar can be determined. The orientation of the trocar can be defined along the axis of the trocar, which preferably corresponds to the perpendicular to the trocar opening. Preferably, the orientation of the trocar is determined from the microscopic image of the trocar. The orientation of the trocar can be determined from the deviation of a trocar structure with a known geometry depicted in the microscopic image. The trocar structure has a known geometry. may include a shape and / or a marking, which may be located, for example, on the trocar funnel or on an edge of the trocar.

[0049] Preferably, the orientation of the trocar can be determined from the deviation of the trocar opening depicted in the trocar's microscopic image from a previously known trocar opening geometry. This has the advantage that the trocar opening, typically the largest structure on the trocar, is used to determine the orientation, allowing for precise determination.

[0050] The trocar opening can have a known two-dimensional geometry. Preferably, the trocar opening has a circular geometry. In an image where the object plane is parallel to the known geometry of the trocar opening during capture, the known geometry is reproduced. If the object plane is not parallel to the known geometry of the trocar opening, the geometry of the depicted trocar opening deviates from the known geometry.

[0051] Given a known circular geometry of the trocar opening, the orientation of the perpendicular to the trocar opening relative to the object plane of the operating microscope can be determined from the ratio of the semi-axes of the ellipse in the trocar's microscope image. This orientation can be specified in the coordinate system defined by the operating microscope in the form of rotational coordinates, for example, for a vector of the perpendicular to the trocar opening.

[0052] In another implementation, position data of the trocar adapter of the robot tip can be determined using the corresponding steps, as described above for determining the position data of the trocar, whereby these steps are based on The trocar adapter can be used. To determine the position of the trocar adapter in a plane perpendicular to the optical axis of the operating microscope, a microscope image of the trocar adapter can be acquired with the image sensor. When determining the position of the trocar adapter parallel to the optical axis of the operating microscope, the focal plane can be set not on the trocar or a part of the trocar, but instead on the trocar adapter or a part of the trocar adapter. The position data can be specified in the coordinate system defined by the operating microscope, preferably as three-dimensional coordinates.

[0053] Alternatively or additionally, position data of the trocar adapter can be determined from another type of sensor data, for example from sensor data acquired with a LiDAR device, a camera and / or an OCT device.

[0054] In another implementation, the orientation of the robot tip can also be determined by applying the corresponding steps. This requires that the robot tip has distinctive points and / or marking points arranged on the robot tip, the position of which relative to the axis of the robot tip is precisely known. The positions of the distinctive points and / or the marking points in a plane perpendicular to the optical axis of the operating microscope can be determined from a microscope image of the robot tip. This microscope image of the robot tip must, in any case, be an image of a portion of the robot tip as well as the distinctive points and / or the marking points. The robot head is comprised of points. The orientation of the robot head can be derived from their positions relative to the axis of the robot tip.

[0055] The orientation can be specified in the coordinate system defined by the sensor device, for example by a vector along the axis of the robot tip.

[0056] Alternatively or additionally, the orientation of the robot tip can be determined in another way, for example, from motion data recorded by motion sensors at the robot tip and / or measurements from other sensors at the robot tip, from position data recorded by position sensors in the joints of the surgical robot, by stereoscopic distance measurement using a stereo camera, and / or by means of LiDAR. In particular, the orientation of the robot tip can be determined from sensor data of the robot tip, whereby the sensor data of the robot tip can be recorded with the sensor device.

[0057] In one implementation, the same corresponding steps can be applied to determine the position data of the trocar adapter and / or the orientation of the robot tip as for the position data of the trocar and possibly additionally the orientation of the trocar. This has the advantage that all coordinates determined in this way can be defined in the same coordinate system established by the sensor device. This can enable a precise determination of the relative position with respect to the coordinates thus determined.

[0058] In another implementation, the trocar's position data can be specified in a coordinate system defined by the sensor device, while the trocar adapter's position data is specified in a coordinate system of the robot tip. A synchronization between the coordinate system defined by the sensor device and the robot tip's coordinate system is then performed. In particular, the trocar adapter's position data and / or the robot tip's orientation can be determined differently and specified in a robot tip coordinate system that differs from the one defined by the sensor device. This has the advantage that the trocar adapter's position data and / or the robot tip's orientation can be determined even if the trocar adapter and / or the robot tip are not, or not completely, within the sensor device's field of view.In this form of implementation, a coordination can take place between the coordinate system defined by the sensor device and the coordinate system of the robot tip.

[0059] The alignment between the coordinate system defined by the sensor device and the coordinate system of the robot tip can be achieved by referencing the coordinate systems. This involves aligning both coordinate systems to a common baseline. The common baseline can include a patient bed and / or a structure within an operating room and / or a mounting point for the surgical robot. In particular, referencing can be accomplished by positioning the robot tip within the field of view of the sensor device.

[0060] Alternatively or additionally, a stereo camera system can be provided to determine the position of the sensor device and the position of the robot tip in space. Both coordinate systems can thus be calibrated to each other.

[0061] The trocar's position data is processed in a control unit, which may have a connection to the sensor device for transmitting the determined position data. The position data of the trocar adapter, as determined in various implementations, and / or the orientation of the trocar and / or the orientation of the robot tip may also be processed. The control unit may include a computer unit for processing the trocar's position data. The control commands for the surgical robot generated by this processing can be transmitted to the surgical robot via a connection between the control unit and the robot. The control unit may be part of the surgical robot. Alternatively or additionally, the control unit may be part of the sensor device.When generating control commands, a synchronization between the coordinate system defined by the sensor device and the coordinate system of the robot tip can be performed as described above. The control commands can include orienting the robot tip and setting an initial distance between the trocar adapter and the trocar. The control unit can be configured to actuate the sensor device. Actuating the sensor device allows for adjustments such as image focus, image magnification, and sensor device displacement.

[0062] In one implementation form, the control commands can include safe trajectory control for the robot tip. Safe trajectory control can include aligning the longitudinal axis of the robot tip parallel to the orientation of the trocar. In particular, safe trajectory control can determine the spatial position of Take into account obstacles, such as components of the surgical robot and / or the sensor device, as well as patient characteristics.

[0063] To bring the trocar adapter close to the trocar, actuators of the surgical robot are controlled with commands that may include safe trajectory guidance. The approach of the trocar adapter to the trocar can occur along a longitudinal trajectory along the axis of the trocar. The robot tip can be aligned along the axis of the trocar during approach.

[0064] The approach by controlling the surgical robot's actuators with the control commands can end when the trocar adapter engages the trocar funnel. This can be achieved when the initial distance between the trocar adapter and the trocar has been overcome. Alternatively, the approach by controlling the surgical robot's actuators with the control commands can end when the trocar adapter is a certain distance from the trocar. This remaining distance can be 0 to 40 mm, preferably 10 to 15 mm. In this case, the robot tip can then be manually guided to the trocar, preferably using a robot tip controller, until the trocar adapter engages the trocar funnel. This has the advantage of keeping the trocar adapter at a safe distance from the trocar. Manually guiding the robot tip to the trocar is quick and easy due to the small remaining distance.Alternatively or additionally, automated guidance of the robot tip to the trocar can be provided.

[0065] After the trocar adapter has been inserted into the A trocar funnel can insert a surgical instrument through the canal the trocar adapter and the trocar neck are inserted into the interior of the eye.

[0066] During interruptions of the operation, particularly interruptions in the use of surgical instruments, the trocar adapter may be removed from the insertion into the trocar funnel during eye surgery. The trocar adapter may be separated from the trocar by a decoupling distance during the interruption. In such cases, the position and / or orientation of the trocar may change, especially due to eye movements and / or patient movements. These changes in the trocar's position and / or orientation can be monitored using the sensor device, whereby the steps described above can be performed to determine the trocar's position and / or orientation data.

[0067] The positional data of the trocar adapter and / or the orientation of the robot tip can be known when the trocar adapter is removed from the insertion into the trocar funnel by registering the displacement of the trocar adapter during removal by the surgical robot. The displacement of the trocar adapter can be registered, in particular, by position sensors in the joints of the surgical robot. Alternatively or additionally, it can be determined using the steps described above.

[0068] Changes in the trocar's position and / or orientation can be monitored from the moment the trocar adapter is removed from the insertion into the trocar funnel and / or from the moment the operator inserts the trocar adapter into the sensor device's field of view. This is achieved by repeatedly determining the trocar's position and / or orientation data at short intervals. Trocars are used. The short time interval can be from 0.001 to 20 seconds, preferably from 0.5 to 10 seconds, and more preferably from 1 to 3 seconds. The tracked changes in the position and / or orientation of the trocar can be processed by the control unit into tracking commands, preferably by correcting the position data and / or orientation of the trocar accordingly.

[0069] The surgical robot's actuators can be controlled using the tracking commands as described above. By controlling the surgical robot's actuators with these tracking commands, the trocar adapter can be repositioned close to the trocar. This control with tracking commands can be continuous, whereby the trocar's position data and / or orientation can be repeatedly processed into corrected tracking commands at short intervals, and the surgical robot's actuators can then be controlled with these corrected commands. This has the advantage that the robot tip remains close to the trocar, and the trocar adapter can be quickly and easily repositioned in the trocar funnel to resume the use of surgical instruments. This saves time when resuming the operation.Alternatively or additionally, the tracking commands can be initiated at a time when the use of surgical instruments in the eye is to be resumed. This can be triggered by a tracking start command sent to the surgical robot. Here, too, time is saved by the robot-controlled approach of the robot tip to the trocar.

[0070] The invention also relates to a surgical robot system comprising a surgical robot, a sensor device, a computer unit, and a control unit. The surgical robot includes a robot tip, which is equipped with a trocar adapter. In a position determination state of the surgical robot system, the sensor device is configured to acquire sensor data from a trocar. When the robot tip moves relative to the trocar from a starting position, the sensor device remains stationary relative to the starting position. The computer unit is configured to derive position data from the microscope image, representing the position of the trocar. The control unit of the surgical robot system is configured to process the position data to generate control commands.In an approach state of the surgical robot system, the actuators of the surgical robot are controlled with the control commands so that the trocar adapter is brought close to the trocar.

[0071] In the approach state, the robot tip can be moved by a force exerted by the actuators on the robot tip so that the trocar adapter formed on the robot tip approaches the trocar. In particular, an action of the actuators on the robot tip cannot have any effect on the sensor device, so that the sensor device remains stationary.

[0072] The disclosure includes further developments of the surgical robot system, which are described in connection with the method according to the invention. The disclosure includes further developments of the method with features that are described in connection with the surgical robot system according to the invention.

[0073] The invention is described below by way of example with reference to the accompanying drawings and advantageous embodiments. The drawings show: Fig. 1: a schematic representation of an operating room in which an operating robot system according to the invention is used; Fig. 2: a schematic representation of an operating robot system according to the invention; Fig. 3a : a schematic representation of the image field captured in the microscope image of the operating microscope at higher magnification; Fig. 3b : a schematic representation of the image field captured in the microscope image of the operating microscope at lower magnification; Fig. 3c : a schematic representation of a camera image taken with a camera showing a trocar with identification marking; Fig. 4a-d : a schematic representation of the procedure steps for determining a position of the trocar parallel to the optical axis of the operating microscope; Fig. 5a : a schematic representation of a microscope image of a trocar adapter at higher magnification; Fig. 5b : a schematic representation of a microscope image of the trocar adapter at lower magnification; Fig. 6a-d : a schematic representation of the procedure steps for aligning and approaching the trocar adapter to the trocar and for inserting surgical instruments into the trocar; Fig. 7: a schematic representation of a surgical robot system with a control unit and a processor.

[0074] Figure 1 schematically depicts an operating room with an arrangement of equipment for eye surgery. The operating room includes an operating microscope 14 and a surgical robot 40. A patient 32 lies on a patient bed 31, and surgery is to be performed on the patient's eye 34.

[0075] The operating microscope 14 is connected to a base 29 located in the operating room 30 by a motion system 18 and is movable in space. By virtue of its position in the operating room 30, the operating microscope 14 defines a coordinate system 33 within which a movement of the operating microscope 14 controlled by the motion system 18 is defined. The coordinate system 33 defined by the operating microscope 14 is a Cartesian coordinate system with x, y, and z coordinates.

[0076] An exemplary stereoscopic beam path 20 of the operating microscope 14 passes from the eye 34 of the patient 32, illuminated by the illumination module 23, through an imaging optic of the operating microscope 14, which includes a fixed focal length objective lens 15, a magnification system 16, a beam splitter module 22 and a viewing port 21. includes . Through the stereoscopic eyepiece 19, the surgeon can view an image of the surgical field produced by the imaging optics of the operating microscope 14 .

[0077] The surgical robot 40 moves a robot tip 43 by means of kinematic elements, of which a ball joint 41 and a robot arm segment 42 are shown. The robot tip 43 is arranged as an end deflector on the kinematic elements of the surgical robot 40. A trocar adapter 45 is formed at the distal end of the robot tip 43. The robot tip 43, moved by actuators 47, has six degrees of freedom of movement, so that both its position and its orientation can be changed completely in three-dimensional space.

[0078] Fig. 2 shows a schematic representation of the surgical robot system according to the invention, in which the surgical microscope 14 comprises a camera 24. An image sensor 26 of the camera 24 is arranged in an image plane 25 of the beam path 20 of the surgical microscope 14, which branches off in the beam splitter module 22. When the image sensor 26 is activated, it captures a microscope image 70 of the image produced by the imaging optics of the surgical microscope 14. The microscope image 70 comprises those objects which are located within the image field 27 of the surgical microscope 14 when the image sensor 26 is activated.

[0079] The operating microscope 14 has an optical axis 17 defined with respect to the imaging optics. In the embodiment shown in Fig. 2, a plane 28 perpendicular to the optical axis of the operating microscope touches a part of the trocar and the eye. The part of the plane 28 located within the image field 27 of the operating microscope that is perpendicular to the optical axis of the The vertical plane 28 of the operating microscope is encompassed by the microscope image 70.

[0080] In the eye 34, a trocar 50 is positioned such that it maintains an open piercing opening in the outer surface of the eye 34, in particular the sclera. A tubular trocar neck 54 of the trocar 50 projects into the interior of the eye, in particular the vitreous body. Adjoining the trocar neck 54 is a narrower opening of a conical trocar funnel 53. The wider opening of this funnel projects out of the eye 34. The outer edge of the trocar funnel 53 forms a trocar opening 52. With respect to the trocar 50, a trocar axis 51 is defined by the common axis of rotation of the trocar neck 54 and the trocar funnel 53. This axis simultaneously forms a perpendicular to a plane spanned by the trocar opening 52.

[0081] With respect to the robot tip 43 with the trocar adapter 45, an axis 44 of the robot tip 43 is defined along the centerline of the trocar adapter 45. In the embodiment according to Fig. 2, the robot tip 44 is oriented such that its axis 44 is aligned along the trocar axis 51.

[0082] Microscope images 70 of an eye 34 with a trocar opening 52 at different magnifications are shown in Figs. 3a and 3b. The position of a point depicted in the microscope image 70 in a plane 28 perpendicular to the optical axis 17 is derived from the image position of the depicted point by inverting the beam path 20. Inverting the beam path 20 results in a transformation from the two-dimensional image coordinate system 71 to the coordinate system 33 defined by the operating microscope 14. The image coordinates of a point in the microscope image 70 are given in the two-dimensional image coordinate system 71. The x and y image coordinates are given. These are converted into the x and y coordinates of the coordinate system 33 defined by the operating microscope 14 using the known transformation. Thus, the position of a point 55 of the trocar opening 52 in a plane 28 perpendicular to the optical axis 17 is determined by calculating the x and y coordinates of point 55 of the trocar opening 52 in the coordinate system 33 defined by the operating microscope 14 from the x and y image coordinates of point 55 of the trocar opening 52 in the microscope image 70.

[0083] Fig. 3c shows a camera image 70a of an eye 34 with a trocar 50; that is, the sensor data is present here as a camera image 70a. The camera image 70a is recorded with an optical camera and includes coordinates of a coordinate system 71 defined by the camera. The trocar 50 has an identification mark 77. The identification mark 77 facilitates the recognition of the trocar 50 during the processing of the camera image 70a.

[0084] Figures 4a-4d show various displacements of the operating microscope 14. The object plane 73 runs perpendicular to the optical axis 17 of the operating microscope 14 through the part of the trocar 50 on which the image is to be focused. In this case, the focus is on a region of the trocar opening 52. The z-coordinate of the trocar 50's position to be determined corresponds to the z-coordinate of the object plane 73, or of the part of the trocar 50 on which the image is to be focused. The working distance 74 of the operating microscope 14 corresponds to a distance between the objective lens of the operating microscope 14 and the focal plane 72. This is a fixed working distance of 20 cm. Objects located within the depth of field 75 are The image field 27 of the operating microscope 14 is displayed sharply. The depth of field 75 depends on the magnification. In this embodiment, the depth of field 75 extends 0.7 mm in the z-direction. In Fig. 4a, the trocar opening 52 and the object plane 73 are located outside the depth of field 75. Therefore, the trocar opening 52 is not sharply imaged by the imaging optics of the operating microscope 14. In Fig. 4b, the object plane 73, and thus the area of ​​the trocar opening 52 to be focused, lies at the lower edge of the depth of field 75. The area of ​​the trocar opening 52 to be focused is therefore sharply imaged, and a contrast level of sharpness is achieved. The displacement of the operating microscope 14 shown in Fig. 4b thus corresponds to the upper limit of the sharp image.

[0085] In Fig. 4c, the focal plane 72, which corresponds to the center of the depth of field 75, lies on the object plane 73. The area of ​​the trocar opening 52 to be focused is imaged sharply. The working distance 74 of the operating microscope 14 determines the position of the trocar 50 parallel to the optical axis 17 of the operating microscope 14. This position is given as the z-coordinate 76 relative to the known position of the operating microscope 14.

[0086] Fig. 4d shows the lower limit of the sharp image. If the operating microscope 14 is moved further downwards, the area of ​​the trocar opening 52 that needs to be focused will no longer be in sharp focus.

[0087] To determine the z-coordinate of the trocar 50's position in this embodiment, the operating microscope 14 is automatically moved in the z-direction. This is done in The displacement increments are 0.1 mm. In the automatic procedure, the operating microscope assumes displacement increments between those shown in Figs. 4a-4d, as well as one displacement increment below that shown in Fig. 4d. By determining the contrast at each displacement increment, the upper and lower limits of the sharp image are determined, beyond which the contrast changes compared to that of the sharp image. The midpoint between the upper and lower limits of the sharp image, as shown in Fig. 4c, is used to determine the z-coordinate of the trocar 50's position.

[0088] Figures 5a and 5b show microscope images 70 of a trocar adapter 45 on a robot tip 43 at different magnifications. At the lower magnification in Figure 5b, the eye 34 with the trocar opening 52 is also included in the image field 27 of the operating microscope 14. The position of the trocar adapter 45 can be determined from the x and y image coordinates of the trocar adapter 45 in the two-dimensional image coordinate system 71 using the same procedures as described above for the position of the trocar opening 52.

[0089] Figures 6a-6d illustrate the sequence of process steps for aligning and approaching the trocar adapter 45 to the trocar 50 and for inserting surgical instruments 46. Figure 6a shows the robot tip 43 with the trocar adapter 45, spaced from the eye 34 and not aligned with the trocar 50. After determining the position and orientation of the trocar 50, the robot tip 43 is aligned so that the axis 44 of the robot tip 43 runs along the axis 51 of the trocar 50, as shown in Figure 6b. The robot tip 43 is then approached to the trocar 50 by axial displacement along the axis 51 of the trocar 50 until the trocar adapter 45 engages the trocar 50, in particular the trocar funnel 53. The approach is automatic until... at a safe distance from the eye. The surgeon guides the robot tip 43 into the trocar 50 using control commands manually entered on the surgical robot 40. The insertion of the trocar adapter 45 into the trocar funnel 53 is shown in Fig. 6c. Once the trocar adapter 45 has inserted into the trocar funnel 53, surgical instruments 46 arranged on the robot tip 43 are inserted through the trocar 50 into the interior of the eye 34, as shown in Fig. 6d. The surgical instruments 46 are ready for use in the procedure.

[0090] Fig. 7 shows the surgical robot system with a control unit 60 and a processor 61. The processor 61 determines the position data of the trocar 50 from the microscope image 70 of the trocar 50 and forwards this data to the control unit 60. The control unit 60 processes the position data of the trocar 50 and generates control commands. These control commands actuate the actuators 47 of the surgical robot 40 so that the trocar adapter 45 is brought close to the trocar 50. To reposition the robot tip 43 during an interruption of the operation, the position data is determined at regular, short intervals. When the position data of the trocar 50 changes, control commands for repositioning are generated, and the actuators 47 of the surgical robot 40 are actuated accordingly. Reference character list: 14 Operating microscope 15. Fixed focal length objective lens 16 magnification system 17 optical axis of the operating microscope 18 Movement system 19 Eyepiece, stereoscopic 20 Beam path 21 Insight 22 Beam splitter module 23 Lighting module 24 camera 25 Image plane 26 image sensor 27 Image field of the operating microscope 28 Plane perpendicular to the optical axis of the operating microscope 29 Base of the operating microscope 31 patient beds 32 patients 33 Coordinate system of the operating microscope 34 Eye 40 surgical robots 41 Ball joint 42 robot arm segments 43 robot tips 44th axis of the robot tip 45 trocar adapters 46 Surgical instruments 47 Actuator 50 trocars 51 Trocar axis 52 Trokaröf opening 53 trocar funnels 54 trocar neck 55 point of the trocar opening 60 Control unit 61 processor 70 Microscope image 70a Camera image 71 two-dimensional image coordinate system 72 Focus plane 73 Object level 74 working distance 75 Depth of field range 76 Z-coordinate 77 Identification mark

Claims

38 Patent claims 1. A method for positioning a robot tip (43) of a surgical robot (40) on a trocar (50), wherein a trocar adapter (45) is formed on the robot tip (43), comprising the following steps: a. Acquiring sensor data (70, 70a) of the trocar (50) with a sensor device (14), wherein, during a movement of the robot tip (43) relative to the trocar (50) from a starting position, the sensor device (14) is arranged to remain stationary relative to the starting position; b. Determining position data of the trocar (50), wherein a position of the trocar (50) is derived from the sensor data (70, 70a); c. Processing the position data of the trocar (50) in a control unit (60) to generate control commands for the surgical robot (40); d. Controlling actuators (47) of the surgical robot (40) with the control commands, so that the trocar adapter (45) is brought close to the trocar (50).

2. Method according to claim 1, wherein the sensor device (14) receives the sensor data (70, 70a) of the trocar (50) while the trocar (50) is located in an image field (27) of the sensor device (14).

3. Method according to one of claims 1 to 2, wherein a data position of the trocar (50) is identified in the sensor data (70, 70a) by digital data processing in order to derive a position of the trocar (50). 39 4. Method according to any one of claims 1 to 3, wherein the sensor device (14) is a camera and wherein the trocar (50) has an identification mark (77), wherein the distance and / or orientation of the identification mark (77) to the sensor device (14) is detected during data processing of the sensor data (70, 70a).

5. Method according to claim 3, wherein the sensor device (14) is an operating microscope (14), wherein the acquisition of the sensor data (70, 70a) comprises acquiring a microscope image (70) with an image sensor (26), wherein the image sensor (26) is arranged in an image plane (25) of the operating microscope (14), wherein, in order to determine position data of the trocar (50), the data position of the trocar (50) is projected back from the image plane (25) to the object plane by inverting an optical beam path (20) in the operating microscope (14).

6. Method according to any one of claims 1 to 5, wherein the sensor device (14) is an operating microscope (14), wherein a position of the trocar (50) is determined in a plane (28) perpendicular to the optical axis of the operating microscope (14) and / or parallel to the optical axis (17) of the operating microscope (14).

7. Method according to claim 6, wherein the position of the trocar (50) parallel to the optical axis (17) of the operating microscope (14) is determined by adjusting a distance between the operating microscope (14) and the trocar (50), wherein the distance between the operating microscope (14) and the trocar (50) is adjusted such that the trocar (50) lies in a focal plane (72) of the operating microscope (14) and the position of the trocar (50) is parallel to the optical axis (17). 40 of the operating microscope (14) from a working distance of the is determined using an operating microscope (14).

8. Method according to any one of claims 1 to 7, wherein an orientation of the trocar (50) is determined from the sensor data (70, 70a) of the trocar (50).

9. Method according to claim 6 or 7, wherein position data of the trocar adapter (45) of the robot tip (43) are determined, wherein a microscope image (70) of the trocar adapter (45) is recorded with an image sensor (26) which is arranged in the image plane (25) of the operating microscope (14) and wherein a position of the trocar adapter (45) in a plane perpendicular to the optical axis (17) of the operating microscope (14) is derived from the microscope image (70) of the trocar adapter (45).

10. Method according to any one of claims 1 to 9, wherein an orientation of the robot tip (43) is determined from the sensor data (70, 70a) of the robot tip (43).

11. Method according to any one of claims 1 to 10, wherein the position data of the trocar (50) are specified in a coordinate system (33) defined by the sensor device (14), wherein position data of the trocar adapter (45) are specified in a coordinate system of the robot tip (43) and a coordination takes place between the coordinate system (33) defined by the sensor device (14) and the coordinate system of the robot tip (43).

12. Method according to any one of claims 1 to 11, wherein the robot tip (43) is aligned along the axis (51) of the trocar (50) when the trocar adapter (45) is approached to the trocar (50).

13. Method according to any one of claims 1 to 12, wherein the approach of the trocar adapter (45) to the trocar (50) by actuating actuators (47) of the surgical robot (40) with the control commands then ends when the trocar adapter (45) is spaced a certain distance from the trocar (50), wherein the remaining distance is 0 to 40 mm, preferably 10 to 15 mm.

14. Method according to any one of claims 1 to 13, wherein, in the event of an interruption of an operation, a change in the position of the trocar (50) is monitored by the sensor device (14).

15. Method according to claim 14, wherein the tracked change in the position of the trocar (50) is processed into tracking commands and wherein a continuous control of actuators (47) of the surgical robot (40) is carried out with the tracking commands.

16. Surgical robot system comprising a surgical robot (40), a sensor device (14), a computer unit (61), and a control unit (60), wherein the surgical robot (40) comprises a robot tip (43) with a trocar adapter (45) formed on the robot tip (43), wherein in a position determination state of the surgical robot system, the sensor device (14) is configured to receive sensor data (70, 70a) of a trocar (50), wherein during a movement of the robot tip (43) relative to the trocar (50) from an initial position, the sensor device (14) is arranged to remain stationary relative to the initial position, and wherein the computer unit (61) is configured to derive position data from the sensor data (70, 70a) of the trocar (50) that represent the position of the trocar (50). wherein the control unit (60) is designed to process the position data in order to generate control commands with which actuators (47) of the surgical robot (40) are controlled in an approach state of the surgical robot system such that the trocar adapter (45) is brought close to the trocar (50).

Images