Method and system for determining relative positions of at least one object in an operating room

Abstract

Method and system for determining the relative position of at least one object in an operating room. The invention relates to a method and system for determining the relative position of at least one object in an operating room (1), in a reference coordinate system of a relative position detection device of a surgical microscope (5), comprising: - determining the relative position of the object in a first coordinate system by means of a movable arranged microscope-external relative position detection device, the first coordinate system being a coordinate system arranged stationary with respect to the operating room (1), - determining the relative position of the reference coordinate system in the first coordinate system by the non-stationary microscope-external relative position detection device, - transforming the relative position of the object from the first coordinate system into the reference coordinate system of the relative position detection device of the surgical microscope (5).

Description

TECHNICAL FIELD

[0001] The present invention relates to a method and a system for determining a relative position of at least one object in an operating room, in a reference coordinate system of a relative position detection device of a surgical microscope. BACKGROUND

[0002] Generally, a plurality of objects is arranged in an operating room for carrying out a (neuro-)surgical intervention, for example. In addition to an operating table, the operating room can comprise, for example, a surgical microscope, a (microscope-external) stereo tracking system, one or more display devices, one or more devices for intraoperative imaging, such as an OCT device, an X-ray device or an ultrasound device.

[0003] In order to make the implementation of the intervention easier and / or to reduce the time required for the intervention, the surgeon can be assisted by a surgical navigation system. This surgical navigation system in particular facilitates the joint representation of intraoperatively generated information and preoperatively generated information in the correct relative position. As an example, intraoperatively obtained data can relate to information about the relative position of a medical instrument or of intraoperatively generated image data. CT or MRI volume data can be an example of preoperatively obtained information.

[0004] So-called tracking systems are used for surgical navigation; these tracking systems facilitate the determination of the relative position of objects, in particular instruments, in the operating room during surgery. In particular, stereo systems for capturing the relative position of an instrument, such as a pointer or other imaging objects / devices, in the operating room are known. In order to determine the relative position by means of such a system, it is generally necessary to arrange a marker or a target having a plurality of markers on the object whose relative position is to be determined. Due to the spatial boundary conditions in the operating room, there is usually a spatial distance of 1 m or up to 2 m between the imaging unit of the tracking system and the object. However, this distance also requires the marker or the target to occupy a certain spatial volume in order to be able to be imaged by the imaging unit with the resolution required for determining the relative position. This is disadvantageous, in particular in the case of microsurgical interventions, since, for example, the instruments are small and the corresponding spatial volume is not available in this case.

[0005] Relative position detection devices integrated into a surgical microscope are also known. These relative position detection devices use an image capturing device for microscopic imaging of the surgical microscope or an environmental camera separate therefrom in order to determine the relative position of an object in the capture area of the respective image capturing device in an image-based manner. Thus, DE 10 2018 206 406 A1 describes a microscopy system having an image capturing device for optically detecting a target. However, the disadvantage of using such tracking systems is that these tracking systems only have a spatially limited capture area and thus no longer detect other objects in a more extensive environment and thus cannot determine their relative position.

[0006] WO 2017 / 157763 A1 discloses tracking the relative position of a medical instrument in a medical work area, wherein a microscope integrated camera is used to determine the relative position of the instrument in the medical work area. A second video camera is provided for determining the relative position between an anatomical structure and the above mentioned microscope integrated camera. This thus allows detecting the surgical microscope by means of this second camera and determining its relative position, while the surgical microscope itself is able to detect the medical instrument by means of the microscope integrated camera and is able to determine the relative position of said medical instrument. The disadvantage in this case is that detecting the relative position of further objects in the environment of the surgical microscope requires all these objects to be in the capture area of the second camera at the same time. These objects must therefore be positioned accordingly and in some cases must also be repositioned during the operation. This generally requires a lot of time and can interrupt or prolong the operation. Another disadvantage is that increasing the distance also requires the size of the appropriate scaling markers / targets. Further, a tracking system operating with such a second camera generally has an undesirably large footprint in the operating room.

[0007] So-called head-mounted displays (HMD) are also known. US 2021 / 090344 A1 discloses an HMD with a mechanism for determining a relative position and an absolute position with respect to a patient, wherein for example a camera is configured in such a way that it detects markers on the patient and based thereon detects the position of the surgeon and the line of sight with respect to the patient. US 2005 / 203380 A1 discloses an HMD with at least one tracking camera. WO 2020 / 102665 A1 discloses so-called inside-out tracking, wherein one or more cameras are fastened to the HMD.

[0008] However, it is desirable to provide information about the relative position of an object in the operating room, which is not in the capture area of the relative position detection device of the surgical microscope, in the reference coordinate system of the relative position detection device of the surgical microscope, in order to be able to merge the relative position of such an object and / or information provided by such an object (for example, intraoperatively generated image data) with the information provided by the relative position detection device of the surgical microscope and / or the (image) information provided by the surgical microscope in a simple manner. In particular, but not exclusively, it is desirable to be able to quickly, accurately, reliably, in particular also with the correct relative position in relation to further information (for example, also preoperatively generated information), display intraoperative image data, if they are generated by the surgical microscope. SUMMARY

[0009] Thus, the technical problem that arises is to develop a method and a system for determining the relative position of at least one object in an operating room, which facilitates a quick, accurate, reliable and correct determination of information about the relative position of the object, in particular in a reference coordinate system of a relative position detection device of a surgical microscope, in terms of relative position.

[0010] The solution to the technical problem is provided by a method according to the invention for determining the relative position of at least one object in an operating room in a reference coordinate system of a relative position detection device of a surgical microscope and a system according to the invention.

[0011] A method for determining the relative position of at least one object in an operating room in a reference coordinate system of a relative position detection device of a surgical microscope is proposed. The relative position detection device of the surgical microscope is also referred to below as the internal relative position detection device. The reference coordinate system of this relative position detection device can be the reference coordinate system of the surgical microscope or can be arranged in a stationary manner with respect to this coordinate system.

[0012] The method comprises:

[0013] - determining the relative position of the object in a first coordinate system by means of a movably arranged microscope external relative position detection device, which first coordinate system is a coordinate system arranged stationary with respect to the operating room,

[0014] - determining the relative position of the reference coordinate system in the first coordinate system by the non-stationary microscope external relative position detection device,

[0015] - transforming the relative position of the object from the first coordinate system into the reference coordinate system of the relative position detection device of the surgical microscope.

[0016] In this context, the operating room can be represented by a room or a similar room area set up for performing a surgery. In particular, in the operating room there are arranged devices or equipment required for performing a surgery, in particular the one or more objects mentioned at the outset, but the list is not exhaustive. In particular, the object can be a device for intraoperative imaging, for example a device for producing X-ray images, such as a C-arm, a device for producing ultrasound images or a device for producing OCT images. The object can also be an instrument, in particular a medical instrument, more particularly a surgical instrument. This includes, for example, instruments such as clamps, holders, syringes, forceps, spatulas, scissors, scalpels, wound hooks, pincers, suction devices, cauterizing tools, retractor, for example a brain retractor, and the like.

[0017] Within the meaning of the present invention, determining the relative position of the object can mean determining the relative position in the target coordinate system completely and one-to-one, for example determining the position values of the origin of the Cartesian object coordinate system with respect to the three spatial axes of the Cartesian target coordinate system and determining the three angle values between the three spatial axes of the Cartesian object coordinate system and the three spatial axes of the Cartesian target coordinate system. The target coordinate system can be the reference coordinate system of the respective relative position detection device. However, determining the relative position can also mean determining the relative position incompletely, for example in this case not all position values and / or not all angle values are determined.

[0018] Within the meaning of the present invention, the surgical microscope can mean a device for magnifying the visual presentation of an examination object or an examination region. The microscope can be a conventional optical microscope, which generates a magnified image representation by making use of optical effects, in particular by means for beam guidance and / or beam shaping and / or beam deflection, for example lens elements. However, the microscope can also be a digital microscope, wherein the image representation to be visualized by the microscope can be generated by means of a microimaging image capture device and can be displayed on a suitable display device, for example a visual display unit. The surgical microscope can comprise a stand for holding an objective, which can be mechanically fastened, in particular mechanically moveably fastened, for example pivotably fastened, to the free end of the stand. In this case, the stand is designed to facilitate movement of the objective in space. The surgical microscope can be part of a surgical microscope system, which can comprise further devices in addition to the surgical microscope, for example a display device, in particular a 3D display device.

[0019] The internal relative position detection device of the microscope is preferably a monoscopic relative position detection device. The monoscopic relative position detection device comprises at least one image capture device, in particular exactly one image capture device, for generating a two-dimensional image representation of the object having the relative position, in particular of the at least one marker, that is to say the three-dimensional position and / or the three-dimensional orientation of the object can be determined by evaluating the two-dimensional image representation, in particular exactly one two-dimensional image representation. In particular, an evaluation of the intensity values of the pixels (image points) of the two-dimensional image representation can be carried out in order to determine the relative position. However, the internal relative position detection device does not necessarily have to be in the form of a monoscopic relative position detection device; the internal relative position detection device can also be designed differently, for example as a stereoscopic relative position detection device.

[0020] In this case, the image capturing device(s) of the internal relative position detection device can be arranged in the housing of the microscope. The image capturing device of the internal relative position detection device or one of these image capturing devices can be the image capturing device for the microscopic imaging. However, the image capturing device(s) of the internal relative position detection device is / are preferably different from the image capturing device for the microscopic imaging. In this case, the beam path of the radiation for the microscopic imaging can be arranged and / or formed differently, in particular spatially separated, from the beam path of the radiation directed to the image capturing device of the internal relative position detection device and used for determining the relative position. These beam paths can be formed in a common housing or microscope body of the microscope. The capturing areas of these image capturing devices can at least partially overlap.

[0021] The microscope external relative position detection device (in the following also referred to as external relative position detection device) can be an optical relative position detection device, in particular a monoscopic or stereoscopic relative position detection device. Further, the microscope external relative position detection device can comprise exactly one or at least two image capturing devices. The image capturing device(s) of the external relative position detection device can be an RGB camera. However, it can also be conceivable to use different sensors, for example at least one time-of-flight sensor or a gyroscope, or a sensor for determining the relative position as described in more detail below, and which is part of the microscope external relative position detection device. The external relative position detection device, in particular its sensor(s) or image capturing device(s), can in this case be portable and for example integrated in or arranged on a device which can be worn in particular by a user or surgeon. In particular, the wearable device can be a device which can be arranged / fixed on the head or arm or any other body part. Further, the relative position detection device, that is to say the internal and / or external relative position detection device, can comprise at least one illumination device.

[0022] The relative position can be determined with the aid of a marker. This can mean that exactly one marker or a plurality of markers are arranged, more particularly fastened, to the object. It is also conceivable that the object is arranged, more particularly fastened, to the object, wherein this object comprises exactly one marker or a plurality of markers. A plurality of markers of the object or the object can be arranged in a predetermined arrangement, that is to say a previously known arrangement, relative to one another, which in particular facilitates the identification of the object. The marker can in this case be optically detectable and thus also detectable in the image representation of the image capturing device of the relative position detection device, in particular thus having an optically detectable pattern. The optically detectable marker can in particular be a so-called ARUCO marker or a marker as specified in US 2017 / 258531 A1. The marker is preferably a passive marker, but can also be an active marker. The marker can also be a reflective or non-reflective marker, a phosphorescent or non-phosphorescent marker and / or a fluorescent or non-fluorescent marker. The relative position can also be determined without a marker. In this case, for example, the geometry of the imaged object can be evaluated in order to determine the relative position. Methods for determining the relative position on the basis of markers and without markers are known to the person skilled in the art.

[0023] In this case, the relative position information can be determined with the aid of an evaluation device of the relative position detection device. The evaluation device can be formed as a microcontroller or an integrated circuit or comprise such a microcontroller or integrated circuit. For this purpose, the output signals (image signals) generated by the image capturing device(s) can be transmitted to the evaluation device and can be evaluated by the evaluation device for determining the relative position. The fact that the first coordinate system is arranged in a stationary manner relative to the operating room can mean that the relative position of this coordinate system, that is to say in particular the position of its origin and the alignment of its axes, does not change relative to stationary components of said operating room. As an example, such stationary components can be the floor, the walls of the room, the ceiling or objects arranged / fastened to these in a stationary manner. In particular, the first coordinate system can be arranged in a stationary manner in the GNSS coordinate system. The first coordinate system can be the reference coordinate system of the external relative position detection device, wherein the relative position is determined by the external relative position detection device in this coordinate system. However, in order to determine the relative position, there can be a transformation from the coordinate system of the detection device of the external relative position detection device to the first coordinate system. This transformation or transformation rule can be determined at runtime - this will still be explained in more detail below. The relative position of the first coordinate system in the operating room can be achieved by evaluating the signals produced by the external relative position detection device, in particular by means for acquiring environmental information of the external relative position detection device.

[0024] The transformation of the relative position of the object from the first coordinate system into the reference coordinate system of the internal relative position detection device can be performed by the evaluation device, in particular by the evaluation device of the surgical microscope. This can be the evaluation device which also determines the relative position with the aid of the internal relative position detection device. However, it can also be an evaluation device which is separate therefrom or an evaluation device external to the microscope, for example an evaluation device of the external relative position detection device or an evaluation device of the overall system. In particular, the reference coordinate system of the internal relative position detection device can be a coordinate system which is arranged in a stationary manner with respect to the objective of the surgical microscope, i.e. a coordinate system which is stationary with respect to the objective.

[0025] The evaluation device can then control the display device, in particular the 3D display device of the surgical microscope or of the surgical microscope system comprising the display device and the surgical microscope, in such a way that there is a representation of the object in its correct relative position and / or a representation of the information, in particular the image data, of the object provided intraoperatively in its correct relative position with respect to the information, in particular the image data, produced preoperatively, for example by overlaying, superimposing and / or using methods for representing virtual or augmented reality. Thus, the microscope, X-ray, ultrasound and / or OCT images produced intraoperatively and the image data such as CT or MRI data produced preoperatively can be represented, for example, in correct relative positions with respect to one another. The evaluation device can also control the display device in such a way that there is a representation of the object in its correct position and / or a representation of the information, in particular the image data, of the object provided intraoperatively in its correct relative position with respect to the microscope images produced intraoperatively, in particular by means of the methods described above. Thus, the X-ray, ultrasound and / or OCT images produced intraoperatively and the microscope images produced intraoperatively can be represented, for example, in correct relative positions with respect to one another.

[0026] The object detected by the internal relative position detection device can also be represented in its correct relative position in the image data produced preoperatively and / or intraoperatively. By determining the relative position of the reference coordinate system of the internal relative position detection device and the relative position of the object in the first coordinate system and by transforming into the reference coordinate system of the internal relative position detection device, it is thus advantageously possible to represent the information quickly, accurately, reliably and in correct relative positions with the aid of the surgical microscope or with the aid of the surgical microscope system comprising the surgical microscope.

[0027] In a further embodiment, at least one sensor of the movable microscope-external relative position detection device is integrated in or arranged on a head-mounted display (HMD). The sensor can represent a detection device which produces output signals which can be evaluated for determining the relative position, in particular with the aid of an evaluation device. The evaluation device can likewise be integrated in or arranged on the HMD. However, the evaluation device can also not be integrated in or arranged on the HMD.

[0028] The HMD can provide the wearer with objects in a virtual and / or augmented reality. The HMD can have a glasses-like design, in particular as a pair of VR and / or AR glasses. In particular, the HMD can be designed in such a way that the wearer can perceive real objects in the operating room that are located in his field of view, in addition to which virtual objects can be overlaid into the field of view, for example in the form of an overlay, in particular a transparent overlay, or in the form of a replacement for real objects. As an example, the wearer can visually capture information that is represented on a display device in the operating room, in particular a microscopic image representation provided by a surgical microscope, wherein at least one additional information item, in particular at least one virtual object, for example information generated preoperatively and / or intraoperatively, can be displayed to the wearer by means of the HMD, in particular in the correct relative position. In addition to this superimposition of information / objects, the HMD comprises a relative position detection device or at least a part thereof, in particular a sensor-type part. This allows partial or complete determination of the relative position of the objects in the capture area, in particular in the reference coordinate system of the HMD. In this case, the capture area of the relative position detection device can comprise or at least partially overlap the field of view of the wearer provided by the HMD. Head movements of the wearer also change the relative position of the capture area. Head movements, for example for scanning the operating room, thus contribute to providing a large spatial capture area of the external relative position detection device in a simple manner, in particular by fusing the capture areas in the various relative positions to form a composite capture area. Thus, by virtue of integrating the relative position detection device into the HMD, an external relative position detection device with a large capture area is advantageously provided in a simple manner.

[0029] In another embodiment, the relative position of the microscope-external relative position detection device or at least the relative position of the sensors of the microscope-external relative position detection device remains constant between determining the relative position of the object by the microscope-external relative position detection device and determining the relative position of the reference coordinate system. In other words, the relative positions of the object and the surgical microscope are determined by the external relative position detection device when both the object and the surgical microscope are located in the (relative position-specific) capture area of the external relative position detection device in their current relative position. Advantageously, this contributes to reliably and accurately determining the relative positions in the common coordinate system, in particular the first coordinate system, since possible changes in the relative position of the external relative position detection device between determining the relative positions have to be taken into account.

[0030] Alternatively, the relative position is changed between the determination of the relative position of the object by the microscope-external relative position detection device and the determination of the relative position of the reference coordinate system. In other words, the relative position of the object and the surgical microscope is determined by the external relative position detection device in different relative positions of the external relative position detection device, wherein for example the object and not the surgical microscope is located in the capture area of the external relative position detection device in the first relative position and the surgical microscope and not the object is located in the capture area in the changed relative position. This advantageously contributes to reliably determining the relative position in the common coordinate system, even if the surgical microscope and the object are not together located in the capture area of the external relative position detection device in their current relative position, for example if they are arranged far away from each other in the operating room. The relative position can be changed in such a way that the capture area of the external relative position detection device before the change in the relative position and the relative position of this capture area after the change in the relative position have a common area, or the capture areas in the two relative positions do not intersect.

[0031] In another embodiment, the microscope-external relative position detection device comprises at least one image capture device and / or at least one time-of-flight sensor and / or at least one depth sensor and / or a device for detecting surfaces to determine the relative position. In particular, they can form the above-mentioned sensors of the external relative position detection device. In other words, the at least one sensor for determining the relative position can be integrated into the external relative position detection device, wherein the determination of the relative position can be performed by evaluating the output signals of the sensors. In the case of an image capture device, in particular an RGB camera or an infrared camera, there can be a single-view determination of this relative position. Furthermore, the sensor can be an ultrasonic sensor, a radar sensor or a lidar sensor. The determination of the relative position using at least one image capture device is advantageous, in particular in the case of a marker-assisted determination of the relative position, since a high lateral resolution (in the image plane) is regularly provided and thus a high accuracy, and / or texture and / or color information, in particular RGB information, which contributes to the reliable identification of the markers, can be determined. This is particularly advantageous when the relative position is determined on the basis of the optical detection of a plurality of different markers. When the relative position is determined without markers, the use of a time-of-flight sensor and / or at least one depth sensor and / or a device for detecting surfaces is advantageous, in particular because this allows the determination of depth information, which in turn contributes to the rapid, accurate and reliable determination of information about the 3D geometry, for example information about the shape of the instrument, wherein this information can then be evaluated to determine the relative position.

[0032] Preferably, the microscope-external relative position detection device comprises a plurality of means or sensors for determining the relative position, in particular at least one image capturing device and at least one sensor for capturing depth information, e.g. a time-of-flight sensor, a depth sensor or a device for capturing the surface. The relative position of the object can be determined by a sensor fusion approach, wherein the relative position information generated by the plurality of sensors is combined in such an approach in order to determine the relative position. In particular, the output signals of the plurality of sensors can be combined / fused and then the relative position is determined based on these fused output signals. It is also possible to combine / fuse the sensor-specific relative position information determined by evaluating the output signals of the corresponding sensors and then determine the relative position based on these fused items of relative position information.

[0033] Further, the external relative position detection device can comprise an illumination device. As an example, this illumination device can be used to generate a structured illumination, wherein the image data generated by the image capturing device under this structured illumination can be evaluated for determining the relative position. The illumination device can also be an IR illumination device.

[0034] In another embodiment, the relative position of the first coordinate system is determined or defined by evaluating the output signal of at least one device for acquiring environmental information of the microscope-external relative position detection device, which device or at least a part thereof is integrated in the microscope-external relative position detection device or arranged thereon. In other words, the external relative position detection device can comprise at least one device for acquiring environmental information. This device for acquiring environmental information is used to acquire information about the environment of the external relative position detection device. As an example, it can generate an output signal which depends on the relative position and / or shape of the object in its capturing region. Then, the evaluation of the output signal helps to determine the relative position of the object in the coordinate system of this device or in the coordinate system of the external relative position detection device, if the transformation from this coordinate system to the coordinate system of the external relative position detection device is known, e.g. as a result of a calibration. The device for acquiring environmental information can be designed as an environmental sensor or ambient sensor or can comprise one or more environmental sensors.

[0035] The evaluation of the output signal of the device is particularly helpful for the determination or creation of a spatial model, wherein the coordinate system of the spatial model can be the first coordinate system. As an example, the spatial model can provide information about objects in the capturing area of the at least one device for acquiring environmental information, in particular about their relative position, preferably in the first coordinate system, and / or with respect to their shape. In particular, the spatial model can be a 3D model of the operating room, which is created on the basis of the output signal of the device for acquiring environmental information. It is conceivable that, in addition to the output signal, also pre-known information is used for creating the spatial model, for example pre-known relative positions of objects arranged in the operating room, in particular objects arranged in a stationary manner with respect to the operating room. As an example, such information can be architectural data, like floor plan data. As an example, this information can be provided by a database.

[0036] In particular, the spatial model can allow to determine the relative position of the external relative position detection device in the first coordinate system by evaluating the output signal of the at least one device for acquiring environmental information. Thus, a transformation rule from the coordinate system of the external relative position detection device to the first coordinate system can be determined. However, in this case, also the relative position of an object determined by the external relative position detection device can be determined in the first coordinate system. In particular, for this the relative position of the object can be determined in the coordinate system of the external relative position detection device, which can then be transformed into the first coordinate system by means of the above-mentioned transformation rule.

[0037] The device for acquiring environmental information can comprise one, a plurality or preferably all sensors for determining the relative position, that is to say one or more sensors whose output signal is used by the external relative position detection device for determining the relative position. However, it is also possible to use sensors whose output signal is not used by the external relative position detection device for determining the relative position, that is to say sensors which are different therefrom. Preferably, the device comprises at least one image capturing device and at least one device for generating depth information. Furthermore, the device for acquiring environmental information can comprise a device for determining the relative position of the microscope external relative position detection device, for example in the form of or comprising a gyroscope. This device for determining the relative position of the microscope external relative position detection device can for example determine the relative spatial position of the portable device, wherein the microscope external relative position detection device or at least one sensor of said relative position detection device is integrated.

[0038] Determining or defining the relative position of the first coordinate system based on the output signals of the device for acquiring environmental information advantageously contributes to the relative position of the surgical microscope and the object being determined by means of the external relative position detection device being able to be achieved quickly, in particular without the need for recalibration or request procedures, even if the relative position of the external relative position detection device changes, in particular during the method run. A further advantage of this definition is that the surgical microscope and the object do not need to be located in the capture region of the microscope external relative position detection device at the same time.

[0039] In a further embodiment, the relative position of the first coordinate system and / or the relative position of the microscope external relative position detection device is updated in the first coordinate system during the run. In particular, the aforementioned spatial model can be updated during the run, for example continuously and after the expiry of a predetermined time interval. This advantageously increases the reliability of the relative position determination, in particular if the reference coordinate system of the internal relative position detection device and / or the relative position of the object has changed in the first coordinate system.

[0040] In a further embodiment, a SLAM method is performed to determine / define the relative position of the first coordinate system. Such a SLAM method (simultaneous localization and mapping method) is known to the person skilled in the art. Thus, the relative position of the first coordinate system, and thus also the relative position of the object in the reference coordinate system of the internal relative position detection device, can advantageously be determined simply, accurately and reliably.

[0041] In a further embodiment, information about the relative position of the reference coordinate system in the first coordinate system and about the relative position of the object in the first coordinate system is transmitted to an evaluation device of the surgical microscope, which transforms the relative position of the object into the reference coordinate system of the internal relative position detection device. The evaluation device can be formed as a microcontroller or an integrated circuit, or comprise such a microcontroller or integrated circuit. The evaluation device can also be an evaluation device for determining the relative position of the internal relative position detection device. Furthermore, the evaluation device can be part of a control device of the surgical microscope, which controls, for example, the operation of the surgical microscope, for example the adjustment of the movement, the magnification and / or the focus of the surgical microscope, and / or the illumination of a microscope illumination device. Furthermore, the control device can generate control data or control signals for controlling a display device in order to achieve the representation of the aforementioned information on the display device with the correct relative position. As a result of the transmission to the evaluation device of the surgical microscope, it is advantageously the case that the surgical microscope uses the relative position information quickly, in particular without undesirable delays, in particular for fusing the relative position information with information generated by the surgical microscope.

[0042] In another embodiment, the information about the relative position of at least one further object in the operating room is determined by a relative position detection device of the surgical microscope. In particular, the further object can be an object whose relative position is determined or determinable by the external relative position detection device. However, the further object is preferably an object whose relative position is not determined or determinable by the external relative position detection device. In particular, the further object can be an object arranged in the capture area of the internal relative position detection device, for example a medical instrument. The relative position of the reference coordinate system of the internal relative position detection device, the relative position of the object and the relative position of the further object in the reference coordinate system can then be determined, in particular by means of the evaluation device of the surgical microscope. As an example, the further object can thus be displayed with the correct relative position to the object detected by the external relative position detection device or the information generated by this object in the operation and / or the information generated by the surgical microscope in the operation and / or the information generated before the operation.

[0043] A system for determining the relative position of at least one object in the operating room, in the reference coordinate system of a relative position detection device of a surgical microscope is also proposed. In particular, the system is configured to carry out the method according to one of the embodiments described in the present embodiment. In other words, such a method can be carried out by the system.

[0044] The system comprises at least a relative position detection device of a surgical microscope and a moveably arranged microscope-external relative position detection device, and an evaluation device. The system is configured to carry out the following steps:

[0045] - determining the relative position of the object in a first coordinate system by means of the moveably arranged microscope-external relative position detection device, the first coordinate system being a reference coordinate system of the microscope-external relative position detection device,

[0046] - determining the relative position of the reference coordinate system in the first coordinate system by the non-stationary microscope-external relative position detection device,

[0047] - transforming the relative position of the object from the first coordinate system into the reference coordinate system of the relative position detection device of the surgical microscope.

[0048] The internal relative position detection device can comprise at least one or exactly one image capturing device for microscopic imaging and / or at least one or exactly one image capturing device for determining the relative position, which is different from the image capturing device for microscopic imaging. Further, the system can comprise data or signal transmission means for transmitting data / signals between the evaluation device, the internal relative position detection device and / or the external relative position detection device. These data or signal transmission means can be designed as wireless or wired data transmission. Further, the system can comprise one or more components of the above-mentioned internal and / or external relative position detection device. The proposed system advantageously contributes to the implementation of one of the embodiments of the method described in the present disclosure and has the corresponding technical advantages. The system can be integrated into an operating room.

[0049] In another embodiment, the movable microscope external relative position detection device is integrated in or arranged on a head-mounted display. This and the corresponding technical advantages have already been explained above.

[0050] In another embodiment, at least one image capturing device and / or at least one time-of-flight sensor and / or at least one depth sensor and / or a device for detecting a surface for determining the relative position is integrated in or arranged on a head-mounted display. This and the corresponding technical advantages have already been explained above.

[0051] In another embodiment, at least one device for acquiring environmental information is integrated in or arranged on a head-mounted display. This and the corresponding technical advantages have already been explained above. In particular, this device can comprise one, more or all devices for determining the relative position, which are integrated in or arranged on the HMD. Thus, in this case, the devices for determining the relative position correspond to the devices for acquiring environmental information or components of this device.

[0052] In another embodiment, the system comprises at least one display device, which can be controlled by the evaluation device, for representing the object in the preoperatively and / or intraoperatively generated data with the correct relative position. This and the corresponding technical advantages have already been explained above. BRIEF DESCRIPTION OF DRAWINGS

[0053] The application will be explained in more detail on the basis of exemplary embodiments. In the drawings:

[0054] Figure 1 a schematic view of an operating room is shown,

[0055] Figure 2 a schematic flow chart of a method according to the application is shown, and

[0056] Figure 3 A schematic flow chart of a method according to the application is shown.

[0057] In the following, identical reference signs indicate elements having identical or similar technical features. DETAILED DESCRIPTION

[0058] Figure 1 A schematic view of an operating room 1 is shown. A surgeon 2 and a patient 3 lying on an operating table 4 in the operating room 1 are depicted. Further, a surgical microscope 5 with a stand 6 and an objective 7 are depicted. The objective 7 is arranged on a free end of the stand 6. A drive device (not depicted here) of the surgical microscope 5 facilitates a change of the position and orientation of the objective 7, in particular in a first coordinate system, which is a Cartesian coordinate system and comprises a longitudinal axis X and a vertical axis Z. The first coordinate system is arranged in a stationary manner with respect to the operating room 1, in particular with respect to a floor 8 of the operating room 1. A transversal axis, which can be oriented perpendicular to the depicted longitudinal and vertical axes X, Z, is not depicted. In particular, the vertical axis Z can be oriented parallel and opposite to the force of gravity.

[0059] Further, a control and evaluation device 9 of the surgical microscope 5, and a display device 10, in particular a 3D display device, which can be part of a surgical microscope system comprising the surgical microscope 5, are depicted. In particular, the control and evaluation device 9 can control the display device 10 to display information, in particular by means of an interface suitable for this purpose.

[0060] The surgical microscope 5, in particular the objective 7, is further depicted as comprising an environmental camera 11, which can be arranged in a housing of the objective 7. A beam path 12 is likewise schematically depicted, along which a light beam captured by the environmental camera 11 extends. Further schematically depicted is an image capturing device 13 of the surgical microscope 5 for microimaging, and a beam path 14, along which a light beam captured by means of this image capturing device 13 extends. The image capturing device 13 and the beam paths 12, 14 are also arranged in the housing of the objective 7, wherein the beam paths are formed separately.

[0061] In this case, the environmental camera 11 is part of a relative position detection device of the surgical microscope 5, that is to say an internal relative position detection device. A data connection 26 is schematically depicted, which can be used to transfer image data of the environmental camera 11 to the control and evaluation device 9, which then determines the relative position. A reference coordinate system of the internal relative position detection device can be a coordinate system stationary with respect to the objective. Thus, the relative position determined by the internal relative position detection device can be determined in this reference coordinate system.

[0062] Thus, for example, the relative position of the instrument 15 held by the surgeon 2 can be determined in the reference coordinate system of the internal relative position detection device. In particular, the relative position can be determined by an exact evaluation of one image representation generated by the two-dimensional environment camera 11. For determining the relative position, the instrument 15 can comprise a marker 16 imaged by the environment camera 11, wherein the respective image information can be evaluated by the control and evaluation device 9 to determine the relative position.

[0063] Further depicted is a head-mounted display 17 worn on the head by the surgeon 2. The HMD 17 comprises an image capturing device 18 and display means / devices for detecting information from virtual and / or augmented reality. The image capturing device 18 can be part of the microscope external relative position detection device (external relative position detection device).

[0064] Not depicted is that the HMD 17 comprises an evaluation device which facilitates the relative position of an object in a coordinate system stationary with respect to the HMD. This coordinate system stationary with respect to the HMD can be the reference coordinate system of the external relative position detection device. Further, this evaluation device can transform the relative position information determined in this way from the coordinate system stationary with respect to the HMD, for example the coordinate system of the image capturing device 18, into a first coordinate system, the result of which is that the relative position of the object can be determined in the first coordinate system by the external relative position detection device. The evaluation device can be integrated into the HMD 17, but can also be formed by the control and evaluation device 9. Alternatively or in addition to the image capturing device 18, the external relative position detection device can also comprise a time-of-flight sensor and / or a depth sensor and / or a device for capturing surfaces to determine the relative position.

[0065] Further, the HMD can comprise at least one device for acquiring environmental information (not depicted), the output signal of which can be evaluated for determining the relative position of the first coordinate system or for defining the relative position of the first coordinate system with respect to the operating room, in particular in a global coordinate system, for example a coordinate system stationary with respect to the ground, for example a geocentric coordinate system.

[0066] Further, the information about the relative position of the object in the first coordinate system or the output signal of the image capturing device 18 can be transmitted to the control and evaluation device 9 by means of a device for data transmission (not depicted here).

[0067] Further depicted is an objective 7 having a marker 20 fastened to the housing of the objective 7. This marker 20 can be imaged by the image capturing device 18 of the HMD 17 and can be used to determine the relative position of the objective 7. As described above with respect to the object, the relative position of the objective 7 of the surgical microscope 5 in the first coordinate system can thus be determined by means of the external relative position detection device. This in turn allows to determine the relative position of the reference coordinate system of the internal relative position detection device in the first coordinate system by means of the external relative position detection device.

[0068] Also depicted is an intraoperative ultrasound device 21 comprising a base 22, a display device 23 and an ultrasound head 24, a marker 25 being fastened to the ultrasound head as well. The surgeon 2 or another surgeon or assistant (not depicted here) can actuate the ultrasound head 24 to perform intraoperative imaging and move the ultrasound head for this purpose relative to the patient 3, in particular in a state of contact with the patient 3. In this case, the external relative position detection device can also determine the relative position of the ultrasound head 24 in the first coordinate system by a precise evaluation of the image representation(s) generated by the image capturing device 18 of the HMD 17.

[0069] This relative position information can also be transmitted to the control and evaluation device 9. Thus, this advantageously allows the image data generated intraoperatively by the intraoperative ultrasound device 21 and the image data generated by the objective 7, in particular of the image capturing device 13 for microscopic imaging, which can be determined in the reference coordinate system of the internal relative position detection device, in particular in a coordinate system stationary with respect to the objective, to be represented on the display device 10 in a correct relative position with respect to each other.

[0070] Further, the relative position of the instrument 15 determined by the internal relative position detection device can likewise be represented in a correct relative position with respect to the image information generated by the ultrasound device 21 and / or in a correct relative position with respect to the image information generated by the image capturing device 13 of the surgical microscope 5. Naturally, alternatively or additionally, preoperatively generated data, in particular image data, can also be represented on the display device 10 in a correct relative position.

[0071] Figure 2 A schematic flow chart of the method according to the present application is shown. In a first step S1, a relative position of an object is determined in a first coordinate system (see Figure 1 ) by means of an external relative position detection device which is movably arranged, the first coordinate system being a coordinate system which is arranged stationary with respect to the operating room 1. In particular, the object can be an intraoperative imaging object, e.g. the imaging ultrasound device 21 depicted in Figure 1 . In particular, the external relative position detection device or a part thereof can be integrated in the HMD 17 (see Figure 1 ).

[0072] In a second step S2, the relative position of the reference coordinate system of the internal relative position detection device of the surgical microscope 5 in the first coordinate system is determined. As mentioned above, this reference coordinate system can in particular be a coordinate system stationary with respect to the objective. Then, in a third step S3, the relative position of the object is transformed from the first coordinate system to the reference coordinate system of the internal relative position detection device.

[0073] In particular, a transformation rule for transforming the relative position in a coordinate system stationary with respect to the object into the first coordinate system can be determined. As a result, for example, the relative position information of the image information generated by the object and thus also with respect to one coordinate system stationary with respect to the object can be transformed into the first coordinate system.

[0074] Determining the relative position of the reference coordinate system of the internal relative position detection device in the first coordinate system facilitates determining a transformation from the first coordinate system to this reference coordinate system. Thus, the relative position information of the above-mentioned image information can be transformed into the reference coordinate system of the internal relative position detection device.

[0075] Between the first step S1 and the second step S2, the relative position of the external relative position detection device, in particular in the first coordinate system, can change or remain constant.

[0076] Figure 3 A schematic flowchart of a method according to the application in another embodiment is shown. As in the embodiment shown above, Figure 2 In contrast to the embodiment shown, the method comprises a relative position determination step S0 in which the relative position of the first coordinate system is defined by the external relative position detection device itself, in particular by evaluating an output signal of at least one device for acquiring environmental information of the external relative position detection device. As an example, for this a so-called SLAM method can be executed.

[0077] With regard to Figure 1 With regard to the exemplary embodiment depicted in

[0078] The relative position of the first coordinate system and / or the relative position of the external relative position detection device in the first coordinate system can also be updated at runtime, for example by means of a continuation of a SLAM method. It is not mandatory to perform the relative position determination step S0 before the sequence of the first, second and third steps S1, S2, S3. Thus, the determination / definition of the relative position of the first coordinate system and / or the relative position of the external relative position detection device in the first coordinate system can also be performed in parallel to the sequence of steps S1, S2, S3. In particular, the first step S1 or information produced in the course thereof, for example an image representation of objects in the operating room 1, can be used to determine the relative position of the first coordinate system and / or the relative position of the external relative position detection device in the first coordinate system.

[0079] List of reference signs

[0080] 1 operating room

[0081] 2 surgeon

[0082] 3 patient

[0083] 4 operating table

[0084] 5 surgical microscope

[0085] 6 holder

[0086] 7 objective

[0087] 8 floor

[0088] 9 control and evaluation device

[0089] 10 display device

[0090] 11 environmental camera

[0091] 12 beam path

[0092] 13 image capturing device for microscopic imaging

[0093] 14 beam path

[0094] 15 instrument

[0095] 16 marker

[0096] 17 HMD

[0097] 18 image capturing device

[0098] 19 display device

[0099] 20 marker

[0100] 21 ultrasound device

[0101] 22 holder

[0102] 23 display device

[0103] 24 ultrasound head

[0104] 25 marker

[0105] 26 data connection

[0106] S1 first step

[0107] S2 second step

[0108] S3 third step

[0109] S0 relative position determination step

[0110] X longitudinal axis

[0111] Z vertical axis

[0112] The same reference signs in the following drawings indicate elements with the same or similar technical features.

Claims

1. A method for determining a relative position of at least one object in an operating room (1) in a reference coordinate system of a relative position detection device of a surgical microscope (5), the method comprising: - determining a relative position of the object in a first coordinate system by means of a movably arranged microscope-external relative position detection device, wherein the first coordinate system is a coordinate system arranged stationary with respect to the operating room (1), - determining a relative position of the reference coordinate system in the first coordinate system by the movably arranged microscope-external relative position detection device, - transforming the relative position of the object from the first coordinate system into the reference coordinate system of the relative position detection device of the surgical microscope (5).

2. The method of claim 1, wherein, At least one sensor of the movably arranged microscope-external relative position detection device is integrated in or arranged on a head-mounted display (17).

3. The method according to any of the preceding claims, characterized in that, The relative position of the microscope-external relative position detection device remains constant or changes between determining the relative position of the object and determining the relative position of the reference coordinate system by the microscope-external relative position detection device.

4. The method according to claim 1 or 2, characterized in that, The microscope-external relative position detection device comprises at least one image capturing device (18) and / or at least one time-of-flight sensor and / or at least one depth sensor and / or a device for detecting a surface to determine a relative position.

5. The method according to claim 1 or 2, characterized in that, The relative position of the first coordinate system is determined or defined by evaluating at least one output signal of a device for acquiring environmental information of the microscope-external relative position detection device, wherein the device for acquiring environmental information is integrated in or arranged on the microscope-external relative position detection device.

6. The method of claim 5, wherein, The relative position of the first coordinate system and / or the relative position of the microscope-external relative position detection device is updated in the first coordinate system at runtime.

7. The method of claim 5, wherein, A SLAM method is performed to determine / define the relative position of the first coordinate system.

8. The method of claim 6, wherein, A SLAM method is performed to determine / define the relative position of the first coordinate system.

9. The method of claim 1 or 2, wherein, Information about the relative position of the reference coordinate system in the first coordinate system and about the relative position of the object in the first coordinate system is transmitted to an evaluation device (9) of the surgical microscope (5), wherein the evaluation device (9) transforms the relative position of the object into the reference coordinate system of the relative position detection device of the surgical microscope (5).

10. The method of claim 1 or 2, wherein, Information about a relative position of at least one further object in the operating room (1) is determined by the relative position detection device of the surgical microscope (5).

11. A system for determining a relative position of at least one object in an operating room (1) in a reference coordinate system of a relative position detection device of a surgical microscope (5), wherein the system comprises at least a relative position detection device of the surgical microscope (5) and a movably arranged microscope-external relative position detection device, and an evaluation device (9), wherein the system is configured to perform the following steps: - determining the relative position of the object in a first coordinate system by means of the microscope-external relative position detection device of the movable arrangement, wherein the first coordinate system is a coordinate system arranged stationary with respect to the operating room (1), - determining the relative position of the reference coordinate system in the first coordinate system by means of the microscope-external relative position detection device of the movable arrangement, - transforming the relative position of the object from the first coordinate system into the reference coordinate system of the relative position detection device of the surgical microscope (5).

Images