Medical system and method for setting a focus

Abstract

The present disclosure relates to a medical system (2) for imaging, having a digital microscope (4) having a focus (8) that can be adjusted along an optical axis (6) of the microscope (4), a camera (10) that is separate from the microscope (4) and designed to determine a distance (14) from the camera (10) to an object surface (16) by means of depth perception (12) and to provide it as depth information (18), and a control unit (20) that is designed to automatically set the focus (8) of the microscope (4) on the object surface (16) on the basis of the depth information (18), wherein the camera (10) has a greater field of view (S) and a greater depth of field (T) than the microscope (4). In addition, the present disclosure relates to a method for setting a focus (8) of a microscope (4) of a medical system (2) and to a use of a camera (10) with depth perception (12) of a medical system (2).

Description

[0001] Medical system and procedures for setting a focus

[0002] Description

[0003] Technical field

[0004] The present disclosure relates to a medical imaging system for a surgical procedure on a patient using a microscope that has a focus adjustable along an optical axis, wherein the focus of the microscope is automatically set or adjusted.

[0005] Background of the Revelation

[0006] Medical microscopes must be precisely focused to provide the user with a sharp view of the object under examination. Especially with surgical microscopes, i.e., microscopes used in surgery such as neurosurgery, precise focus and thus a sharp image are essential for accurate operation. If the distance between a patient and the microscope changes, for example, by moving the microscope or removing tissue from the patient, the focus must be readjusted.

[0007] Various methods for automatically adjusting the focus are known from the state of the art: These include active methods in which light or a laser is shone onto the object to be focused as a reference, and passive methods that work without affecting the object to be focused.

[0008] An example of passive autofocus is classic, image-based autofocus, where the sharpness of an image or recording is examined at different focus settings. Various focus points are tried to determine which one produces maximum image sharpness. This has the disadvantage that the focus needs time to stabilize, during which the image is blurry. This is particularly problematic when the distance between the microscope and the object of interest changes frequently or continuously, for example, during microscope movement, meaning the focus never reaches its optimal sharpness during the distance change. By the time a suitable focus is found for a given distance, the distance has already changed, and the image is blurry again.Therefore, this method always lags slightly behind, and a continuously sharp image is not available while the microscope is moving.

[0009] Alternatively, the distance between the microscope and the object surface to be focused can also be determined actively, for example using laser radiation. A disadvantage of active methods is that they require additional components and involve acting on and projecting the beam onto the object under investigation.

[0010] US 4,639,587 A discloses a microscope with automatic focusing that projects images as a reference onto an object under investigation and uses these images to adjust the focus.

[0011] US 4,958,920 A discloses an autofocus function of a microscope in which an active autofocus, in which light is shone onto an object to be examined and a focus signal is created based on the reflection, and a passive autofocus are combined.

[0012] US 5,288,987 A discloses a microscope in which a bar-shaped marker is projected for autofocusing.

[0013] US 5,345,334 A discloses a control system that limits the movement of a microscope to a spherical motion around an object, thus maintaining the distance and eliminating the need to readjust the focus. US 2015 / 0164329 A1 further discloses a surgical microscope with an optical unit for generating an image of an object plane in the microscope's eyepieces. DE 10 2015 117 824 A1 also discloses a device for adjusting a focus distance in an optical observation device. Finally, DE 10 2022 109 398 A1 discloses a method for accelerating focus adjustment during a Z-scan for an imaging system.

[0014] Another possibility is to adjust the focus after measuring the relative movement between the microscope and the object to be focused, using a reference configuration. If the focus was correctly set (in focus) at a specific distance to the object, the new distance and thus the new, correct focus can be determined using the measured relative movement, ensuring the object remains in focus. The disadvantage of this method is that the focus must initially be set as a reference using another method or by trial and error, and errors can propagate with many relative movements.

[0015] Brief description of the Revelation

[0016] In view of the problems described above, it is therefore a task of disclosure to overcome or at least alleviate the disadvantages immediately and / or to show an alternative for adjusting the focus of a microscope, and in particular to make it possible to adjust the focus of the microscope automatically and as quickly, easily, reliably and / or precisely as possible, especially in order to enable sharp microscope images as continuously or permanently as possible during a relative movement between the microscope and an object to be focused.

[0017] This problem is solved by a medical system according to claim 1 and a method and use according to the dependent claims. Advantageous embodiments are the subject of the dependent claims and / or described below.

[0018] The disclosed medical imaging system, in particular for surgical intervention on a patient, comprises a digital microscope with a focus adjustable along an optical axis, a camera in addition to the digital microscope (i.e., the camera is specifically not part of the microscope but is designed separately from it or has a different field of view than the microscope), which is set up by means of depth perception to determine a distance from the camera to an object surface, in particular a patient surface, and to provide this as depth information, and a control unit which is set up to automatically adjust the focus of the microscope on the object surface based on the depth information, in particular in real time, or to position / move it to the object surface.

[0019] Setting the focus on the object surface is not purely a geometric process, but rather aims to focus (as precisely as possible) on the object surface. For example, the focus can be set close to (and / or at a defined distance from) the object surface and then fine-tuned using another method or step. In other words, the goal is (ultimately) to see and / or image the object surface sharply.

[0020] In other words, a core idea of ​​the revelation is to adjust the focus of the microscope using the depth perception of an additional or separate camera (in addition to the microscope), or more generally, to use a camera with depth perception (in addition to a microscope) to adjust the focus of the microscope. The camera detects, in particular, a relative position and / or a change or shift in a relative position between the object surface and the camera. Based on the detected relative position and / or change or shift in the relative position, the focus of the microscope can be adjusted / moved accordingly; specifically, a focal point of the microscope can be shifted along the optical axis of the microscope by an amount corresponding to a distance between the focal point and the object surface or to a change in the relative position with respect to the optical axis. In particular, the focus is...The focal point is moved accordingly, taking into account a (known or recorded) position of the microscope or a (known or recorded) orientation of the optical axis, in order to achieve a desired focusing of the microscope.

[0021] The term microscope specifically includes digital microscopes and exoscopes. In other words, a microscope can also be a digital microscope and / or an exoscope.

[0022] The optical axis is understood as an axis along which the focus, in particular a focal plane and / or a focal point, can be varied. It should be understood that the optical axis of the microscope or camera can also be changed or moved, for example, when the optics are adjusted to change the field of view.

[0023] It is understood that the control unit can, for example, be included in the microscope, be designed as a separate component, or have several individual components.

[0024] Real-time focus adjustment means that the focus is set (correctly) with as little delay as possible, i.e., as close to real-time as possible. In other words, the focus can be adjusted based on the current distance between the camera and the object's surface. Of course, due to machine limitations, a certain delay or latency may occur, but this should ideally be imperceptible and negligible.

[0025] The object surface can be the surface of any object (on which the focus is to be placed), in particular the surface of a patient or the surface of an anatomical region. It is important to understand that the depth information can contain not just one precise distance, but many potentially locally varying distances (to individual points or areas of the object surface), since the distance from the camera to the object surface varies locally (depending on the surface's features, e.g., curvature or profile, and / or size). If the camera is stereoscopic, for example, the distance of each point on the object surface to the camera can be determined from the disparity (or transverse disparity / deviation) of two so-called half-images representing a specific location or area of ​​the object surface.The depth information can therefore contain the depth (coordinate away from the camera) (ideally) of each individual pixel of the camera, so that (ideally) for every point on the object surface in the camera's field of view, the distance of this point to the camera is known.

[0026] It is important to understand that the camera should ideally update the depth information in real time or at a high frequency / sampling rate, essentially continuously and consistently. It should also be able to determine and provide the distance between the camera and the object surface, ensuring that the instantaneous distance between the camera and the object surface is always available in real time. This allows the microscope's focus to be adjusted as continuously and in real time as possible, without any blurring or noticeable blurring of the object surface to the user. Therefore, the focus can be described as continuously sharp, and the process can be termed real-time, continuous autofocus.

[0027] The advantages of this revelation lie in the fact that the microscope's focus can be easily and automatically adjusted. The focus is set based on the depth perception of the additional camera. This is a comparatively cost-effective solution. The focus can be adjusted passively without illuminating the object surface, for example, with a laser. Furthermore, the focus can be readjusted at any time without requiring an initial reference configuration or risking error propagation. Additionally, the microscope's focus can be set quickly and without extensive trial and error, approximation, or stabilization. This enables (nearly) continuous sharpness and real-time focus adjustment in response to relative movements between the microscope and the object surface. Moreover, the additional camera can also be used to monitor the object surface and, preferably, an area larger than the microscope's field of view, for example.to provide a better overview. This means the camera can also be used for other optical applications besides just focusing the microscope.

[0028] The microscope is a digital microscope (an image is captured and made available digitally, specifically displayed on a screen rather than viewed through eyepieces), and preferably an exoscope. It can also be a hybrid microscope, which is at least partly digital.

[0029] Preferably, the microscope is a surgical microscope or an operating microscope, or in other words, designed, equipped and / or suitable for surgery or operating.

[0030] The camera has a wider field of view than the microscope. Consequently, the camera has a different, and in particular wider, angle of view than the microscope. Therefore, the camera can capture a larger area of ​​the object's surface (or even the entire object's surface) compared to the microscope. As a result, the camera can have a lower resolution.

[0031] The camera has a greater depth of field than the microscope. This allows the camera to focus sharply on objects over a larger depth range, such as the object's surface, without needing to adjust the focus if the surface is within that depth range and / or moving. If, for example, the distance between the microscope and the camera changes, it may be necessary to readjust the microscope's focus to avoid blurring, whereas the camera's focus may not require adjustment or may even be fixed. Therefore, it can be easier to adjust the microscope's focus using the camera's greater depth of field, as it is more likely to already have the object's surface in focus and may require further adjustments.It is easier to adjust the focus appropriately due to the greater depth of field.

[0032] Preferably, the camera has a fixed focus. The camera's focus is therefore preferably fixed, unchangeable, or preset. In other words, the camera is preferably designed to always focus sharply on objects at the same specific distance or range of distances. The camera does not need to have any moving lenses.

[0033] Preferably, the camera and the microscope are calibrated relative to each other and preferably also rigidly arranged relative to each other (so that a relative transformation between the camera and the microscope is known). Thus, the distance from the camera to the object surface, determined by the camera's depth perception, can be transferred to a coordinate system of the microscope and / or used to calculate the distance or deviation of the microscope's focus from the object surface along the optical axis and to adjust the focus accordingly. In particular, the focus can be adjusted by shifting it by this distance and setting it so that the object surface is then in focus of the microscope.A rigid arrangement is understood to mean a fixed relative position between the camera and the microscope, such that a transformation is known, whereby optical elements such as the optical axis or lenses can also move independently, and their relative position or transformation can be determined by calibration. It should be understood that the microscope and camera do not necessarily have to be rigidly arranged relative to each other; for example, they can also be movable relative to each other and tracked to determine the relative position of the camera to the microscope. Preferably, the optical axis of the microscope (along which the focus of the microscope can be set or adjusted) is at least partially within the camera's field of view. In other words, preferably at least a part, preferably the entire optical axis of the microscope, or possibly only a small part of the optical axis near the microscope, is within the camera's field of view.of the initial part of the optical axis.

[0034] Preferably, at least part of the microscope's optical axis lies within the camera's field of view where the object surface is located or may be located. In other words, the microscope is preferably positioned and / or moved relative to the object surface such that the object surface is at a depth to the microscope where the microscope's optical axis is within the camera's field of view. Thus, depending on the object surface, the camera can then potentially see the point on the object surface where the optical axis intersects it.

[0035] Preferably, the control unit is configured to calculate or determine the distance of a focus point of the microscope to the object surface along the optical axis of the microscope, based on the depth information from the camera. This means that the control unit is specifically configured to calculate the distance between the camera and an intersection of the microscope's optical axis with the object surface and to provide this distance as depth information. A focus point is understood to be a point that is in focus, i.e., sharply imaged, and preferably lies on the optical axis. Therefore, if such a distance or deviation exists, the object surface (a point on the object surface along the optical axis distanced from the focus point) would not be rendered sharply.

[0036] It is important to understand that the term focal plane refers to an area, typically perpendicular to the optical axis, in which an object located therein is sharply focused, even though the focal plane can be practically a line or a point. Particularly in stereoscopic cameras and microscopes, the optical axis of the camera or microscope, usually as a center line, typically results from the individual optical axes. Likewise, the focal plane (as a cross-section) typically results from the individual focal planes. Thus, strictly speaking, the focal plane can be a point or a line, but it is still referred to as the focal plane. In other words, the term focal plane is functional and not purely geometric.

[0037] In other words, the control unit is preferably configured to calculate the distance from a focal point towards the object surface along the optical axis. Based on this distance information along (i.e., on or parallel to) the optical axis, the focus can be set particularly easily. In other words, the control unit can be configured to calculate the distance from a point in the focal plane, or a focal point, to the object surface in the direction of the optical axis.

[0038] Preferably, the control unit is configured to control the microscope in such a way that the focal point is shifted by this calculated distance along the optical axis relative to the object surface. In other words, the control unit is configured to place the focal point on the object surface using the calculated distance (so that the object surface, or at least a point that now coincides with the focal point, is rendered sharply).

[0039] In other words, the control unit can preferably adjust the focus of the microscope so that the focus is varied by the calculated distance, and thus brought into sharp focus.

[0040] Typically and preferably, the focal point is the center of the microscope image or the center point of a microscope image. This has the advantage that a user typically positions, or wants to position, the microscope so that an area of ​​interest is usually as close to the center as possible, and thus in focus. If the intersection of the microscope's optical axis and the object surface is not within the camera's field of view—that is, if the camera cannot see the point on the object surface that is to be focused—the distance from the camera to a point on the object surface that is closest to the optical axis and still within the camera's field of view can be determined and provided as depth information to adjust the microscope's focus.

[0041] Preferably, the camera is stereoscopic. Depth information is preferably obtained by passive 3D reconstruction based on / from the stereoscopy. In other words, depth information is preferably derived (similar to human vision) from the disparity between several, at least two, images. However, the disclosure is not limited to this; the depth or depth information can also be obtained by active stereoscopy, for example, using a structured-light method in which light is projected onto the object surface as a reference for the disparity, or, for example, using a time-of-flight method in which the time it takes for an emitted light beam to return to the camera is measured.

[0042] Preferably, the control unit is further configured to fine-tune the microscope's focus based on the microscope's own depth perception and / or an image taken by the microscope (preferably in addition to and after focusing via the camera's depth perception). In other words, the microscope's focus can preferably be fine-tuned without requiring the camera, for example, by means of a classic, image-based autofocus (trying out different focuses or shifting the focal planes and evaluating the resulting sharpness), as described above, or a method for depth perception or obtaining depth information, as already mentioned above (regarding the camera), such as stereoscopy, time-of-flight, etc.This can be particularly advantageous if the intersection of the microscope's optical axis and / or focal plane with the object surface is not within the camera's field of view, and the focus needs to be set on this intersection point. In this case, the focus can be roughly set using the camera's depth perception, but then fine-tuned to this intersection point using the microscope.

[0043] Preferably, the medical system includes a robot with a flange. Preferably, the camera and the microscope are each, both, or both together attached to the end-effector end of the robot's flange. A (robot) flange is typically defined as the final section or end piece (away from the robot's base) that serves as the connection point for an end effector. In other words, preferably, the camera and the microscope are both attached to the robot or a robot arm as end effectors. Preferably, the camera and the microscope are both attached to a common, preferably the only, robot arm as end effectors. It should be understood that the camera and the microscope themselves do not necessarily have to be directly connected to or mounted on the flange.Preferably, the microscope and the camera are integrated into a common head or (end-effector) housing, which is mounted to the flange. Preferably, the robot is configured to position and / or move the camera and the microscope. This allows for easy positioning and also has the advantage that the camera and microscope cannot move relative to each other or independently.

[0044] Preferably, the control unit is configured to adjust the microscope's focus (based on depth information from the camera) only upon receiving a trigger command. In other words, the control unit can be configured to adjust the focus only after receiving a trigger command. This has the advantage that movement of an instrument or, for example, a user's hand in front of the object surface does not cause the focus to be incorrectly shifted. Such a trigger command could be, for example, user input or a movement of the robot. Preferably, the control unit is configured to adjust the microscope's focus based on depth information from the camera, depending on robot movement. It is particularly preferred that the control unit be configured to automatically adjust the focus when the robot is moving.Triggered by a movement of the robot and / or continuously during a movement of the robot, the focus of the microscope is adjusted based on depth information. This has the advantage that the focus can be adapted to a changing distance of the microscope to the object surface due to the movement and / or can also be appropriately adjusted during the movement to enable a sharp image of the object surface continuously during the movement.

[0045] Alternatively or additionally, the control unit can be set up to receive an input command from a user and adjust the focus of the microscope in response to the input command.

[0046] In particular, when the camera and the microscope are each, or both, mounted on the end-effector side of the robot's flange, as described above, the camera is preferably calibrated to the flange (so that a relative pose or transformation between the camera and the flange is known), the microscope is calibrated to the flange (so that a relative pose or transformation between the microscope and the flange is known), and the focus of the microscope is calibrated (so that a relative pose or transformation between the microscope and the focal plane or focal point is known). It should be understood that the focus depends on the magnification; that is, if, for example, the microscope zooms in closer to something, the focus also changes. To know and determine this relationship, the focus of the microscope can be calibrated. The calibrations mentioned above can initially be performed and recorded once.

[0047] Calibrations between the flange and the microscope, as well as between the flange and the camera, can each be performed, for example, using hand-eye calibration. Calibration of the microscope's focus can be achieved through optical (internal) calibration of the microscope.

[0048] Alternatively or additionally, the camera and the microscope can also be calibrated (directly) to each other, for example by means of an eye-eye calibration, so that a transformation between camera and microscope (without going through e.g. the flange) is known.

[0049] Furthermore, the disclosure relates to a method for adjusting the focus of a microscope of a medical system, preferably as described above, comprising the steps:

[0050] -Determining and providing a distance from the camera to an object surface, especially in real time, as depth information using the camera's depth perception, and

[0051] -Adjusting the focus of the microscope based on the depth information on the object surface.

[0052] Setting the focus on the object surface is not purely geometric, but rather the goal of focusing (as precisely as possible) on the object surface. For example, the focus can be set close to (and / or at a defined distance from) the object surface and then fine-tuned using another method or step. In other words, the goal is to (ultimately) see and / or image the object surface sharply.

[0053] Preferably, the method further includes the step:

[0054] - Fine-tuning of the microscope's focus based on the microscope's own depth perception and / or a microscope image, for example using classic, image-based autofocus (trying out different focuses or shifting the focus planes and evaluating the achieved sharpness), as described above, or a method for depth perception or for obtaining depth information, as already mentioned above (regarding the camera), for example stereoscopy, time-of-flight, etc.

[0055] Preferably, the step "Adjusting the focus of the microscope based on the depth information" includes the following:

[0056] -Calculate, based on the depth information, a distance of a focal point of the microscope to the object surface along the optical axis of the microscope, and

[0057] -Controlling the microscope in such a way that the focal point is shifted by this calculated distance.

[0058] Furthermore, the disclosure relates to the use of a camera with depth perception, particularly in a medical system, preferably as described above, for determining a distance from the camera to an object surface, particularly a patient surface, and for providing the distance as depth information in order to focus a separate digital microscope, particularly as described above, particularly the medical system, on the object surface along an optical axis of the microscope (automatically by a control unit), based on the depth information, particularly in real time. In particular, the camera can have a larger field of view and a greater depth of field than the microscope, as already described above.

[0059] Setting the focus on the object surface is not to be understood in a purely geometric sense, but rather as the goal of setting the focus (as precisely as possible) on the object surface. For example, the focus can also be set close to (and / or at a defined distance from) the object surface and then fine-tuned using or in another method or step. In other words, the goal is (ultimately) to see and / or image the object surface sharply. According to a potentially independent aspect, the disclosure further relates to a computer-readable storage medium that has functions which cause a medical system (in particular a control unit of the medical system), preferably as described above, to execute at least one or more steps of a method for setting the focus of a microscope of the medical system, preferably as described above.

[0060] According to an aspect that may be claimed independently, the disclosure further relates to a medical system, preferably as described above, wherein the control unit is configured to perform at least one or the steps of the method for adjusting the focus of a microscope of a medical system, as described above, and / or to control a camera and / or a microscope of the medical system, preferably as described above, in such a way as to perform at least one or the steps of the method.

[0061] Brief description of the characters

[0062] The disclosure is explained in more detail below with reference to preferred embodiments and the accompanying figures.

[0063] Fig. 1 shows a medical system according to the present disclosure;

[0064] Fig. 2 shows transformations of the medical system from Fig. 1 that can be obtained by means of calibration;

[0065] Fig. 3 shows a method according to the present disclosure;

[0066] Fig. 4 shows a medical system according to the present disclosure. The figures are schematic and serve only to illustrate the disclosure. The features of the different embodiments can be interchanged.

[0067] Detailed description of preferred embodiments

[0068] Fig. 1 shows a medical imaging system 2, in particular for a surgical procedure E on a patient P, comprising a microscope 4 with a focus 8 adjustable along an optical axis 6, a camera 10 in addition to the microscope 4, which is equipped by means of a depth sensing 12 to determine a distance 14 from the camera 10 to an object surface 16, in particular a patient surface 16P, and to provide it as depth information 18, and a control unit 20, which is equipped to adjust the focus 8 of the microscope 4 on the object surface 16 based on the depth information 18, in particular in real time.

[0069] Furthermore, camera 10 has a larger field of view S and a greater depth of field T than microscope 4. The fields of view of camera 10 and microscope 4 are indicated by dashed lines.

[0070] Here (i.e., in the preferred embodiment), the camera 10 and the microscope 4 are arranged and / or can be arranged relative to each other such that at least a part of the optical axis 6 (shown as a dashed line) lies within the field of view S of the camera 10. The optical axis 6 of the microscope 4 lies within the field of view S of the camera 10, except for an initial portion (due to the design or a distance between the camera 10 and the microscope 4), or in other words, from a certain distance from the microscope 4.

[0071] Here, the control unit 20 is set up to calculate, based on the depth information 18, a distance 22 of a focus point FP of the microscope 4 to the object surface 16 along the optical axis 6 of the microscope 4 and to control the microscope 4 in such a way that the focus point FP is moved by this calculated distance 22 along the optical axis 6 to the object surface 16.

[0072] To this end, the camera 10 or the control unit 20 determines, if possible, the distance 14 between the camera 10 and the intersection point SP of the optical axis 6 of the microscope 4 with the object surface 16 and provides this as depth information 18. The intersection point SP is the target focus point ZP, i.e., the point to which the focus point FP should ideally be moved. If the camera 10 cannot see this intersection point SP, for example, because it is obscured behind a raised area on the object surface, the distance 14 of the nearest point on the object surface 16 to the camera 10 that is within the camera 10's field of view S can be provided as depth information 18, and the focus 8 of the microscope 4 (by the control unit 20) can be set based on this. In other words, if the intersection point SP or target focus point ZP is not visible to the camera 10, the distance 14 of another point or...The control unit can provide a different point on the object surface 16 to adjust the focus 8 and at least approach the target focus point ZP. Alternatively, if the intersection point SP is not visible to the camera 10, the control unit can shift the focus point FP along the optical axis 6 towards the object surface 16 until the shifted or new focus point FP is just visible to the camera 10, thus bringing the focus point FP closer to the target focus point ZP.

[0073] Here, camera 10 is stereoscopic for depth perception 12. The depth (dimension away from camera 10) is determined here by means of two simultaneous images (so-called half-images) and the disparity between them.

[0074] Here, the microscope 4 is stereoscopic and thus possesses its own depth perception 24, in which the depth (dimension away from the microscope 4) is also determined by means of two images (so-called half-images) and the disparity between them. The control unit 20 is configured here to fine-tune the focus 8 of the microscope 4 based on this own depth perception 24 of the microscope 4. Additionally or alternatively, the control unit 20 can be configured to fine-tune the focus 8 of the microscope 4 based on an image A of the microscope 4. In doing so, the control unit 20 determines the correct / best focus 8 or position of the focus point FP by evaluating the sharpness of image A for various foci 8 and selecting the focus at which image A is sharpest.

[0075] The medical system 2 preferably includes a robot 28 with a flange 26. The camera 10 and the microscope 4 are both attached to the end effector side of the flange 26 of the robot 28. In other words, the camera 10 and the microscope 4 are attached to the robot 28 as end effectors. Here, the robot 28 has exactly one robot arm 30, at the end of which (distance from a base of the robot 28) is the flange 26, to which the camera 10 and microscope 4, here in a common housing 32, are attached. The housing 32 is mounted to the flange 26.

[0076] The control unit 20 is configured here to adjust the focus 8 of the microscope 4 based on the depth information 18 from the camera 10, depending on a movement B of the robot 28. Specifically, the control unit 20 is configured to adjust the focus 8 of the microscope 4 preferably triggered by a movement B of the robot 28. In other words, a robot movement B triggers an adjustment of the focus 8 of the microscope 4. Furthermore, the control unit 20 is configured here to continuously adjust the focus 8 of the microscope 4 during a movement B of the robot 28. Thus, the image from the microscope remains sharp, even during the movement of the microscope. It should be understood that the movement B of the robot 28 refers to a robot movement B in which the pose of the microscope 4 (relative to the object surface 16) is changed. Fig. 2 shows the medical system 2 from Fig.1 different transformations T between the flange 26, the microscope 4, the camera 10 and the focus (point) of the microscope 4.

[0077] Here, the camera 10 is calibrated to the flange 26 of the robot 28, as is the microscope 4, which is calibrated to the flange 26 of the robot 28. The transformation FTK between camera 10 and flange 26 is known through the calibration of the camera 10 to the flange 26. The transformation FTM between microscope 4 and flange 26 is known through the calibration of the microscope 4 to the flange 26. Furthermore, the stereoscopic microscope 4 is internally calibrated, so the transformation MkiTMk2 between the two microscope cameras 34 of the microscope 4 is known. It should be understood that in this embodiment, the camera 10 is also stereoscopic and has two cameras, which, however, are not shown individually in Fig. 1 for the sake of simplicity. Furthermore, the focus 8 or focus point FP of the microscope 4 is calibrated, so the transformation MTFP between the microscope and the focus point FP is known. Thus, the transformations can be used to determine coordinates.The purpose is to convert poses from a camera coordinate system, a microscope coordinate system, and a flange coordinate system (coordinate systems of the respective components) into one of the other coordinate systems. It should be understood that the transformations are preferably transformation matrices, but the disclosure is not limited to this; for example, quaternions are also possible.

[0078] Thus, the distance 14 measured by the camera 10 from the camera coordinate system can be used to adjust the focus 8 of the microscope 4 on the object surface 16 by minimizing the distance 22 of the focus point FP to the object surface 16. For this purpose, the (instantaneous) distance 22 of the focus point FP to the object surface 16 is determined using the described transformations and the distance 14 between the camera 10 and the object surface 16. Fig. 3 shows a method for adjusting a focus 8 of a microscope 4 of a medical system 2 as disclosed herein, for example, the medical system 2 from Fig. 1, comprising the steps:

[0079] -S1: Determining and providing a distance 14 from the camera 10 to an object surface 16, especially in real time, as depth information 18 using the depth perception 12 of the camera 10,

[0080] -S2: Setting the focus 8 of the microscope 4 based on the depth information 18 on the object surface 16.

[0081] The procedure also includes the optional step S3:

[0082] -S3: Fine adjustment of the focus 8 of the microscope 4 based on its own depth perception 24 of the microscope 4 and / or a recording A of the microscope 4.

[0083] Fig. 4 shows an embodiment of a medical system 2 according to the present disclosure. The microscope 4 is a surgical operating microscope 4 and a digital microscope 4 configured to create and provide a digital image A. The camera 10 is also configured to create and provide a digital image A. The medical system 2 has a display 36 configured to display digital images A from the microscope 4 and the camera 10, and preferably further images and / or videos in 2D and / or 3D (for example, pre- or intraoperative 3D recording data of a patient P). Thus, a user, for example, a surgeon, can see the microscope image, which is focused on the object surface 16 by means of the camera 10 in the manner described in this disclosure. The microscope 4 is, as shown in Fig. 4, an exoscope.

[0084] In this case, the camera 10 is externally attached to a housing 32 of the microscope 4. Furthermore, a computer-readable storage medium 38 is shown, which has functions that cause the medical system 2 (in particular the control unit 20) to execute the procedure from Fig. 3.

[0085] List of reference symbols

[0086] 2 medical system

[0087] Patient

[0088] A surgical procedure

[0089] 4 Microscope

[0090] 6 optical axis

[0091] 8 Focus

[0092] 10 Camera

[0093] 12 Depth perception camera

[0094] 14 Distance between camera and object surface

[0095] 16 Object surface

[0096] 16P Patient surface

[0097] 18 In-depth information

[0098] 20 control unit

[0099] 5 field of vision

[0100] Depth of field

[0101] 22 Distance between the focus point of the microscope and the object surface

[0102] FP Focus Point

[0103] 24 Depth perception microscope

[0104] SP Intersection of optical axis of microscope with object surface

[0105] ZP Target Focus Point

[0106] A recording

[0107] 26 flange

[0108] 28 robots

[0109] 30 robot arms

[0110] 32 cases

[0111] B Robot movement

[0112] 34 Microscope camera

[0113] 36 ads

[0114] 38 computer-readable storage medium

Claims

Claims 1. Medical system (2) for imaging, in particular for a surgical procedure (E) on a patient (P), comprising a digital microscope (4) with a focus (8) adjustable along an optical axis (6) of the microscope (4), a camera (10) separate from the microscope (4) which is equipped by means of a depth sensing (12) to determine a distance (14) from the camera (10) to an object surface (16), in particular a patient surface (16P), and to provide it as depth information (18), and a control unit (20) which is equipped to automatically adjust the focus (8) of the microscope (4) on the object surface (16) based on the depth information (18), in particular in real time, wherein the camera (10) has a larger field of view (S) and a greater depth of field (T) than the microscope (4).

2. Medical system (2) according to claim 1 , characterized in that the camera (10) has a fixed focus.

3. Medical system (2) according to claim 1 or 2, characterized in that the optical axis (6) of the microscope (4) lies at least partially in the field of view (S) of the camera (10).

4. Medical system (2) according to one of claims 1 to 3, characterized in that the control unit (20) is configured to calculate a distance (22) of a focal point (FP) of the microscope (4) to the object surface (16) along the optical axis (6) of the microscope (4) based on the depth information (18) and to control the microscope (4) such that the focal point (FP) is moved by this calculated distance (22) along the optical axis (6) to the object surface (16).

5. Medical system (2) according to one of the preceding claims, characterized in that the camera (10) for depth perception (12) is stereoscopic.

6. Medical system (2) according to one of the preceding claims, characterized in that the control unit (20) is further configured to finely adjust the focus (8) of the microscope (4) based on its own depth perception (24) of the microscope (4) and / or a recording (A) of the microscope (4), in particular when an intersection point of the optical axis (6) of the microscope (4) with the object surface (16) is not in the field of view (S) of the camera (10).

7. Medical system (2) according to one of the preceding claims, characterized in that the system (2) comprises a robot (28) with a flange (26), wherein the camera (10) and the microscope (4) are each attached to the flange (26) on the end effector side.

8. Medical system (2) according to claim 7, characterized in that the control unit (20) is configured to adjust the focus (8) of the microscope (4) based on the depth information (18) from the camera (10) depending on a movement (B) of the robot (28), preferably triggered by a movement (B) of the robot (28) and / or continuously during a movement (B) of the robot (28).

9. Medical system (2) according to claim 7 or 8, characterized in that the camera (10) is calibrated to the flange (26) of the robot (28), the microscope (4) to the flange (26) of the robot (28), and the focus (8) of the microscope (4).

10. Method for adjusting a focus (8) of a microscope (4) of a medical system (2) according to any one of claims 1 to 9, comprising the steps: -Determining and providing a distance (14) from the camera (10) to an object surface (16), especially in real time, as depth information (18) using the depth perception (12) of the camera (10), -Adjusting the focus (8) of the microscope (4) based on the depth information (18) on the object surface (16).

11. Method according to claim 10, further characterized by the step: - Fine adjustment of the focus (8) of the microscope (4) based on its own depth perception (12) of the microscope (4) and / or a recording (A) of the microscope (4).

12. Use of a camera (10) with depth perception (12), in particular a medical system (2) according to any one of claims 1 to 9, for determining a distance (14) from the camera (10) to an object surface (16), in particular a patient surface (16P), and for providing the distance (14) as depth information (18) in order to set a focus (8) of a separate digital microscope (4) on the object surface (16) based on the depth information (18), in particular in real time, along an optical axis (6) of the microscope (4), wherein the camera (10) has a larger field of view (S) and a greater depth of field (T) than the microscope (4).

Images