Method for calibrating a stereoscopic medical microscope, and medical microscope arrangement

EP4469753A1Pending Publication Date: 2024-12-04CARL ZEISS MEDITEC AG
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
EP2023702401
Authority / Receiving Office
EP · EP
Patent Type
Applications
Current Assignee / Owner
Priority Date
2022-01-25
Filing Date
2023-01-24
Publication Date
2024-12-04

AI Technical Summary

Technical Problem

Current methods for calibrating stereoscopic medical microscopes do not achieve optimal three-dimensional image visualization and overlay during surgeries, leading to potential disruptions and fatigue due to poor image superimposition and errors.

Method used

A method combining three-dimensional and two-dimensional calibrations using specific calibration objects, where images from both cameras are linked and calibration data is generated and stored for correction, improving overall calibration and image correction during surgeries.

Benefits of technology

This approach enhances image superimposition and reduces disruptions by providing corrected images to surgeons, improving information transfer and workflow efficiency during operations.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 1.1
    Figure 1.1
Patent Text Reader

Abstract

The invention relates to a method for calibrating a stereoscopic medical microscope (1), comprising: a) capturing images (20) of at least one three-dimensional calibration object (40) using cameras (3l, 3r) of a stereo camera system (3) of the medical microscope (1); b) generating calibration data (30) on the basis of the captured images (20), the generated calibration data (30) being stored for correction; c) capturing further images (21) of at least one two-dimensional calibration object (41) using the cameras (3l, 3r) of the stereo camera system (3); d) generating further calibration data (31) on the basis of the captured further images (21), the further calibration data (31) being stored for correction. The invention further relates to a medical microscope arrangement (100).
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Method for calibrating a stereoscopic medical microscope and medical microscope arrangement

[0002] The invention relates to a method for calibrating a stereoscopic medical microscope and a medical microscope arrangement.

[0003] The goal of digital visualization in surgery and microsurgery is to provide a surgeon with an optimal three-dimensional image of the surgical field during an operation, as well as, if necessary, an optimal three-dimensional overlay of additionally generated information onto the field of view on the medical microscope monitors. To achieve this, captured camera images are processed using digital image processing with defined calibration data.

[0004] Methods for the extrinsic and intrinsic calibration of cameras combined with the generation of distortion fields are known, for example, from Reg G. Willson, Modeling and Calibration of Automated Zoom Lenses, PhD dissertation, The Robotics Institute, Carnegie Mellon University, Pittsburgh, Pennsylvania, USA, 1994. Furthermore, methods for calibrating multiple operating point settings (zoom and focus) with the aim of "stereo augmented reality" are known from AP King et al., Stereo Augmented Reality in the Surgical Microscope, Presence, Vol. 9, No. 4, August 2000, 360-368, Massachusetts Institute of Technology.

[0005] Three-dimensional calibration objects are known from DE 102019 131 646 A1. These three-dimensional calibration objects comprise, for example, a transparent body and calibration marks embedded in the volume of the transparent body.

[0006] The invention is based on the object of improving a method for calibrating a stereoscopic medical microscope and a medical microscope arrangement.

[0007] The object is achieved according to the invention by a method having the features of patent claim 1 and a medical microscope arrangement having the features of patent claim 11. Advantageous embodiments of the invention emerge from the subclaims.

[0008] A fundamental concept of the invention is to link a three-dimensional calibration with a two-dimensional calibration and thereby achieve an overall improved, in particular optimal, calibration result. This is achieved by acquiring images of at least one three-dimensional calibration object using cameras of a stereo camera system of the medical microscope. In particular, each of the cameras of the stereo camera system acquires at least one image. The images are acquired in particular such that images from the two cameras of the stereo camera system for the same operating points of the medical microscope are linked to one another, i.e., in particular, are assigned to one another or can be assigned to one another. Based on the acquired images, calibration data is generated and stored for correction.Furthermore, further images of at least one two-dimensional calibration object are acquired using the cameras of the stereo camera system. These further images are also acquired in particular in such a way that the further images of the two cameras of the stereo camera system are linked with one another for the same operating points, i.e., in particular, they are assigned to one another or can be assigned to one another. The images and the further images are acquired in particular at the same operating points. Based on the acquired further images, further calibration data is generated and stored for correction. Storing the calibration data here particularly comprises storing the calibration data in a memory provided for this purpose so that it can be retrieved and applied from this memory as needed.The stored calibration data and the stored additional calibration data can be applied to images (and sequences of images, in particular videos) subsequently acquired, in particular during an operation, so that these images can be corrected by the calibration data and additional calibration data and can then be acquired by a surgeon and / or other persons.

[0009] In particular, a method for calibrating a stereoscopic medical microscope is provided, comprising: a) capturing images of at least one three-dimensional calibration object by means of cameras of a stereo camera system of the medical microscope, b) generating calibration data based on the captured images, wherein the generated calibration data are stored for correction, c) capturing further images of at least one two-dimensional calibration object by means of the cameras of the stereo camera system, d) generating further calibration data based on the captured further images, wherein the further calibration data are stored for correction.

[0010] Furthermore, in particular, a medical microscope arrangement is created, comprising a stereoscopic medical microscope with a stereo camera system comprising cameras and stored calibration data, and a data processing device, wherein the data processing device is configured to generate calibration data based on images of at least one three-dimensional calibration object captured by means of the cameras of the stereo camera system and to store them for correction, and to generate further calibration data based on further images of at least one two-dimensional calibration object captured by means of the cameras of the stereo camera system and to store them for correction.

[0011] An advantage of the method and the medical microscope arrangement is that a holistic calibration of the stereoscopic medical microscope can be achieved through a combination of three-dimensional and two-dimensional calibration and associated two-dimensional and three-dimensional calibration objects. After the holistic calibration data has been generated, corrected images can be provided to a surgeon and / or other persons using this calibration data. An overlay for a stereoscopic viewing of these corrected images is improved compared to the uncorrected images, so that the transfer of information during an operation is also improved and disruptions caused by inadequate overlay and / or image errors, as well as the associated impairments in the workflow during an operation and fatigue, can be reduced.

[0012] The calibration data and the additional calibration data are generated, in particular, by evaluating the acquired images and additional acquired images. In particular, features on the respective calibration objects are identified and evaluated. Since the calibration objects have known properties, the calibration data and the additional calibration data can be generated, in particular determined, based on the known properties and the evaluated acquired images and additional images. In particular, the respective features and their properties, in particular a location (position and orientation), geometric arrangement, shape, color, brightness, etc., are known and can therefore be identified in the images and additional images.By comparing the known target properties of the respective features in the images and other images with the actual properties, the calibration data and the additional calibration data can be determined in a conventional manner using known calibration methods. The evaluation is carried out, in particular, by means of the data processing device. In particular, known methods of computer vision, pattern recognition, and / or machine learning can be used for this purpose.

[0013] A medical microscope is, in particular, a surgical microscope. However, a medical microscope can also be a microscope used for medical examinations and / or diagnostic purposes, for example, in ophthalmology or other fields. A medical microscope arrangement is, in particular, a surgical microscope arrangement.

[0014] Calibration data and additional calibration data can, in principle, be of the same or different types and relate to different properties and / or effects and / or disturbances and / or features of the medical microscope. The distinction between calibration data and additional calibration data was chosen primarily for linguistic purposes.

[0015] A two-dimensional calibration object can have known shapes and patterns, for example a two-dimensional checkerboard pattern, line markings, crosshairs and the like. A three-dimensional calibration object has, in particular, three-dimensional structures. In particular, a three-dimensional calibration object can have structures that are arranged in several planes perpendicular to this axis with respect to an optical axis of the imaging system of the medical microscope and can be captured, for example, by irradiation and / or by defined intrinsic illumination with the medical microscope or by means of the cameras. For example, these can be high-contrast markings arranged in two planes perpendicular to one another in order to create depth information with respect to the optical axis (see, for example, Fig. 2 in King et al.).Three-dimensional calibration objects are also described, for example, in DE 102019 131 646 A1. A three-dimensional calibration object therefore has, for example, a transparent body and calibration marks embedded in the volume of the transparent body. A three-dimensional calibration object in one of the embodiments described in DE 102019 131 646 A1 is essentially a cube-shaped 3D calibration body constructed from transparent layers. The 3D calibration body is constructed from stacked light guides. The light guides are formed by alternating transparent layers, wherein the layers have a higher refractive index than layers arranged between them. Light is coupled into the layers by means of selectively switchable light sources in such a way that it is totally reflected at the interfaces between the layers. Due to the total reflection at the interfaces between the layers, light propagation occurs, i.e.a propagation of the electromagnetic waves of light, only within the respective light guides. In order to be able to represent calibration marks in the light guides of the 3D calibration body, thin films are applied to the layers at specific intervals. The refractive index of these films is selected such that total internal reflection is suppressed at these points, allowing light to propagate into the layers with a low refractive index. When light is coupled into one of the light guides, the light is coupled out of the light guide at the points where the films are applied, creating luminous spots in the volume of the transparent body made up of the layers that serve as calibration marks. Depending on which light source is switched on, the luminous spots appear at different depths of the transparent body.

[0016] It can also be provided that at least one two-dimensional calibration object and at least one three-dimensional calibration object are combined to form a combined calibration object. The procedure is then fundamentally the same.

[0017] Parts of the medical microscope arrangement, in particular the data processing device, can be implemented individually or collectively as a combination of hardware and software, for example as program code executed on a microcontroller or microprocessor. For example, the data processing device can comprise a computing device in the form of a microprocessor or microcontroller and a memory. However, it can also be provided that parts are implemented individually or collectively as an application-specific integrated circuit (ASIC) and / or a field-programmable gate array (FPGA). The data processing device can also be part of the medical microscope. In principle, however, the data processing device can also be implemented separately from the medical microscope, for example as a desktop, laptop, or tablet computer, or even as a cloud-based solution.

[0018] In one embodiment, it is provided that step a) and / or step c) are carried out at different operating points of the medical microscope, wherein the calibration data in step b) and / or the further calibration data in step d) are generated for each of the different operating points. This allows calibration data to be generated over a larger, in particular over an entire, working and / or operating range of the medical microscope. An operating point here comprises in particular one or more of the following parameters: a position (position and / or orientation) of the camera(s) and / or a magnification (zoom) and / or a working distance (focus or a position of the focusing lens) and / or a position of an aperture and / or a value of an aperture (aperture diaphragm) and / or the presence of a drape lens on the microscope (yes / no).It can be provided that calibration data and additional calibration data for operating points for which no images and / or additional images have been acquired are generated from calibration data and / or additional calibration data from adjacent operating points, in particular by interpolation and / or extrapolation. It can also be provided that calibration data for such operating points are estimated using a function that is adapted to the calibration data generated from the measured operating points.

[0019] In one embodiment, it is provided that the calibration data generated in step b) and / or the further calibration data generated in step d) are applied at least partially to the acquired images and / or the acquired further images for correction, wherein step b) is carried out and / or repeated for the corrected images and / or step d) is carried out and / or repeated for the corrected further images. As a result, the calibration data and / or the further calibration data can be optimized step by step, since already generated calibration data and / or further calibration data are applied to the acquired images and / or the acquired further images. In particular, this allows several different effects that lead to interference between the images and / or the further images and that can influence one another to be taken into account.For example, a distortion error in an image captured by a left camera of the stereo camera system can influence an offset between the images captured by the left and right cameras of the stereo camera system, and vice versa. If the distortion error is corrected before determining the offset, the effect of the distortion error can be minimized or even eliminated when determining the offset.

[0020] In particular, it is provided that the calibration data generated in step b) are first applied, at least in part, to the additional acquired images for correction before step d) is executed. This allows step d) to be performed with additional acquired images already corrected by the calibration data. In particular, the effect of mutually influencing interference can be reduced.

[0021] In a further embodiment, it is provided that the correction and steps c) and d) are repeated until at least one predefined optimization criterion is met. This allows the further calibration data to be improved step by step. The at least one predefined optimization criterion is, for example, a predefined maximum value (e.g., + / -1 pixel) for an (x / y) offset between the captured further images of the two cameras of the stereo camera system and / or a predefined maximum value (e.g., + / -0.001° with respect to an image center) for a rotation between the captured further images of the two cameras of the stereo camera system.

[0022] If the steps of the method are carried out in the order c), d), a), b), it is also possible to correct the images acquired in step a) using the further calibration data generated in step d) before step b) is carried out. The procedure is fundamentally analogous to the procedure described above. In one embodiment, it is provided that the generated calibration data are checked by at least partially applying them to the acquired images and / or the acquired further images and evaluating a result of the application, wherein steps a) to d) are at least partially repeated if an evaluation result does not meet at least one predetermined criterion. An evaluation result can, for example, comprise values ​​for deviations that are determined for selected variables.The selected variables can, for example, include one or more of the following variables: an offset, a rotation, a difference in brightness, a distortion, etc. A predefined criterion includes, for example, predefined limit values ​​for the respective evaluation results, i.e., in particular, maximum permissible values ​​for the respective deviations. It can also be provided that individual evaluation results for the selected variables are aggregated into a single evaluation result and compared with the at least one criterion (e.g., with a maximum value for an aggregated deviation).

[0023] In one embodiment, it is provided that step b) and / or d) comprises one or more of the following measures: determining extrinsic calibration data, determining intrinsic calibration data, determining directory correction field data, determining edge drop-off correction data, determining chromatic shift field correction data.

[0024] For the extrinsic calibration data and / or intrinsic calibration data, camera projection matrices are determined using known methods. Methods for extrinsic and intrinsic calibration of cameras are known, for example, from one of the publications mentioned above (Reg. G. Willson, 1994).

[0025] To determine the index correction field data, a distortion of the acquired images is determined using a suitable calibration object—in the simplest case, for example, a calibration object with several perpendicular straight lines within a plane perpendicular to the optical axis of the medical microscope, or a checkerboard pattern. Based on the determined distortion, a de-distortion field is determined, from which the index correction field data is then derived. To improve the determination of the index correction field data, three-dimensional structures of the three-dimensional calibration object are recorded and evaluated. This is described, for example, in King et al.

[0026] To determine the edge falloff correction data, in particular a (camera) vignetting in the acquired images is determined, which is corrected by the edge falloff correction data in such a way that brightness values ​​in the acquired images do not fall off towards the edge of the acquired images. In particular, images of a homogeneously self-luminous three-dimensional calibration object are acquired for different working points (working distance, zoom, etc.). As part of an evaluation carried out in step b), a brightness curve in the acquired images is determined and, based on the determined brightness curve, the edge falloff correction data are determined in such a way that acquired images of the homogeneously self-luminous three-dimensional calibration object corrected by means of the edge falloff correction data have a constant, i.e. homogeneous brightness, across the entire calibration object.

[0027] The chromatic shift field correction data is intended, in particular, to correct lateral chromatic aberration, i.e., in particular, to correct a color fringe that is particularly visible at edges. To determine the chromatic shift field correction data, a color-channel-dependent evaluation of the acquired images of a three-dimensional calibration object is carried out. In particular, an offset (e.g., x / y offset) is determined depending on the color channel, and an offset field for the acquired images is determined for each color channel. Due to chromatic aberration, for example, an edge with a color fringe has a slightly different position in the acquired image in each color channel.The correction field generated from the offset field then specifies how the captured image in the respective color channel must be shifted so that the images of all color channels overlap exactly to the pixel, thereby eliminating the color fringing. The correction fields of all color channels then produce the shift field correction data.

[0028] In one embodiment, step b) comprises determining the respective focus position of the cameras of the stereo camera system and adjusting the respective focus position. The adjustment of the focus position takes place in particular on the basis of the focus positions determined for the cameras of the stereo camera system from the images. The aim of the determination and adjustment is that the cameras always have the same (actual) focus position at each operating point. The adjustment of the focus position is carried out, for example, by a technician to whom the values ​​for the focus positions determined on the basis of the acquired images are displayed on a display device of the medical microscope. The technician then adjusts the focus positions such that the (actual) focus positions are always the same for both cameras of the stereo camera system at the various operating points.In principle, however, automated adjustment can also be carried out, for example by means of an actuator system set up for this purpose in the medical microscope.

[0029] In one embodiment, step d) comprises determining an offset between the captured additional images of the cameras of the stereo camera system and determining offset correction data and / or determining a rotation between the captured additional images of the cameras of the stereo camera system and determining rotation correction data. An offset (e.g., x / y offset) and / or a rotation between the captured additional images of the cameras are determined in particular with reference to a respective image center of the captured additional images.

[0030] In one embodiment, it is provided that, within the scope of step a) or c), when capturing at least some of the images, different operating points of a movable lens system of the stereoscopic medical microscope are selected, wherein, within the scope of step b) or d), an offset value is determined for each of the operating points of the movable lens system, wherein the calibration data and / or further calibration data are generated taking into account the respectively determined offset values. In this way, an offset caused by the movable lens system can also be corrected across different operating points of the movable lens system. An offset value refers in particular to an (x / y) offset value of image elements in images captured by the two cameras. By selecting different operating points, a position of the lenses within the lens system changes, in particular.If an optical axis of the lens system changes, for example due to tolerances and inaccuracies in the mechanics or in the actuators used, this can lead to imaging errors, which can result in an offset between the acquired images and / or additional images acquired by the two cameras of the stereo camera system. Such an offset can then be corrected at each operating point using the generated calibration data and / or additional calibration data. For operating points for which no offset values ​​have been determined, calibration data and / or additional calibration data can be generated through interpolation or extrapolation.

[0031] In one embodiment, it is provided that at least one additional image of a homogeneously reflecting and / or remitting calibration object is captured by means of the stereo camera system, wherein, based on the captured at least one additional image, illumination correction data for correcting an illumination geometry of at least one light source of the medical microscope is generated and taken into account when generating the calibration data and / or the further calibration data. This also makes it possible to correct an inhomogeneous illumination geometry of the at least one light source. The illumination correction data can be included in the calibration data or further calibration data. The homogeneously reflecting and / or remitting calibration object can also be part of the at least one three-dimensional or the at least one two-dimensional calibration object.

[0032] Further features of the medical microscope arrangement will become apparent from the description of embodiments of the method. The advantages of the medical microscope arrangement are the same as those of the embodiments of the method.

[0033] The invention will be explained in more detail below using preferred embodiments with reference to the figures.

[0034] Fig. 1 is a schematic representation of an embodiment of the medical microscope arrangement;

[0035] Fig. 2a schematic representations of a three-dimensional calibration object and a two-dimensional calibration object in a side view;

[0036] Fig. 2b shows schematic representations of the three-dimensional calibration object and the two-dimensional calibration object in a top view; Fig. 3 shows a schematic flow diagram of an embodiment of the method for calibrating a stereoscopic medical microscope.

[0037] Figure 1 shows a schematic representation of an embodiment of a medical microscope assembly 100. The medical microscope assembly 100 comprises a stereoscopic medical microscope 1, for example, a surgical microscope, and a data processing device 2. The method described in this disclosure is explained below using the medical microscope assembly 100.

[0038] The stereoscopic medical microscope 1 comprises a stereo camera system 3 comprising a left camera 3I and a right camera 3r. The cameras 3I, 3r capture a capture area imaged via a stereoscopic imaging optics 4 (shown only schematically) of the medical microscope 1. To correct, in particular, errors in an adjustment, calibration data 30 and further calibration data 31, for example, for an actuator 5 configured to change properties of the imaging optics 4, can be stored.

[0039] The medical microscope 1 further comprises a signal processing device 6, which processes a raw signal 101 provided by an image sensor of the left camera 3I and a raw signal 10r provided by an image sensor of the right camera 10r, and generates and provides images 20 and further images 21 from the raw signals 101, 10r. The images 20, 21 can be displayed, for example, on at least one display device 7, for example one or more monitors or a head-mounted display (HMD), which can in particular also be part of the medical microscope 1. The signal processing can in particular comprise a pixel-dependent change in brightness and / or a generation of an offset and / or a rotation around an image center (or another point), as well as further manipulations (filtering, color correction, etc.).To correct the raw signals 101, 10r, calibration data 30, 31 can be stored in the signal processing device 6.

[0040] It can also be provided that the calibration data 30, 31 are stored only in the image processing device 6. The data processing device 2 comprises a computing device 2-1, for example, a microprocessor or microcontroller, and a memory 2-2. The data processing device 2 can also be part of the medical microscope 1.

[0041] The data processing device 2 is configured to generate the calibration data 30 based on images 20 of at least one three-dimensional calibration object 40 captured by the cameras 3I, 3R of the stereo camera system 3 and to store them for correction. Furthermore, the data processing device 2 is configured to generate the additional calibration data 31 based on further images 21 of at least one two-dimensional calibration object 41 captured by the cameras 3I, 3R of the stereo camera system 3 and to store them for correction.

[0042] In particular, a method for calibrating the stereoscopic medical microscope 1 is carried out by means of the medical microscope arrangement 100, comprising: a) capturing images 20 of the at least one three-dimensional calibration object 40 by means of the cameras 3I, 3r of the stereo camera system 3 of the medical microscope 1, b) generating the calibration data 30 based on the captured images 20, wherein the generated calibration data 30 are stored for correction, c) capturing further images 21 of the at least one two-dimensional calibration object 41 by means of the cameras 3I, 3r of the stereo camera system 3, d) generating the further calibration data 31 based on the captured further images 21, wherein the further calibration data 31 are stored for correction.

[0043] It can be provided that step a) and / or step c) are carried out at different operating points of the medical microscope 1, wherein the calibration data 30 in step b) and / or the further calibration data 31 in step d) are generated for each of the different operating points. An operating point can in particular comprise parameters for the following settings: a position (position and / or orientation) of the camera(s) and / or a magnification (zoom), a working distance (focus or a position of the focusing lens), a position of an aperture, a value of an aperture stop (aperture diaphragm), the presence of a drape lens (yes / no), etc. The various operating points can be set both manually and automatically. During manual setting, it can be provided that parameters of the medical microscope 1 are displayed, for example, to a technician so that the technician can set the parameters for each operating point.During automated adjustment, the data processing device 2 can be configured to generate suitable control parameters and to supply them to the medical microscope 1, in particular the actuator 5, for adjusting the various operating points.

[0044] It can be provided that the calibration data 30 generated in step b) and / or the additional calibration data 31 generated in step d) are applied at least partially to the acquired images 20 and / or the acquired additional images 21 for correction, wherein step b) is performed and / or repeated for the corrected images 20 and / or step d) is performed and / or repeated for the corrected additional images 21. In this case, the correction can take place both in the signal processing device 6 and in the data processing device 2.

[0045] It can be provided that the correction and steps c) and d) are repeated until at least one predefined optimization criterion is met. For this purpose, in particular, the additional calibration data 31 are stored in the medical microscope 1, in particular in the image processing device 6, before each repetition. Steps c) and d) are then repeated with modified additional calibration data 31. This allows the calibration of the medical microscope 1 to be successively and iteratively improved.

[0046] In particular, it is provided that step b) comprises one or more of the following measures: determining extrinsic calibration data, determining intrinsic calibration data, determining directory correction field data, determining edge falloff correction data, and determining chromatic shift field correction data. The generated calibration data 30 then comprises the extrinsic calibration data and / or intrinsic calibration data and / or directory correction field data and / or edge falloff correction data and / or chromatic shift field correction data. It can be provided that step b) comprises determining the respective focus position of the cameras 3I, 3R of the stereo camera system 3 and adjusting the respective focus position. The (actual) focus position can be set or adjusted based on a value determined for the respective focus position.This can be done either manually by a technician or automatically and / or motorized using actuator 5. The goal is to ensure that the focus position of both cameras 3I, 3r is always the same at different operating points.

[0047] It can be provided that step d) comprises determining an offset between the acquired further images 21 of the cameras 3I, 3R of the stereo camera system 3 and determining offset correction data and / or determining a rotation between the acquired further images 21 of the cameras 3I, 3R of the stereo camera system 3 and determining rotation correction data. The generated further calibration data 31 then comprises the offset correction data and / or the rotation correction data.

[0048] It can be provided that, within the scope of steps a) or c), when capturing at least some of the images 20, 21, different operating points of a movable lens system of the stereoscopic medical microscope 1 are selected, wherein, within the scope of steps b) or d), an offset value is determined for each of the operating points of the movable lens system, wherein the calibration data 30 and / or the further calibration data 31 are generated taking into account the respectively determined offset values.

[0049] It can be provided that at least one additional image 22 of a homogeneously reflecting and / or remitting calibration object 42 is further acquired by means of the stereo camera system 3, wherein, based on the acquired at least one additional image 22, illumination correction data for correcting an illumination geometry of at least one light source (not shown) of the medical microscope 1 are generated and taken into account when generating the calibration data 30 and / or the further calibration data 31. The illumination correction data are determined for this purpose by means of the data processing device 2. For this purpose, the data processing device 2 determines a brightness curve in the acquired at least one additional image 22, in particular for different operating points.Since the brightness in the captured at least one additional image 22 should be constant, the illumination correction data can be generated based on the respective brightness profile, in particular determined from the brightness profile. The illumination correction data are stored in the signal processing device 6 for correction during operation, in particular as calibration data 30, 31. The signal processing device 6 then applies the illumination correction data to the raw signals 101, 10r.

[0050] The acquisition of the at least one additional image 22 of the homogeneously reflecting and / or remitting calibration object 42 and the generation of the illumination correction data occur, in particular, following steps a) to d). In particular, a correction based on the edge falloff correction data (for correcting camera vignetting) may already have been performed before the brightness profile and the illumination correction data are determined. However, it may also be provided that the homogeneously reflecting and / or remitting calibration object 42 is part of the two-dimensional calibration object 41. The acquisition of the at least one additional image 22 of the homogeneously reflecting and / or remitting calibration object 42 can then also occur simultaneously with other steps, for example, with step c).

[0051] Figures 2a and 2b show schematic representations of embodiments of the three-dimensional calibration object 40 and the two-dimensional calibration object 41. Fig. 2a shows a side view, Fig. 2b a top view, which corresponds to a view of how the medical microscope 1 would capture the calibration objects 40, 41 using the cameras 31, 3r. The three-dimensional calibration object 40 has a triangular shape with an inclined surface (it can therefore also be referred to as an "inclined target"), so that different distances (e.g., z-direction) can be captured simultaneously. The two-dimensional calibration object 41 has a checkerboard pattern on its upper side. With the help of the checkerboard pattern, in particular, an offset (e.g., x / y-direction) and a rotation between the (further) images 20, 21 captured by the cameras 31, 3r of the stereo camera system 3 can be determined.In principle, the two-dimensional calibration object 41 can additionally or alternatively have other suitable structures, such as rectangles, lines, and / or other suitable two-dimensional shapes, etc. In the example shown, the calibration objects 40, 41 are shown together as a combined object. However, the calibration objects 40, 41 can also be formed separately from one another and / or used separately from one another.

[0052] In particular, it can also be provided that the three-dimensional calibration object 40 is designed according to one of the embodiments as described in DE 10 2019 131 646 A1, in particular according to an embodiment of Fig. 5 or Fig. 6 shown therein.

[0053] Figure 3 shows a schematic flow diagram of an embodiment of the method for calibrating a stereoscopic medical microscope. The method is carried out, for example, using a medical microscope arrangement as shown in Figure 1.

[0054] After the method is started 200, which can be triggered, for example, by a technician, images of at least one three-dimensional calibration object are captured in a method step 201 using cameras of a stereo camera system of the medical microscope. Before the capture, (at least) one three-dimensional calibration object is placed in the detection range of the medical microscope. In particular, it is provided that method step 201 is performed for various operating points of the medical microscope.

[0055] In a method step 202, calibration data are generated, in particular determined, from the acquired images using a data processing device. The generated calibration data are stored for correction, in particular in a memory provided for this purpose in the medical microscope, for example in a memory of a signal processing device and / or a control device and / or an actuator system of the medical microscope. The generation, in particular determination, of the calibration data takes place in particular for the various operating points. The generated, in particular determined, calibration data comprise in particular camera projection matrices (extrinsic calibration data and / or intrinsic calibration data) and / or directory correction field data and / or edge falloff correction data and / or chromatic shift field correction data.Depending on the type of calibration data to be determined, different three-dimensional calibration objects may be used. Images are then captured for each of the three-dimensional calibration objects in method step 201, particularly at different operating points.

[0056] In a method step 203, images of at least one three-dimensional calibration object are captured using the cameras of the stereo camera system. This is done in particular with the aid of the three-dimensional calibration object 40 ("inclined target"), which is shown in Fig. 2a. Due to the inclined surface, a blur gradient in a captured image along the optical axis (z-direction) can be evaluated, so that a respective focus position of the cameras can be determined. The capture can be carried out in particular for different working points, in particular at different working distances from the three-dimensional calibration object. Based on the captured images, the respective focus positions of the cameras of the stereo camera system are determined.

[0057] In a method step 204, the focus positions of the cameras are adjusted or aligned based on the determined values. This is done, in particular, manually by a technician. In particular, this is done in such a way that the focus positions are always the same for both cameras at the various working points (in particular, working distances).

[0058] In a method step 205, further images of at least one two-dimensional calibration object are captured by means of the cameras of the stereo camera system.

[0059] In a method step 206, further calibration data are generated, in particular determined, based on the acquired further images. In particular, it is provided that an offset (x / y offset) between the acquired further images of the cameras of the stereo camera system and offset correction data, as well as a rotation between the acquired further images of the cameras of the stereo camera system and rotation correction data, are determined. The generated, in particular determined, further calibration data are stored for correction, in particular in a memory of the medical microscope provided for this purpose, for example in a memory of a signal processing device and / or a control device and / or an actuator system of the medical microscope.

[0060] Depending on the type of calibration data to be determined, different two-dimensional calibration objects may be used. Images are then acquired for each of the two-dimensional calibration objects in method step 205, particularly at different operating points.

[0061] In a method step 207, the calibration data and the additional calibration data are applied to the acquired additional images of the two-dimensional calibration object; in particular, the acquired additional images are thereby corrected. This can be done, for example, by means of the signal processing device 6 (Fig. 1), which corrects the raw signals 101, 10r using the stored calibration data and the stored additional calibration data and provides the results, i.e., the images thus corrected.

[0062] It may be provided that method steps 205, 206, and 207 are subsequently repeated. In particular, it is provided that method steps 205, 206, and 207 are repeated until at least one predefined optimization criterion is met. Predefined optimization criteria may, for example, be maximum values ​​for an offset and / or a rotation (e.g., + / -1 pixel and / or + / -0.001°) between the images of the two cameras of the stereo camera system.

[0063] In method step 207, it can be provided that the generated calibration data are checked by at least partially applying them to the acquired images and / or the additional acquired images and evaluating a result of the application, wherein method steps 201 to 207 are at least partially repeated if an evaluation result does not meet at least one predetermined criterion. The at least one predetermined criterion can, for example, include maximum values ​​for an offset and / or a rotation between the images of the two cameras of the stereo camera system, maximum values ​​for a distortion of the cameras and / or maximum values ​​for an edge drop in brightness (camera vignetting), etc. Once the calibration data have been optimized and / or checked, the calibration data and the additional calibration data are stored in the medical microscope in a method step 208 for use in the field.The procedure is then terminated 209.

[0064] List of reference symbols

[0065] 1 Medical Microscope

[0066] 2 Data processing facility

[0067] 2-1 Calculation device

[0068] 2-2 memory

[0069] 3 stereo camera system

[0070] 3I left camera

[0071] 3r right camera

[0072] 4 Imaging optics

[0073] 5 Actuators

[0074] 6 Signal processing device

[0075] 7 Display device

[0076] 101 Raw signal (left camera)

[0077] 10r raw signal (right camera)

[0078] 20 Figure

[0079] 21 further illustration

[0080] 22 Additional figure

[0081] 30 calibration data

[0082] 31 additional calibration data

[0083] 40 three-dimensional calibration object

[0084] 41 two-dimensional calibration object

[0085] 42 homogeneously reflecting and / or remitting calibration object

[0086] 100 Medical Microscope Arrangement

[0087] 200-209 Procedural steps of the procedure

Claims

Patent claims 1. Method for calibrating a stereoscopic medical microscope (1), comprising: a) capturing images (20) of at least one three-dimensional calibration object (40) using cameras (3l, 3r) of a stereo camera system (3) of the medical microscope (1), b) generating calibration data (30) from the captured images (20), wherein the generated calibration data (30) are stored for correction, c) capturing further images (21) of at least one two-dimensional calibration object (41) using the cameras (3l, 3r) of the stereo camera system (3), d) generating further calibration data (31) from the captured further images (21), wherein the further calibration data (31) are stored for correction.

2. Method according to claim 1, characterized in that step a) and / or step c) are performed at different operating points of the medical microscope (1), wherein the calibration data (30) in step b) and / or the further calibration data (31) in step d) are generated for each of the different operating points.

3. Method according to claim 1 or 2, characterized in that the calibration data (30) generated in step b) and / or the further calibration data (31) generated in step d) are applied at least partially to the captured images (20) and / or the captured further images (21) for correction, wherein step b) is performed and / or repeated for the corrected images (20) and / or step d) is performed and / or repeated for the corrected further images (21).

4. Method according to claim 3, characterized in that the correction and steps c) and d) are repeated until at least one predetermined optimization criterion is met. A method according to any of the preceding claims, characterized in that the generated calibration data (30) are verified by at least partial application to the acquired images (20) and / or the acquired further images (21) and evaluation of a result of the application, wherein steps a) to d) are at least partially repeated if an evaluation result does not meet at least one predetermined criterion. A method according to any of the preceding claims, characterized in that step b) and / or d) comprises one or more of the following measures: determining extrinsic calibration data, determining intrinsic calibration data, determining directory correction field data, determining edge falloff correction data, determining chromatic displacement field correction data.A method according to any of the preceding claims, characterized in that step b) comprises determining the respective focus position of the cameras (3l, 3r) of the stereo camera system (3) and adjusting the respective focus position. A method according to any of the preceding claims, characterized in that step d) comprises determining an offset between the captured further images (21) of the cameras (3l, 3r) of the stereo camera system (3) and determining offset correction data and / or determining a rotation between the captured further images (21) of the cameras (3l, 3r) of the stereo camera system (3) and determining rotation correction data.Method according to one of the preceding claims, characterized in that, in step a) or c), when capturing at least a part of the images (20, 21), different operating points of a movable lens system of the stereoscopic medical microscope (1) are selected, wherein, in step b) or d), an offset value is determined for each of the operating points of the movable lens system, wherein the calibration data (30) and / or further calibration data (31) are generated taking into account the respective determined offset values. Method according to one of the preceding claims, characterized in that at least one additional image (22) of a homogeneously reflecting and / or remitting calibration object (42) is captured by means of the stereo camera system (3), wherein starting from the captured at least one additional image (22) illumination correction data for correcting an illumination geometry of at least one light source of the medical microscope (1) are generated and taken into account when generating the calibration data (30) and / or the further calibration data (31).Medical microscope arrangement (100) comprising: a stereoscopic medical microscope (1) with a stereo camera system (3) comprising cameras (3l,3r) and storable calibration data (30,31), and a data processing device (2), wherein the data processing device (2) is configured to i) generate and store for correction calibration data (30) based on images (20) of at least one three-dimensional calibration object (40) captured by the cameras (3l,3r) of the stereo camera system (3), and ii) generate and store for correction further calibration data (31) based on further images (21) of at least one two-dimensional calibration object (41) captured by the cameras (3l,3r) of the stereo camera system (3).