FOCAL LENGTH PARAMETER VALUE FOR CAMERA MODEL

Abstract

Various examples of the disclosure relate to techniques for calibrating a camera model. The disclosed calibration techniques enable simple and rapid calibration, which can also be applied to a surgical microscope in the field. In the various techniques disclosed herein, a pair of images is acquired and a displacement of an imaging position of one or more objects in the image pair is determined. Based on this displacement, a focal length parameter value for the camera model, for example, a pinhole camera model, is then determined.

Description

TECHNICAL FIELD

[0001] Various examples of the disclosure relate to techniques for determining a focal length parameter value of a camera model that models an optical channel of a surgical microscope. In particular, various examples of the disclosure relate to determining the focal length parameter value by varying an objective focal length of an objective of the optical channel. BACKGROUND

[0002] Calibrating the optical channel is helpful for various functions of surgical microscopes. This involves determining various parameter values of a camera model, such as distortion, optical centers, image scales, or focal length. The camera model then provides a mapping from the object space to the image space on the camera: it is therefore possible to specify which camera pixels specific objects in the object space are imaged on. The calibrated camera model can then be used to de-sketch images, determine poses (relative position and orientation) of objects, create topographies, or incorporate data or information (e.g., preoperative data) into the images.

[0003] The exact determination of these parameter values presents a challenge in practice, as these parameter values must be recalculated in the field each time individual components are replaced. Furthermore, it is important to determine the parameters quickly and / or with the simplest tools possible, or even without any tools.

[0004] US 2014 / 0362186 A1, for example, concerns the field of electronic stereo-optical three-dimensional surface mapping. A pre-calibration phantom with reference markers at known positions in 3D space is prepared. A stereo image pair is acquired from the pre-calibration phantom, assuming that the pre-calibration phantom has a known surface profile, providing a large number of known points in three dimensions. A total of eleven camera parameters are then determined by pre-calibration using a least-squares approach.

[0005] US 6,677,565 B1 discloses a calibration method that uses the positions of the surface centroids corresponding to a microscope focal plane, as well as the centroid positions corresponding to a series of z-shifts away from the focal plane. The centroid positions for any surface are then compared to the calibrated centroid positions to determine the focus correction in a single step.

[0006] Pinhole camera models are often used to model the optical channels. See, for example, DE 10 2014 210 099 B3. An important parameter of a pinhole camera model is the focal length, i.e., the vertical distance between the camera chip and the modeled optical center of the pinhole camera.

[0007] Approaches to determining the focal length parameter value of a pinhole camera model are known in the prior art. For example, a checkerboard pattern can be used. Images of this pattern are acquired from various perspectives using the surgical microscope, which are then processed. To acquire the images, the checkerboard pattern is moved or, if the surgical microscope has a robotic stand, the surgical microscope is automatically moved relative to the checkerboard pattern. This calibration procedure is relatively time-consuming, as it usually has to be performed for varying zoom positions and working distances.

[0008] Camera calibration typically involves taking multiple images at a single operating point (zoom and focus). A pinhole camera model with the corresponding parameters is then created using bundle adjustment algorithms. All parameters of the pinhole camera model are determined at once. An advantage of this method is that one parameter, namely the focal length of the pinhole camera model, can be specifically determined and used for calibration or monitoring.

[0009] Another calibration method for determining the focal length parameter value is based on the use of a three-dimensional (3D) calibration object (also referred to as a 3D target). See, for example, DE 10 2019 131 646 A1. To cover large zoom and working distance ranges, various 3D targets of different sizes are required. Since the costs for measuring such 3D targets are high, this method is also only of limited use for determining the focal length parameter value in the field. SUMMARY

[0010] Therefore, there is a need for improved techniques for calibrating a camera model for a digital optical channel of a surgical microscope. In particular, there is a need for a calibration method for determining the focal length parameter value of a camera model. In particular, there is a need for techniques that mitigate at least some of the limitations or disadvantages mentioned above.

[0011] This problem is solved by the features of the independent patent claims. The features of the dependent patent claims define embodiments.

[0012] A method for calibrating a camera model for a digital optical channel of a surgical microscope is disclosed. The surgical microscope has a zoom lens. A main ray of the optical channel is spaced apart from a main axis of the zoom lens (decentering). The method comprises controlling the surgical microscope to acquire a first image of at least one object. The first image is acquired at a first focal length of the zoom lens. The method also comprises controlling the surgical microscope to acquire a second image of the at least one object. The second image of the at least one object is acquired at a second focal length of the zoom lens. The second focal length is different from the first focal length. The method further comprises evaluating the first image and the second image to determine a displacement of the at least one object between the first image and the second image.A focal length parameter value of the camera model is then determined based on the respective displacement of each of the at least one object and the change in the decentering angle. The focal length parameter value in the camera model is associated with a specific focal length of the zoom lens, and the focal length parameter value of the camera model is further determined based on an angular change in the angle between the chief ray of the optical channel and a central ray of an object beam path based on the change in the first and second focal lengths.

[0013] The angles can be obtained directly from the optical model based on the set focal lengths. The camera model describes the generation of an image by the optical channel. A 3D scene is mapped onto an image plane. Since this is a digital optical channel, this image generated in the image plane can be captured by the camera. The camera model describes a mapping of 3D points of the 3D scene in object space (space in front of the microscope) into image space (pixel coordinates of the camera). The camera model can, for example, be a pinhole camera model (also known as a "pinhole camera model"). In the pinhole camera model, for example, a transformation mapping in space consisting of rotation and translation can be taken into account. This allows a reference coordinate system to be converted into a camera coordinate system, where the camera coordinate system is characterized by the optical axis and an optical center.The pinhole camera model can further describe the mapping from object space to image space using a central projection along rays through the origin of the camera coordinate system (optical center) into a virtual plane at a defined distance from the optical center (focal length). Distortion parameters can also be considered. However, the camera model could also be a lens-based camera model. In this case, aberrations due to specific lens properties are taken into account.

[0014] Determining the shift between the two images can be particularly easy. Furthermore, changing the lens focal length between the acquisition of the two images can be particularly quick and easy with a motorized varifocal lens. This means these techniques can be performed particularly quickly and easily.

[0015] With a varifocal lens, the lens focal length can be adjusted (for example, by shifting two lenses relative to each other along the optical axis). The varifocal lens can, in particular, be motorized. When the lens focal length changes, the focal length parameter value of the camera model also changes. The focal length parameter value of the camera model can be determined for a specific lens focal length (calibration reference point) by, in particular, incrementally changing the lens focal length around the reference point. This means that by slightly changing the lens focal length, the relationship between the lens focal length and the focal length parameter value of the pinhole camera model is sampled.Thus, the first lens focal length (at which the first image is captured) and the second lens focal length (at which the second image is captured) can be larger and smaller than the specific lens focal length with which the focal length parameter value is associated.

[0016] The zoom lens allows for positioning of the lens or lens group. For example, an optomechanism can be provided by which the lens focal length can be finely adjusted. For example, controlled fine adjustment of the lens focal length can be possible. This enables particularly precise determination of the focal length parameter value. Suitable sensors can be provided for such a control loop. This can ensure that the lenses or lens groups move as closely as possible along the ideal optical axis. Possible inaccuracies can be partially or fully compensated for through calibration. The mechanical and optical design can ensure that the decentration angle can be precisely set or read out using the selected sensor technology. Calibrating the main lens can improve the setting quality of the focal length or decentration angle.Main lenses, for example, can be measured in the factory, and the associated data can be stored on an EEPROM, in the cloud, or on a system. This measurement can also be performed on a surgical microscope in the field or during assembly. The relationship between the electromechanical actuator for adjusting the decentration angle and the actually measured decentration angle can then be stored in a lookup table. This relationship can also be approximated using functions, such as a polynomial function.

[0017] Such an incremental change in the lens focal length around the reference point for determining the focal length parameter value can mean, for example, that the difference between the first focal length and the second focal length is no greater than 1% of each of the first focal length and the second focal length. For example, the distance between the first focal length and the second focal length could correspond to a minimum step size of a focal length setting of the zoom lens. This means that the smallest possible change in the lens focal length is made between the capture of the first image and the second image. The step size can be specified, for example, by a stepper motor.On the other hand, it can be considered that the distance between the first focal length and the second focal length is large enough so that, during the evaluation of the first image and the second image, a displacement of the at least one object between the first image and the second image can be determined. This means that in other examples, it is also conceivable that the distance between the first focal length and the second focal length is selected according to a specification and is larger than the minimum possible step size. From the above, it can be seen that by using a change in the lens focal length that is as small as possible, but as large as necessary, particularly good results can be achieved for determining the focal length parameter value of the camera model.

[0018] In other examples, however, focal length changes can also be selected to be greater than 1%, for example when calibration is carried out in the field. If larger changes in focal length are selected around a corresponding reference point, non-linear equations can be used. Such non-linear equations take into account that pixels of objects that are a greater distance from the center in the direction of movement of the pixels cover greater distances in the image than pixels of objects that are closer to the image center (optical center). If pixels that lie on a common straight line perpendicular to the direction of movement of the pixels are selected, they cover the same distance in the image, assuming that the image is free of distortion.

[0019] Furthermore, it is possible to correct distortion for off-center pixels. For example, it could be checked whether distortion needs to be corrected. The distortion can be determined through calibration or taken from the optical model.

[0020] To obtain more information from two images, the displacement of several objects arranged at different image positions can be determined. It would then be possible to relate these displacements to one another, i.e. to determine and evaluate a distribution of the displacements. It would also be conceivable to determine temporary focal length parameter values based on the displacements and to compare these temporary focal length parameter values with one another, i.e. to determine and evaluate their distribution. For example, the (final) focal length parameter value can be calculated by averaging the displacements or temporary focal length parameter values. Alternatively, outliers in the respective distribution can be identified and deleted using sorting methods. A large width of the distribution indicates a defective zoom lens, for example.

[0021] To detect even small changes in the lens focal length in the image, checkerboard patterns, exposed calibration objects, or other contrast features can be used. Such contrast features make it possible to precisely determine a position in the image. Methods with sub-pixel analysis are particularly conceivable. Alternatively, methods from computer vision or artificial intelligence can be used for this purpose.

[0022] The focal length parameter value is also determined based on the angle between the principal ray of the optical channel and a central ray of the object beam path (decentering angle). The decentering angle is not equal to 0° because the principal ray of the optical channel's beam path and the principal axis of the zoom lens are offset from each other. The decentering angle regulates the degree of displacement of the field of view and thus of objects located within the field of view when the focal length changes. This makes the determination of the focal length parameter value particularly precise.

[0023] The relationship between the decentration angle and the lenses or lens groups of the zoom lens cannot be linear. To account for this, polynomial functions or other nonlinear functions can be used instead of assuming linear relationships. These relationships can be determined using the optical model by varying the positions of the zoom lens.

[0024] Different objects can be used for calibration. For example, 2D objects can be used, such as a flat checkerboard pattern. More generally, the object whose displacement is determined between the two acquired images can, for example, be one contrast feature from a large number of contrast features of a flat calibration sample. If multiple objects are used, multiple contrast features can be selected. For example, the black-to-white transition of the corner of a specific checkerboard square of a 2D checkerboard sample could be used as the object. Such a calibration sample, which provides a large number of objects as candidates for determining the displacement, has the advantage that one or more suitable objects can be selected from these candidates. For example, one or more clearly visible objects can be selected.One or more particularly prominent objects can be selected. One or more objects can also be selected depending on their position in the images. When evaluating the first image and the second image, for example, a contrast feature can be selected from a plurality of available contrast features. This can be done based on the distance of the contrast feature to the image center of the first image or the second image. Multiple contrast features can also be selected. In other words, this means that if a plurality of contrast features are available, one or more such contrast features can be selected to determine the shift if they are at a certain distance - for example, a small or even minimal distance - from the image center.For example, determining the focal length parameter value of the camera model can be particularly accurate and easy if the contrast feature is selected as close to the center of the image as possible.

[0025] The evaluation of the first image and the second image can be limited to the displacement of the object between the first image and the second image. In other words, this means that prior knowledge about the arrangement or shape of the plurality of contrast features of the calibration sample, or more generally about properties of the object, does not need to be available. This simplifies the techniques associated with image evaluation. For example, it is not necessary to perform the calibration based on prior knowledge regarding the size or relative arrangement of the contrast features to one another. A simple displacement can be determined. This further enables the method to be used for monitoring purposes during operation. During operation, for example, after activating the autofocus function, the image before and after performing the autofocus function can be used to determine a focal length parameter value.The focal length parameter value determined during operation can then be compared with the stored focal length parameter value. This allows the microscope's operational capability to be monitored during operation. For example, a failure or quality assurance of the zoom lens could be detected or provided during operation. An automatic service request could be created or sent.

[0026] Between the acquisition of the first image and the acquisition of the second image, the pose of the surgical microscope can be fixed on the object. This enables particularly fast acquisition of the first image and the second image. For example, no manual or automated adjustment (using a robotic tripod) is required. This means that the focal length parameter value can be adjusted for a particularly large number of lens focal lengths or zoom settings of the varifocal lens, for example, without requiring a lot of time. This enables repeated calibration, even in the field, which overall improves imaging using the surgical microscope. One or more imaging parameters of the surgical microscope can be fixed accordingly between the acquisition of the first image and the acquisition of the second image.The imaging parameters can include, for example, the direction of illumination of the object. A magnification factor / zoom factor of a zoom lens (if present) can be fixed.

[0027] An electronic data processing device is disclosed. The electronic data processing device comprises a processor and a memory. The processor is configured to load and execute program code from the memory, wherein execution of the program code by the processor causes the processor to perform a method as described above.

[0028] The features set forth above and features described below may be used not only in the corresponding explicitly set forth combinations, but also in further combinations or in isolation, without departing from the scope of the present invention. SHORT DESCRIPTION OF THE CHARACTERS Fig. 1 schematically illustrates a stereoscopic surgical microscope with two optical channels whose beam bundles each have main rays that are arranged offset from a main axis of a common objective of the surgical microscope. Fig. 2 is a flowchart of an exemplary method. Fig. Figure 3 schematically illustrates a pinhole camera model and the shift of the field of view for different lens focal lengths according to various examples. Fig. 4 schematically illustrates an electronic data processing device and a processor with a memory according to various examples. DETAILED DESCRIPTION

[0029] The present invention is explained in more detail below using preferred embodiments with reference to the drawings. In the figures, identical reference numerals designate identical or similar elements. The figures are schematic representations of various embodiments of the invention. Elements shown in the figures are not necessarily drawn to scale. Rather, the various elements shown in the figures are depicted in such a way that their function and general purpose will be understood by those skilled in the art. Connections and couplings between functional units and elements shown in the figures can also be implemented as an indirect connection or coupling. A connection or coupling can be implemented wired or wirelessly. Functional units can be implemented as hardware, software, or a combination of hardware and software.

[0030] The following describes techniques related to calibrating a camera model for a digital optical channel. In particular, an optical channel of a surgical microscope with a zoom lens can be calibrated. However, it would also be conceivable to calibrate the optical channel of another microscope type (e.g., a microscope with Köhler illumination) using the techniques described herein.

[0031] Based on the camera model, auxiliary information can then be inserted into a camera image captured by the camera of the optical channel, with positional accuracy. For example, the auxiliary information in connection with the surgical microscope could be determined based on preoperative volume image data, such as magnetic resonance volume image data or computed tomography volume image data. Such positional accuracy incorporation of auxiliary information should ensure that corresponding 3D points associated with specific 3D positions in object space are inserted at the correct position in image space, i.e., at the correct pixel coordinates of the camera. This is made possible by the camera model. The camera model maps 3D points in object space into image space.

[0032] For example, precise calibration of the camera model - and in particular, precise determination of the focal length parameter value - can enable precise insertion and overlay of such auxiliary information into the microscope images.

[0033] Fig. 1 schematically illustrates a surgical microscope 2. The surgical microscope 2 has two stereoscopic optical channels 90A, 90B. The surgical microscope 2 comprises an objective lens 5 directed towards an object field 3. The object field 3 is arranged in the focal plane of the objective lens 5, so that an observation object located in the object field 3 is imaged by the objective lens 5 towards infinity. The objective lens 5 is a zoom objective lens. This means that the focal length of the objective lens 5 can be adjusted. The distance to the focal plane can also be varied. For this purpose, for example, several lenses of the objective lens 5 can be shifted relative to one another (in Fig. 1 not shown).

[0034] Out of Fig. 1 shows that the main axis 80 of the lens 5 is spaced from the main rays 81A, 81B of both optical channels 90A, 90B. The corresponding offsets 95A, 95B are shown. The decentration angle 150 is also shown.

[0035] A magnification changer 11 is arranged on the observer side of the objective 5. This can be configured either as a zoom system for continuously changing the magnification factor, as in the illustrated embodiment, or as a so-called Galilean changer for stepwise changing the magnification factor. Setting a magnification factor using the magnification changer 11 is achieved, for example, via a motor-driven actuator, which, together with the magnification changer 11, forms part of a magnification change unit for setting the magnification factor.

[0036] On the observer side, the magnification changer 11 is connected to an optical interface arrangement 13A, 13B, which includes beam splitter prisms 15A, 15B. In principle, however, other types of beam splitters can also be used, such as semi-transparent mirrors. In the present example, the optical interfaces 13A, 13B serve to decouple a respective beam of rays in order to capture images using cameras 37A, 37B. The two optical channels 90A, 90B are therefore digital.

[0037] A binocular tube 27 is connected to the interface arrangements 13A, 13B on the observer side. This tube has two tube objectives 29A, 29B, which focus the respective parallel beam bundles 9A, 9B onto an intermediate image plane 31, thus imaging the object field 3 onto the respective intermediate image plane 31A, 31B. The intermediate images located in the intermediate image planes 31A, 31B are then imaged to infinity by ocular lenses 35A, 35B, so that an observer can view the intermediate image with a relaxed eye. Furthermore, the distance between the two partial beam bundles 9A, 9B is increased in the binocular tube by means of a mirror system or prisms 33A, 33B in order to adapt it to the observer's interpupillary distance. The mirror system or prisms 33A, 33B also erect the image.

[0038] The surgical microscope 2 is also equipped with an illumination device with which the object field 3 can be illuminated with illuminating light. For this purpose, the illumination device in the present example comprises a white light source 41, such as a halogen lamp or a gas discharge lamp. The light emitted by the white light source 41 is directed toward the object field 3 via a deflecting mirror 43 or a deflecting prism in order to illuminate it. The illumination device also includes an illumination optics 45, which ensures uniform illumination of the entire observed object field 3.

[0039] The surgical microscope 2 in Fig. 1 is just one of many possible design variants. For example, instead of a surgical microscope with two optical channels, a variant in which only one optical channel is used would be conceivable. Furthermore, the binocular tube 27 is optional: one or more purely digital optical channels can be used. Optionally, it would also be conceivable to have one or more further interface arrangements configured to couple beams of light, so that, for example, auxiliary information based on a camera model of the respective optical channel can be superimposed onto the beam of light viewed through an eyepiece.

[0040] Fig. 2 is a flowchart of an exemplary method. The method of Fig. 2 can be executed by a processor of an electronic data processing device. For this purpose, the processor can, for example, load and execute program code from a memory. For example, the method from Fig. 2 be implemented by a control system of a surgical microscopy system. The surgical microscopy system can be the control system and a surgical microscope, for example the surgical microscope from Fig. 1, include.

[0041] The procedure from Fig. 2 serves to calibrate a camera model for a digital optical channel of a surgical microscope having a zoom lens. In particular, for example, a camera model for the optical channel 90A of the surgical microscope 2 or the optical channel 90B of the surgical microscope 2 can be Fig. 1. Calibration takes advantage of the fact that there is a dependency between the lens focal length and the field of view or the positioning of an object in corresponding images for a zoom lens in which the main ray of the optical channel is spaced from the main axis of the zoom lens, ie the decentration angle is not equal to 0°.

[0042] In particular, the method from Fig. 2, a value for a focal length parameter of the camera model is determined. The camera model can, for example, be a pinhole camera model: there, the focal length parameter is the distance between the virtual optical center and the virtual image plane.

[0043] In box 505, a first image of an object is captured at a first focal length of the zoom lens of the surgical microscope. For this purpose, the surgical microscope can be controlled accordingly. For example, a specific target value for the focal length of the zoom lens can be specified, and a corresponding motor for adjusting the lenses of the zoom lens can be controlled accordingly. For example, a camera chip of the surgical microscope can then be controlled to capture the first image.

[0044] Then, in box 510, a second image of the object is captured at a second focal length of the zoom lens. The second focal length is different from the first focal length. Also in box 510, one or more components of the surgical microscope can be controlled accordingly, as described above in connection with box 505.

[0045] The focal length of the zoom lens can be changed incrementally between box 505 and box 510. This means that the difference between the two lens focal lengths from box 505 and box 510 is small compared to the two lens focal lengths, for example, no more than 1 percent of each of the first focal length and the second focal length of the zoom lens.

[0046] Through this incremental change in the lens focal length, the relationship between the lens focal length and the focal length parameter value of the camera model can be sampled. This sampling occurs in the range of the first and second focal lengths of the varifocal lens. The focal length parameter value of the camera model can, for example, be associated with the mean value of the first and second focal lengths.

[0047] In box 515, the first image and the second image are then evaluated to determine a displacement of the object between the first image and the second image. Box 515 may involve the application of various image evaluation algorithms, depending on the implementation variant. A SIFT algorithm could be used. For example, object localizations could be performed. A machine-learned model could be used. Object tracking could be used. A structure-based object localization algorithm could be used to locate the object in each of the first image and the second image and then determine the displacement (for example, in units of image pixels). In principle, previously known techniques for image evaluation can be used in box 515, so further details are not described in this context.

[0048] In the various examples, it would be conceivable to locate multiple objects in the images and determine a corresponding displacement of the image position for each of the multiple objects. In this way, multiple displacements of the image position are obtained for the multiple objects. It would then be conceivable to evaluate a corresponding distribution of the displacements of the image position, for example by determining the maximum of the distribution (corresponding to averaging) or by removing outliers. It would also be conceivable to determine and analyze a corresponding distribution of temporary focal length parameter values. In this way, a reliable determination of the focal length parameter value can be achieved using statistical methods by evaluating the displacement of the image of multiple objects.

[0049] Techniques have been described above for determining the displacement of the image position of one or more objects between two images acquired at two zoom lens focal lengths. As a special case, it would be conceivable to determine the displacement of the image position of an object between these two images and one or more additional images acquired at different zoom lens focal lengths. For example, the incremental change in the image position as a function of the change in lens focal length ∂s / ∂f can be determined using curve fitting.

[0050] Then, in box 520, the focal length parameter value of the camera model is determined based on the lateral displacement. Details of this are given below in connection with Fig. 3 described.

[0051] Optionally, in box 525, it would be possible to compare the focal length parameter value with one or more specifications. For example, the focal length parameter value could be compared with one or more previously determined focal length parameter values and / or stored tolerance ranges. If the focal length parameter value is thus classified as acceptable in box 525, a further iteration of box 505 and subsequent steps could be performed. Alternatively, a service request could optionally be submitted in box 530. This makes it possible to monitor the operational capability of the microscope based on the focal length parameter value.

[0052] Fig. Figure 3 schematically illustrates the beam path in a pinhole camera model 110. Fig. Figure 3 illustrates two beams of rays (dashed and dotted-dashed lines), both passing through the optical center 113. The two beams correspond to different lens focal lengths and define different fields of view 115, 116. The fields of view 115, 116 shift due to the offset between the main ray of the optical channel and the main axis of the zoom lens (cf. Fig. 1, offsets 95A, 95B).

[0053] Also shown are object rays (dotted-dotted-dashed lines) for an object 125. The corresponding imaging positions 121, 122 of the object 125 on the camera chip 111 change along with the shift of the fields of view 115, 116. Shown are corresponding distances 131, 132 of the imaging positions 121, 122 to the shortest connection between the camera chip 111 and the optical center 113; this shortest connection defines the focal length parameter value 112 (also referred to as f). The distance 131 is referred to as s'; the distance 132 as s. The shift of the imaging position of the object is s - s'.

[0054] The angles 141 (denoted as α) and 142 (denoted as β) are also shown. The difference angle 143 β - α = Δ corresponds to the change in the decentration angle resulting from the different lens focal lengths and depends on the offset between the principal ray and the principal axis of the zoom lens.

[0055] The result is: tan β≈β=sf as well as tan α≈α=s'f

[0056] In equations (1) and (2), a small-angle approximation is assumed. This is valid for imaging positions 121, 122 that are a short distance 131, 132 from the image center, i.e., from the center of the camera chip 111. For larger angles—for example, due to a larger difference in lens focal lengths—the small-angle approximation can be dispensed with, i.e., non-linear equations can be used. It would also be conceivable to consider multiple objects. In this case, at least one of these multiple objects may be a significant distance from the image center, so that non-linear equations can also be used in such a case.

[0057] The observed object 125 can be selected from a multitude of candidate objects if a corresponding flat calibration sample with point-like (0-D) or line-like (1-D) contrast features is used. However, prior knowledge of the sample is not required. It is assumed that the change in the decentration angle, i.e., the difference angle Δ 143, is known.

[0058] Then the following applies: Δ=β−α=sf−s'f=s−s'f

[0059] The focal length parameter value f can be determined from the object displacement s - s' and the known change in the decentration angle Δ 143. The advantage of this solution is that a large amount of information can be generated quickly (the incremental change in the object focal length can be carried out quickly), and calibration can be performed quickly for many zoom positions and working distances. Furthermore, a simple checkerboard pattern can be used for calibration and is therefore also suitable for determining the focal length parameter value in the field.

[0060] The change in the decentration angle Δ 143 is information from the zoom lens. The accuracy of the decentration angle or its change is system-dependent and can be improved by calibrating the zoom lens.

[0061] Fig. Figure 4 schematically illustrates an electronic data processing device 200 comprising a processor 211, a memory 212, and a communication interface 213. For example, the electronic data processing device 200 can be part of a control system for a surgical microscope, for example, for the surgical microscope 2 of Fig. 1. The processor 211 is configured to load and then execute program code from the memory 212. When the processor 211 executes the program code, this causes the processor to perform techniques as described herein, for example, in connection with the method of Fig. 2, in particular: controlling a camera of the surgical microscope to capture images; controlling one or more components of the surgical microscope to change imaging parameters, for example the working distance or the lens focal length, a magnification value, etc.; evaluating captured images, for example to localize objects or to determine a displacement between objects in different images; determining a parameter value for the focal length of a camera model; displaying auxiliary information in images captured by a camera of the surgical microscope based on the camera model; etc.

[0062] In summary, techniques have been described above by means of which it is possible to determine a focal length parameter value of a camera model, in particular a pinhole camera model. The various examples described herein exploit the fact that, due to an offset between the principal axis of a lens and the principal ray of the beam path of an optical channel, changing the lens focal length causes a change in the field of view and thus the imaging position of an object. The relationship between the lens focal length and the focal length parameter value of the camera model is determined by incrementally varying the lens focal length. This does not require any prior knowledge of the calibration object. In particular, a 3D calibration object is not necessary. The object does not have to be imaged from different perspectives; rather, the same perspective on the object can be used.This means the object does not need to be moved, nor does a robotic or manual movement of the surgical microscope relative to the object become necessary. This makes the procedure particularly quick and easy to perform and also suitable for field use.

[0063] Of course, the features of the previously described embodiments and aspects of the invention can be combined with one another. In particular, the features can be used not only in the described combinations, but also in other combinations or on their own, without departing from the scope of the invention.

[0064] The described method for calculating the focal length can also be used to monitor stored focal lengths during operation. To calculate the focal length, images can be acquired in the field at two decentration angles, and identical features can be detected in these two images. In the field or during operation, it must be ensured that the observed object does not move relative to the surgical microscope. The focal length value is calculated as described, and this can be compared with the stored focal length. If the newly calculated focal length parameter initially deviates from the stored value, the user can be warned, a service technician can be called, or certain functions that use the calibration data can be deactivated.

[0065] For example, a small-angle approximation was used above in connection with Equations 1 and 2. Such a small-angle approximation is optional. For larger angles α and β, it would be conceivable to use corresponding nonlinear equations.

[0066] Furthermore, for example, in connection with Fig. 1 shows a stereoscopic surgical microscope with two optical channels. However, it would also be possible to use a surgical microscope with only one optical channel.

[0067] In various examples above, the determination of the displacement for a single object was discussed. However, corresponding techniques can also be used to determine the displacements of multiple objects in a pair of images. Statistics can then be collected, allowing a more accurate determination of the focal length parameter value.

Claims

[1] Method for calibrating a camera model (110) for a digital optical channel (90A, 90B) of a surgical microscope (2) with a zoom lens (5), wherein a main ray (81A, 81B) of the digital optical channel (90A, 90B) is spaced (95A, 95B) from a main axis (80) of the zoom lens (5), the method comprising: - controlling the surgical microscope (2) to capture a first image of at least one object at a first focal length of the zoom lens (5), - controlling the surgical microscope (2) to capture a second image of the at least one object at a second focal length of the zoom lens (5), wherein the second focal length is different from the first focal length, - evaluating the first image and the second image to determine a respective displacement of an imaging position (131, 132) of each of the at least one object between the first image and the second image, and - Determining a focal length parameter value (112) of the camera model (110) based on the respective displacement of each of the at least one object, wherein the focal length parameter value (112) in the camera model is associated with a specific focal length of the zoom lens (5), and wherein the focal length parameter value (112) of the camera model (110) is further determined based on an angular change in the angle (150) between the chief ray (81A, 81B) of the optical channel (90A, 90B) and a central ray (80) of an object beam path based on the change in the first and second focal lengths. [2] A method according to claim 1, wherein the determined focal length of the zoom lens (5) is between the first focal length and the second focal length. [3] The method according to claim 1 or 2, wherein a difference between the first focal length and the second focal length is not greater than 1% of each of the first focal length and the second focal length. [4] Method according to one of the preceding claims, wherein a distance between the first focal length and the second focal length corresponds to a minimum step size of a focal length setting of the zoom lens. [5] Method according to one of the preceding claims, wherein the at least one object comprises at least one contrast feature of a plurality of contrast features of a flat calibration sample. [6] The method of claim 5, wherein evaluating the first image and the second image comprises: Selecting the at least one contrast feature from the plurality of contrast features based on a distance of the contrast feature to an image center of the first image or the second image. [7] The method of claim 5 or 6, wherein the evaluation of the first image and the second image is not based on prior knowledge of an arrangement or a shape of the plurality of contrast features. [8] Method according to one of the preceding claims, wherein the at least one object comprises several objects, wherein the respective displacement of the image position is determined for each of the plurality of objects, wherein the focal length parameter value (112) of the camera model is determined based on a distribution of the displacements of the imaging position for the plurality of objects or the distribution of corresponding temporary focal length parameter values. [9] Method according to one of the preceding claims, wherein a pose of the surgical microscope (2) with respect to the at least one object is fixed between the acquisition of the first image and the acquisition of the second image. [10] Method according to one of the preceding claims, wherein imaging parameters of the surgical microscope are fixed between the acquisition of the first image and the acquisition of the second image, wherein the imaging parameters are optionally selected from the group comprising: zoom setting of a zoom optics of the optical channel; illumination direction of an illumination of the at least one object. [11] Method according to one of the preceding claims, wherein the camera model (110) is a pinhole camera model. [12] A method according to any one of the preceding claims, wherein the method further comprises: - Monitoring the operational capability of the surgical microscope based on the focal length parameter value (112). [13] A method according to any one of the preceding claims, wherein the method further comprises: - controlling the surgical microscope (2) to capture a third image of the at least one object at a third focal length of the zoom lens (5), wherein the third focal length is different from the second focal length, wherein the first image, the second image and the third image are evaluated to determine the respective displacement of the imaging position (131, 132) of each of the at least one object between the first image and the second image and the third image. [14] An electronic data processing device comprising a processor and a memory, the processor being arranged to load and execute program code from the memory, wherein execution of the program code by the processor causes the processor to carry out the method according to any one of the preceding claims.

Images