Method and system for generating expanded image of at least partial section of row of teeth, and computer program product

Abstract

To provide a method and a system for generating an expanded image of at least a partial section of a row of teeth, and a computer program product.SOLUTION: A method for generating an expanded image AA of at least a partial section of a row of teeth comprises: a) detecting a mirror image S5 imaged in an image generated by the image capture device and being provided by mirror elements arranged in an intake region of an image capture device so as to image a mirror image of at least one partial section of a row of teeth with an image capture device; b) determining a posture P of a mirror element; c) determining a section KA corresponding to the partial section of a row of teeth in accordance with a posture of at least the mirror elements in preoperative generation information PI; and d) generating an expanded image by the corresponding section being reflected and superimposed on the at least imaged mirror image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a method and system, and a computer program product, for generating an enhanced image of at least a partial section of a dentition. [Background technology]

[0002] So-called preoperative information useful for diagnosis and / or treatment planning is determined, particularly in the case of medical procedures, for example in the field of implantology, or for restorative / aesthetic treatment of patients. In dental applications, such preoperative information can be generated, for example, by what are known as intraoral scans, which detect information about the geometry of the teeth. This can then be useful for determining the condition of the teeth or jaw, particularly for recording this at various points during treatment. Furthermore, such intraoral scans may be useful for planning or constructing implants or crowns.

[0003] The generation of preoperative information by methods based on computed tomography or magnetic resonance imaging is also known, which, in particular in relation to teeth, allows internal structures such as tooth roots or nerves to be detected or imaged, which information may also be useful for planning the procedure, for example to determine the desired position and orientation of a drill head for drilling purposes.

[0004] Thus, in addition to anatomical information, pre-operative information may also include further additional information, such as planning information, which may, for example, specify which areas of the tooth can be removed to securely seat a crown, or where sensitive nerves run within the tooth that must not be damaged during the procedure.

[0005] During the procedure, pre-operative information can be displayed, for example, on a display device, particularly to augment the information generated intraoperatively, e.g., to provide an augmented image representing information generated both intraoperatively and pre-operatively.

[0006] The use of so-called medical navigation systems is also known, which can be used to detect, among other things, the orientation of the instrument being used. Depending on this orientation, the instrument can be displayed in relation to preoperative data, e.g., superimposed on an image generated from the preoperative information. Thus, the treating physician can obtain information about, for example, the relative orientation between the instrument and a non-visible structure. This can help, for example, to determine the correct drilling angle and drilling depth for a drill. A drawback of such navigation systems is that, when the preoperative data is displayed on a monitor, the treating physician must watch the monitor while working, which may not represent the surgical site in real time. Therefore, the treating physician must alternate his or her field of vision between the monitor and the examination / surgical area. This is particularly disadvantageous when the treating physician works using magnifying glasses, which are often used to observe the surgical area, and which generally have a considerable weight and do not allow for good visual detection of the examination area in the event of head tremors.

[0007] Surgical microscopes are also known. These surgical microscopes are used by the user during treatment to provide an image, especially a magnified image, of the treatment area, especially in situ. So-called surgical stereomicroscopes generally have two separate optical channels for guiding beams, which can provide the user with an impression of the depth of the examination area. For this purpose, the beams guided in the two channels can be captured by the user's eyes through eyepieces. Alternatively or additionally, surgical digital microscopes have two image capture devices, each capturing the beam of one of the optical channels to generate an image. A three-dimensional image is then provided to the user via a suitable display device based on the two images, hereinafter also referred to as corresponding images. Furthermore, other surgical microscopes are also known that can optically capture information, especially depth information, and in particular provide a three-dimensional image of the treatment area. For this purpose, optical detection systems that can provide depth information based on, for example, interferometry, triangulation, time-of-flight (TOF), or microlens arrays can also be used in surgical microscopes, especially as an alternative to stereoscopic systems.

[0008] To ensure accurate representation, accurate calibration of the stereo camera system is required, and known calibration methods are used to determine internal and external camera parameters, which are then used by the image processing process to ensure accurate representation. Internal camera parameters describe parameters related to each camera / image capture device itself, such as its distortion. External camera parameters describe the relationship, especially the spatial relationship, of the image capture devices and therefore the camera images to each other. Such internal and external camera parameters are known to those skilled in the art.

[0009] Known prior art includes US Pat. No. 5,629,999, which discloses a method and an intraoral scanner for detecting the surface topography of semi-transparent, particularly dental, objects.

[0010] Also known is US Pat. No. 5,629,999, which also discloses an intraoral scanner for three-dimensional scanning of the upper or lower jaw with or without teeth together with the jaw components, particularly in relation to implant prostheses.

[0011] Also known is US Pat. No. 5,629,999, which also discloses an intraoral scanner for digital dental impressions in the dental field and a method for producing a digital dental impression by means of an intraoral scanner.

[0012] Also known is US Pat. No. 5,629,999, which discloses a dental observation device in which a so-called dental microscope is used. [Prior art documents] [Patent documents]

[0013] [Patent Document 1] German Patent Application Publication No. 102020133627A1 [Patent Document 2] German Patent Application Publication No. 102019008510A1 [Patent Document 3] German Patent Application Publication No. 102016121687A1 [Patent Document 4] European Patent Application Publication No. 3689295A1 Summary of the Invention [Problem to be solved by the invention]

[0014] Commonly known as dental mirrors are used in treating the backs or molars of teeth, and optionally also the chewing surfaces of teeth. They can provide a mirror image of the teeth / dentition. Such mirror elements are also used whenever image capture devices, especially surgical microscopes, are used to generate images, especially magnified images, of the oral cavity during the procedure.

[0015] This presents a technical challenge of developing a method and system for generating an augmented image of at least a partial section of the dentition, as well as a computer program product, that allows accurate augmentation of the image with pre-operative information when mirror elements are used. [Means for solving the problem]

[0016] The solution to the technical problem is evident from the subject matter having the features of the independent claims. Further advantageous configurations of the invention are evident from the dependent claims.

[0017] A method for generating an augmented image of at least a partial section of a dentition is proposed. In particular, the augmented image may be an image of a real partial section of a dentition, which is augmented with computer assistance, in particular by, for example, overlaying or superimposing at least one virtual object and / or additional information on the image of the real partial section. The augmented image can be displayed to a user, e.g., a treating physician or surgeon, in particular using a suitable display device, e.g., a monitor or a head-mounted display. As will be explained in more detail later, if the image of the real partial section of a dentition is generated using an image capture device of a surgical microscope, the augmented image can also be provided to the user so that it is optically detectable by the user through the eyepieces of the surgical microscope. For this purpose, preoperative information can be introduced, e.g., reflected, into the beam path leading to the eyepieces. In the case of a surgical microscope having a stereo camera system with two image capture devices, the augmented image can be generated from the images generated by the two image capture devices in both cases. Thus, an augmented image with depth information, i.e., an augmented three-dimensional representation, can be provided to the user on a suitable display device or similarly through the eyepieces.

[0018] In a first step, an imaged mirror image is detected in an image generated by an image capturing device, the mirror image being provided by a mirror element, which is positioned in a capturing area of ​​the image capturing device so that the mirror image of at least one partial section of the dentition can be imaged by the image capturing device.

[0019] A partial section of the dentition may in particular comprise at least one tooth or part of one tooth. Teeth may also refer to dentures within the meaning of the present invention.

[0020] Therefore, in particular, before the imaged mirror image is detected, the above-mentioned mirror element can be placed in the capture area of ​​the image capture device, where the image capture device can image a mirror image of at least one partial section of the dentition. Furthermore, before the detection, an image can be generated by the image capture device. In particular, the mirror element can be a dental mirror or a part thereof. In particular, the mirror element can have or take the form of a mirror surface that reflects radiation. The image of the mirror surface generated by the image capture device, i.e., the imaged mirror image, is generated by capturing this reflected radiation. Therefore, the mirror image generated / provided by the mirror element is sensed by capturing the reflected radiation. The capture by the image capture device generates the imaged mirror image. However, the image generated by the image capture device may include, in addition to the imaged mirror image, further areas that do not include a mirror image. In other words, the mirror image may be imaged in a partial area of ​​the image generated by the image capture device. In the imaged mirror image, at least one partial section of the dentition is imaged. The mirror element may further include a frame section that surrounds the mirror surface. The mirror element may also include a handle section that allows a user to position the mirror element in space. The mirror element is preferably a mirror element having a non-curved mirror surface. Preferably, the mirror surface is a circular surface. However, it is also contemplated that a polygonal mirror surface may be used.

[0021] However, surgical microscopes are also described that use, in addition to or instead of the described image capture devices, a further optical capture system that is in particular, although not necessarily, able to provide depth information. Such optical capture systems have already been described above in the introduction. In particular, this further optical capture system may be different from a stereo camera system.

[0022] In a second step, the orientation of the mirror element, particularly the mirror surface, is determined. The orientation includes a translational component and a rotational component. For example, the position of a reference point of the mirror element, e.g., the center of the mirror surface, and the orientation of the mirror element, e.g., the orientation of the normal to the mirror surface, can be determined as the orientation of the mirror element. It is obvious that the positions of multiple reference points of the mirror element or the orientations of multiple subsections can also be determined as the orientation, particularly in the case of a curved mirror surface. Exemplary methods for determining the orientation will be described in detail below. The orientation can be determined in a reference coordinate system. For example, the reference coordinate system may be the reference coordinate system of an image capture device. The reference coordinate system of the image capture device may be the image coordinate system of an image generated by the image capture device. The reference coordinate system may be the reference coordinate system of preoperative information, the reference coordinate system of a surgical microscope, or a coordinate system serving as a reference, which will be described in more detail below.

[0023] In a third step, a section corresponding to the partial section of the dentition is determined in the preoperatively generated information depending on at least the orientation of the mirror element, in other words, a portion of the preoperative information containing information about the section of the dentition reflected by the mirror element is thus determined.

[0024] The preoperative information can be generated, in particular, in the form of image data or volume data. As already mentioned at the beginning, such information can be generated, in particular, using CT-based or MRI-based methods, but can also be generated using other imaging methods, in particular, ultrasound-based methods. A reference coordinate system can be assigned to the preoperative information, so that the preoperatively generated information can also include spatial information. The orientation of the mirror elements forms an input variable for determining the corresponding section. To perform the third step, it may be necessary to perform an alignment of the reference coordinate system of the preoperatively generated information with the reference coordinate system of the image capture device. This alignment can be performed before the third step is performed, in particular before the first step is performed. The alignment determines the reference of both the preoperative information and the image, in particular for information in the image generated by the image capture device, to a shared reference coordinate system. In particular, this shared reference coordinate system can be the reference coordinate system of the preoperatively generated information, the reference coordinate system of the image capture device, or a different reference coordinate system, for example, a global coordinate system serving as a reference. If a surgical microscope is used, the shared reference coordinate system can be the reference coordinate system of the surgical microscope. This will also be explained in more detail below.

[0025] In this regard, registration methods are known to those skilled in the art. For example, model-based registration can be performed. In this process, features, e.g., geometric features, corresponding to already known features in the preoperative information, in particular of the jaw or tooth sections, can be detected in the image, and then the registration can be determined in a known manner based on these corresponding features. For example, the registration can be determined in the form of a transformation matrix including a rotational and / or translational component. Edge-based registration can be an exemplary model-based registration, where corresponding features are formed, for example, by the characteristics of at least one edge, preferably multiple edges, in both the image and the preoperative information. In particular, topography-based registration can also be performed, especially when the topography can be determined using, for example, a stereo camera system of a surgical microscope. In this way, topographic information can be determined in at least one image, and corresponding features, points, or sections can be detected in both the preoperative information and this topographic information and can subsequently be used to determine the registration. For example, if a stereo camera system, in particular of a surgical microscope, is used to generate at least one image, it is possible to generate a three-dimensional image of at least a partial section of the dentition when a corresponding image is generated by the stereo camera system, which three-dimensional image forms or provides topographical information for registration purposes. However, it is also possible to use only one of the two corresponding images for registration purposes.

[0026] For example, stereo reconstruction methods can be used to generate such three-dimensional images, with the imaged mirror images forming the input image for the method. Such methods are known to those skilled in the art. In particular, such methods can determine corresponding pixels in two input images. For example, such corresponding pixels or pixel sets can be determined using feature matching methods. Corresponding methods and features are known to those skilled in the art. Exemplary features are known as SIFT features, i.e., features for / of scale-invariant feature transforms. However, it is obvious that other methods, such as variational methods or AI-based methods, can also be used for the determination. Next, for the object points or object sections imaged at the corresponding pixels or pixel sets, three-dimensional coordinates can be determined in the reference coordinate system of the three-dimensional image; possible reference coordinate systems have already been described above. This can also be referred to as reconstruction.

[0027] The three-dimensional image can be generated based on the orientation of the mirror elements. For example, a reconstruction method can be performed based on the orientation of the mirror elements. In particular, at least one method step of the reconstruction method can be performed based on the orientation. In particular, the orientation can be represented by at least one parameter, and at least one method step is performed based on the parameter or takes the parameter into account during its execution. Preferably, a stereo triangulation reconstruction method is performed to determine the three-dimensional image. Stereo triangulation reconstruction methods are known to those skilled in the art. In this case, a projection matrix used during reconstruction, which describes the perspective transformation of three-dimensional object coordinates in a reference coordinate system into two-dimensional image coordinates, can be determined based on the orientation of the mirror elements and the (known) laws of reflection. In other words, the orientation of the mirror elements affects the projection matrices of both image capture devices and therefore also affects the stereo triangulation reconstruction performed based on or in response to these projection matrices. For example, so-called homogeneous or so-called non-homogeneous methods can be applied to determine the three-dimensional coordinates. In particular, a rectification method for correcting or removing nonlinear distortions in the image can be performed before the three-dimensional image is determined.

[0028] It may also be advantageous to perform a calibration of at least one image capture device. For example, known calibration methods can be used to determine the internal and external parameters of the image capture device, which are used by the image processing process to ensure accurate representation. The internal parameters describe parameters related to the image capture device itself, such as its distortion. The external parameters describe the relationship, especially the spatial relationship, between multiple image capture devices and therefore their images. Such internal and external parameters are known to those skilled in the art. Preferably, the above-mentioned parameters are determined for all operating states of the image capture device or for a predetermined operating state, where the operating state is characterized by the set (adjustable) parameters of the image capture device (e.g., zoom, focus, capture area). If only one image capture device is used, only the internal parameters need to be determined for calibration purposes. If a stereo camera system with two image capture devices is used, the external parameters must also be determined for calibration purposes.

[0029] In a fourth step, a corresponding section of the preoperatively generated information is reflected and superimposed on the imaged mirror image, thereby generating an augmented image. For the purpose of augmentation, methods known to those skilled in the art can be used. In other words, the preoperatively generated information corresponding to the section reflected by the mirror element is superimposed on a partial region of the image in which the mirror image provided by the mirror element is imaged. For this purpose, the preoperatively generated information is also reflected. The preoperatively generated information can be reflected by applying a transformation matrix, which is determined based on the orientation of the mirror element. In other words, in this way, it is possible to determine how a given section of the preoperatively generated information is reflected by the mirror element. For this purpose, a virtual reflection image of the preoperatively generated information, which will later be used for the purpose of augmentation, can be determined in a computer-aided manner, for example, in a model-based manner. The virtual reflection image can be an image of the preoperative information reflected by a virtual mirror element, which is taken into account in the model-based determination.

[0030] By determining the pose, it is possible, in particular, to determine the mirror plane in the reference coordinate system and to limit it by the variables identified (by the image scale). The laws of reflection can be applied to this plane, so that this limited area can be projected onto the preoperative information. The resulting intersection of the projection and the preoperative information can then be appropriately reflected and virtually superimposed on the imaged mirror image.

[0031] Advantageously, when mirror elements are used, this results in an easy-to-implement and accurate extension of the image with preoperative information. In particular, information about the structure of a partial section that cannot be imaged by the image capture device, such as nerve pathways inside a tooth, can be overlaid on the image of the image capture device. Furthermore, it is no longer necessary to provide the preoperative and intraoperative information to the user on different output devices.

[0032] Alternatively or additionally, to generate the augmented image, not only preoperatively generated information but also intraoperative information, i.e., information recorded during treatment, can be used. Thus, for example, information about a partial section of the dentition can be collected and stored during treatment, and then this information can be used to generate the augmented image. Thus, in this case, depending at least on the orientation of the mirror element, a section corresponding to the partial section of the dentition can be determined in the intraoperatively generated information, and the corresponding section is at least reflected and superimposed on the imaged mirror image to generate the augmented image. The use of intraoperatively generated information is particularly advantageous when different visualization options are activated at different times. For example, information about a portion of the dentition can be obtained in fluorescence mode, and this information can then be used for augmentation in normal vision mode or white light mode. In particular, this information can be processed, analyzed, and, in particular, classified before further use.

[0033] In a further embodiment, the image capture device is an image capture device of a surgical microscope, which can serve, in particular in medical applications, to provide a magnified representation of an object or area to be examined, and thus in particular a sub-area of ​​or inside the mouth.

[0034] The surgical microscope may include an image capture device. The image capture device may be designed to generate a two-dimensional image. In this case, the image may be generated with a predetermined number of pixels and thus a predetermined resolution. For example, the image sensor of the image capture device may be a CMOS sensor or a CCD sensor. Obviously, other types of sensors may also be used.

[0035] The surgical microscope may also include a stereo camera system including a first image capture device and a further image capture device, the image capture device generating the mirror image being one of the image capture devices. The stereo camera system may be a calibrated stereo camera system.

[0036] A surgical microscope with a stereo camera system may include two optically separated beam paths, with the first image capture device positioned and / or designed to generate an image based on a beam directed in the first beam path. The further image capture device may be positioned and / or designed to generate a further image based on a beam directed in the further beam path. In particular, these images may be generated simultaneously. Furthermore, the images generated by the first image capture device and the further image capture device may be referred to as corresponding images. As explained above, these may serve to generate a three-dimensional image.

[0037] Furthermore, the surgical microscope may comprise at least one optical element for beam guidance and / or beam shaping, which may in particular take the form of a lens element, and which may serve, for example, to generate a magnified image. The optical properties of the surgical microscope, such as magnification, focus, zoom, exposure time, and capture area size, may be adjustable.

[0038] The surgical microscope further comprises at least one eyepiece through which or into which a user can gaze to visually capture an image produced by the surgical microscope. In particular, the user can also view the examination area through the eyepiece. The surgical microscope may comprise at least one objective lens or objective lens system, the objective lens system including at least one optical element for beam guidance and / or beam shaping. The eyepiece may or may be optically connected to the objective lens.

[0039] The surgical microscope may be part of a microscope system, which may comprise not only the surgical microscope but also a stand for holding the surgical microscope. In this regard, the stand may be designed to allow movement of the surgical microscope in space with at least one degree of freedom, preferably six degrees of freedom, where the degrees of freedom may be translational or rotational. The relevant degrees of freedom may be related to a reference coordinate system. The vertical axis (z-axis) of this reference coordinate system may be oriented parallel to and opposite to gravity. The longitudinal axis (x-axis) and the transverse axis (y-axis) of this reference coordinate system may span a plane oriented perpendicular to the vertical axis. Furthermore, the longitudinal and transverse axes may be oriented perpendicular to each other. Furthermore, the stand may comprise at least one drive device, e.g., a servomotor, for moving the surgical microscope. The stand may also comprise means for transmitting forces / moments, e.g., gears and / or coupling units. Thus, the surgical microscope may be mounted or held in a movable manner. Among other things, this allows the user to change the attitude, i.e. the position and / or orientation, of the surgical microscope, for example to change the viewing angle in the examination area or to view other examination areas.

[0040] The operating microscope may in particular be a dental operating microscope, which is designed to generate images for dental applications.

[0041] Since surgical microscopes are often used for magnified representations in the field of dental applications as well, this advantageously leads to an increase in the functionality of these surgical microscopes: the user can still be provided with extended magnification in a simple and reliable manner, in particular without having to switch between a magnified representation of intraoperative information, for example by means of a magnifying glass, and the display of preoperative information in this case.

[0042] In a further embodiment, the overlay is further performed based on at least one optical property of the mirror element. In this connection, the at least one optical property may be predetermined. Alternatively, the at least one optical property may be determined in a further step of the proposed method, in particular in an image-based manner, i.e., by evaluating at least one image of the mirror element. In particular, the optical property of the mirror element may be a magnification or reduction property. It is obvious that other optical properties of the mirror element that affect the mirror imaging may also be taken into account. For example, the optical property may be represented by the described projection matrix. This advantageously leads to a good accuracy of the overlay.

[0043] Alternatively or additionally, the registration is further performed based on at least one imaging property of the surgical microscope. In particular, the imaging property may be a set magnification (zoom), a set focus, or any other imaging property that affects the imaged mirror image. This imaging property may also be represented by the described projection matrix. This also results in a very accurate registration.

[0044] In a further embodiment, a portion of the pre-operative information is determined as a function of at least the orientation of the mirror element, the portion of the pre-operative information is assigned to the image of the virtual image capturing device, and at least one corresponding section of the dentition reflected in the catadioptric system is imaged in the image of the virtual image capturing device. Further, the augmented image is generated by superimposing the portion of the pre-operative information on the section of the dentition reflected in the catadioptric system.

[0045] The catadioptric system refers to a relay optical system comprising at least mirror elements and optical elements of an image capture device, such as an objective lens. If a surgical microscope is used, the catadioptric system may comprise optical elements of the objective lens of the surgical microscope. The beam path through the catadioptric system can be determined based on the optical properties of the optical elements of the catadioptric system, which are known or determinable in advance, as well as on the orientation of the mirror elements and known laws of optics.

[0046] The virtual image capture device is a mathematical or physical model of the image capture device, which can be evaluated, in particular, with computer support. Depending on the model, it is possible to generate a virtual image generated by the virtual image capture device, in particular by computer-implemented calculation of pixels. This virtual image depends, inter alia, on the parameters of the (modeled) image capture device and the pose of the (modeled) image capture device. If the mirror surface is, for example, not curved and does not have magnification properties, the internal parameters of the virtual image capture device may be equal to the internal parameters of the modeled image capture device. In particular, the pose of the virtual image capture device can be determined based on the pose of the mirror element, such that the virtual image of the virtual image capture device in this pose images the non-reflected section of the dentition that is reflected by the catadioptric system and thus also by the mirror element. This non-reflected section represents the corresponding section.

[0047] In addition to the orientation of the mirror elements, the generation of such a virtual image also depends on (further) properties of the catadioptric system, for example the set zoom of the objective lens.

[0048] The augmented image can then be generated by superimposing the portion of the preoperative information onto the section of the dentition reflected by the catadioptric system. This can be done by reflecting the portion of the preoperative information based in particular on the properties of the catadioptric system, and thus on the orientation of the mirror elements, and then superimposing it onto the imaged mirror image. In other words, the portion of the preoperative information can thus be transformed into the described reference coordinate system, where the transformation is determined by the properties of the catadioptric system. Advantageously, this results in a superposition that is accurate and easy to perform from a computational standpoint.

[0049] In a further embodiment, the pose is determined by evaluating at least one characteristic of the imaged mirror image or the imaged mirror element (or a section thereof). The at least one characteristic can be determined in an image-based manner, in particular by evaluating an image generated by an image capture device. In particular, the imaged mirror surface or the imaged mirror element can be recognized in such an image, for example, by an object recognition method known to those skilled in the art. For example, the object recognition method may be a segmentation method. Thus, for example, the imaged mirror surface, the imaged frame section, or the imaged handle section can be recognized in an image-based manner. For example, it is possible to form a section of the mirror element, for example, the frame section, from a material with predetermined optical properties, for example, from a matte material, in order to enable reliable detection of this section in the image. Alternatively, detection can also be performed by a user selecting, for example by means of a suitable input device, an image area in which the mirror image or the section to be detected is imaged.

[0050] The property of the imaged mirror image or imaged mirror element may be a geometric property of the imaged mirror image, e.g., a dimensional property such as a dimensional variable. The dimensional variable may be a width, a height, a diameter, or any other dimensional variable. Furthermore, the property may be a shape property, e.g., a geometric shape such as a circle, an ellipsoid, a rectangle, or any other geometric shape. In particular, a shape factor may be determined that represents a relationship between the imaged shape and the actual shape, and the pose is determined based on the shape factor.

[0051] As explained above, the pose of a mirror element can affect its imaging by an image capture device. Therefore, the pose can also affect how the actual properties of a mirror image or mirror element are mapped to the properties of the imaged mirror image or mirror element. If the relationship between the actual properties and the properties of the imaged mirror image can be described by a transformation matrix that depends on the pose, the pose can be determined based on the actual properties and the properties of the imaged mirror image or mirror element. The actual properties may already be known and may be determinable, for example, from a model of the mirror element, particularly a CAD model.

[0052] If the mirror element or part thereof, in particular the mirror surface, is circular and the imaged mirror image is elliptical, the pose can be determined based on the properties of the ellipse, such as the orientation and length of the ellipse axis, and on the properties of the circular mirror element that are known in advance, in such a way that the known properties are transformed into properties of the imaged mirror image. If polygonal mirror elements, in particular regular polygonal mirror elements, are used, the pose can be determined at least in part by the ratio of the lengths of the edges in the image and the relative positions of the edges in the image with respect to each other.

[0053] Furthermore, the characteristic may be the orientation of the imaged mirror image or the imaged mirror element or part thereof in the image coordinate system. For example, the position may be determined as the position of a reference point, e.g., a geometric center. For example, the orientation may be determined as the orientation of an axis of a reference section. For example, if the mirror element includes a handle section, the handle section may be recognized in the image, and its position and / or orientation may be determined. For example, the orientation of the longitudinal axis of the handle section may be determined.

[0054] If the attitude is determined by evaluating at least one characteristic of the imaged mirror image or of the imaged mirror element, this advantageously results in a simple determination of the attitude, since the image generated in any case can be evaluated for the purpose of determining the attitude.

[0055] Alternatively, the attitude may be determined based on markers. To this end, the mirror element may include or take the form of at least one marker element for determining the attitude of the mirror element. Obviously, the mirror element may also include or take the form of multiple markers, and the attitude of the mirror element may be determined based on the relative positions of these markers, which are known in advance.

[0056] The marker elements may be active or passive. They may be designed to be captured by a capture device. In particular, the capture device may be an image capture device. Accordingly, in this case, the marker elements may be optically captureable marker elements. For example, the optically captureable marker may include a predetermined pattern that allows the orientation of the marker, and therefore the orientation of the mirror element, to be determined. For example, such an optically captureable pattern may take the form of a QR code. Furthermore, such optically captureable markers may be reflective marker elements, which are designed to reflect, for example, radiation from a predetermined wavelength range, such as an infrared wavelength range. The image capture device that optically captures the marker elements may be an image capture device that images a mirror image or a different image capture device. If the mirror element includes multiple marker elements, the orientation of the mirror element may also be determined based on the relative positions of the imaged marker elements in the image. Additionally, the pose of the mirror elements may be determined, at least in part, by determining the stereoscopic pose of at least the marker elements.

[0057] The image capture device and the marker elements can be used to perform what is known in particular as monoscopic pose determination. In this case, the pose can be determined by evaluating a two-dimensional image, in particular exactly one two-dimensional image of exactly one image capture device. In particular, an evaluation of the intensity values ​​of the pixels of the two-dimensional image can be performed to determine the position. Such methods of image-based position detection using exactly one image capture device and / or based on exactly one two-dimensional image are known to those skilled in the art. However, if a stereo camera system is used, the pose can also be determined by evaluating corresponding images of the image capture device.

[0058] In particular, the pose can thus be determined by optical tracking methods, in particular using an image capture device, which may be marker-based tracking, in which case specific visually or optically captureable marker elements, such as QR codes or optical patterns of different designs, are used to determine the pose.

[0059] Alternatively, in particular in determining the above-mentioned pose by evaluating at least one property of the imaged mirror image or the imaged mirror element, markerless tracking methods can also be applied, in which features are captured and used to determine the pose.

[0060] However, instead of optically capturable marker elements, it is also possible to use marker elements that can be capturable in other ways to determine the orientation of the mirror element, for example magnetically, capacitively, inductively or in a radio-based manner, for example the marker elements may be designed as RFID tags.

[0061] The mirror element may also include an orientation sensor, such as an initial sensor or a GNSS sensor, and the orientation may be determined based on the output signal of such a sensor. In such an embodiment, the surgical microscope may include or be connected to a receiving device for the output signal generated by the orientation sensor.

[0062] Obviously, it is also possible to determine pose using a hybrid method, which combines at least two of the methods for determining pose described above.

[0063] Marker-based pose determination advantageously provides very accurate pose determination, resulting in accurate three-dimensional images.

[0064] The marker elements may be identifiable, in particular bijectively. For example, a pattern of optically identifiable markers may encode the marker's identity. Thus, the marker elements or mirror elements may be made identifiable by incorporating the marker elements. The thus determinable identity may be assigned to the properties of the mirror elements, in particular the optical properties described above. This assignment, as well as the identity and the properties, may be stored in a retrievable or readable manner, for example in a memory device. This advantageously allows the optical properties of the mirror elements to be simply determined.

[0065] For the purpose of marker-based pose determination, in a further embodiment, at least one marker element is imaged by the at least one image capturing device or by a further image capturing device. The at least one marker element is arranged on or formed by a mirror element. The pose is then determined based on at least one property of the imaged marker element, as already explained above. The further image capturing device may in particular be a tracking camera or an environmental camera of the microscope system, which takes a different form from the image capturing device of the surgical microscope intended for magnified imaging. This tracking camera or environmental camera can in particular serve for marker-based tracking of further instruments. In any case, this advantageously results in the simplest possible integration of optical pose determination, especially when using a surgical microscope or a microscope system comprising a surgical microscope.

[0066] In a further embodiment, the focal position of the image capture device is set based on the orientation of the mirror element. In particular, this allows the focal position to be set on a point on the mirror surface or on a point spaced a predetermined distance from the mirror surface. In particular, the predetermined distance depends on the depth of field of the image capture device or of the surgical microscope and can be smaller than the depth of field range. In this case, the depth of field is known or can be determined in advance. As a result, a high imaging quality can be obtained for the mirror image, which advantageously increases the accuracy of the generated three-dimensional image. In particular, this also allows the imaged mirror image to be more easily and reliably detected in the image of the image capture device.

[0067] In a further embodiment, the radiation captured to generate the image of the image capture device is filtered. In a preferred embodiment, the filtering is polarization filtering. However, it is clear that other radiation filters may also be used. Advantageously, this allows for the suppression of unwanted reflections from the tooth surface in the image, which in turn allows for a better visualization of the dilation. In this case, polarization filtering allows for the greatest possible or complete suppression of reflections.

[0068] In further embodiments, the filter element is arranged in the illumination beam path and / or in the imaging beam path, in particular of the image capture device or of the surgical microscope. If multiple imaging beam paths are present, a respective filter element can be arranged in each imaging beam path. In either case, a good structural integration of the filter element in the image capture device or surgical microscope or microscope system is provided, thereby enabling the generation of high-quality augmented images.

[0069] Further alternatively or additionally, a filter element may be arranged on the mirror element. For example, the filter element may be arranged on a mirror surface of the mirror element. Advantageously, the mirror element provides the desired filter characteristics, and as a result, there is no need to integrate an additional radiation filter element in the image capture device or in the surgical microscope or microscope system.

[0070] In a further embodiment, at least one partial section of the dentition is illuminated by radiation having predetermined radiation characteristics. For example, such radiation characteristics may be (a) a predetermined wavelength of the radiation used for illumination purposes, a predetermined intensity, or, in a preferred embodiment, predetermined polarization or further characteristics of the radiation used for illumination purposes. Advantageously, this also allows for reduced reflections from the tooth surfaces, which further has a beneficial effect on the quality of the augmented image.

[0071] In a further embodiment, based on the images generated by the image capture device, a registration is performed in the reference coordinate system of the pre-operatively generated information and the reference coordinate system of the image capture device. This has already been explained above and advantageously allows a simple integration of the registration in the proposed method. In particular, corresponding features in the image and the pre-operative information can be detected for the purpose of registration, for example by detection methods known to those skilled in the art, and the registration is then performed based on these detected features.

[0072] In a further embodiment, an image is generated at a first magnification, and an augmented image is generated at a further magnification different from the first magnification, in particular greater than the first magnification. In other words, the alignment can be performed based on an image generated using a magnification smaller than the image having the magnification used to determine the augmented image. This advantageously results in accurate and reliable alignment, since a smaller magnification allows a larger area to be imaged, thereby making more information available for alignment. The magnifications for generating images at different magnifications can be generated optically or digitally. For example, an image at a first magnification can be generated at an optically set magnification, and an image at a further magnification can be generated using optically and digitally set magnifications. This further improves the quality of the augmentation.

[0073] In a further embodiment, the registration is performed based on the mirror image. In particular, corresponding features in the detected mirror image and the preoperative information can then be detected, and the registration is then performed based on these detected features. For this purpose, it may be necessary to reflect the preoperative information. This advantageously allows for a simple integration of the registration in the proposed method. If only the reflected section is imaged by the image capture device's image, the detection of the mirror element may also be omitted for the registration in this procedure.

[0074] A system for generating an enhanced image of at least a partial section of a dentition is also proposed, the system comprising at least one image capturing device and at least one evaluation device.

[0075] The system may comprise a stereo camera system having a first image capture device and a further image capture device.

[0076] The system may comprise a method according to one of the embodiments described in the present disclosure, in particular a) detecting an imaged mirror image in an image generated by an image capture device, the mirror image being provided by a mirror element, the mirror element being positioned in a capture area of ​​the image capture device and reflecting at least one partial section of the dentition; b) determining the pose of the mirror element; c) determining in the preoperatively generated information a section that corresponds to a partial section of the dentition depending on at least the orientation of the mirror element; d) generating an expanded image by reflecting and superimposing the corresponding section onto at least the imaged mirror image; is configured to run

[0077] The evaluation device may take the form of or comprise a computing device, which may further comprise or take the form of a microcontroller or an integrated circuit, in which case the evaluation device is capable of performing at least one of steps a), b), c) and d), but preferably all of these steps.

[0078] The system may be a component of a surgical microscope or a microscope system, which may comprise a stereo camera system and an evaluation device. Furthermore, the system may comprise a capture device for capturing marker elements. Furthermore, the system may comprise a filter element for filtering radiation that serves to generate the image of the stereo camera system. Furthermore, the system may comprise an illumination device for illuminating the partial section with a predetermined radiation characteristic.

[0079] The system advantageously allows the implementation of a method according to one of the embodiments described in the present disclosure, with the advantages also already described.

[0080] In a further embodiment, the system comprises a mirror element.

[0081] A computer program product is also proposed having a computer program, the computer program comprising software means for performing one, some or all steps of the method according to one of the embodiments described in the present disclosure when the computer program is executed by or in a computer or automation system.

[0082] A mirror element for generating an extended image of at least a partial section of the dentition is also described. According to the invention, the mirror element comprises or forms at least one marker element for determining the orientation of the mirror element. Alternatively or additionally, the mirror element comprises or forms at least one filter element for filtering reflected radiation. This and the corresponding advantages have already been explained above.

[0083] The present invention will now be described in detail based on exemplary embodiments. [Brief explanation of the drawings]

[0084] [Figure 1] 1 shows a schematic flow chart of a method according to the present invention. [Figure 2] 4 shows a schematic flow chart of a method according to the invention in a further embodiment; [Figure 3a] 4 shows a schematic block diagram of a system according to the invention in a further embodiment; [Figure 3b] 4 shows a schematic block diagram of a system according to the invention in a further embodiment; [Figure 4a] 1 shows a mirror element in a first position. [Figure 4b] 4b shows the mirror element shown in FIG. 4a in a further position; [Figure 4c] 4b shows the mirror element shown in FIG. 4a in a further position; [Figure 4d]4b shows the mirror element shown in FIG. 4a in a further position; [Figure 5] 1 shows a schematic diagram of a mirror element according to the present invention; [Figure 6] FIG. 1 shows a schematic block diagram of a virtual image capturing device. [Figure 7] 1 shows an exemplary augmented image. DETAILED DESCRIPTION OF THE INVENTION

[0085] Hereinafter, the same reference numerals refer to elements having the same or similar technical features.

[0086] FIG. 1 shows a schematic flowchart of a method according to the present invention for generating an augmented image AA of at least a partial section of dentition Z (see FIG. 3a). In a first step S1 of the method, an imaged mirror image S5 is detected in an image I5 generated by an image capture device 5 (see, for example, FIG. 3a). This mirror image is provided by a mirror element 3, which is arranged in a capture area 4 of the image capture device 5 so that the image capture device 5 can image a mirror image of at least one partial section of dentition Z. This detection in the first step S1 can be performed using object recognition methods known to those skilled in the art. In addition to the partial area in which the mirror image provided by the mirror element 3 is imaged, the image I5 generated by the image capture device 5 may in this case include further partial areas, for example, imaging other partial sections of dentition Z, in particular partial sections not reflected by the mirror element 3. The thus detected imaged mirror image S5 forms an input variable for determining the augmented image AA.

[0087] In a second step S2, the pose P of the mirror element 3 is determined. The reference coordinate system may be the coordinate system of the image capturing device 5 serving as a reference or a global coordinate system serving as a reference. Obviously, other reference coordinate systems are also conceivable. In addition to the imaged mirror image S5, the pose P forms a further input variable for determining the augmented image AA.

[0088] Then, in a third step S3, based on this orientation P of the mirror element 3, a section in the preoperatively generated information PI that corresponds to a partial section of the dentition Z can be determined based on the preoperatively generated information PI that was generated before the third step S3 was performed and the alignment information RI that was generated before the first step S1 was performed. The alignment information RI allows the preoperatively generated information PI and the image to be transformed into a common reference coordinate system, which may for example be the coordinate system of the image capture device 5. The orientation P of the mirror element also advantageously allows for the transformation into this reference coordinate system.

[0089] In a fourth step S4, the corresponding section in the preoperatively generated information is reflected and superimposed on at least the imaged mirror image to generate the augmented image AA. This superimposition can be performed using image fusion methods known to those skilled in the art. In other words, the corresponding section in the preoperatively generated information is superimposed on a partial section of the image in which the mirror image is imaged.

[0090] Prior to a first step S1 of the method, a mirror element 3 is positioned in the capture area 4 of the stereo camera system 1 so as to be able to image a mirror image of at least a partial section of the dentition Z by the image capture device 5 (positioning step). Following this positioning, an image I5 is generated using the image capture device 5 (image generation step).

[0091] 2 shows a schematic flow chart of a method according to a further embodiment. Prior to the implementation of the four steps S1, S2, S3, S4 of the method, an image I5_1 is generated at a first magnification using an image capture device 5 (see, for example, FIG. 3a) in an image generation step BSR. Based on the image I5 thus generated, in an alignment step RS, alignment information RI shown in FIG. 1 is determined. Those skilled in the art can use known methods of image-based alignment for this purpose.

[0092] In that case, before the first step S1, the magnification ratio for generating the image can be changed, and then the first step S1 is performed based on image I5_2 generated using a further magnification ratio different from the first magnification ratio, in particular greater than the first magnification ratio.

[0093] 3a shows a schematic block diagram of a system according to the invention in a first embodiment, the system comprising an image capture device 5 and an evaluation device 10. Also shown are a dentition Z having a front side 7, a back side 8 and a chewing surface 9, and a mirror element 3 arranged in a capture area 4 (shown in dashed lines) of the image capture device 5. Also shown are the mirror elements 3, which reflect a beam from the dentition Z, in particular from its back side 8, in particular into the imaging beam path of the image capture device 5.

[0094] A memory device 16 for the pre-operative information PI is also represented, from which the pre-operative information PI can be retrieved, for example, by the evaluation device 10. The evaluation device 10 can then be used to detect the imaged mirror image provided by the mirror element 3 in the image generated by the image capture device 5.

[0095] The orientation P of the mirror element 3 can also be determined by the evaluation device 10 or a further device different therefrom. Furthermore, steps S3, S4 shown in Fig. 1 can be performed by the evaluation device. The alignment information required for this purpose can likewise be retrieved from the memory device 16, but can also be retrieved from a separate memory device.

[0096] Figure 3b shows a schematic block diagram of a system according to the invention for generating an augmented image AA of at least a partial section of the dentition Z, the system comprising a stereo camera system 1 and at least one evaluation device 10. The system is configured to perform at least steps S1, S2, S3, S4 shown in Figures 1 and 2. It will be appreciated that the system can also be configured to perform steps BSR, RS shown in Figure 2, in which case these steps or at least some of them can be performed by the evaluation device 10.

[0097] FIG. 3b schematically illustrates the capture area EB of the image capture devices 5a, 5b and the optically separated beam paths 11a, 11b of the surgical microscope 2. The illumination device of the surgical microscope 2, which can illuminate the dentition Z, is not shown. In this case, the illumination device can generate radiation with predetermined radiation characteristics, particularly predetermined polarization characteristics. The radiation reflected by the dentition Z reaches the image sensors of the image capture devices 5a, 5b via the beam paths 11a, 11b, thereby generating images I5a, I5b of the dentition Z, which can be evaluated by the evaluation device 10. Also shown is the mirror element 3, which reflects the beam from the dentition Z, particularly its back side 8. This reflected radiation also reaches the image sensor via the beam paths 11a, 11b and is imaged there as a mirror image. This image can then be detected as a mirror image by the evaluation device 10.

[0098] The radiation captured to generate the images I5a, I5b of the stereo camera system 1 can be filtered. This can be done, for example, by filter elements, which are arranged in each case in the beam paths 11a, 11b. Furthermore, filter elements may be arranged in the illumination beam path of an illumination device (not shown) of the surgical microscope 1. Furthermore, filter elements may be arranged on / at the mirror element 3. In particular, such filter elements may be polarizing filter elements. It is also possible for the evaluation device 10 to filter the images I5a, I5b generated by the image capture devices 5a, 5b, for example to suppress reflections.

[0099] A memory device 16 for the preoperative information PI and the alignment information RI is also represented, from which the evaluation device 10 can retrieve, for example, in particular when the third step S3 is performed. In a fourth step S4, an augmented image AA can be generated by the evaluation device 10.

[0100] Figure 4a shows an image of a mirror element 3 in a reference position. The mirror element 3 comprises a handle section 12 and a circular mirror section 13, which further comprises a mirror surface 14. For example, the centre of this mirror surface 14 is the reference point P3 of the mirror element 3. A coordinate system at rest relative to the mirror is depicted, which has longitudinal, horizontal, y and vertical axes z3 (see Figure 4b).

[0101] Figure 4b shows an image of the mirror element 3 in a position P, which occurs when the mirror element 3 is rotated about the longitudinal axis x3 from the reference position shown in Figure 4a. It is clear that the round mirror surface 14 shown in Figure 4a is now imaged as an ellipse. Depending on the orientation and length of the major and minor axes of this ellipse, which can be detected, for example, by object recognition methods, it is possible to determine the rotation angle by which the mirror element 3 has rotated about the longitudinal axis x3, and again the current position P of the imaged mirror element 3 shown in Figure 4b.

[0102] In a similar manner, Figures 4c and 4d show the imaged mirror element 3 rotated about the horizontal axis y3 (Figure 4c) or about the vertical axis z3 (Figure 4d) relative to the reference position shown in Figure 4a. For example, the corresponding rotation angle can be determined based on the orientation of the vertical axis x3 (Figure 4d) and / or based on the orientation and length of the axes of the elliptical image of the mirror surface 14.

[0103] It can be seen from Figures 4a to 4d that a shape-based determination of the pose P of the mirror element 3 can be made, that is, the shape characteristics of the imaged mirror element 3 can be determined and then the pose P can be determined based on these characteristics.

[0104] It is also clear that the center of the mirror surface 14 can be detected. If the focus is directed not only at the point reflected at the mirror surface 14, for example at the center, but also at the non-reflected edge of the mirror surface 14, it is possible to ascertain the distance of the mirror surface 14, and in particular the reflecting point, from the dentition Z by the difference in focus position.

[0105] In particular, it is possible to determine the difference between the focal position when the stereo camera system 1 or the surgical microscope 2 is focused on a point of an edge that is not reflected, e.g., a point of the frame section 13, and the focal position when the stereo camera system 1 or the surgical microscope 2 is focused on a point of the object that is reflected on the mirror surface 14, e.g., a point that is reflected on the center of the mirror surface 14. This difference in focal positions can represent the distance of the mirror element 13 from the object, in this case, a point on the dentition Z, and the distance can therefore be determined based on this difference. Furthermore, the distance can also be determined based on the orientation P of the mirror element 13. This distance information can be used for scaling within the described stereo reconstruction, in particular to match the magnification when reconstructing a reflected section, e.g., the back side 8 of the dentition Z, to the magnification when reconstructing a non-reflected section, e.g., the front side 7 of the dentition Z, i.e., to perform scaling.

[0106] 5 shows a schematic diagram of a mirror element 3 according to the present invention. The mirror element 3 includes or forms, on the handle section 12, marker elements 6 in the form of optically captureable bar codes. The mirror element 3 also includes, on the frame section 13 of the mirror surface 14, further optically captureable marker elements 6 in the form of bar codes. These can be captured in an image of the mirror element 3, which marker elements allow, among other things, to identify the mirror element 3 and to determine the pose P of the mirror element 3. In particular, each marker element 6 can be detected in the case of the mirror element 3 shown in FIG. 5, after which the pose P can be determined by the relative placement of the marker elements in the image.

[0107] Figure 6 shows a schematic diagram of an image capture device 5 (see Figure 3a) and a mirror element 3 arranged in the capture area 4 of the image capture device 5. A point of interest OP to be imaged, e.g. a point on the surface of the dentition Z (see Figure 3a), is also represented. The normal n of the mirror surface 14 of the mirror element 3 is also represented.

[0108] A virtual image capture device 15 is also represented. The (virtual) image of this virtual image capture device 15 can be determined by evaluating a mathematical or physical model. In particular, the model is determined in such a way that a virtual image of the preoperative information PI is generated, the virtual image imaging object points OP corresponding to sections of the dentition Z in a non-reflective manner, i.e., in particular sections reflected by a catadioptric system, however taking into account the properties of the catadioptric system. Then, based on the virtual image, an augmented image AA can be determined, for example, by superimposing the virtual image on a reflected and detected mirror image S5.

[0109] 7 shows an exemplary augmented image AA in which a portion of the dentition Z is imaged, along with a mirror image S5 of the back side 8 of the dentition Z provided by the mirror element 3 having the mirror surface 14. Superimposed on this imaged mirror image S5 is pre-operative information PI in the form of a circle, characterizing a predetermined section of the tooth.

[0110] Instead of capture by an image capture device, in a similar manner, a different capture system used in a surgical microscope can be used to optically capture information, in particular depth information, and provide it in particular in the form of an image. In this optical information, it may also be possible to detect a mirror image by means of a mirror element arranged in the capture area of ​​the surgical microscope, in particular in the capture area of ​​the capture system, which mirror image makes it possible, inter alia, to capture optical information of a partial section of the dentition for the surgical microscope, in particular for another capture system. Furthermore, in the preoperatively generated information, it is possible to determine a section of the dentition that corresponds to this partial section, in particular based on a previously determined orientation of the mirror element, and to generate an extended image by superimposing the corresponding section at least reflected on the imaged mirror image.

[0111] Instead of or in addition to using preoperatively generated information to generate the augmented image, intraoperative information, i.e., information recorded or generated during treatment, can also be used to generate the augmented image. For example, information about a partial section of the dentition can be collected and stored in this way during treatment, and this information can then be used to generate the augmented image. This is particularly advantageous when there are different visualization options that are activated at different times. For example, information about a portion of the dentition, in particular image information, can be obtained in fluorescence mode, and this information can then be used in normal vision mode or white light mode for augmentation. In particular, this information can be processed and analyzed, and in particular classified, before further use.

[0112] Thus, for example, a point on the surface of a partial section of the dentition can be touched or marked using an instrument, and the surface of the partial section can be captured by an image capture device using a mirror element. This is one possibility for generating the intraoperative information described above. For example, the marking can then be shown in the augmented image, particularly based on the orientation of the mirror element. For example, the marking can be shown as a predetermined geometric element. Alternatively or additionally, a different sensor system can be used to capture information about the partial section of the dentition. This is a further option for generating the intraoperative information described above, which can then be shown in the augmented image. [Explanation of symbols]

[0113] 1 Stereo camera system 2 Surgical microscope 3 Mirror Elements 4 Capture Area 5, 5a, 5b Image Capture Device 6 Optically Captivable Marker Elements 7 Front side 8 Back side 9 Chewing surface 10 Evaluation Devices 11a, 11b Beam path 12 Handle Sections 13 Frame Section 14 Mirror surface 15 Virtual Image Capture Device 16 Memory Devices SBV image generation step SRV Reconfiguration Step I5, I5a, I5b images I5_1, I5_2 images S5, S5a, S5b Imaged mirror image P posture A1, A2, A images OP target point EB capture area S1 First step S2 Second step S3 Third step S4 Fourth step AA Extended Image PI preoperative information RI alignment information KA corresponding section

Claims

1. 1. A method for generating an augmented image (AA) of at least a partial section of a dentition (Z), comprising: a) detecting an imaged mirror image (S5) in an image generated by an image capture device (5, 5a, 5b), said mirror image being provided by a mirror element (3) arranged in a capture area (4) of said image capture device (5, 5a, 5b) such that said mirror image (S5) of said at least one partial section of said dentition (Z) can be imaged by said image capture device (5), b) the orientation (P) of said mirror element (3) is determined; c) a section (kA) corresponding to said partial section of said dentition (Z) is determined in preoperatively generated information (PI) depending on at least said position (P) of said mirror element (3), d) said augmented image (AA) is generated by reflecting and superimposing said corresponding section (KA) onto at least said imaged mirror image (S5).

2. 2. The method according to claim 1, characterized in that the image capture device (5) is an image capture device (5a, 5b) of a surgical microscope (2).

3. 3. The method according to claim 1 or 2, wherein the superposition is further performed based on at least one optical property of the mirror element (3) and / or at least one imaging property of the image capturing device (5).

4. A method according to any one of claims 1 to 3, characterized in that an image of a virtual image capturing device is determined depending on at least the position of the mirror element, the at least one partial section of the dentition is imaged without being reflected in the image, the corresponding section is superimposed on the non-reflected partial section, thereby generating a virtual augmented image, and the virtual image is converted into the augmented image.

5. 5. The method according to any one of claims 1 to 4, characterized in that the pose (P) is determined by evaluating at least one property of the imaged mirror image (S5) or the imaged mirror element (3), or in a marker-based manner.

6. 6. The method according to claim 5, characterized in that for the marker-based determination of the pose (P), at least one marker element (6) is imaged by the image capturing device (5) or a further image capturing device, the at least one marker element (6) being arranged on or formed by the mirror element (6), and the pose (P) being determined depending on at least one characteristic of the imaged marker element (6).

7. The method according to any one of claims 1 to 6, characterized in that the focus position of the image capturing device (5) is set based on the orientation (P) of the mirror element (3).

8. Method according to any one of claims 1 to 7, characterized in that the radiation captured for the purpose of generating the image of the image capture device (5) is filtered.

9. 9. The method of claim 8, wherein the filtering is polarization filtering.

10. 10. Method according to claim 8 or 9, characterized in that a filter element is arranged in the illumination beam path and / or in the imaging beam path (11a, 11b) and / or above the mirror element (3).

11. Method according to any one of the preceding claims, characterized in that the at least one partial section of the dentition (Z) is irradiated with radiation having a predetermined radiation characteristic.

12. 12. The method of claim 11, wherein the radiation is generated with predetermined polarization characteristics.

13. 13. The method according to any one of claims 1 to 12, characterized in that the alignment of the reference coordinate system of the preoperatively generated information (PI) with the reference coordinate system of the image capture device (5) is performed on the basis of the images generated by the image capture device (5).

14. 14. A method according to claim 13, characterized in that the image is generated at a first magnification and the augmented image (AA) is generated at a further magnification different from the first magnification, in particular greater than the first magnification.

15. 15. Method according to claim 13 or 14, characterized in that the alignment is performed on the basis of the mirror image (S5).

16. A system for generating an augmented image (AA) of at least a partial section of a dentition (Z), comprising at least an image capture device (5) and at least one evaluation device (10), a) detecting an imaged mirror image (S5) in an image generated by the image capture device (5), the mirror image (S5) being provided by a mirror element (3) arranged in a capture area (4) of the image capture device (5) and reflecting the at least one partial section of the dentition (Z); b) determining the pose (P) of said mirror element (3); c) determining in preoperatively generated information (PI) a section (KA) corresponding to said partial section of said dentition (Z) depending on at least said pose (P) of said mirror element (3); d) said corresponding section (KA) is reflected and superimposed on at least said imaged mirror image (S5) to generate said extended image (AA); A system configured to execute the

17. 17. A system according to claim 16, characterized in that it comprises a mirror element (3).

18. 16. A computer program product having a computer program, said computer program comprising software means for performing one, some or all of steps a) to d) of the method of any one of claims 1 to 15 when said computer program is executed by or in a computer or automation system.

Images