Methods, systems, and computer program products for generating at least partial dentition enhanced images

Abstract

The invention relates to a method for generating an enhanced image (AA) of an at least partially dentition (Z), comprising the following steps: a) detecting an imaged image (S5) in an image generated by an image acquisition device (5, 5a, 5b), which image is provided by a mirror element (3) arranged in an image acquisition region (4) of the image acquisition device (5, 5a, 5b), enabling the image acquisition device (5) to acquire a mirror image (S5) of at least part of the dentition (Z); b) determining a posture (P) of the mirror element (3); c) determining a region (KA) corresponding to the partial region of the dentition (Z) in the preoperatively generated information (PI), at least on the basis of the position (P) of the mirror element (3); d) generating an enhanced image (AA) by at least reflecting the corresponding region (KA) and superimposing it on the imaged image (S5).

Description

Technical Field

[0001] The present invention relates to a method, system and computer program product for generating an enhanced image of at least a portion of a dentition. Background Art

[0002] Preoperative information refers to information used for diagnosis and / or surgical planning, particularly in medical procedures such as implant surgery or restorative / aesthetic treatments for patients. In dental applications, this preoperative information can be generated using intraoral scans, which are used to obtain information about the geometry of the teeth. This information can then be used to determine the state of the teeth or jaw, particularly to document the different stages of treatment. Furthermore, this intraoral scan can be used to plan and fabricate implants or crowns.

[0003] Preoperative information can also be generated, particularly in dentistry, through methods such as computed tomography (CT) or magnetic resonance imaging (MRI), which can be used to examine or image the internal structures of teeth, such as the tooth roots or nerves. This information can also be used for surgical planning, for example to determine the target position and orientation of the drill for drilling operations.

[0004] In addition to anatomical information, preoperative information can also include other additional information, such as planning information. For example, planning information can indicate which areas of the tooth should be removed to ensure a secure fit for the crown, or indicate the location of sensitive nerves in the tooth to avoid damage during surgery.

[0005] During surgery, preoperative information can be displayed on a display device, for example, to enhance information generated during surgery. For example, a single enhanced image can be provided that simultaneously presents both preoperative and intraoperative information.

[0006] Medical navigation systems are well known for detecting information such as the posture of the instrument being used. Based on its posture, the instrument can be displayed relative to preoperative data, for example, by overlaying it on an image generated from that data. This allows the physician to determine the relative posture of the instrument to invisible structures. This can be used, for example, to determine the correct drilling angle and depth for a drill bit. However, a disadvantage of this type of navigation system is that if the preoperative data is displayed on a monitor, the physician must maintain constant eye contact with the monitor during the procedure. In some cases, the monitor does not display the surgical site in real time. Consequently, the physician must switch their gaze back and forth between the monitor and the examination / surgical area. This disadvantage is particularly pronounced when the physician uses conventional magnifying glasses to view the surgical area, as these are often heavy and can hinder a clear view of the examination area due to head shaking.

[0007] Surgical microscopes are also known. Such microscopes can be used during treatment procedures to provide users with images, particularly magnified images, of the treatment area (particularly the surgical site). So-called stereoscopic surgical microscopes typically include two independent optical channels for beam guidance and provide the user with a sense of depth in the area being examined. To this end, the user observes the beams in both channels through eyepieces. Alternatively or in addition, digital surgical microscopes typically include two image acquisition devices, each capturing the beams in its own optical channel to generate an image. Based on these two images (hereinafter referred to as "corresponding images"), the user can obtain a three-dimensional image using an appropriate display device. Other types of surgical microscopes are also known that acquire optical data (particularly depth information) and can provide three-dimensional images of the treatment area. To this end, surgical microscopes may also employ optical detection systems that provide depth information based on methods such as interferometry, triangulation, time-of-flight (TOF), or microlens arrays, particularly as an alternative to stereoscopic vision systems.

[0008] To ensure accurate image presentation, the stereo camera system requires precise calibration, employing known calibration methods to determine the camera's intrinsic and extrinsic parameters. These parameters are then used in image processing to ensure accurate image presentation. Intrinsic parameters describe parameters specific to the individual cameras / image acquisition devices, such as lens distortion. Extrinsic parameters describe the relationships between image acquisition devices, particularly spatial relationships, and are used to determine the relative positions of images captured by the cameras. These intrinsic and extrinsic parameters are well known to those skilled in the art.

[0009] Known prior art includes German patent document DE 10 2020 133 627 A1, which discloses a method for detecting the surface morphology of a transparent object (especially a dental object) and an intraoral scanner.

[0010] Known prior art also includes German patent document DE 10 2019 008 510 A1, which also discloses an intraoral scanner, which is particularly suitable for performing three-dimensional scanning of the upper or lower jaw and its jaw structure with or without teeth during implant restoration.

[0011] There is also DE 10 2019 008 510 A1, which also discloses an intraoral scanner, particularly for three-dimensional scanning of toothed or edentulous upper or lower jaws and jaw components in the field of implant restorations.

[0012] German patent document DE 10 2016 121 687 A1 discloses an intraoral scanner for use in the dental field and a method for generating a digital dental impression using the intraoral scanner.

[0013] European Patent Document EP 3 689 295 A1 discloses a dental observation device in which a dental microscope is used.

[0014] Dental mirrors are typically used to treat the backs of teeth or molars, as well as to view the chewing surfaces of the teeth. They provide a mirrored view of individual teeth or entire rows of teeth. These mirrors are also used when generating oral images (especially magnified images) using image acquisition devices, particularly surgical microscopes. Summary of the Invention

[0015] The resulting technical problem is how to develop a method and system for generating at least partially enhanced images of the dentition, as well as a corresponding computer program product, so that when using a mirror component, correct image enhancement and superposition of preoperative information can still be achieved.

[0016] The solution to this technical problem is reflected in the features defined in the independent claim of the present invention. Further preferred embodiments of the present invention are detailed in the dependent claims.

[0017] The present invention proposes a method for generating an enhanced image of at least a portion of the dentition. The enhanced image can be an image obtained by computer-assisted enhancement of an image of a partial area of the real dentition, in particular by superimposing or covering at least one virtual object and / or additional information on the image of the partial area of the real dentition. The enhanced image can be presented to a user, such as a physician or surgeon, in particular through a suitable display device, such as a monitor or a head-mounted display. If (as described in detail below) the image of the partial area of the real dentition is generated by an image acquisition device of a surgical microscope, the enhanced image can also be optically presented to the user through the eyepiece of the surgical microscope. To this end, preoperative information can be introduced (for example by reflection) into the optical path leading to the eyepiece. If a surgical microscope equipped with a stereo camera system includes two image acquisition devices, corresponding enhanced images can be generated based on the images acquired by these two devices respectively. Therefore, an enhanced image with depth information, i.e. an enhanced three-dimensional presentation, can be provided to the user through a suitable display device or eyepiece.

[0018] In a first step, a mirror image is detected in an image generated by an image acquisition device, the mirror image being provided by a mirror element arranged in an image acquisition area of the image acquisition device such that the image acquisition device can acquire a mirror image of at least a portion of the dentition.

[0019] The partial area of the dentition may include at least one tooth, or a portion of a tooth. In the present invention, "tooth" may also refer to a denture.

[0020] Therefore, particularly before detecting the imaged mirror image, the mirror element can be positioned within the image capture area of the image capture device, enabling the image capture device to capture a mirror image of at least a portion of the dentition. Furthermore, before performing the detection, the image capture device can generate an image. The mirror element can be a dental mirror or a component thereof. The mirror element can include or utilize a mirror surface capable of reflecting radiation. The mirror image generated by the image capture device (i.e., the imaged mirror image) is formed by capturing this reflected radiation. Therefore, the image generated / provided by the mirror element is perceived by capturing the reflected radiation. The image capture process by the image capture device generates the imaged mirror image. However, in addition to the imaged mirror image, the image generated by the image capture device may also include other areas that are not represented by the imaged mirror image. In other words, the imaged mirror image may only appear within a certain portion of the image generated by the image capture device. In the imaged mirror image, at least a portion of the dentition image area is imaged. The mirror element can also include a frame portion surrounding the mirror surface. The mirror element can also include a handle portion that allows a user to adjust the position of the mirror element in space. The mirror element is preferably a mirror with a non-curved surface. The mirror is preferably a circular mirror. However, a polygonal mirror can also be used.

[0021] However, the present invention also relates to a surgical microscope that, in addition to or as an alternative to the aforementioned image acquisition device, may employ another optical acquisition system, particularly (but not limited to) an optical acquisition system capable of providing depth information. Such optical acquisition systems have been described in the introduction above. In particular, such optical acquisition systems may differ from stereo camera systems.

[0022] In the second step, the posture of the mirror element, in particular the posture of the mirror surface, is determined. The posture may include a translational component and a rotational component. For example, the position information of a reference point of the mirror element (e.g., the center of the mirror surface) and the orientation of the mirror element (e.g., the direction of the mirror surface normal) may be determined to determine the posture of the mirror element. Obviously, in some cases (especially for curved mirror surfaces), the positions of multiple reference points or the orientations of multiple partial areas of the mirror element may also be determined as its posture. An example method for determining the posture will be described in detail below. The posture may be determined in a reference coordinate system. For example, the reference coordinate system may be the reference coordinate system of the image acquisition device described above. The reference coordinate system of the image acquisition device may be the image coordinate system of the image generated by the device. The coordinate system may also be a reference coordinate system of preoperative information, a reference coordinate system of a surgical microscope, or other coordinate systems used for reference. The specific contents of these coordinate systems will be described in detail later.

[0023] In the third step, based on at least the posture of the mirror element, an area corresponding to the partial area of the dentition is determined in the preoperative information. In other words, the preoperative information is determined to contain information of the partial area of the dentition reflected by the mirror element.

[0024] Preoperative information can be generated in the form of image data or volume data. As mentioned above, this information can be generated using methods based on CT (computed tomography), MRI (magnetic resonance imaging), or other imaging methods (such as ultrasound). The preoperative information can be assigned a reference coordinate system, so that the preoperative information also contains spatial information. The pose of the mirror element constitutes the input variable for determining the corresponding region. To perform the third step, it may be necessary to align the reference coordinate system of the preoperative information with the reference coordinate system of the image acquisition device. This registration can be performed before the third step, and in particular, before the first step. The purpose of registration is to establish the correspondence between the preoperative information and the image, particularly the information in the image generated by the image acquisition device, in a shared reference coordinate system. The shared reference coordinate system can be the reference coordinate system of the preoperative information, the reference coordinate system of the image acquisition device, or another reference coordinate system, such as a global coordinate system used as a reference. If a surgical microscope is used, the shared reference coordinate system can also be the reference coordinate system of the surgical microscope. This will be discussed in detail later.

[0025] Known registration methods are well known to those skilled in the art. For example, model-based registration can be performed. In this process, features corresponding to known features in the preoperative information (e.g., geometric features, particularly geometric features of the jaw or tooth portion) can be detected in the image. Registration can then be determined using known methods based on these corresponding features. For example, registration can be determined using a transformation matrix containing rotational and / or translational components. Edge-based registration is an exemplary form of model-based registration, where corresponding features can be formed by properties of at least one edge (preferably multiple edges) in the image and preoperative information. Surface morphology-based registration can also be performed, particularly where surface morphology can be determined, such as using a stereo camera system of a surgical microscope. Thus, surface morphology information can be acquired from at least one image, and corresponding features, points, or areas can be detected in the preoperative and surface morphology information, which can then be used to determine the registration. For example, if at least one image is generated using a stereo camera system (particularly a stereo camera system of a surgical microscope), then, when the stereo camera system generates corresponding images, a three-dimensional image of at least a portion of the dentition can be generated, where the three-dimensional image can constitute or provide surface morphology information for registration. However, in the registration process, only one of the two corresponding images may be used.

[0026] For example, stereo reconstruction methods can be used to generate such three-dimensional images, with the imaged mirror images serving as input images for the method. Such methods are well known to those skilled in the art. In particular, in such methods, corresponding image elements can be determined in the two input images. For example, feature matching methods can be used to determine these corresponding pixels or sets of pixels. Related methods and features are well known to those skilled in the art. Exemplary features include SIFT (Scale-Invariant Feature Transform) features. However, other methods, such as variational methods or methods based on artificial intelligence (AI), can also be used for determination. Subsequently, the three-dimensional coordinates of the target point or target area imaged in the corresponding pixel or set of pixels can be determined in the reference coordinate system of the three-dimensional image. Possible reference coordinate systems have been described above. This process can also be referred to as reconstruction.

[0027] The three-dimensional image can be generated based on the pose of the mirror element. For example, reconstruction can be performed based on the pose of the mirror element. In particular, at least one step of the reconstruction method can be performed based on the pose. Specifically, the pose can be characterized by at least one parameter, and the at least one step can be performed based on the parameter or take the parameter into account during execution. Preferably, a stereo triangulation reconstruction method can be used to determine the three-dimensional image. Stereo triangulation reconstruction methods are well known to those skilled in the art. In this method, a projection matrix can be determined based on the pose of the mirror element and (known) laws of reflection. This projection matrix describes the relationship between the perspective transformation of three-dimensional object coordinates in a reference coordinate system and two-dimensional image coordinates and is used in the reconstruction process. In other words, the pose of the mirror element affects the projection matrices of the two image acquisition devices, and thus also affects the stereo triangulation reconstruction performed based on or dependent on these projection matrices. For example, when determining the three-dimensional coordinates, a homogeneous solution or an inhomogeneous solution can be used. Before determining the three-dimensional image, a correction method can be implemented to compensate for or eliminate nonlinear distortions in the image.

[0028] It may also be advantageous to calibrate at least one of the image acquisition devices. For example, known calibration methods can be used to determine the internal and external parameters of the image acquisition device and use them during image processing to ensure correct image presentation. Internal parameters describe parameters related to the image acquisition device itself, such as lens distortion. External parameters describe the relationships between multiple image acquisition devices, in particular spatial relationships, and the correspondence between the images generated by these devices. These internal and external parameters are well known to those skilled in the art. Preferably, the above parameters can be determined for all operating states or preset operating states, where the operating state is characterized by set (adjustable) parameters of the image acquisition device, such as zoom, focus and image acquisition area. If only a single image acquisition device is used, only the internal parameters need to be determined during calibration. If a stereo camera system comprising two image acquisition devices is used, the external parameters should also be determined during calibration.

[0029] In the fourth step, an enhanced image is generated by reflecting the corresponding area in the preoperatively generated information and superimposing it on the imaged mirror image. The enhancement can be achieved using methods known to those skilled in the art. In other words, the content of the preoperatively generated information corresponding to the portion reflected by the mirror element is superimposed on the mirror image area formed by the mirror element in the image. To this end, the preoperatively generated information also needs to undergo a reflection process. This reflection process can be achieved by a transformation matrix, wherein the transformation matrix is determined based on the posture of the mirror element. In other words, it can be determined how a specific area in the preoperatively generated information is reflected by the mirror element. To this end, a virtual reflected image of the preoperatively generated information can be determined based on a model method and with the help of a computer, and used for the enhancement process. The virtual reflected image can be an image formed by reflecting the preoperatively generated information by a virtual mirror element, wherein the virtual mirror element has been taken into account in the model-based calculation process.

[0030] By determining the pose, a mirror plane is determined in the reference coordinate system and defined using acquired variables (such as the imaging scale). Reflection laws are applied to this plane, projecting the defined area onto the preoperative information. The intersection of the projection and the preoperative information is appropriately reflected and superimposed on the imaged mirror image in the virtual environment.

[0031] The advantage of this approach is that, when using mirror elements, it allows for simple and accurate image enhancement, allowing preoperative information to be superimposed on the image. In particular, parts of the dentition that cannot be directly imaged by the image acquisition device (e.g., information about nerve pathways within the teeth) can be superimposed on the image captured by the device. Furthermore, there is no need to provide preoperative and intraoperative information to the user separately via separate output devices.

[0032] As an alternative or in addition, the generation of the enhanced image can use not only the information generated before the operation, but also the intraoperative information, i.e. the information recorded during the treatment. For example, information about a partial area of the dentition can be collected and stored during the treatment, and then used to generate the enhanced image. Therefore, in this case, the part corresponding to the partial area of the dentition can be determined in the information generated during the operation, and this determination process depends at least on the posture of the mirror element. Subsequently, the enhanced image is generated by at least reflecting the corresponding part and superimposing it on the imaged mirror image. In particular, the use of information generated during the operation has advantages when different visualization modes are activated at different times. For example, information about a partial area of the dentition can be acquired in fluorescence mode and enhanced using this information in normal view mode or in white light mode. The information can also be processed and analyzed, in particular classified, before further use.

[0033] In another embodiment of the present invention, when generating an image, the image acquisition device is positioned outside the oral cavity, particularly outside the oral cavity encompassing a portion of the dentition. The oral cavity may refer to the space bounded by the lips in front, the hard and soft palates above, the cheeks on both sides, and the floor of the mouth below. Within the scope of the present invention, the oral cavity may include the oral vestibule. Anatomical structures such as the teeth and tongue are located within the oral cavity.

[0034] The oral cavity is connected to the external environment (extraoral space) via the so-called oral fissure. When generating images, the image acquisition device is located in this external environment. Therefore, the image acquisition device can also be referred to as an extraoral scanner. Preferably, when generating images, the distance between the device and the oral fissure along its optical axis is greater than 1 cm, greater than 5 cm, or greater than 10 cm. In this case, the distance can be defined as the distance between the intersection of the optical axis and the oral fissure and a predetermined reference point of the image acquisition device, such as the intersection of the optical axis and the terminal glass panel of the image acquisition device. Therefore, the image acquisition device can be designed (particularly with respect to its dimensions) so that it cannot be placed inside the oral cavity for image acquisition.

[0035] Thus, enhanced images can be easily and reliably provided to the user via a plurality of image acquisition devices, which do not have to be specifically designed for placement within the oral cavity.

[0036] In another embodiment of the present invention, the image acquisition device is an image acquisition device for a surgical microscope. Such a surgical microscope can be used to magnify an object or area under examination, particularly in medical applications. Therefore, the device is particularly suitable for imaging the interior of the oral cavity or portions of the oral cavity.

[0037] The surgical microscope may include an image acquisition device, which may be designed to generate a two-dimensional image. In this case, the generated image has a predetermined number of pixels, thereby determining its resolution. For example, the image sensor of the image acquisition device may be a CMOS sensor or a CCD sensor. Obviously, other sensor types may also be used.

[0038] The surgical microscope may further include a stereo camera system comprising a first image acquisition device and another image acquisition device, wherein one image acquisition device is configured to generate the mirror image. The stereo camera system may be a calibrated stereo camera system.

[0039] A surgical microscope equipped with a stereo camera system may include two optically independent beam paths, wherein a first image acquisition device is arranged and / or designed to generate an image based on a beam in the first beam path. Another image acquisition device is arranged and / or designed to generate an image based on a beam in the other beam path. In particular, these images may be generated synchronously. Furthermore, the images generated by the first image acquisition device and the other image acquisition device may be referred to as corresponding images. As previously described, these corresponding images may be used to generate a three-dimensional image.

[0040] Furthermore, the surgical microscope may include at least one optical element for beam guidance and / or beam shaping, in particular in the form of a lens element. This at least one optical element may be used to generate a magnified image. For example, the optical properties of the surgical microscope (e.g., magnification, focus, zoom, exposure time, and imaging area size) may be adjustable.

[0041] The surgical microscope may further comprise at least one eyepiece through which a user can observe an image generated by the surgical microscope. In particular, the user can also observe the examination area through the eyepiece. The surgical microscope may comprise at least one objective lens or objective lens system, wherein the objective lens or objective lens system comprises at least one optical element for beam guidance and / or beam shaping. The eyepiece may be optically connected to the objective lens, or may be optically connected to the objective lens.

[0042] The surgical microscope may be part of a microscopy system that includes not only the surgical microscope but also a stand for supporting the surgical microscope. In this case, the stand may be designed to allow the surgical microscope to move in space, particularly with at least one degree of freedom, preferably six degrees of freedom, where the degrees of freedom may be translational or rotational. These degrees of freedom may be defined relative to a reference coordinate system. The vertical axis (z-axis) of this reference coordinate system may be parallel to the direction of gravity and point in the opposite direction to gravity. In this reference coordinate system, the longitudinal axis (x-axis) and the transverse axis (y-axis) may together form a plane perpendicular to the vertical axis (z-axis). Furthermore, the longitudinal and transverse axes may be orthogonal to each other. Furthermore, the stand may include at least one drive device, such as a servo motor, for moving the surgical microscope. The stand may also include devices for transmitting force / torque, such as gears and / or coupling components. Thus, the surgical microscope can be mounted or supported in a movable manner. This design allows the user to adjust the posture, ie, position and / or orientation, of the surgical microscope, for example to change the viewing angle of the examination area, or to view other examination areas.

[0043] The surgical microscope may be used in particular in dental surgery and is designed to generate images in dental applications.

[0044] Since surgical microscopes are commonly used for imaging magnification in dentistry, the present invention can enhance the functionality of such surgical microscopes. It can provide users with enhanced magnification using a simple and reliable method. In particular, the present invention eliminates the need to switch between a magnified view of intraoperative information (e.g., through a magnifying lens) and a view of preoperative information.

[0045] Alternatively, the image acquisition device may also be a head-mounted image acquisition device, for example, fixed by a suitably designed bracket system. Obviously, other extraoral image acquisition devices may also be used. Other types of extraoral observation devices may also be considered, in particular those that can be worn on the head, carried on the body or attached to a part of the body. These devices should be equipped with appropriate image acquisition devices and, in particular, be used in conjunction with a display device (such as a display screen). The display device is also arranged outside the oral cavity and is preferably fixedly mounted in a position relative to the image acquisition device. Particularly preferably, the display device may take the form of glasses, a head-mounted device, a virtual reality (VR) headset, an augmented reality (AR) headset or a mixed reality (MR) headset.

[0046] In another embodiment of the present invention, the superposition process can also be based on at least one optical property of the mirror element. In this case, the at least one optical property can be preset. Alternatively, the at least one optical property can be determined in a subsequent step of the proposed method, particularly in an image-based manner, i.e., by evaluating at least one image of the mirror element. Specifically, the optical property of the mirror element can be a magnification or reduction property. Obviously, other optical properties that affect the mirror image can also be considered. For example, this optical property can be represented by the aforementioned projection matrix. In this way, the accuracy of the superposition can be significantly improved.

[0047] As an alternative or cumulative approach, the overlay process can also be based on at least one imaging property of the surgical microscope. Specifically, this imaging property can be a set magnification (zoom), a set focus, or any other property that influences the mirror image. This imaging property can also be represented by the aforementioned projection matrix. In this way, high-precision overlay can also be achieved.

[0048] In another embodiment of the present invention, a portion of the preoperative information is determined based on at least the pose of the mirror element and assigned to an image of a virtual image capture device. In the image of the virtual image capture device, at least one portion corresponding to the dentition reflected by the catadioptric system is imaged. Furthermore, an enhanced image is generated by superimposing the portion of the preoperative information on the portion of the dentition reflected by the catadioptric system.

[0049] This catadioptric system is a relay optical system that includes at least the mirror element and the optical elements of an image acquisition device (e.g., an objective lens). If a surgical microscope is used, the catadioptric system may also include the optical elements of the surgical microscope objective lens. The beam path in this catadioptric system can be determined based on the optical properties (known or determinable properties) of the optical elements within the system, the orientation of the mirror element, and known optical laws.

[0050] The virtual image acquisition device is a mathematical or physical model of an image acquisition device, particularly one that can be evaluated with the aid of a computer. Based on this model, a virtual image can be generated by the virtual image acquisition device, particularly by computer-implemented pixel calculation. This virtual image depends on the parameters of the (modeled) image acquisition device and the posture of the (modeled) image acquisition device. For example, if the mirror surface is non-curved and does not have magnifying properties, the internal parameters of the virtual image acquisition device can be the same as the internal parameters of the modeled image acquisition device. In particular, the posture of the virtual image acquisition device can be determined based on the posture of the mirror element, so that in this posture, the virtual image generated by the virtual image acquisition device images the part of the dentition that is not directly reflected, which is first reflected by the catadioptric system and then reflected by the mirror element. This part is the corresponding part.

[0051] Besides the pose of the mirror element, the generation of such a virtual image also depends on (other) properties of the catadioptric system, such as the set zoom of the objective.

[0052] An enhanced image can then be generated by superimposing this portion of the preoperative information onto the portion of the dentition reflected by the catadioptric system. This can be achieved by reflecting this portion of the preoperative information, specifically based on the characteristics of the catadioptric system and therefore depending on the orientation of the mirror element, and then superimposing it onto the imaged mirror image. In other words, this portion of the preoperative information can be transformed into the aforementioned reference coordinate system, with this transformation process depending on the characteristics of the catadioptric system. In this way, a computationally precise and easy-to-implement superposition process can be achieved.

[0053] In another embodiment of the present invention, the pose can be determined by evaluating at least one property of the imaged mirror or mirror element (or a portion thereof). This at least one property can be determined based on an image, in particular by evaluating an image generated by an image acquisition device. Specifically, the imaged mirror or mirror element can be identified in the image, for example, using object recognition methods known to those skilled in the art. For example, the object recognition method can be a segmentation method. Thus, for example, the imaged mirror surface, the imaged frame portion, or the imaged handle portion can be identified based on the image. For example, a portion of the mirror element (e.g., the frame portion) can be manufactured from a material with predetermined optical properties (e.g., a matte material), particularly to ensure reliable detection of this portion in the image. Alternatively, detection can be achieved through user operation, for example, by selecting a region in the image containing the imaged mirror or the portion to be detected using a suitable input device.

[0054] The characteristic of the imaged mirror or the imaged mirror element can be a geometric characteristic, such as a dimensional characteristic, such as a size variable. The size variable can be width, height, diameter, or any other dimensional variable. Furthermore, the characteristic can be a shape characteristic, such as a geometric shape, such as a circle, an ellipse, a rectangle, or any other geometric shape. In particular, a shape factor can be determined that represents the relationship between the imaged shape and the actual shape, and the pose can be determined based on the shape factor.

[0055] As mentioned above, the pose of the mirror element may affect the image captured by the image acquisition device. Therefore, the pose may also affect how a certain characteristic of the actual mirror image or mirror element is mapped onto the corresponding characteristic of the imaged mirror image or mirror element. If the relationship between the actual characteristic and the characteristic of the imaged mirror image can be described by a pose-dependent transformation matrix, the pose can be determined based on the actual characteristic and the characteristic of the imaged mirror image or mirror element. The actual characteristic can be known, for example, from a model of the mirror element (e.g., a CAD model).

[0056] If the mirror element or a portion thereof (particularly the mirror surface) is circular and the imaged image is elliptical, the pose can be determined based on the properties of the ellipse (e.g., the orientation and length of the ellipse's axes) and known properties of the circular mirror element, such that these known properties are converted to the properties of the imaged image. If a polygonal mirror element (particularly an equilateral polygonal mirror element) is used, at least part of the pose can be determined based on the proportional relationship between the side lengths in the image and the relative position of the edges in the image.

[0057] Furthermore, the aforementioned characteristics may also include the pose of the imaged mirror image, the imaged mirror element, or a portion thereof in the image coordinate system. For example, the position of a reference point (e.g., the geometric center) may be determined as the position. For example, the axial direction of the reference portion may be determined as the orientation. For example, if the mirror element includes a handle, the handle may be identified in the image and its position and / or orientation determined. For example, the longitudinal axis of the handle may be determined.

[0058] If the pose is determined by evaluating at least one property of the imaged mirror image or the imaged mirror element, the pose can be evaluated using the generated image, thereby enabling a simple pose determination.

[0059] Alternatively, the attitude can also be determined based on markers. To this end, the mirror element can include or employ at least one marker element for determining its attitude. Obviously, the mirror element can also include or employ multiple markers, wherein the attitude of the mirror element can be determined based on the known relative positions of these markers.

[0060] The marker element can be an active marker element, preferably a passive marker element. It can be designed to be captured by a capture device. In particular, the capture device can be an image capture device. Therefore, in this case, the marker element can be an optically captureable marker element. For example, it is conceivable that an optically captureable marker includes a predetermined pattern that can be used to determine the marker's pose, and thus the pose of the mirror element. For example, such an optically captureable pattern can take the form of a QR code. Furthermore, such an optically captureable marker can also be a reflective marker element, for example, one designed to be reflective to radiation from a predetermined wavelength range (e.g., an infrared wavelength range). The image capture device used to optically capture the marker element can be the same image capture device used to image the mirror image, or a different image capture device. If the mirror element includes multiple marker elements, the pose of the mirror element can also be determined based on the relative positions of the imaged marker elements in the image. Furthermore, the pose of the mirror element can be at least partially determined by stereoscopically measuring the pose of at least one marker element.

[0061] The image acquisition device and the marking element can be used to perform monocular pose measurement. In this case, pose can be determined by evaluating a two-dimensional image (particularly a single two-dimensional image acquired by a single image acquisition device). In particular, position can be determined by evaluating pixel intensity values in the two-dimensional image. Methods for detecting position using a single image acquisition device and / or performing image processing based on a single two-dimensional image are well known to those skilled in the art. If a stereo camera system is used, pose can also be determined by evaluating corresponding images from the image acquisition device.

[0062] In particular, the pose can be determined by optical tracking methods, in particular using an image acquisition device. Such optical tracking can employ marker-based tracking methods, i.e., the pose is determined by using specific visible or optically captureable marker elements (e.g., QR codes or optical patterns of various designs).

[0063] Alternatively, particularly in the case of determining the pose by evaluating at least one characteristic of the imaged mirror or mirror element, a markerless tracking method may also be applied to determine the pose by capturing feature information.

[0064] However, in addition to using optically detectable marker elements, marker elements that can be detected by other means can also be used to determine the posture of the mirror element, such as marker elements that can be detected magnetically, capacitively, inductively, or radio-frequency. For example, the marker element can be designed as a radio frequency identification (RFID) tag.

[0065] The mirror element may further comprise a posture sensor, such as an inertial sensor or a GNSS sensor, wherein the posture may be determined based on an output signal of the sensor. In this embodiment, the surgical microscope may comprise a receiving device for receiving the output signal of the posture sensor, or be connected to the receiving device.

[0066] Obviously, a hybrid method can also be used to determine the posture, which combines at least two of the above-mentioned posture determination methods.

[0067] In marker-based pose determination, high-precision pose determination can be achieved, which in turn helps generate high-precision three-dimensional images.

[0068] The marking element can also be identifiable, in particular identifiable in a one-to-one correspondence. For example, the pattern of an optically identifiable marking can encode a unique identifier of the marking element. Thus, the marking element or the mirror element can be identified by capturing the marking element. The unique identifier thus determined can be used to correlate the properties of the mirror element, in particular the aforementioned optical properties. This correlation information, the unique identifier, and the related properties can be stored in a retrievable or readable form, for example, in a storage device. This has the advantage of making it possible to easily determine the optical properties of the mirror element.

[0069] In another embodiment of the present invention, for marker-based posture determination, at least one marker element is imaged by at least one image acquisition device or another image acquisition device. The at least one marker element is arranged on or formed by the mirror element. Subsequently, the posture is determined based on at least one characteristic of the imaged marker element. This has been explained above. The other image acquisition device can be a tracking camera or an environmental camera in a microscope system, which is different from the image acquisition device of the surgical microscope used for magnified imaging. The tracking camera or the environmental camera can be used in particular for tracking other instruments based on markers. In both cases, the method enables optical posture determination that is as simple as possible, which is particularly efficient and advantageous when using a surgical microscope or a microscope system containing a surgical microscope.

[0070] In another embodiment of the present invention, the focus position of the image capture device is set based on the posture of the mirror element. Thus, the focus position can be set at a point on the mirror surface, or at a point no more than a predetermined distance from the mirror surface. The predetermined distance may depend on the depth of field of the image capture device or surgical microscope; in particular, the distance may be less than the depth of field range. In this case, the depth of field is known or measurable. This therefore helps improve the image quality of the mirror image, thereby further enhancing the accuracy of the generated three-dimensional image. Furthermore, this also makes it easier and more reliable to detect the imaged mirror image in the image captured by the image capture device.

[0071] In another embodiment of the present invention, the radiation captured by the image acquisition device used for imaging is filtered. Preferably, the filtering is polarization filtering. However, other radiation filters may also be used. This approach can advantageously suppress unwanted reflections from the tooth surface in the image, thereby further enhancing the visualization of the enhanced information. In this case, polarization filtering can minimize or completely suppress reflections.

[0072] In another embodiment of the present invention, the filter element is disposed in the illumination beam path and / or the imaging beam path, particularly in the beam path of an image acquisition device or surgical microscope. If multiple imaging beam paths are present, a corresponding filter element can be disposed in each imaging beam path. Both approaches facilitate the structural integration of the filter element in the image acquisition device, surgical microscope, or microscopy system, thereby enabling the generation of high-quality enhanced images.

[0073] As an alternative or in combination with the aforementioned solution, a filter element may be provided on the mirror element. For example, the filter element may be arranged on the mirror surface of the mirror element. This advantageously eliminates the need for integrating a separate filter element for radiation filtering into the image acquisition device, surgical microscope, or microscopy system, as the mirror element itself already possesses the necessary filtering properties.

[0074] In another embodiment of the present invention, at least a portion of the dentition can be illuminated using radiation having predetermined radiation characteristics. For example, such radiation characteristics may include a predetermined wavelength, a predetermined radiation intensity, or, in a preferred embodiment, a predetermined polarization or other radiation characteristics. Advantageously, this also reduces reflections from the tooth surface, further improving the quality of the enhanced image.

[0075] In another embodiment, registration can be performed between the reference coordinate system of the preoperative information and the reference coordinate system of the image acquisition device based on the image generated by the image acquisition device. This process has been described above and has the advantage of being easily integrated into the method of the present invention. In particular, the registration can be performed by detecting corresponding features in the image and the preoperative information, for example using detection methods known to those skilled in the art, and then performing the registration based on these detected features.

[0076] In another embodiment of the present invention, an image is generated at a first magnification, while an enhanced image is generated at another magnification, wherein the other magnification is different from the first magnification, in particular greater than the first magnification. In other words, the above-mentioned registration can be performed based on an image with a smaller magnification, which is smaller than the magnification used to determine the enhanced image. The advantage of this method is that a more accurate and reliable registration can be achieved, because a smaller magnification can image a larger area, thereby providing more information for registration. The magnification process for generating images of different magnifications can be achieved optically or digitally. For example, an image with a first magnification can be generated by an optically set magnification, while an image with another magnification can be generated by a combined optical and digital magnification method. This method further improves the quality of the enhanced image.

[0077] In another embodiment of the present invention, the registration is performed based on mirror images. In particular, corresponding features can be identified in the detected mirror image and preoperative information, and registration can be performed based on these detected features. To achieve this, the preoperative information may need to be reflectively processed. This facilitates the simple integration of registration into the method of the present invention. If the image generated by the image acquisition device only contains a reflective portion, the step of detecting the mirror element can be omitted during the registration process.

[0078] The present invention further provides a system for generating an enhanced image of at least a portion of a dentition, the system comprising at least one image acquisition device and at least one evaluation device. In a preferred embodiment, the image acquisition device is designed to be arranged outside the oral cavity or at least to be arranged outside the oral cavity when acquiring the oral cavity image.

[0079] The system may include a stereo camera system having a first image capture device and another image capture device.

[0080] The system is used to implement the method of any embodiment described in this specification, and particularly includes the following steps:

[0081] a) detecting an imaged mirror image in an image generated by an image acquisition device, wherein the mirror image is provided by a mirror element arranged in an image acquisition area of the image acquisition device and reflects at least a portion of the dentition area;

[0082] b) determining the pose of the mirror element;

[0083] c) determining, in the preoperatively generated information, an area corresponding to the partial area of the dentition based at least on the posture of the mirror element;

[0084] d) generating an enhanced image by at least reflecting the corresponding area and superimposing it on the imaged mirror image.

[0085] The evaluation device may be designed as a computing device or include a computing device. The computing device may in turn be designed as a microcontroller or an integrated circuit or include a microcontroller or an integrated circuit. In this case, the evaluation device may perform at least one of steps a), b), c), and d), but preferably performs all of these steps.

[0086] The system can be part of a surgical microscope or microscopic system, wherein the surgical microscope or microscopic system can include the stereo camera system and the evaluation device. Furthermore, the system can include an acquisition device for capturing the marking element. The system can also include a filter element for filtering out radiation used to generate the image of the stereo camera system. Furthermore, the system can include an illumination device for illuminating the portion with predetermined radiation characteristics.

[0087] The system is conducive to implementing the method of any embodiment described in this specification and has the advantages described above.

[0088] In another embodiment of the present invention, the system includes a mirror element.

[0089] In addition, the present invention also proposes a computer program product, wherein the computer program product includes a software module for executing one, multiple or all steps of the method of any embodiment described in this specification, and when the computer program is executed by a computer or an automated system, the corresponding functions can be realized.

[0090] The present invention also provides a mirror element for generating an enhanced image of at least a portion of a dentition. The mirror element is used to generate an enhanced image of at least a portion of a dentition using a system according to any of the embodiments described herein. According to the present invention, the mirror element includes or forms at least one marking element for determining its posture. As an alternative or cumulative solution, the mirror element includes or forms at least one filter element for filtering out reflected radiation. The above and its corresponding advantages have been described in detail above. BRIEF DESCRIPTION OF THE DRAWINGS

[0091] The present invention will be described in detail based on exemplary embodiments and with reference to the following drawings:

[0092] FIG1 shows a schematic flow chart of the method according to the present invention;

[0093] FIG2 shows a schematic flow chart of a method according to another embodiment of the present invention;

[0094] FIG3a shows a schematic block diagram of a system according to another embodiment of the present invention;

[0095] FIG3 b shows a schematic block diagram of a system according to another embodiment of the present invention;

[0096] FIG4 a shows the mirror element in a first posture;

[0097] FIG4 b is a schematic diagram showing the mirror element shown in FIG4 a in another posture;

[0098] FIG4c is a schematic diagram showing the mirror element shown in FIG4a in another posture;

[0099] FIG4 d is a schematic diagram showing the mirror element shown in FIG4 a in another posture;

[0100] FIG5 shows a schematic diagram of a mirror element according to one embodiment of the present invention.

[0101] FIG6 shows a schematic block diagram of a virtual image acquisition device;

[0102] FIG7 shows an exemplary enhanced image. DETAILED DESCRIPTION

[0103] Hereinafter, the same reference numerals denote elements having the same or similar technical features.

[0104] FIG1 shows a schematic flow chart of a method according to the present invention for generating an enhanced image AA of at least a portion of the dentition Z (see FIG3a ). In a first step S1 of this method, a mirror image S5 is detected in an image I5 generated by an image acquisition device 5 (see, for example, FIG3a ). The mirror image is provided by a mirror element 3 , which is arranged within an image acquisition area 4 of the image acquisition device 5 , such that the image acquisition device 5 can capture a mirror image of at least a portion of the dentition Z. The detection in the first step S1 can be performed using target recognition methods known to those skilled in the art. In addition to the partial area imaged by the mirror image provided by the mirror element 3 , the image I5 generated by the image acquisition device 5 may also include other partial areas, such as other parts of the dentition Z, in particular parts not reflected by the mirror element 3 . The detected mirror image S5 is therefore used as an input variable for determining the enhanced image AA.

[0105] In the second step S2, the pose P of the mirror element 3 is determined. The reference coordinate system can be the coordinate system of the image acquisition device 5 or a global coordinate system. Obviously, other reference coordinate systems can also be used. In addition to the imaged mirror image S5, the pose P also constitutes another input variable in the process of determining the enhanced image AA.

[0106] In a third step S3, the region of the preoperative information PI corresponding to the portion of the dentition Z can be determined based on the preoperative information PI and the registration information RI. The preoperative information PI and the registration information RI have been generated prior to the execution of the third step S3 and the first step S1, respectively, and this determination is based on the pose P of the mirror element 3. The registration information RI allows the preoperative information PI and the image to be transformed into a common reference coordinate system, such as the coordinate system of the image acquisition device 5. The pose P of the mirror element can also be used to achieve this transformation into this reference coordinate system, thereby providing advantageous results.

[0107] In step S4, the enhanced image AA is generated by at least reflecting the corresponding region in the preoperative information and superimposing it onto the imaged mirror image. This superposition can be achieved using image fusion methods well known to those skilled in the art. In other words, the corresponding region in the preoperative information is superimposed onto the portion of the image containing the mirror image.

[0108] Before step S1, an arrangement step is performed. This involves placing the mirror element 3 within the image acquisition area 4 of the stereo camera system 1 so that the image acquisition device 5 can capture a mirror image of at least a portion of the dentition Z. After this arrangement, an image generation step is performed, where the image acquisition device 5 generates an image I5.

[0109] Figure 2 is a schematic flow chart of a method according to another embodiment of the present invention. Before executing steps S1, S2, S3, and S4 of this method, an image I5_1 with a first magnification factor is generated using image acquisition device 5 in an image generation step BSR (see Figure 3a). Subsequently, in a registration step RS, registration information RI (see Figure 1) is determined based on the generated image I5. Those skilled in the art can implement this step using known image-based registration methods.

[0110] In this case, before executing the first step S1 , the magnification for generating the image may be adjusted so that the first step S1 is performed based on the image I5_2 having another magnification, wherein the other magnification is different from the first magnification, in particular greater than the first magnification.

[0111] Figure 3a is a schematic block diagram of a system according to a first embodiment of the present invention, comprising an image acquisition device 5 and an evaluation device 10. As shown in Figure 3a, the image acquisition device 5 is located outside the oral cavity. The figure also shows the dentition Z, which includes a front surface 7, a back surface 8, and a chewing surface 9, as well as a mirror element 3 located within the image acquisition area 4 (indicated by dashed lines) of the image acquisition device 5. The mirror element 3 also reflects light beams from the dentition Z, particularly from its back surface 8, and directs them into the imaging beam path of the image acquisition device 5. As is apparent from Figure 3a, the dentition Z and the mirror element 3 are located inside the oral cavity, while the image acquisition device 5 is located outside the oral cavity. For clarity, the figure also shows the oral cavity 17 and the oral cleft 18, which connects the interior and exterior of the oral cavity. The image acquisition device 5 is located within this exterior space, that is, outside the oral cavity or oral cavity 17. There are many advantages to placing the image acquisition device outside the oral cavity. For example, it can provide more space inside the oral cavity for the placement and movement of other instruments, and the image acquisition device is not easily exposed to oral fluids, bacteria, and other substances that may cause contamination or damage. Therefore, additional protective or cleaning measures can be reduced or avoided. In addition, the use of other extraoral observation devices can also be considered, especially head-mounted, portable, or attached to a part of the body, equipped with appropriate image acquisition devices, especially in combination with display devices that are also placed outside the oral cavity, and particularly preferably in the form of glasses, head-mounted devices, virtual reality (VR) headsets, augmented reality (AR) headsets, or mixed reality (MR) headsets.

[0112] Also shown is a storage device 16 for storing pre-operative information PI, from which the evaluation device 10 can retrieve the pre-operative information PI, for example. The evaluation device 10 can then be used to detect the mirror image provided by the mirror element 3 in the image generated by the image acquisition device 5 .

[0113] Furthermore, the pose P of the mirror element 3 can be determined by the evaluation device 10 or a different device. The evaluation device 10 can also perform steps S3 and S4 shown in Figure 1. The registration information required for this can also be retrieved from the storage device 16 or obtained from another storage device.

[0114] Figure 3b is a schematic block diagram illustrating a system for generating an enhanced image AA of at least a portion of a dentition Z according to the present invention. The system comprises a stereo camera system 1 and at least one evaluation device 10. The system is configured to perform at least steps S1, S2, S3, and S4 shown in Figures 1 and 2 . Obviously, the system can also be configured to perform steps BSR and RS shown in Figure 2 . In this case, these steps, or at least some of them, can be performed by the evaluation device 10.

[0115] Figure 3b shows the acquisition area EB of image acquisition devices 5a and 5b and the optically independent beam paths 11a and 11b of surgical microscope 2, which, during normal use, is positioned outside the oral cavity, as is conventional in the art. Other extraoral observation devices are also conceivable, particularly those that are head-mounted, worn on the body, or attached to a body part, equipped with a suitable image acquisition device, particularly when combined with a display device such as glasses, head-mounted devices, virtual reality (VR) headsets, augmented reality (AR) headsets, or mixed reality (MR) headsets. The illumination device of surgical microscope 2, not shown in the figure, is capable of illuminating the dentition Z. In this case, the illumination device generates radiation with predetermined radiation characteristics, particularly radiation with predetermined polarization characteristics. The radiation reflected from the dentition Z travels through beam paths 11a and 11b to the image sensors of image acquisition devices 5a and 5b, thereby generating images I5a and I5b of the dentition Z. The evaluation device 10 can analyze these images. The figure also shows a mirror element 3, which also reflects the beam from the tooth row Z, in particular from its back side 8. This reflected radiation also reaches the image sensor via beam paths 11a and 11b, where it is imaged as a mirror image. The evaluation device 10 can then detect this image as the imaged mirror image.

[0116] The radiation used to generate images I5a and I5b of the stereo camera system 1 can be filtered. This can be achieved by providing filter elements, for example, arranged in beam paths 11a and 11b, respectively. Filter elements can also be provided in the illumination beam path of the illumination device of the surgical microscope 1 (not shown). Furthermore, filter elements can be provided on or near the mirror element 3. In particular, such filter elements can be polarization filters. Furthermore, the evaluation device 10 can also filter the images I5a and I5b generated by the image acquisition devices 5a and 5b, for example to suppress reflections.

[0117] The figure also shows a storage device 16 for storing preoperative information PI and registration information RI, which the evaluation device 10 can retrieve from the storage device 16 when performing the third step S3. The enhanced image AA can be generated by the evaluation device 10 in a fourth step S4.

[0118] Figure 4a shows an image of mirror element 3 in a reference pose. Mirror element 3 comprises a handle portion 12 and a circular mirror portion 13, which further comprises a mirror surface 14. For example, the center of mirror surface 14 can be used as reference point P3 for mirror element 3. A coordinate system fixed relative to the mirror surface is shown, having a longitudinal axis x3, a transverse axis y3, and a vertical axis z3 (see Figure 4b).

[0119] Figure 4b shows an image of mirror element 3 in pose P, which results from rotating mirror element 3 around longitudinal axis x3 relative to the reference position shown in Figure 4a. Clearly, the circular mirror surface 14 shown in Figure 4a is imaged as an ellipse in this situation. Based on the directions and lengths of the major and minor axes of this ellipse (which can be detected using object recognition methods), the angle of rotation of mirror element 3 around longitudinal axis x3 can be determined, thereby further determining the current pose P of mirror element 3 shown in Figure 4b.

[0120] Similarly, Figures 4c and 4d illustrate the image formation of mirror element 3 after rotation about transverse axis y3 (Figure 4c) or about vertical axis z3 (Figure 4d), respectively, relative to the reference position shown in Figure 4a. The corresponding rotation angle can be determined, for example, based on the direction of longitudinal axis x3 (Figure 4d) and / or the axial direction and length of the ellipse formed by mirror element 14.

[0121] As can be seen from FIG. 4 a to FIG. 4 d , the pose P of the mirror element 3 can be determined based on the shape, wherein the shape properties of the imaged mirror element 3 are first determined and then its pose P is determined based on these properties.

[0122] In addition, the center of the mirror surface 14 can also be detected. If the focus is not only on the reflective point on the mirror surface 14 (such as the center) but also on a point on the non-reflective edge of the mirror surface 14, the distance between the mirror surface 14 (especially the reflective point) and the tooth row Z can be determined by the difference in focus position.

[0123] In particular, the difference between the focal position when the stereo camera system 1 or surgical microscope 2 is focused on a point on a non-reflective edge (e.g., a point on the frame portion 13) and the focal position when the system is focused on a point on the mirror 14 reflecting an object (e.g., the reflection point at the center of the mirror 14) can be determined. This difference in focal position represents the distance between the mirror element 13 and the target object (in this example, a point on the dentition Z), and the distance can be determined based on this difference. Furthermore, this distance can also be determined based on the pose P of the mirror element 13. This distance information can be used for scaling adjustments during the stereo reconstruction process described above, particularly for matching the magnification of the reflective area (e.g., the back surface 8 of the dentition Z) to that of the non-reflective area (e.g., the front surface 7 of the dentition Z) when reconstructing the reflective area (e.g., the back surface 8 of the dentition Z), i.e., for scaling correction.

[0124] Figure 5 schematically illustrates a mirror element 3 according to the present invention. Mirror element 3 includes or forms a marking element 6 on handle portion 12, which takes the form of an optically detectable barcode. Furthermore, mirror element 3 includes multiple optically detectable marking elements 6 on frame portion 13 of mirror surface 14, also in the form of barcodes. These marking elements can be captured in an image of mirror element 3 and are used, in particular, to identify mirror element 3 and determine pose P. Specifically, in the mirror element 3 shown in Figure 5, each marking element 6 can be detected, allowing pose P to be determined based on the relative arrangement of these marking elements in the image.

[0125] Figure 6 schematically illustrates an image capture device 5 (see Figure 3a ) and a mirror element 3 disposed within its capture area 4 . Also shown is a target point OP to be imaged, for example, a point on the surface of the tooth arch Z (see Figure 3a ). Also indicated is the normal n to the mirror surface 14 of the mirror element 3 .

[0126] Figure 6 also shows a virtual image acquisition device 15. The (virtual) image of the virtual image acquisition device 15 can be determined by evaluating a mathematical or physical model. Specifically, the model is configured to generate a virtual image of the preoperative information PI that directly displays a region of the dentition Z without reflection, specifically the target point OP corresponding to the portion reflected by the catadioptric system, while taking into account the optical properties of the catadioptric system. An enhanced image AA can then be determined based on this virtual image, for example by reflecting the virtual image and superimposing it on the detected mirror image S5.

[0127] FIG7 shows an exemplary enhanced image AA containing a portion of the dentition Z. Furthermore, a mirror image S5 of the dorsal side 8 of the dentition Z, provided by a mirror element 3 having a mirror surface 14, is shown. Preoperative information PI, presented in circular form and marking predetermined areas of the teeth, is superimposed on the imaged mirror image S5.

[0128] As an alternative to the image acquisition device, another acquisition system in the surgical microscope can similarly be used to capture optical information, particularly depth information, and to detect within this optical information (particularly optical information provided in the form of an image) an image provided by a mirror element disposed within the acquisition area of the surgical microscope, particularly within the acquisition area of the acquisition system. This image allows optical information of a portion of the dentition to be captured by the surgical microscope, particularly the other image acquisition system. Furthermore, based on the previously determined pose of the mirror element, a region corresponding to this portion of the dentition can be determined in preoperatively generated information, and an enhanced image can be generated by at least reflecting this corresponding region and superimposing it on the imaged image.

[0129] As an alternative to or in addition to using preoperatively generated information to generate enhanced images, it is also possible to use intraoperative information (i.e. information recorded or generated during the treatment process) to generate enhanced images. For example, information about parts of the dentition can be collected and stored during the treatment process, and this information can then be used to generate enhanced images. This approach is particularly suitable when different visualization modes are activated at different times. For example, information (in particular image information) about parts of the dentition can be acquired in fluorescence mode and then used for enhancement in normal view mode or white light mode. This information can also be processed and analyzed, in particular classified, before further use.

[0130] For example, an instrument can be used to contact or mark the surface of a partial area of the dentition, and at the same time, the surface information of this partial area can be captured by an image acquisition device through a mirror element. This is a way to generate the aforementioned intraoperative information. For example, the mark can then be displayed in an enhanced image, in particular based on the posture of the mirror element. For example, the mark can be presented in the form of a preset geometric element. As an alternative or in addition, different sensor systems can also be used to capture information of the partial area of the dentition. This is another way to generate the aforementioned intraoperative information, which can then be displayed in an enhanced image.

[0131] Reference Signs List

[0132] 1 Stereo Camera System

[0133] 2 surgical microscopes

[0134] 3 mirror elements

[0135] 4 Collection Area

[0136] 5, 5a, 5b Image acquisition equipment

[0137] 6 Optically captureable marking elements

[0138] 7 Positive

[0139] 8 Back

[0140] 9 chewing surface

[0141] 10 Evaluation Equipment

[0142] 11a, 11b Beam paths

[0143] 12 handle part

[0144] 13 frame part

[0145] 14 mirrors

[0146] 15Virtual image acquisition device

[0147] 16 storage devices

[0148] 17 Oral cavity

[0149] 18 mouth cleft

[0150] SBV image generation steps

[0151] SRV Reconstruction Steps

[0152] I5, I5a, I5b images

[0153] I5_1, I5_2 images

[0154] Mirror images of S5, S5a, and S5b

[0155] P-stance

[0156] A1, A2, A images

[0157] OP target point

[0158] EB collection area

[0159] S1 first step

[0160] S2 Step 2

[0161] S3 Step 3

[0162] S4 Step 4

[0163] AA enhanced image

[0164] PI preoperative information

[0165] RI registration information

[0166] KA corresponding area

Claims

1. A method for generating an enhanced image (AA) of at least a portion of a dentition (Z), comprising the following steps: a) detecting an imaged mirror image (S5) in an image generated by an image acquisition device (5, 5a, 5b), the mirror image being provided by a mirror element (3) arranged in an image acquisition region (4) of the image acquisition device (5, 5a, 5b), so that the image acquisition device (5) can acquire a mirror image of the at least part of the dentition (Z) (S5); b) determining the posture (P) of the mirror element (3); c) determining, in the preoperatively generated information (PI), an area (KA) corresponding to a partial area of the dentition (Z) based at least on the posture (P) of the mirror element (3); d) generating an enhanced image (AA) by reflecting at least the corresponding area (KA) and superimposing it on the imaged mirror image (S5).

2. The method according to claim 1, characterized in that The image acquisition device (5) is arranged outside the oral cavity (17).

3. The method according to claim 1 or 2, characterized in that: The image acquisition device (5) is an image acquisition device (5a, 5b) of a surgical microscope (2), or an image acquisition device that can be worn on the head.

4. The method according to any of the preceding claims, characterized in that The superposition is also based on at least one optical property of the mirror element (3) and / or at least one imaging property of the image acquisition device (5).

5. The method according to any of the preceding claims, characterized in that An image generated by a virtual image acquisition device (15) is determined based at least on the posture (P) of the mirror element (3), wherein at least a partial area of the tooth row (Z) is imaged in a non-reflective manner in the image, and a virtual enhanced image is generated by superimposing a corresponding area (KA) on the non-reflective partial area, and the virtual image is subsequently converted into an enhanced image (AA).

6. A method according to any preceding claim, characterised in that The pose (P) is determined by evaluating at least one property of the imaged mirror (S5) or the imaged mirror element (3), or in a marker-based manner.

7. The method according to claim 6, characterized in that In order to determine a pose (P) based on a marker, at least one marker element (6) is imaged by an image acquisition device (5) or another image acquisition device, wherein the at least one marker element (6) is arranged on a mirror element (3) or is formed by a mirror element (3), and the pose (P) is determined based on at least one characteristic of the imaged marker element (6).

8. A method according to any preceding claim, characterised in that The focus position of the image acquisition device (5) is set based on the posture (P) of the mirror element (3).

9. A method according to any preceding claim, characterised in that Radiation captured by an image acquisition device (5) for generating an image is filtered.

10. The method according to claim 9, characterized in that The filtering is polarization filtering.

11. The method according to claim 9 or 10, characterized in that The filter element is arranged on the illumination beam path and / or the imaging beam path (11a, 11b) and / or the mirror element (3).

12. A method according to any preceding claim, characterised in that At least a partial area of the dentition (Z) is irradiated with radiation having predetermined radiation characteristics.

13. The method according to claim 12, characterized in that The radiation has a predetermined polarization characteristic.

14. A method according to any preceding claim, characterised in that Based on the image generated by the image acquisition device (5), the reference coordinate system of the preoperatively generated information (PI) is registered with the reference coordinate system of the image acquisition device (5).

15. The method according to claim 14, characterized in that The image is generated at a first magnification and the enhanced image (AA) is generated at another magnification, wherein the other magnification is different from the first magnification, in particular greater than the first magnification.

16. The method according to claim 14 or 15, characterized in that The registration is performed based on mirroring ( S5 ).

17. A system for generating an enhanced image (AA) of at least a portion of a dentition (Z), comprising at least one image acquisition device (5) and at least one evaluation device (10), characterized in that The system is used to perform the following steps: a) detecting an imaged mirror image (S5) in an image generated by an image acquisition device (5), wherein the mirror image (S5) is provided by a mirror element (3) arranged in an image acquisition region (4) of the image acquisition device (5) and reflects at least a portion of the dentition (Z); b) determining the posture (P) of the mirror element (3); c) determining, in the preoperatively generated information (PI), an area (KA) corresponding to a partial area of the dentition (Z) based at least on the posture (P) of the mirror element (3); d) generating an enhanced image (AA) by reflecting at least the corresponding area (KA) and superimposing it on the imaged mirror image (S5).

18. The system according to claim 17, wherein: The image acquisition device (5) is designed to be located outside the oral cavity during the imaging process.

19. The system according to claim 17 or 18, characterized in that The system comprises a mirror element (3).

20. A computer program product comprising a computer program comprising software modules for executing steps a) to d) of the method according to any one of claims 1 to 16, wherein the computer program is capable of implementing corresponding functions when executed by a computer or an automated system.

21. A mirror element for generating an enhanced image of at least a portion of a dentition by means of a system according to any one of claims 17 to 19, characterized in that The mirror element (3) comprises or forms at least one marking element for determining its position and / or at least one filter element for filtering out reflected radiation.

Images