Determining Tooth Position and Generating 2D Re-Slice Images Using Artificial Neural Networks

Abstract

The present disclosure relates to a computer-implemented method for automatically generating and displaying a 2D image derived from 3D image data, the 2D image representing a portion of a patient's maxillofacial anatomy, the method comprising: receiving 3D image data including a voxel image volume representing the maxillofacial anatomy, the voxel image volume including the patient's teeth, each voxel of the voxel image volume being associated with a radiation intensity value; detecting one or more anatomical landmarks in the voxel image volume using a first artificial neural network; determining a crown center position for each tooth of a plurality of teeth included in the voxel image volume using the detected anatomical landmarks as reference points using a second artificial neural network; and generating a 2D image as a re-slice image from the 3D image data based on the at least one crown center position.

Description

[Technical field]

[0001] The present disclosure relates to a computer-implemented method and data processing system for automatically generating and displaying 2D images derived from 3D image data, representing a portion of a patient's maxillofacial anatomy, using an artificial neural network. The present disclosure also relates to a method for training an artificial neural network in this regard. The present disclosure further relates to a computer program product including instructions for automatically generating and displaying 2D images derived from 3D image data, representing a portion of a patient's maxillofacial anatomy. [Background technology]

[0002] The location of anatomical landmarks and views of the patient's maxillofacial anatomy allow medical professionals, such as orthodontists, dentists, and dental or maxillofacial surgeons, to make a diagnosis of the patient's condition. This diagnosis allows the medical professionals to plan, monitor, and evaluate the appropriate treatment. The anatomical landmarks are typically depicted while the dental clinician navigates through a two- or three-dimensional x-ray image derived from a digitized record of the patient's corresponding anatomical cross-section. Such a digitized record is, for example, a 3D rendering of a cone-beam computed tomography scan (CBCT scan) with orthogonal slices through the scanned volume. In addition to depicting the desired landmarks of the patient's anatomy in 3D data, the dental clinician also needs appropriate imaging for easy and efficient planning, monitoring, and evaluation of the desired treatment. However, manually performing navigation and annotation tasks is a tedious and time-consuming task that requires a significant amount of training and practice before a satisfactory result is achieved.

[0003] Especially in the case of dental procedures, it is of great interest to efficiently determine maxillofacial anatomical structures for dental procedures. These procedures require detailed planning and are frequently performed. That is, the demand for these procedures is high. Therefore, despite the great advances made in recent years, there is still a need for enhanced methods and systems that provide efficient support in planning, monitoring, and evaluating dental procedures. Summary of the Invention

[0004] In view of the above, it is of particular interest to dental clinicians to have available systems and methods that do not impose additional task burdens when applying the functionality of the software. Instead, functionality is sought that allows dental clinicians to dispense with input that they must provide apart from the essence of their training, i.e., their role in actually diagnosing or treating patients.

[0005] The objective was therefore to provide a method and system that takes the above into account and in particular reduces the amount of effort and work that a dental clinician must put in before being in a position to be able to decide on dental treatment options to meet the needs of a patient.

[0006] In response, the present disclosure provides a computer-implemented method for automatically generating and displaying a 2D image representative of a portion of a patient's maxillofacial anatomy derived from 3D image data. The method includes receiving 3D image data including a voxel image volume representative of the patient's maxillofacial anatomy, the voxel image volume including the patient's teeth. Each voxel of the voxel image volume is associated with a radiation intensity value. As part of the method, one or more anatomical landmarks within the voxel image volume are detected using a first artificial neural network. Further, a second artificial neural network is used to determine a crown center position of each tooth of a plurality of teeth included in the voxel image volume using the detected anatomical landmark as a reference point. The 2D image is generated as a re-slice image from the voxel image volume based on at least one crown center position.

[0007] The computer-implemented method allows for the automatic generation of 2D images showing a single tooth or multiple teeth from 3D image data received from the imaging means. Thanks to this process, the workload of the dental clinician while navigating the 3D image data is significantly reduced. In other words, the dental clinician can start working on the dental treatment with 2D images that have already been generated or are automatically generated upon the dental clinician's request. Instead of having to extract the desired views from the image data, the dental clinician has the option of receiving these views automatically.

[0008] By using a first artificial neural network to detect one or more anatomical landmarks, reference points are identified within the voxel image volume of the 3D image data. These reference points are used, in particular, to identify image regions in which the patient's teeth are present or expected to be present. The detected anatomical landmarks can be used to provide an orientation to the 3D image data (e.g., how the maxillofacial anatomical regions are positioned within the 3D image data, or how the maxillofacial anatomical structures are positioned and oriented relative to a scanner, in particular a CT scanner).

[0009] A second artificial neural network is used to determine a crown center location for each tooth of the plurality of teeth included in the voxel image volume. The crown center location is specifically a location within the crown volume along a central axis that intersects with the occlusal plane. The crown center location preferably represents a center of the crown of the tooth in three dimensions. The crown center location is specifically determined within an image region that includes the patient's teeth as indicated by detecting one or more anatomical landmarks.

[0010] Preferably, the second neural network is configured to identify the crown center position depending on the location of the tooth, in particular the anatomical structure of the tooth, or the type of tooth expected to be present at this position (e.g. tooth position / anatomical structure as defined in the FDI World Dental Federation notation). In other words, the second neural network is preferably configured or trained to determine the crown center position of each tooth individually and depending on the type of tooth or the anatomical structure of the tooth.

[0011] The first and / or second artificial neural networks are preferably convolutional neural networks, which have been found to be particularly efficient in analysing image data.

[0012] The tooth crown center positions determined in the voxel image volume may serve as reference points for the 2D image. In particular, the determined tooth crown center positions are used as reference points for defining a (e.g., planar or curved) re-slice plane through the voxel image volume to generate a re-slice image.

[0013] The method may further comprise calculating a curved path that follows the patient's dental arch using at least one of said one or more anatomical landmarks as a reference point and / or by fitting a curvature to said determined crown center location. Preferably, the curved path is used as a reference for generating a 2D image.

[0014] The curved path represents the orientation or placement of the teeth along the dental arch. A dental clinician may use the curved path as a reference to provide dental restorations for missing teeth that are well-matched to the patient's dental condition.

[0015] The curved path is preferably adapted to pass through or intersect all teeth of the mandible or maxilla contained within the voxel image volume.

[0016] A curved path can be calculated for each of the mandibular and maxillary sides.

[0017] The curved path may be calculated based on one or more anatomical landmarks identified using the first artificial neural network. This may provide the curved path as another reference and may help determine the crown center location contained within the voxel image volume and / or help visualize the 3D data. Furthermore, when multiple teeth are missing, especially at the ends of the dental arch, the anatomical landmarks may be used to estimate the course of the curved path of the dental arch.

[0018] Preferably, the curved path is calculated by fitting the curvature to the determined crown center positions. The curved path using the crown center positions as reference points can be reliably and accurately derived based on the patient's teeth represented in the voxel image volume and / or at least one anatomical landmark. In this way, accuracy and reliability are achievable even when one or more teeth are missing.

[0019] Furthermore, particularly high reliability and accuracy are achieved when both anatomical landmarks and crown center positions are used as reference points to calculate a curved path that follows the patient's mandibular and / or maxillary dental arches.

[0020] For example, the method may further include detecting the center of the left and / or right condyle as an anatomical landmark by inputting the voxel image volume into a first artificial neural network, where a curved path tracing the dental arch is calculated using the center of the left and / or right condyle as a point, in particular as a reference point defining the left and right extreme segments of the curved path.

[0021] If the centre of the left and / or right condyle is represented within the voxel image volume of the patient's maxillofacial anatomy, then using at least one condyle as a reference point allows reliable determination of the end(s) of the curved path following the patient's dental arch.

[0022] The re-slice image may be a panoramic image generated based on the curved path such that it follows the arch form of the patient's teeth.

[0023] Such re-slice images in particular provide the dental clinician with an automatically generated overview of the teeth represented in the 3D image data, facilitating at least an initial assessment of the patient's overall dental situation and allowing the dental clinician to review the presented results.

[0024] Preferably, the panoramic view is generated based on a curved path. Additionally or alternatively, slice images may also be generated (directly or indirectly) using the determined crown center positions. The panoramic image may include the maxillary dental arch and / or the mandibular dental arch.

[0025] The method may include analysing the 3D image data, and in particular the voxel image volume, in at least an image region surrounding a determined crown centre position of one of the teeth to derive a tooth axis, which intersects the coronal crown surface and extends from this intersection towards the apex of the tooth and includes the determined crown centre position for the tooth. This step also preferably includes setting a re-slice plane including the tooth axis for projecting the re-slice image onto the re-slice plane.

[0026] Furthermore, deriving the tooth axis preferably includes the steps of starting from a cross-section of the tooth that includes the estimated crown center position, tracing corresponding positions in the next adjacent cross-section, and continuing to trace corresponding positions of adjacent cross-sections along the tooth, particularly in a direction towards the apical side of the tooth, and calculating the tooth axis based on these corresponding positions.

[0027] This is a particularly rapid technique for providing tooth axes and can therefore be easily performed for each tooth in the patient's image voxel volume. This provision can be implemented to occur automatically in the background, allowing dental clinicians immediate access to tooth axes upon request.

[0028] Once corresponding locations in adjacent cross sections of the voxel image volume throughout the tooth crown have been tracked (i.e., identified), the tooth axis is calculated based on these corresponding locations, preferably using linear regression.

[0029] Preferably, the corresponding positions are tracked in adjacent cross sections perpendicular to the axis of the coordinate axis of the voxel image volume that most closely approximates the orientation of the teeth (coronal-apical) or maxillofacial anatomy (cranial-caudal). This allows a particularly fast and easy tracking of corresponding positions in cross sections across the teeth, since the tracking is based directly on the 3D image data.

[0030] The derived tooth axis orientation is preferably calculated relative to a curved path that follows the dental arch.

[0031] Furthermore, the tooth axis represents a useful reference that is preferably included when setting a re-slice plane for projecting a re-slice image onto the re-slice plane based on the voxel image volume, in other words, the tooth axis particularly facilitates the derivation of re-slice images showing cross-sections of individual teeth.

[0032] For example, a re-slice image may be generated and displayed as a cross-sectional re-slice image that includes the crown center, where the plane of the cross-sectional re-slice image is perpendicular to the tooth axis of the tooth.

[0033] Thus, the tooth axis can serve as a reference for presenting the dental clinician with a cross-sectional view of the tooth at the height of the crown center.

[0034] Additionally, the re-slice image may be generated and displayed as a mesial-distal re-slice image that includes the tooth axis and is oriented essentially parallel to the curved path at one of the tooth locations, or as a buccal / labial-oral re-slice image that includes the tooth axis and is oriented essentially perpendicular to the curved path at one of the tooth locations.

[0035] Thus, a curved path extending along the dental arch may be used to orient a re-slice plane of a re-slice image representing a cross-sectional view. More specifically, the plane extends along the tooth axis in a mesial-distal direction, and the mesial-distal direction is oriented parallel to the curved path at the tooth location, particularly at the crown center location or the location on the curved path closest to the crown center location.

[0036] Additionally or alternatively, the curved path may be used to orient the re-slice plane of the cross-sectional re-slice image such that, on the one hand, the plane extends in the buccal-oral or labial-oral direction and, on the other hand, extends along the tooth axis, with the buccal-oral or labial-oral direction being oriented perpendicular to the curved path at the tooth position (in particular, the crown center position or the position on the curved path closest to the tooth axis or crown center position).

[0037] This significantly reduces the dental clinician's workload in terms of having to navigate to each tooth to create an individual re-slice image. This feature allows the dental clinician to directly select and evaluate each tooth.

[0038] Furthermore, starting from a preferably automatically generated re-slice image, the dental clinician can move and / or rotate the re-slice plane onto which the re-slice image is projected. The tooth axis can be used as a reference for moving the re-slice plane through the tooth voxel image volume. The tooth crown center position can be used as a reference for rotating the re-slice plane within the tooth voxel image volume.

[0039] Before the second artificial network determines the crown center positions, the method preferably includes the steps of defining, for each tooth assumed to be present in the patient's dentition, an image region expected to contain a representation of this tooth, and assigning a label to the image region, the image region being specifically defined as referenced to at least one of the anatomical landmarks.

[0040] The number and types of teeth assumed to be present in the patient's dentition are preferably based on a complete (i.e., normal) child or adult dentition appropriate to the patient's age. In other words, the system or method preferably assumes that the patient has a complete dentition. Based on this assumption and the detected anatomical landmarks, the system assigns the above-mentioned image regions and complete preliminary labels to each tooth of the complete dentition.

[0041] To determine the crown center position of each of the teeth represented in the 3D image data, the second artificial neural network may include a module trained to estimate three components of an offset vector from a position in the voxel image volume to a predicted crown center position.

[0042] This module is preferably trained to determine the crown center location of the teeth based on the assigned (i.e., preliminary) tooth label. In other words, the module can be trained to determine the crown center location of each tooth according to the tooth label of this tooth.

[0043] In particular, the second artificial neural network may be trained to estimate a number of locations in the voxel image volume where the patient's teeth must be located, in particular offset vectors from image regions in the voxel image volume of teeth that may have been assigned a tooth label. Such a configuration results in the generation of a number of crown center predictions in each of said image regions using the second artificial neural network. All crown center predictions determined for a given image region may be marked with the tooth label of said image region or the tooth label of the crown center prediction determined by the second artificial neural network.

[0044] If multiple crown center predictions have been generated for an image region, the (most likely) crown center location may be determined by clustering the crown center predictions, and an initial tooth label is assigned to the determined crown center location based on the tooth labels of the clustered crown center predictions.

[0045] The initial tooth labels of the determined crown center positions, assigned based on the original tooth labels under the assumption that the dentition is complete, may then be revised based on an analysis of the relative positions of the determined crown center positions. Preferably, the relative positions of the determined crown center positions are compared with the crown center positions of a model dentition, which preferably comprises, for each tooth type, average crown center positions obtained from a number of reference dentitions. This allows for the assignment of revised tooth labels as tooth labels taking into account the differences from the complete dentition.

[0046] Thus, the dental clinician does not need to manually assign tooth labels, but is already presented with the tooth labels. Nevertheless, manual revision by the dental professional may be performed as an option.

[0047] The method may further comprise deriving, from the determined crown center positions and assigned tooth labels, virtual crown center positions and associated virtual tooth labels for any teeth missing from the patient's 3D image data.

[0048] In other words, based on the determined crown center position and the revised tooth label assigned thereto, any teeth missing from the patient's dentition may be identified. In particular, if the determined crown center position and tooth label indicate a gap along a curved path that traces the dental arch of the jawbone (maxilla or mandible), for example, one or more teeth are determined to be missing accordingly. The location of the virtual crown center may be estimated based on the location of the teeth identified in the voxel image volume as being present and the type of tooth specified by the virtual tooth label. Another input source that may be used in deriving the virtual crown center position is the curved path described above (if calculated).

[0049] Thus, the overview presented to the dental clinician can automatically provide a complete overview of the patient's dental status, which may, for example, facilitate manual revision of tooth labels by the dental clinician, if necessary.

[0050] Further, the method may include displaying a dental chart generated using the detected crown center positions and associated tooth labels. The dental chart may be a personalized chart indicating missing teeth and reflecting the relative placement of the patient's teeth based on the distances between the detected crown center positions. Preferably, such a dental chart includes selectable icons indicating teeth represented in the 3D image data and / or teeth missing from the 3D image data, and when one of the icons is selected, a 2D image of the tooth position of the selected icon is displayed, including at least one reslice image.

[0051] Preferably, the 2D image includes two or more re-slice images, and in particular mesial-distal (coronal), buccal / labial-oral or labial-oral (sagittal), and / or cross-sectional (transverse) re-slice images, which may be generated as described above.

[0052] Selecting a tooth chart icon displays a 2D image containing at least one re-slice image, allowing the dental clinician to quickly and efficiently evaluate each tooth represented in the voxel image volume as at least one re-slice image is automatically generated.

[0053] The re-slice images can be generated in advance or by selecting a tooth icon on the dental chart.

[0054] The method may further include displaying the 3D image data and displaying a marker at the location of each determined crown center position and / or each virtual crown center position.

[0055] The 3D image data is preferably displayed as a three-dimensional representation of the jawbone and teeth. Such a display of the data allows a dental clinician to evaluate the patient's dental condition from different angles and / or magnifications according to the clinician's preferences. Thus, the clinician can preferably manipulate the display after it has been automatically generated.

[0056] The method preferably includes the steps of extrapolating a virtual tooth axis of at least one missing tooth, where the virtual tooth axis includes a virtual crown center position of the missing tooth, and calculating a direction of the virtual tooth axis based on a local orientation of a curved path following the dental arch and / or a direction of the tooth axis of at least one adjacent tooth.

[0057] The direction or angle of the virtual tooth axis is preferably calculated with respect to a curved path that follows the dental arch. In particular, in this case, the direction of the tooth axis may also be based on the local direction of the curved path. For example, the local direction and the curved path may be used to identify a lingual, buccal or labial direction, based on which the apical-coronal direction of the virtual tooth may be derived to define the virtual tooth axis. Additionally or alternatively, the virtual tooth axis may be based on the direction of the tooth axis of one adjacent tooth, preferably two adjacent teeth. One adjacent tooth may be used as a basis, in particular if two or more teeth are missing or if the missing tooth is at the end of the patient's dental arch.

[0058] By providing the virtual tooth axis, the dental clinician is given at least a good starting point for planning the restoration and a direction for the insertion of one or more implants that will support the restoration.

[0059] Calculating a curved path that follows the dental arches of the patient's teeth may include fitting the curvature of the path to both the determined crown center positions of the teeth represented in the 3D image data and to each virtual crown center position.

[0060] The method may include generating and displaying a re-slice image of a re-slice plane that includes a virtual tooth axis of the missing tooth.

[0061] Such re-slice images, for example mesial-distal, buccal-oral, or labial-oral re-slice, may be useful to show the placement site for the dental implant and the immediate surroundings of the envisaged restoration. The re-slice images in particular allow the dental clinician to check whether the virtual tooth axis can be used as the placement axis for these surroundings. Thus, the re-slice images help to avoid problems during the treatment of the patient by enhanced planning, without generating more functional workload for the dental clinician.

[0062] The 3D image data may be displayed with displayed lines corresponding to the tooth axes for each crown center position and / or lines corresponding to the virtual tooth axes for each virtual crown center position.

[0063] In general, the above methods and features provide dental clinicians with enhanced capabilities for diagnosing dental conditions, planning, implementing and / or reviewing treatments, without increasing the clinician's workload through increased interaction with computer-implemented methods to utilize this functionality. Quite to the contrary, the methods replace a significant portion of previous interaction, particularly in terms of having to navigate 3D image data for imaging.

[0064] Thus, the present disclosure provides, among other things, a method for generating 2D re-slice images from volumetric image data in a different manner that is substantially user-independent. Existing user-facing workflows typically required the user to find fiducials in the image data, such as tooth position indicators, tooth axes, or dental curves for the upper and / or lower jaw. However, identifying these fiducials is a tedious task that involves scrolling through various views / slices of the image data and is subject to intra- and inter-individual variability. Furthermore, if the fiducials to be identified in the generation of the 2D re-slice images were not clearly identifiable anatomical landmarks, their display was prone to change based on, among other things, the user's experience and personal preferences. It is also noted that the user's already selected view / slice of volumetric data to show such fiducials essentially influences the results and inputs for the subsequent treatment plan execution and review.

[0065] The method of the present disclosure has the advantage that it is user independent and allows uniform and consistent generation of 2D reslice images without interference. Thus, when reviewing the reslice images generated according to the present disclosure, the clinician can be assured that the reslice images are the result of an objective workflow with less errors and does not have to take into account whether the pre-processing of the volumetric data was performed by the clinician, a colleague or an assistant. This consistency of reslice image generation is particularly relevant and improves results when comparing reslice images from subsequent volumetric image data of the same patient at different time points.

[0066] For panoramic re-slice images, the disclosed method utilizes the detected tooth crown center positions to fit dental curves to the upper and lower jaws. This approach provides a particular advantage that it immediately generates a curved re-slice plane through a voxel image volume suitable for projecting a panoramic re-slice image, the curved re-slice plane passing through substantially all tooth volumes of imaged teeth. In contrast, user-directed workflows generally require iterative adjustment of dental curves to obtain a panoramic re-slice plane with such a desired path. [Brief description of the drawings]

[0067] The following drawings illustrate preferred embodiments of the present invention. These embodiments should not be construed as limiting, but should be understood to merely enhance the understanding of the present invention in conjunction with the following description. In these figures, the same reference numerals refer to features having the same or equivalent functions and / or structures throughout the drawings. It should be noted that repeated descriptions of these components are generally omitted for reasons of brevity.

[0068] [Figure 1] 1 is a flow chart illustrating a series of steps performed by a computer-implemented method of the present disclosure to generate a 2D image from 3D image data of a patient's dentofacial anatomy. [Diagram 2] 1 is a flowchart illustrating a series of steps performed by a computer-implemented method of the present disclosure to detect anatomical landmarks in a voxel image volume using a first neural network. [Diagram 3] 3 is a schematic overview of a first neural network for detecting anatomical landmarks that may be implemented in the method shown in FIG. 2. [Figure 4] FIG. 1 shows three occlusal landmarks detected by the first neural network and initially used in the method for determining the crown center position. [Diagram 5]13 is a flow chart showing steps of a computer-implemented method for determining crown center location using a second neural network. [Figure 6] 1 shows two volume renderings of a patient's dentofacial image data from different directions. [Figure 7] This shows the spatial ordering of the cluster centers relative to the center point c. [Figure 8] 2 shows two exemplary visualizations of the mandible extracted from 3D image data, with the tooth crown center positions indicated by dots and the detected tooth lines showing the estimated tooth axes. [Figure 9] 13A-13C show panoramic images automatically generated based on detected tooth crown center positions, where (a) shows a panoramic image generated using dental curves determined for the mandible, and (b) shows a panoramic image generated using dental curves determined for the maxilla. [Figure 10] An exemplary screenshot showing windows respectively displaying lingual-buccal, mesial-distal, and axial re-slices of tooth positions automatically derived from volumetric 3D image data, with the upper right portion of the screenshot showing a tooth chart window for selecting and displaying tooth re-slice images, such as (a) a re-slice image of tooth position 23 for an adult patient, and (b) a re-slice image of tooth position 54 for a pediatric patient. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0069] The flowchart in Figure 1 provides an overview of a computer-implemented method according to the present disclosure. With reference to Figure 1, after describing the general method of the present disclosure, more details are provided regarding method options for implementing aspects of the method.

[0070] The 3D image data 110 to be processed includes a voxel image volume 120 representing at least a portion of the patient's dentofacial anatomy. The 3D image data 110 may further include additional information, such as information about the patient, e.g. age, sex, etc.

[0071] The voxel image volume 120 is preferably derived from CT data (Computed Tomography data) or CBCT data (Cone Beam Computed Tomography data). Thus, the voxels of the voxel image volume 120 of the 3D image data 110 represent intensity values, each of which corresponds to a density value of the scanned volume represented by one of these voxels.

[0072] Prior to using the voxel image volume 120 as an input, the voxel image volume is preferably subjected to pre-processing 115 (shown by dashed lines in FIG. 1 ) according to the voxel image volume used to train the neural network that is applied to the voxel image volume 120 in steps 130 and 150. As will be explained in more detail below, the pre-processing may include an intensity normalization step and / or a resizing step.

[0073] The voxel image volume 120, with or without pre-processing, enters as input into step 130. At step 130, at least one anatomical landmark 130 represented in the voxel image volume 120 is detected by using a first neural network, preferably using a convolutional neural network (CNN), that has been previously trained to predict the location of the at least one detected landmark 130. Examples of anatomical landmarks 130 are listed in Table 1. [Table 1-1] [Table 1-2]

[0074] The method further uses the voxel image volume 120 as an input for determining, at step 150, a crown center position 160 of each tooth represented in the voxel image volume 120. This process relies on a second neural network, in particular a convolutional neural network, that has been previously trained to predict the position of each tooth represented in the voxel image volume 120. In addition to the voxel image volume 120, the process of step 150 has at least one detected anatomical landmark 130 as an input for determining the crown center position 160.

[0075] Finally, the method generates at least one 2D image that is presented to the dental clinician in step 170. As mentioned above, the 2D image is essentially generated automatically at least upon request of the user, i.e. the dental clinician, to provide the 2D image without the need to manually or interactively navigate the display of the voxel image volume 120 on the screen. Nevertheless, the method may provide the dental clinician with tools to adapt or fine-tune the position and / or orientation of the 2D image within the voxel image volume 120. In other words, the method may include a user interface for input by the dental clinician to modify the orientation and / or position of a 2D slice extending within the voxel image volume 120. This planar or curved 2D slice serves as a surface onto which the 2D image 170 is projected based on the information (intensity, density) of the voxels.

[0076] 2, the 3D image data 110 may be pre-processed when extracting the voxel image volume 120. In particular, the 3D image data may be normalized and / or rescaled as described in more detail below. This pre-processing preferably conforms the 3D image data 110 to the voxel image volume 120, such that the characteristics of the voxel image volume 120 correspond to (e.g., essentially match) the characteristics of the voxel image volume of the data that has been used to train the neural network.

[0077] As mentioned above, the first and / or second network is specifically a convolutional neural network, although other types of neural networks may be used. However, convolutional neural networks have been shown to be particularly accurate, efficient, and reliable for image analysis. Nevertheless, in light of this preference, the neural network is referred to as a convolutional neural network or "CNN", although other types of neural networks may be implemented.

[0078] [Preprocessing - Intensity normalization] For example, the 3D image data may contain voxels with intensity values ​​ranging from -1000 for air, to about +200 for soft tissue, and about +600 for bone tissue. On the screen, these values ​​are preferably displayed using a grey scale.

[0079] Before being used as input for the neural network, the voxel values ​​are preferably rescaled so that the background grey intensity levels, the soft tissue grey intensity levels (soft tissue values), and / or the hard tissue grey intensity levels (hard tissue values) each correspond to the levels of the training data for the neural network applied to the voxel image volume 120.

[0080] In a first step, to normalize the intensity values ​​I(v)=I(i,j,k) of the original volume of 3D image data 110, an upper bound I max Threshold and Lower Limit I min The threshold can be calculated as follows:

number

[0081] In this example, the intensity values ​​are chosen to be bounded by values ​​of -1000 and 3500.

[0082] These values ​​are then used to rescale the original intensity values ​​of the volume, I(v), as follows:

number

[0083] where s is a scaling factor that is initially set to 1. The remaining negative intensity values ​​(3) are set to 0. Then, a scaling factor s is determined to scale the intensity values, so that the soft tissue values ​​or the hard tissue values ​​are scaled to a fixed value. For example, the soft tissue values ​​may be scaled to a fixed value, e.g., 0.15.

[0084] [Preprocessing - Volume Resampling] Before or after rescaling the original volume intensity values, the volume may be resampled to a fixed voxel size, preferably corresponding to the voxel size used in the training data. Although the number of voxels may be increased (i.e., to a smaller voxel size), it is advantageous if the resizing step includes subsampling the original volume to a smaller number of voxels (i.e., a larger voxel size), preferably without cropping the image volume. As a reference, it is shown that a CBCT scanning device imaging the entire skull usually produces a volume image having a voxel size of about 0.4 mm or less.

[0085] In particular, depending on the landmarks of interest, the volume image may be resampled to a voxel size of about 0.4 mm to 3 mm, preferably to a voxel size of about 0.4 to 2 mm, more preferably to a voxel size of about 0.4 mm to 1 mm, or to a voxel size of about 0.4 mm to / or 0.8 mm, if necessary, especially taking into account the voxel size of the provided 3D image data. For example, to detect the frontal foramen of the mandibular nerve, the use of smaller voxel sizes, for example between 2.0 and 0.4 mm, in both the detection data and the training data, is preferred. For other landmarks listed in Table 1, relatively larger voxel sizes, for example between 0.8 and 3.0 mm, may be used.

[0086] The resizing process can be performed as explained below. If the original volume is resized to (N x ,N y ,N z ) voxels, we define the volume as Δv (t) =(Δx (t) ,Δy (t) ,Δz (t) ) to the target voxel size of the resampled volume.

number

number

[0087] In (4), "r" stands for "resampled" and "t" stands for "target", where the operator [:] corresponds to integer rounding. Thus, rounding in (4) results in an output volume with an integer number of voxels. Thus, the voxel size of the resampled volume, Δv (r) =(Δx (r) ,Δy (r) ,Δz (r) ) may differ slightly from the target voxel size.

[0088] The exact voxel size consistent with (4) is given by:

number

[0089] Following the calculation of the size of the resampled volume, the volume itself is resampled. The voxel index of the resampled volume, v (r) =(i (r) ,j (r) ,k (r) ) is 0 to

number

number

number

[0090] where voxel index v of the resampled volume (r) is converted to a voxel index v=(i,j,k) in the original volume. For this, the index is converted to a Cartesian position p=(x,y,z) using the following formula: p=(v (r) +0.5)Δv (r) (6)

[0091] These distances can be converted back to non-integer voxel indices in the original volume using the following formula:

number

[0092] Since the corresponding volume index v of the original volume may have non-integer values, nearest neighbor interpolation may be used to determine the intensity values ​​of the resampled volume.

[0093] [Landmark detection] The resampled volume data I(v), representing a portion of the patient's skull, may be used in a neural network-based process 130 (FIG. 1) to detect anatomical landmarks, in particular at least one of the anatomical landmarks listed in Table 1. This process 130 for detecting at least one anatomical landmark is shown in the form of a flow chart in FIG.

[0094] [Landmark detection: volume entity generation] From the volume I(v), a volume entity is generated ( FIG. 2 : step 220) having a predetermined number of voxels, such as 25×25×25 voxels. This procedure results in a voxel index v=(v x ,v y ,v z ) (where C ∈ [1,...,C]) gives rise to a volume entity of C centered on C.

[0095] These volumetric entities are then used as input to a neural network, in particular a convolutional neural network (FIG. 2: step 230). An exemplary schematic overview of a preferred 3D convolutional neural network for detecting landmarks is shown in FIG. 3. For each anatomical landmark to be detected, such a neural network is trained on the corresponding anatomical landmark before being applied to the patient's voxel image volume. It is noted that both the first neural network and the second neural network may include a 3D convolutional neural network as diagrammatically represented in FIG. 3.

[0096] The neural network of FIG. 3 includes three consecutive blocks: a convolutional layer, a ReLu layer, and a max pooling layer. As shown, the network ends with a fully connected layer with three output values. The volume entity is sent through the convolutional layer, the ReLu layer, and the max pooling layer. At the end of the network, the fully connected layer has three outputs corresponding to the components of an offset vector, which estimates the location of the 3D landmark relative to the center of the volume entity in the voxel image volume.

[0097] The network is trained to estimate the three components of an offset vector v0 from the volume entity center v towards the voxel index to which the predicted landmark belongs. As a result, C distinct landmark position predictions are obtained at the voxel index (FIG. 2: step 240). v i =v+v0(9)

[0098] Note that because the components of the estimated offset vector are floating point, these voxel indices may have non-integer values. These resulting non-integer voxel indices are then transformed to Cartesian coordinates p using (6).

[0099] [Landmark detection: Cluster and filter predictions] 2 may be implemented to filter out from the landmark location predictions of C those that are deemed less accurate than a predefined threshold. To this end, the predictions are first clustered using a brute force technique.

[0100] First, the number of inliers for each predictor p is counted, and for a selected predictor, an inlier is any other predictor located at a distance less than a given threshold distance from the selected predictor, which may be set between 8 and 16 mm, more preferably between 8 and 12, such as at 10 mm. For the predictor with the most inliers, the predicted landmark positions of the inlier volume entities are averaged to indicate the cluster center k.

[0101] Next, the distance from each predicted value towards this cluster center k is calculated. Predictions at distances from the cluster center k that exceed the threshold distance are discarded. This results in a length vector that contains the remaining predicted values ​​p of the landmark positions.

number

number

[0102] A high probability means that a reliable estimate of the landmark position is obtained, whereas a low probability indicates an unreliable estimate of the landmark position. In particular, an estimate is considered unreliable when its probability is lower than between 0.2 and 0.05, preferably between 0.15 and 0.05, e.g., less than 0.1. Unreliable estimates usually occur when the retrieved anatomical landmarks are not present in the 3D image data of the patient's dentofacial anatomy, e.g., when the skull is only partially scanned.

[0103] Some of these detected landmarks may be useful for determining the crown center position, as described in more detail below. Furthermore, the detected landmarks may be used to reorient the coordinate systems of the dentofacial anatomical structures and the 3D image data or voxel image volume relative to each other.

[0104] [Landmark detection: training] In this example, the neural networks for detecting anatomical landmarks were each trained using a training data set 231 based on 225 different scans, where CBCT scans are used, although as noted above, other CT scanning techniques may be used.

[0105] Each scan showed the landmarks listed in Table 1. To increase the amount of training data, each CBCT scan was split in half along the x-axis and mirrored to the sagittal plane, where x=0 (FIG. 2: step 232). Consequently, 450 symmetric skull data sets were acquired.

[0106] For each landmark, the network was trained using 150 volume entities per dataset, each of which was randomly selected with a maximum distance of 12 voxels from the annotated anatomical landmark location, resulting in a maximum number of volume entities used to train the network of 67,500.

[0107] In this example, all networks were trained in MATLAB® (The MathWorks, Inc., Natick, Massachusetts, USA) using stochastic gradient descent with momentum. Specifically, 100 epochs were used with a mini-batch size of 100 volume entities (an epoch in MATLAB® is a measure of the number of times all of the training vectors are used once to update the weights). The order of all volume entities was shuffled after every epoch to ensure that volume entities from a variety of training data were included in each mini-batch. This improved the convergence of the optimization. The initial learning rate was equal to 0.0001.

[0108] To standardize the training input data, intensity normalization and volume resampling with predefined parameters were applied, as described above.

[0109] [Determining the crown center position] This portion of the method uses a second neural network (Figure 1: step 150) to determine the crown center location for each tooth based on anatomical landmarks (Figure 5: step 510) and the voxel image volumes (Figure 5: step 515).

[0110] [Determining the crown center position: Bounding box placement] First, in determining the crown center position, one or more anatomical landmarks p ∈ [1, 2, 3] that are located on the occlusal plane or that indicate the occlusal plane detected by the first neural network described above are used. o is used (Figure 2).

[0111] In an exemplary embodiment, these occlusal landmarks correspond to point p1 between the tips of the upper incisors, occlusal landmark p2 between the right molars, and occlusal landmark p3 between the left molars (see FIG. 4). An example of a transparent iso-surface rendering of the teeth region of a 3D image dataset with all three landmarks visualized as points is shown in FIG.

[0112] Nevertheless, other landmarks or alternative landmarks may be used as long as they allow to indicate the occlusal plane of the maxillary or mandibular teeth, which may be necessary or advantageous, for example, if one or more of the above-mentioned occlusal landmarks are not represented in the patient's 3D image data.

[0113] In the next steps (FIG. 5: steps 520 and 525), these landmarks are preferably located at the estimated crown centers.

number

[0114] Presumed crown center position

number

number

number

number

[0115] For each of the determined occlusal landmarks, an estimated crown center position

number

number

number

[0116] Presumed crown center position

number

number

[0117] Presumed crown center position

number

number

[0118] [Determining the crown center position: offset voting algorithm] Within each of the image regions defined by the bounding boxes 610, a second neural network approach is used that is the same or similar to the offset voting approach applied in the landmark detection. In other words, the second neural network preferably also uses an offset voting algorithm to determine the crown center predictions (see FIG. 3).

[0119] Prior to detection, the intensity of the data is preferably normalized and / or the volume is resampled according to the training data as described above. In particular, resampled volumes of up to 2 mm voxel size are used, preferably the volume is resampled to a voxel size of about 0.4 mm to 1 mm, more preferably to a voxel size of about 0.4 mm to 0.5 mm, such as a voxel size of about 0.4 mm.

[0120] Then, for each possible tooth number, for example, 500 volume entities are generated, preferably with a size of 25x25x25 voxels. For the first neural network, a different number of volume entities may be selected, in particular in the range of 250-650 volume entities. Also, volume entities with different sizes may be used, in particular in the range of 15-40 voxels along each dimension of the volume entity (FIG. 5: step 530). The voxel index of the voxel entity center is randomly selected from the voxels inside the bounding box 610 with the corresponding tooth number. In other words, voxels are randomly selected from the bounding box 610, each selected voxel representing the center of a volume entity with a predetermined size, for analysis by the neural network, as described below.

[0121] All these volume entities are then processed by a second neural network trained to estimate the offset towards the crown center position of the tooth with the tooth number corresponding to the tooth number of the bounding box label (FIG. 5: step 540). This results in a number of offset vectors corresponding to the number of volume entities, i.e. 500 in this example, which, when added to the volume entity centers, provide a corresponding number of predictions of the crown center positions (here 500 predictions, FIG. 5: step 545).

[0122] [Determining the crown center position: Clustering] As explained above, the method for detecting the crown center location results in a corresponding number of crown center predictions for each tooth number. In the above example, there are 32 x 500 = 16000 predictions, all with corresponding tooth numbers (32 teeth x 500 volume entities of a complete adult dentition). In order to obtain the exact coordinates of the crown centers, these predictions are preferably clustered (Figure 5: step 550).

[0123] For the purpose of clustering, the predictions may be split into two sets based on the predicted tooth numbers corresponding to the upper and lower jaws, in other words the upper and lower jaws are preferably analyzed separately.

[0124] First, the clustering step 550 determines the tooth number for which the cluster with the most inliers is found. This can be done using a brute force clustering approach, as described above. In this approach, a predefined inlier threshold distance (e.g., 1.4 mm) is used, resulting in a number of inliers associated with the averaged crown center c k The number of inliers divided by the number of volume entities (500 in this example) gives the cluster probability P k Represents.

[0125] Center of the tooth crown c k Within the inlier set (having size L) associated with, the occurrences of each tooth number are counted. Together, these occurrences divided by L are calculated as cluster centers c k A discrete probability distribution q of possible tooth numbers assigned to k (j). Then, all such inlier predictors are removed from the list of predictors.

[0126] This procedure is repeated for the remaining set of predictors until 16 cluster centers per jaw are found (FIG. 5: step 555).

[0127] [Determining the crown center position: obtaining optimal tooth configuration]" Optimal tooth configuration C m Next, the 16 cluster center positions c k This tooth configuration labels adjacent crown centers with their respective tooth numbers, which preferably take into account possible missing teeth.

[0128] First, clusters with an insufficient number of inliers are considered to be related to missing teeth and are removed. Typically, a threshold ratio in the range of 5%-15% inliers, especially 7% inliers, is used.

[0129] The remaining probabilities are then divided into two parts by the maximum cluster probability Pmax = max(P k : k = 1, ..., 32), resulting in the cluster with the highest probability having probability equal to 1. All other probabilities are scaled accordingly.

[0130] Furthermore, the cluster center c k are divided into two sets, one set related to teeth represented in the image data and one set of cluster centers related to teeth that may be represented. The upper and lower probability limits used for this distinction can be set, for example, between 0.6 and 0.4 and between 0.2 and 0.05, respectively, such as 0.5 and 0.1. Cluster centers with a relative probability lower than the lower probability limit are considered not related to the represented tooth and are discarded.

[0131] The optimal tooth configuration C is then searched for by first considering only those cluster centers that are considered to be related to the teeth represented in the image data, i.e., cluster centers that have a probability exceeding the upper probability limit. Such a tooth configuration includes a list of crown centers and their corresponding tooth numbers. Then, the remaining cluster centers c for K≦16 are searched for by considering only those cluster centers that are considered to be related to the teeth represented in the image data, i.e., cluster centers that have a probability exceeding the upper probability limit. k All the different possibilities are explored to find the best tooth configuration.

[0132] Mathematically, there are M possibilities to choose K tooth numbers out of 16.

number

[0133] For example, if 15 cluster centers are found in the upper jaw (K=15), then there are M=16 possible tooth configurations.

number

[0134] Table 2 lists maxillary tooth numbers according to the universal tooth numbering system, in which the last upper right molar has tooth number 1 and the last lower right molar has tooth number 32, assuming a complete adult dentition.

[0135] The combination of these M teeth

number

number

number

number

number

number

number

[0136] Once the optimal tooth configuration has been calculated for the set of cluster centers deemed certain, an additional cluster is selected from the list of possible cluster centers. This cluster center is now added to the configuration, so that the configuration now has N+1 cluster centers. For this updated list, the optimal tooth configuration is again calculated using (20).

[0137] The total probability of this configuration

number

number

[0138] In the next step, the procedure is repeated and subsequent possible tooth clusters are explored until the calculated configuration probability falls below, for example, 0.7 times, the previous best configuration probability of all possible additional cluster centers.

[0139] Therefore, the optimal configuration

number

[0140] Cluster center sorting. To calculate the probability of the different configurations, all the crown centers are preferably ordered along the dental arch in space (see FIG. 7). For this purpose, the center position p of the occlusal dental curve dccan be roughly estimated or obtained by adding a fixed offset to the most reliable of the occlusal landmarks.

[0141] Next, the crown center is the cluster center c k and dental curve center p dc The angle Θ between the vector connecting i (See the default world axis configuration shown in Figure 7.) In this example, the upper jaw cluster centers are ordered according to increasing angles, and the lower jaw cluster centers are ordered according to decreasing angles Θ i are sorted according to

number

[0142] Permutation probability. As mentioned above, for every cluster center c k is a discrete probability distribution q that aggregates the discrete probability of the kth cluster having a particular tooth number j, j ∈ [1,...,32]. k (j) Having a specific tooth configuration

number

number

number

number

[0143] Here, the power 1 / K is used to make the probability of configurations with different numbers of cluster centers equal.

[0144] Distance probability. Preferably, the Euclidean distance d between adjacent cluster centers k,k+1 is the ordered cluster center c of K k These distances are then calculated for each configuration with the mesiodistal averaged distances obtained by averaging the distances between adjacent tooth centers in the training data (taking into account missing teeth and hypodontia).

number

[0145] Various d k,k+1 and

number

number

number

[0146] In this example, σ is equal to 4 mm and α m is a rescaling factor used to compensate for the size of the patient's jaw, which may be larger or smaller than the average reference jaw. The rescaling factor is calculated to minimize the difference between the scaled distance and the reference distance.

number

[0147] Scaling factor α mThe value of is preferably constrained to be in the interval [0.8,1.2].

[0148] Probability of edentulism. The previous section always assumes a complete dentition as the reference for the studied jaw, meaning that the reference configuration includes all possible tooth numbers. However, in hypodontia, one or more teeth are missing without a physical gap between the adjacent teeth. Therefore, the average distance in hypodontia

number

number

[0149] The different oligodontia cases investigated in this exemplary embodiment are missing second incisors or missing second premolars. These cases can occur on the right, left or both sides of the jaw. Including complete dentition, this represents seven different oligodontia configurations to investigate. Since oligodontia occurs less frequently than complete dentition, the oligodontia probability is equal to 1 for the complete dentition case and 0.8 for the other six oligodontia cases.

number

[0150] [Determining the crown center position: training a neural network] Similar to anatomical landmarks, crown center locations were manually indicated in the database of 225 3D images (Figure 5: step 541). For each crown center location, the network was trained using 150 volume entities per dataset. Each volume entity was randomly selected with a distance of up to 12 voxels from each annotated crown center. The neural network was trained to estimate offset vectors from the volume entities towards the respective crown centers in the same way as described above for anatomical landmarks. In order to make all input data as similar as possible, intensity normalization and volume resampling were applied, as described above.

[0151] [Estimation of the axis of detected teeth] By analyzing the patient's dental condition to determine anatomical landmarks and crown center positions, there are further method steps that can be performed, particularly to eliminate manual input steps that previously had to be performed by dental clinicians.

[0152] The previous section explained how the tooth configurations for both the maxilla and mandible can be calculated from 3D image data that represents at least a portion of the patient's dentofacial anatomy. These tooth configurations are determined by the crown center t j and its assigned tooth number j. In an additional step, the major tooth axis in the apical-coronal direction can be calculated.

[0153] In the following exemplary embodiment, this is the crown center t iThis is done by starting from a 2D axial z slice, where the voxel positions are located, and tracking the corresponding voxel positions across subsequent axial z slices in the direction of the root. Preferably, a so-called Kanade-Lucas-Tomasi tracker is used with adjacent axial z slices serving as image frames. As already mentioned above, the axial z slice is the slice perpendicular to the axis of the voxel image volume closest to the apical-coronal direction. The original axis of the 3D image data and the axial voxel image volume may have been set by the scanner acquiring the 3D image data.

[0154] The path acquired by this tracker essentially provides the basic main tooth axis. For each axial slice, the previous slice Z s-1 (x,y) crown center

number

number

number

number

[0155] Using this method, the center point of each tooth is tracked over a predetermined number of slices (24 in this exemplary embodiment) in the direction from the occlusal plane to the root. The tooth orientation is then estimated by fitting a 3D line through the tracked points. Preferably, the line begins at the starting point r oand the direction e. r=r o +γe (33)

[0156] To eliminate the influence of outliers, a RANSAC procedure is preferably used, for example with 200 iterations. At each iteration, three random points are

number

[0157] Then, all points located at a distance from this line smaller than a predefined inlier threshold, for example 1 mm, are considered as inliers, and the line is fitted again through these inliers. Again, the number of inlines is calculated for this line. The iteration with the largest number of inliers is considered the best configuration, resulting in the final 3D line. From this fitted line, a direction may be used in particular. A line representing the tooth axis direction passing through the detected crown center positions is shown as a white line 810 for the exemplary case of the mandible of a patient in FIG. 8. In this figure, the crown center positions of the teeth known to be represented in the voxel image volume are shown as white dots 820, while the estimated crown center positions of the missing teeth are shown as black dots 830.

[0158] Alternatively, the tooth axis can be determined using segmentation methods. Tooth voxels in (CB)CT images are usually brighter than the surrounding tissue and air voxels due to the relatively high density of dental tissue. Using this property of tooth voxels, a rough segmentation of the tooth can be obtained from the detected crown center position by identifying brighter voxels connected to the voxel containing the crown center position.

[0159] Preferably, this segmentation is performed in a cropped image region, which can be set from the detected crown center location by defining a volume surrounding the crown center, the size, shape and location of which can be selected to narrowly encompass the tooth whose crown center is detected. Later, tooth axes can be determined for the segmented teeth, for example using principal component analysis.

[0160] In yet another embodiment, determining the tooth axis includes the use of a template tooth, which may be either a crown model or a complete tooth (i.e., crown and root) model.

[0161] Based on the tooth label assigned to a given crown center, a template tooth may be selected having a tooth type corresponding to the tooth type represented at the crown center location, which may then be aligned and / or matched to the represented tooth.

[0162] Preferably, the template tooth and the represented tooth are aligned in an initial step by aligning the detected crown center position with a crown center position determined for the template tooth. The tooth axis at the detected crown center position can then be derived from the tooth axis determined for the aligned and / or matched template tooth.

[0163] [Add missing tooth position] Once the crown center positions are detected together with their tooth numbers, the position of each missing tooth in the imaged patient's dentition can also be determined. The crown center positions of missing teeth are preferably extrapolated based on (i) the crown center positions of the teeth represented in the 3D image data and the associated tooth labels, and (ii) information about the average crown center position of each of the teeth (e.g. 32 teeth for an adult patient).

[0164] The average crown center position can be determined from the training data by calculating the average crown center position and the variation from the average crown center position observed in the x, y, and z directions for each of the 32 tooth numbers. An example is presented in FIG. 8, where the crown center positions are shown as dots in the imaged tooth volume. Also in this figure, the crown center positions of teeth that are missing from the patient but whose crown center positions have been estimated are visualized as dots.

[0165] Therefore, the tooth axis directions may also be estimated for these missing teeth, preferably using linear interpolation based on the tooth axis directions of the adjacent teeth, as shown in Figure 8. If a missing tooth has only one adjacent tooth, linear interpolation between the tooth axes of the adjacent teeth cannot be performed. Instead, the directions of the adjacent teeth are copied.

[0166] [Tooth Plane Estimation] The crown center positions detected for the upper and lower jaws make it possible to determine the tooth planes of the upper and lower jaws, respectively. The tooth plane of a jaw is estimated by fitting a plane passing through the crown center positions determined for that jaw. The crown center positions of wisdom teeth are preferably not included in this fitting process.

[0167] If both the upper and lower jaws contain teeth, the averaged tooth planes can be fitted through the complete set of crown center locations from both the upper and lower jaws. When fitting the average tooth planes to the crown center locations, the crown center locations of wisdom teeth are preferably not considered.

[0168] After determining the maxillary dental plane, the mandibular dental plane, and / or the average dental plane, a dental transformation D can be determined for any one of the dental planes, which transforms the original volumetric image with the dental plane parallel to the z-plane of the volumetric image, and which preferably orients the imaged skull anatomy with the incisors facing forward and at their base.

[0169] Applying the tooth transformation D to the crown center positions of the detected teeth in the upper or lower jaw provides the corresponding crown positions in the 2D planar reslice plane. From the (x,y) components of these transformed crown center positions, a 2D spline curve can be calculated that lies in the plane of the tooth. In this way, 2D dental curves can be determined for the upper and lower jaw, respectively.

[0170] [Generate reslices] The 2D or 3D dental curve (FIG. 9: reference number 910) allows defining a curved re-slice plane through the voxel image volume suitable for projecting a panoramic re-slice image 920 onto it.

[0171] Determining the tooth axis may further serve as a reference for generating a mesial-distal re-slice image 1010, a buccal / lingual-oral re-slice image 1020, and / or an axial re-slice image 1030 at a given or selected tooth location (see FIG. 10). The mesial-distal re-slice image 1010 includes the tooth axis and is essentially parallel to the 2D or 3D dental curve at the selected tooth location. The buccal / lingual-oral re-slice image 1020 also includes the tooth axis but is essentially perpendicular to the 2D or 3D dental curve at the tooth location. The axial re-slice image 1030 includes the detected or possible crown center location at the tooth location, and the plane of the axial re-slice is perpendicular to the tooth axis.

[0172] [Generation of dental chart using the detected crown center position] The detected crown center positions and associated tooth labels may further enable automatic entry of an icon into a patient-specific dental chart indicating the presence or absence of a tooth posterior to a given tooth number in the patient's dentition. Optionally, the dental chart may further reflect the relative positioning of the patient's teeth based on the distance between the detected crown center positions.

[0173] [Using dental charts to navigate volumetric image data and generate tooth-specific reslices] The computer-implemented method may be embodied as a computer program product. The computer-implemented method may be implemented as a software application including a user interface displaying a dental chart including selectable icons representing tooth positions. When a user selects one of the icons, the application may automatically generate and display the mesial-distal, buccal / lingual-oral, and / or axial re-slice images of the selected tooth position derived from volumetric 3D image data representative of the patient's dentofacial anatomy. In one embodiment, the application may display the volumetric 3D image data and automatically orient the 3D images within a viewing window such that the selected tooth position is visible.

Claims

1. A computer implementation method for automatically generating and displaying a 2D image of a portion of a patient's maxillofacial anatomical structure derived from 3D image data, wherein the method includes the following steps: Receive 3D image data including a voxel image volume representing the maxillofacial anatomical structure of the patient, wherein the voxel image volume includes the patient's teeth, and each voxel in the voxel image volume is associated with a radiation intensity value. Using a first artificial neural network, detect one or more anatomical landmarks within the voxel image volume. Using the detected anatomical landmarks as reference points, a second artificial neural network is used to determine the central position of the crown of each tooth among the multiple teeth included in the voxel image volume, and The 2D image is generated as a reslice image from the voxel image volume based on at least one crown center position.

2. The method according to claim 1, further comprising the step of calculating a curved path along the dental arch of the patient's teeth by using one or more anatomical landmarks and / or fitting the curvature to the determined crown center position, and preferably using the curved path as a reference for generating the reslice image.

3. The method according to claim 2, further comprising the step of detecting the centers of the left and / or right condyles using the first artificial neural network, wherein a curved path following the dental arch is calculated, preferably using the centers of the left and / or right condyles as endpoints.

4. The method according to claim 2 or 3, wherein the reslice image is a panoramic image generated based on a curved path that follows the shape of the patient's dental arch.

5. The method according to claim 2, further comprising the step of analyzing 3D image data within at least one image region surrounding a determined crown center position of one tooth in order to derive a tooth axis, wherein the tooth axis intersects the crown-shaped tooth surface, extends from this intersection toward the apex of the tooth, and includes the determined crown center position for the tooth.

6. The method according to claim 5, wherein deriving the tooth axis includes the steps of starting from a tooth cross section containing the estimated crown center position, tracking corresponding positions in subsequent adjacent cross sections, and continuing to track corresponding positions in adjacent cross sections along the tooth, particularly in the direction toward the apex of the tooth, and calculating the tooth axis based on these corresponding positions.

7. The method according to claim 5 or 6, dependent on claim 2, wherein the orientation of the derived tooth axis is calculated with respect to a curved path following the dental arch.

8. The method according to claim 5 or 6, wherein the reslice image is generated and displayed as a cross-sectional reslice image including the center of the tooth crown, and the plane of the cross-sectional reslice image is perpendicular to the tooth axis of the tooth.

9. The method according to claim 2 or 3, wherein the reslice image is generated and displayed as a mesial-distal reslice image that includes the tooth axis and is oriented essentially parallel to a curved path at one position of the tooth, or as a buccal / labial-oral reslice image that includes the tooth axis and is oriented essentially perpendicular to a curved path at one position of the tooth.

10. The method according to claim 1 or 2, wherein the second artificial neural network comprises a module trained to estimate three components of an offset vector directed from a position in the voxel image volume to a predicted crown center position.

11. The method according to claim 1, further comprising determining the central position of the tooth crown by assigning a tooth label to each determined central position by analyzing the relative position of the central position of the tooth crown using the detected anatomical landmarks.

12. The method according to claim 11, further comprising the step of deriving a virtual crown center position and associated virtual tooth label for any tooth missing from the patient's 3D image data from the determined crown center position and the assigned tooth label.

13. The method according to claim 12, further comprising the step of generating and displaying a dental chart including selectable icons indicating teeth represented in the 3D image data and / or any teeth missing from the 3D image data, wherein when one of the icons is selected, a 2D image of the tooth of the selected icon is displayed, including at least the reslice image.

14. The method according to claim 12 or 13, further comprising displaying the 3D image data and displaying markers at the positions of each determined crown center position and / or each virtual crown center position.

15. The steps of extrapolating the virtual tooth axis of at least one missing tooth, wherein the virtual tooth axis includes the virtual crown center position of the missing tooth, and A step of calculating the direction of the virtual tooth axis based on the local direction of the curved path following the dental arch and / or the direction of the tooth axis of at least one adjacent tooth. The method according to claim 12 or 13, further comprising:

16. The method according to claim 12 or 13, dependent on claim 2, wherein calculating a curved path along the dental arch of the patient's teeth includes fitting the curvature of the path to both the determined crown center position of the tooth represented in the 3D image data and each virtual crown center position.

17. The method according to claim 15, further comprising the step of generating and displaying a reslice image of a reslice plane that includes the virtual tooth axis of the lost tooth.

18. The method according to claim 15, further comprising displaying the 3D image and displaying a line corresponding to the tooth axis for each crown center position, and / or displaying a line corresponding to the virtual tooth axis for each virtual crown center position.

Images