Automatic assesment of soft tissue recession in an oral cavity

Abstract

The disclosure generally relates to a computer-implemented method for determining a health state of soft tissue in an oral cavity. In more detail, the method may utilize a trained machine learning model that has been trained on the basis of annotated ground truth data to determine a measure for soft tissue recession in an oral cavity.

Description

[0001] AUTOMATIC ASSESMENT OF SOFT TISSUE RECESSION IN AN ORAL

[0002] CAVITY

[0003] FIELD

[0004] The disclosure relates to computer-implemented method(s) for determining and displaying soft tissue recession of an oral cavity of a patient, wherein the method may use a trained machine learning model as part of determining the soft tissue recession. The disclosure furthermore relates to method(s) for generating a suitable training dataset for training a machine learning model in learning to detect soft tissue recession in an oral cavity to assist in automating soft tissue recession assessment. In addition, the disclosure relates to systems and computer-programs suitable for performing an automatic assessment of soft tissue recession in an oral cavity.

[0005] BACKGROUND

[0006] Oral health conditions in general pose great health problems across the globe, where most oral health conditions are largely preventable or treatable if observed in early stages of development. The prevalence of oral health conditions increases globally, where some of the most occurring conditions include dental caries (tooth decay), periodontal diseases, tooth loss and oral cancers.

[0007] Periodontal disease, which is generally a result of early infection of the tissue (e.g., the soft tissue) that holds the teeth in place, affects the tissues that both surround and support the teeth. The disease is characterized by bleeding or swollen gums (also known as gingivitis), pain and sometimes bad breath. If none treated, early stages of periodontal disease may develop into a more severe form, where the gum may pull away from the tooth and supporting bone, causing teeth to become loose and sometimes fall out. Anatomically, in healthy teeth, the gum generally attaches to the healthy tooth at the location of the tooth where the cementum and enamel meet, also called the cementoenamel junction (CEJ), as illustrated in Figure 1. In Figure 1, the healthy teeth situation is illustrated on the left side of the tooth 100 and a deceased situation is illustrated on the right side of the tooth 100. The right and left side is “divided” by a dotted line 101. Accordingly, in healthy teeth, the cementoenamel junction (CEJ) may not be visible as it may be covered by the gum (also denoted gingiva), where the border between the gum and the tooth is generally defined as the gingival margin. In early stages of periodontal diseases, an apical shift, arrow 103, (i.e., towards the root area 102 of a tooth 100) of the gingival margin with respect to the CEJ may start appearing, indicating early signs of soft tissue recession. This apical shift if developing further may be associated with a resulting exposure of the root surface to the oral environment, where the amount of the apical shift is defining the soft tissue recession measure. In later stages of recession, this apical shift 103 may cause periodontal pockets to be created causing teeth to become loose and sometimes fall out. The combined periodontal measure utilizing a soft tissue recession measure and a measure of pocket depth is known as the clinical attachment loss (CAL).

[0008] As soft tissue recession is not easily reversible, spotting it early is clinically very important. If spotted early, patients can stop further damage, or even recover, by changing their brushing habits, not chewing on hard objects, adjusting their ortho treatment etc. Moreover, surgical intervention used to treat recession (such as grafting surgery) would also benefit from an objective and precise method for monitoring. Therefore, to assess accurately the early stages of periodontal diseases development and the potential progression thereof, an identification of the position of the CEJ may be of high relevance in clinical practices.

[0009] In clinical practice today, periodontal disease assessment is generally performed using visual and / or tactile methods, such as using a probe positioned in the mouth during clinical examination. The probe measurements are generally considered to have a large error margin and can alter the soft tissue recession measure by applying pressure directly on the gingiva. Furthermore, probe measurements comprise major periodontal probe evaluation limitations including amongst other things, rounding off the measurements, inter-operator variations in position and angulation of the probe, which all result in inaccurate and non-replicable measurement of gingival recession for each patient and in-between patient. Accordingly, there are currently no good clinical methods for quickly, objectively, and accurately detecting and quantifying soft tissue recession and subsequent changes over time which could assist in preventing pocket depths from developing and teeth to fall out.

[0010] With the evolvement and increasing popularity of digital dentistry, digital approaches of assessing gingival recession are appearing. Some approaches rely on 3D models obtained by intraoral scanners and some approaches rely on radiation modalities, such as X-rays, CBCT, etc. As radiation solutions expose the patient to radiation, digital dentistry has an increasing focus on intra-oral scanner applicability to aiding and / or directly diagnosing different dental conditions, including soft tissue recession. The digital methods applying intra-oral scan data in assessing soft tissue recession generally comprises a plurality of manual steps, where the dental practitioner evaluates the obtained scan data by working with a 3D model in a user interface of a software application of a digital dental system. The 3D model is generally generated from the scan data, and the digital dental software may be equipped with different tools allowing a dental practitioner to assess using the digital tools more accurately in a digital environment the soft tissue recession measure. Accordingly, the dental practitioner may with such digital method manually assess, via a 3D model of an intra-oral scan of the oral environment, a measure of soft tissue recession.

[0011] Such manual assessment may be more accurate than the tactile-visual approach using the probe but may require extensive time for the dental practitioner. The dental practitioner first needs to learn how to use the tools associated with aiding in the determination of soft tissue recession measure and secondly spend extensive time to create the digital measures. In addition, such manual measures may still be considered at least partly subjective measure as they rely on the dentist positioning digital tools on e.g., a 3D model to assess recession. Furthermore, the known approaches may not reveal sufficiently the early stages of recession, as the visual inspection, whether it being direct in-mouth clinical examination or the manual assessment via the 3D model, may require detailed assessment of each tooth in view of assessing the color of the gum, apical shift changes etc. at for example central and interdental sides of each tooth. Accordingly, soft tissue recession may with the current solutions potentially only be discovered when the recession has advanced in stage and a risk with the current methods is that the advancement of soft tissue recession is only detectable over several clinical visits.

[0012] For the above reasons a need for solutions for accessing automatically soft tissue recession, allowing early detection thereof while optimizing the dental practitioners time spent in doing so, while at the same time improving the accuracy and objective measure of soft tissue recession is needed for. SUMMARY

[0013] The solutions described herein addresses the above-mentioned challenges by providing a computer-implemented method for automatically determining, quantifying, and displaying soft tissue recession of an oral cavity of a patient, wherein the method comprises retrieving in a processor intraoral scan data of a patient, wherein the intraoral scan data comprises a plurality of dental features. The intraoral scan data may be obtained by an intraoral scanner. In one embodiment, the method provides for an automatic determination of a health state of the soft tissue in the intra-oral scan data using a trained machine learning model. That is, in one embodiment, a health state of the soft tissues (also denoted the gum or gingiva) in the intra-oral scan data may be determined from e.g., a single scan without having any past datapoints to compare with. To be able to automatically detect the health state of the soft tissue in the intra-oral data read into the processor, the method may be configured to generate an input structure suitable for being processed by a trained machine learning model. The input structure may be generated by performing a data transformation of the intra-oral scan data from a 3D space to a plurality of descriptive representations of each of the dental features in the intra-oral scan data. To allow processing by the trained machine learning model, the method comprises feeding the input structure to a trained machine learning model, configured to generate a health state prediction structure representing at least two or more health state(s) of each of the descriptive representation of the intraoral scan data. Based on the health state prediction structure as output from the trained machine learning model, the method is configured to determining a soft tissue recession measure by identifying in the 3D representation the health state labels associated with soft tissue recession and evaluating a measure of the health state labels associated with soft tissue recession. In this way, it is possible from a single scan data set of an oral cavity of a patient to evaluate a health state of the soft tissue automatically without using any visual -tactile or subjective measures. The method provides for a fast and reliable method to assess soft tissue health as it has been trained on a training dataset comprising a plurality of samples representing different stages of soft tissue health, as will be apparent from the description.

[0014] To ensure that a dentist may be able to asses and even communicate the findings of soft tissue health state with a patient, the method also provides for displaying in a graphical user interface a representation of the determined soft tissue recession measure. The dental features described herein are substantially construed as one or more teeth and associated gingiva for each of the teeth in an intra-oral scan data.

[0015] The trained machine learning model, as previously mentioned have been trained on the basis of a training dataset of intra-oral scan data comprising a plurality of data samples, wherein each of the plurality of data sample may comprise a 3D model comprising one or more dental features having associated dental feature labels. Each of the dental features represents teeth and associated gingiva in the training data samples and the associated feature labels represents one or more annotated labels considered relevant for identifying a health state of the soft tissue in an oral cavity. The annotated labels may comprise any of a group of labels chosen from: no recession, recession, buccal recession, lingual recession, interdental recession. The number of annotated labels used depends on whether the neural network is trained to perform a classification or a segmentation task. In case of a classification task more than two of the above-mentioned annotated labels are taken into consideration when training the machine learning model. On the other hand, when the machine learning model is used to perform a segmentation task, resulting in no recession or recession for each point in e.g. a point cloud, as an output, only such two ground truth labels is needed. In more detail, in case of segmentation the annotated labels may be considered as ideal gingiva or tooth for each point in e.g. a point-cloud based solution. A further annotated label with respect to the segmentation may comprise the actual gingiva, which may be provided from a segmentation task segmenting the 3D model into teeth and gingiva, where the border between teeth and gingiva defines the actual gingiva. Similar considerations apply if the input to the machine learning model is a whole jaw scan and not only a single tooth. The training of the machine learning model will be described in further detail throughout.

[0016] The annotations have been made by experts within dental health and have been applied to a larger training data set used to train the machine learning model. The dental experts have been trained to classify each facet into tooth or ideal-gingiva low or ideal-gingiva high, where high and low signify confidence. Low is usually used when the expert does not see the CEJ landmark and high is when the CEJ may be seen in the training data by a dental expert. From the output of the trained machine learning model, the computer-implemented method is configured to determining from the assigned health state labels a pointset representing soft tissue recession points of each dental feature in the intra-oral scan data and quantifying the soft tissue recession measure for each dental feature based on the determined pointset. The term dental feature may be construed as either a single tooth and associated gingiva or a full jaw-scan of a set of teeth. The determination of a pointset representing soft tissue recession points of each dental feature (i.e., each tooth and associated gingiva or full jaw scan) ensures that the output from the trained machine learning model is translated back into the 3D space of the intra-oral scan data, where a quantification of the soft tissue may be determined automatically by the processor of a computer performing the computer- implemented method described herein. In other words, the pointset representing soft tissue recession points of each dental feature may comprise tooth, ideal gingiva and / or actual gingiva for each point of a dental feature.

[0017] The pointsets may represent one or more of an apical and / or coronal margin of the soft tissue for each of the dental features. That is, the pointset may define the border between the substantially exposed or healthy gum and crown (i.e., the coronal margin) and / or the border between a recessed gum and the exposed part of the gum (i.e., the apical margin). To automatically assess a measure of soft tissue recession, the method may be configured to determining the soft tissue recession measure by measuring a distance between a most apical point of the apical margin and the most coronal point of the coronal margin for each dental feature. In other words, the method may comprise identifying per tooth, the most apical point of the coronal margin and the apical margin and calculating the distance between these most apical points. In this way, only one point may be used to describe the recession measure. In one example, the method described herein may comprise determining corresponding points by identifying a coronal -apical direction of each tooth; identifying the most apical point of the gingival margin for each tooth; identifying the most apical point of the coronal margin for each tooth; and calculating a distance between the most apical point of the gingival margin and the most apical point of the coronal margin for each tooth. This may be repeated for all teeth in the intra-oral scan data.

[0018] In an example, the method may comprise determining the coronal -apical direction using a tooth-pose estimation, which is configured to calculate a local coordinate system defining the X, Y, Z direction for each tooth in a jaw or in a single tooth sample. From this coordinate system, the coronal-apical direction may be determined, and the most apical and coronal points may be determined. The coronal-apical direction may be determined using the segmented point into tooth, ideal gingiva and actual gingiva classes as output from the neural network. That is in an embodiment, when the segmented classes of ideal gingiva, actual gingiva and tooth has been proj ected back to the 3D model the processor is configured to determine the most apical and coronal point of the segmented points and to determine the distance there between.

[0019] The method described above may work both in the case when the dental features are configured as a whole jaw scan and when the dental features are configured as a single tooth and associated gingiva.

[0020] The coronal margin may in embodiments described herein be construed as a detection of the cemento-enamel junction, and / or as a predicted healthy state of the soft tissue of the oral cavity. That is, referring back to Figure 1 or Figure 2, the coronal margin 31 may be the margin where the gum in a healthy tissue meets the crown of the tooth. The apical margin 30 on the other hand represents the position of a recessed gingiva, where a part of the gum has gone through an apical shift 103 from the predicted coronal margin. In other words, the Figure similarly illustrates that the coronal margin is where teeth meets the ideal gingiva and the apical margin is where the actual gingiva meets the crown of a tooth.

[0021] The soft tissue recession measure may in an embodiment be determined by determining corresponding points for each tooth in the intraoral scan data using the output from the trained machine learning model or using a two-scan solution. That is, on one embodiment, the method comprises determining a middle point of one of the coronal or apical margin for each tooth in the intra-oral scan data, determining a direction of the tooth and determining a center of the tooth, wherein these determinations are used to generating a first plane using the determined middle point, direction, and center. In this way a first reference plane may be generated to assist in determining the soft tissue recession measure as will be explained in the following. From the first plane, the method comprises generation by a processor at least one second plane parallel to the first plane, wherein the at least one second plane is positioned at x-distance from the first plane. In this way it is possible to identify for the first and second plane a point on the coronal margin and apical margin at which the first and second plane intersects. Accordingly, the method comprises determining from the first and second planes intersecting points on the coronal and apical margins. The determined points on the coronal and apical margin intersection with the plane may define the points for measuring a distance defining a soft tissue recession. Accordingly, if applying a multiple of planes by updating the second plane position x-distance from the first or second plane generating a new second plane position a multiple of points may be generated on the coronal and apical margin line defining the points for which a recession measure is calculated. Accordingly, identifying a point on the apical and coronal margin, at which the new second plane position intersects the distance between the identified points may be calculated. Repeating the steps with a plurality of generated planes along the coronal and apical margin for all teeth represented in the intra-oral scan data, recession measure may be calculated across a plurality of teeth in the intra-oral scan data.

[0022] In a multi-scan solution (to be explained in more detail), the determination of the soft tissue recession measure may in accordance with the previous described example utilize a determined gingival margin instead of either the coronal or apical margin as just described to generate the planes and identify intersecting points. The point-to-point correspondence utilizing planes provides for at least two points for calculating soft tissue recession across a tooth in the intra-oral scan data. To ensure that a number of points are represented for the point-to-point correspondence, the method comprises updating the second plane position with x-distance from the first or second plane generating a new second plane position, where new points may be identified. In this way a plurality of point-to-point correspondence between the gingival margin and the e.g., the coronal margin may be found. The soft tissue recession measure may then be determined by calculating a distance between the identified points on the coronal margin and the determined gingiva margin and repeating the steps for each of the plurality of teeth in the 3D intraoral scan data. With these steps according to the method described herein, it is possible to determine accurately a soft tissue recession measure along the margin line of a tooth and along the plurality of teeth in the intra-oral scan data. The point-correspondence determination described provides an accurate and clinically valuable measurement for the determination of soft tissue recession measure, which may be presented to the dental practitioner in a software application through the previously mentioned representation of the soft tissue recession measure on the or in connection with a 3D model generated from the intra-oral scan data.

[0023] The distance x at which the second parallel plane is generated and subsequently moved may depend on the number of points which is used represent the soft tissue recession measure. This number may be pre-defined to be 1 to 100, 10 to 50, 10 to 40 or less or more. The numbers define the number of points used to determine a measure for recession in the data. Alternatively, this number may be adjusted or manually set by a dental practitioner depending on how accurate the spacing between the point-correspondences is considered to be relevant.

[0024] In an embodiment, the parallel planes are generated and moved based on based on points. In an example, the margin line may be divided into 10 equally sized parts, which generates 10 points on which the planes are generated. It is important to note that the 10 points need not be equally separated from each other.

[0025] For any of the previous described methods for determining soft tissue recession, it is common that no manual assessment is needed, and that the margin boarders (i.e., the coronal and apical boarders or the determined gingival margin) are automatically identified using e.g., a trained machine learning model or other computer processing methods.

[0026] Furthermore, common to the embodiment described herein, is that the determined soft tissue recession measure may be represented as a projection of the soft tissue recession measure onto the 3D model, wherein the 3D model is generated from the intraoral scan data. In this way, the method provides for an accurate visualization of the calculated soft tissue recession measure allowing a dental practitioner to assess the automatically calculated soft tissue recession measure without requiring any manual interactions necessarily and to communicate to a patient the findings.

[0027] In an embodiment, also common to the embodiments described herein, the representation of the determined soft tissue recession measure is configured as a representation in a window of the graphical user interface, which window is different from where the 3D model, generated from the intraoral scan data, is presented. Accordingly, the representation may for example be an overall recession score or number defining the amount of soft tissue recession in the oral cavity of the patient. This overall recession score may be represented in a part of the graphical user interface different from a representation of the intra-oral scan data.

[0028] In an embodiment, the soft tissue recession measure may be represented in a separate window than the 3D model in a graphical user interface as images of a single tooth, activated when a dental practitioner hoovers over a tooth on the 3D model or alternatively selects a specific tooth. This allows single tooth assessment, where each tooth, when assessed may be visualized by the processor on the screen of the user interface. This may allow the dentist to see for example the raw images of that specific tooth, while at the same time assessing the recession measure provided automatically by the method(s) described herein. In this way, the dental practitioner with the method described may be provided with an automatic determination of a soft tissue recession measure per tooth and at the same time be presented with the actual raw image data as obtained by the 3D intra-oral scanner to evaluate the soft tissue recession visually. Such evaluation could be comprised of the possibilities in the user interface to modify, accept and or add to the findings of the automated generated results.

[0029] In an embodiment, the method comprises displaying the apical margin and the coronal margin of the health state prediction superimposed onto the 3D representation of the intra- oral scan data in the graphical user interface. In this way it may be possible for the user, such as a dentist or dental practitioner, to evaluate visually the output from the trained machine learning model and asses the quality thereof considering the actual intra-oral data loaded into the piece of software running the computer-implemented method(s) described herein. Such visualization of the findings from the trained machine learning model may also provide an effective communication tool allowing the dental practitioner to visually explain to the patient the health state of the soft tissue.

[0030] To provide an even clearer visual aid, the method comprises assigning a coloring or a line pattern to the apical margin, wherein the coloring defines the distance between the most apical point of the apical margin and the most coronal point of the coronal margin for each dental feature falling within one or more set threshold(s). The coloring may assist in showing a quantitative measure of the health state of the soft tissue to e.g., a patient and / or to assess by the dental practitioner him / her-self. In an embodiment, the method comprises displaying in the graphical user interface a continuous color map as a part of the graphical user interface and projecting the apical margin and the coronal margin onto the 3D representation of the intra-oral scan data, wherein the continuous color map comprises a slider, which upon activation by a user, causes the processor to adjust the threshold defining the amount of soft tissue recession to visualize. In this way the method allows for an easy adjustment of the threshold of soft tissue recession to be visualized on the 3D model of the intra-oral scan data.

[0031] In an embodiment, the method comprises projecting the apical margin and the coronal margin onto the 3D representation of the intra-oral scan data, and determining, by the processor, a maximum distance between the apical margin and the coronal margin for each tooth on the 3D model. The maximum distance calculated for a local area of each tooth may be displayed with the 3D model. In this way only a single number for soft tissue recession for each tooth may be represented on the 3D model of the intra-oral scan data. This may allow a simple and fast overview of soft tissue recession.

[0032] In an embodiment, the trained machine learning model is configured to output a confidence label vector for each of the dental features, wherein a confidence label falling under a first class represents a high degree of confidence that the health state predictions is accurately detected by the trained machine learning model, and wherein a confidence label falling under a second class represents a low degree of certainty that the healthy state predictions is accurately detected by the trained machine learning model, wherein the processor is configured to utilizing the confidence label to output a recession confidence measure for each dental feature of the intraoral scan data. Providing a confidence label as part of the output of the trained machine learning model ensures that the build machine learning models provides an automatic assessment of the quality of the output based on the learning process that the machine learning model has gone through. In this way the user, such as a dentist, can get an overview of the performance of the machine learning model to ensure accurate soft tissue recession measures. The confidence level is configured to describe the certainty that the machine learning model is on having correctly predicted the health state of the soft tissue. The confidence level could be represented on the mouth, jaw and / or tooth level and the confidence level could change depending on what view the dental practitioner selects. In an embodiment, the computer-implemented method comprises calculating a weighted severity score by utilizing the recession confidence measure.

[0033] The method(s) described herein may also be used to determine a soft tissue recession measure based on two consecutive scans taken at different points in time. The determination of the soft tissue recession measure may in a multi-scan embodiment be determined by using the health state prediction vector and the associated health state labels, as stored from a first scan taken at a past point in time, together with a determined gingival margin from a second scan. Accordingly, at a subsequent dental visit, different from a first dental visit where a single-scan recession has been evaluated using only the trained machine learning model, a new intra-oral scan may be taken and the gingival margin for the subsequent visit may be determined. To assess the development of soft tissue recession, at least the health state prediction describing the coronal margin from the first scan (stored in the memory) may be used together with the determined gingival margin as will be described in the following.

[0034] To allow for a multi-scan soft tissue recession evaluation, the method may comprise to determining from the 3D intra-oral scan data a gingival margin and to load from a storage media the past acquired health state prediction vector and / or associated health state labels. The health state label or vector information in general may be processed to generate the coronal margin representing in one embodiment the CEJ. The determination of the gingival margin ensures that the actual current border between the teeth and the gingiva in the 3D intra-oral scan data is found. This to be able to compare the actual gingival margin border, reflecting the current state of the intra-oral cavity, with the past acquired estimation of a soft tissue recession using e.g., the previously described trained machine learning model method. In this way it may be possible to assess if the current state is healthy or sick, i.e., if soft tissue recession has evolved since a past point in time, where a first scan (and only single scan) was acquired.

[0035] In accordance herewith, the method may furthermore comprise determining a soft tissue recession measure by calculating a difference between the loaded past health state labels (such as the coral and / or apical margin) and the determined gingival margin. In this way a difference measure between the past health state labels defining the past soft tissue recession for each tooth, and the current state of the soft tissue (i.e., the determined gingival margin) is provided for, where the difference may reflect soft tissue recession progressing. In this way it may be possible to determine from 3D intra-oral scan data loaded into a processor configured to perform the method(s) described herein, a change in soft tissue from an initial single scan assessment at a first point in time to a second visit, the progression of sickness of the soft tissue in early stages. In other words, by providing a trained machine learning model described herein to generate a single-scan measure for soft tissue recession at a first point in time, and using a second later point in time scan it is possible to evaluate the progression of soft tissue recession over time. This allows not only for an early detection of soft tissue recession, but also for a more precise, fast, and objective measure as no manual assessment needs to be performed by the dental practitioner.

[0036] The difference calculations, between a past and a present time and / or from a single scan, may with the method(s) disclosed herein objectively be based on the output from the machine learning model, reflecting the estimated past health state and the current situation reflected by the determined gingival margin.

[0037] The health state predictions as output from the trained machine learning model, may as previously described be used to generate health state labels for each of the dental features (i.e., teeth and gingiva) to represent the prediction in the intra-oral scan data. The health state labels may be represented as pointsets representing soft tissues recession points of the teeth in the intra-oral scan data acquired at a first point in time. These health state label pointset may in a multi-scan situation (past and current time scan) be loaded into a processor together within tooth identification information to be able to compare the past evaluation of soft tissue recession with a current. The health state labels may comprise information regarding a position of at least the coronal margin of the previously acquired scan data. In this way for each tooth, it is possible to identify not only the presence but also the actual position of the coronal margin in the 3D intra-oral scan data.

[0038] In an embodiment, the determined gingival margin may comprise a plurality of gingival margin pointsets and a tooth identification for each of the plurality of gingival margin pointset, wherein the plurality of gingival margin pointsets represents the positions of the determined gingival margin for each tooth in the 3D intraoral scan data (e.g., taken a second later point in time). The two measures, i.e., the health state label pointset and the gingival margin pointsets may then be compared to assess the soft tissue recession measure in a situation where past scan data is present and current scan data is present. Providing the possibility of a tooth-to-tooth time comparison of the health state labels from a past scan data and the determined gingival margin of a current scan data ensures that a soft tissue recession measure may be given for not only the entire teeth set (i.e., all teeth in the oral cavity) but also allows for single teeth assessment progression over time. This ensures that accurate soft tissue recession measures of progression over time may be provided. Furthermore, it assists in ensuring that the dental practitioner gets the full picture of the state of each tooth rather than providing one single soft tissue recession measure for all teeth in the oral cavity.

[0039] A comparison between the past health state labels (e.g., in the form of pointsets of e.g., the coronal and / or apical margin) may comprise calculating the soft tissue recession measure by determining a distance between the health state label pointsets and the determined gingival margin. The calculated soft tissue recession measure may be displayed in the graphical user interface as a representation of the calculated soft tissue recession measure.

[0040] An example of calculating the soft tissue recession measure using a plane generation has previously been described, and the method applies also to the just described situation where data from a past and current scan is used. Accordingly, for details reference is made to previous sections of the disclosure describing different methods of calculating the soft tissue recession.

[0041] Accounting for all the embodiments described herein, the method may comprise assessing soft tissue recession on both sides of a tooth in the intra-oral scan data, where each tooth comprises a buccal, lingual, and interdental side(s).

[0042] Accordingly, to assess soft tissue recession on both sides of a tooth in the intra-oral scan data, the determined gingival margin described in embodiments herein, may comprise a determined buccal and lingual margin line. Correspondingly, the health state label may comprise pointsets for a buccal and lingual health state label for each tooth in the intraoral scan data. In this way it is possible to assess on a tooth-by-tooth basis and a side-by-side (i.e., buccal, and lingual sides) basis the soft tissue recession for all teeth in the oral cavity. This ensures that accurate measures of soft tissue recession on all sides of the teeth are accurately acquired and allows for an overall assessment of the soft tissue recession of the oral cavity defined by the intra-oral scan data input into the processor.

[0043] In one embodiment, the trained machine learning model is configured to output a health state prediction structure comprising a position label for each for the descriptive representations, wherein each of at least one of the position labels represents at least one of a buccal, lingual, or inter-dental side identification of the descriptive representations. Such output from the machine learning model allows the computer to assess from which side of the tooth potential soft tissue recession (i.e., the health state labels as out from the machine learning model) is configured to belong to.

[0044] To determine the gingival margin, as described in an embodiment, of the 3D intra-oral scan data, the processor may be configured to segmenting the intra-oral data into gingiva and teeth, wherein the segmentation represents the intraoral data as a plurality of connected teeth points assigned a set of tooth values, and a plurality of connected gingiva points assigned a set of gingiva values. In other words, the intra-oral data may be segmented into a plurality of facets (i.e., connected teeth points) assigned a tooth value and a plurality of facets (i.e., connected gingiva points) assigned a gingiva value. This representation may be preferred as the intra-oral scan data may be represented as a mesh. Other possibilities could include a point-cloud representation of the intra-oral scan data, where each point representing tooth may be assigned a tooth value and each point representing gingiva may be assigned a gingiva value.

[0045] To determine the gingival margin from the plurality of connected points of the segmentation, the processor may be configured to determine if a point in the plurality of connected teeth points and gingiva points comprises at least one point from the set of tooth values and gingiva values and assigning a margin label to the respective points, thereby identifying the points in the intra-oral scan data forming the margin of the gingival margin line. Where each of the teeth in the oral cavity is neighboring each other, a junction may be identified by the processor by determining if a point in the plurality of connected teeth points and gingiva points comprises at least one point from the set of gingiva values and at least one point from at least two different sets of tooth values and assigning a junction label to the respective points. With this method it is possible to determine the gingival margin line for each tooth and the connection points with neighboring teeth in the 3D intra-oral scan data.

[0046] In an example, the processor may be configured to identify for each tooth in the intraoral scan data, one or more margin labels for the buccal and / or lingual side of the tooth. In this way, the margin labels may be marked as a buccal and / or lingual margin label ensuring that both sides of the tooth are covered and / or for presentation purposes.

[0047] Furthermore, in an example, the processor may be configured to identify for each tooth in the intraoral scan data, one or more junction labels for the buccal and / or lingual side of the tooth. Also, the junction labels may be marked for buccal and / or lingual side to ensure both sides are covered and / or for presentation purposes.

[0048] As the input data to the processor, being the intra-oral scan data during processing as previously described may appear in random order, the junction and margin labels may correspondingly be randomly organized. Accordingly, to sort the randomly ordered data points (in the form of margin and junction label) to accurately determine the gingival margin line in relation to the input data, the processor may be configured to sorting the margin and junction labels into a structured order by utilizing a random determined junction label from the junction labels as a starting point. Based on the determined starting point traversing each margin label to determine an edge orientation of the each of the margin labels, wherein the determined edge orientation of the margin label is used to determine the subsequent margin label in the collection of margin labels.

[0049] The sorted margin and junction labels may in an example be configured to represent a collection of margin line splines for each of the teeth in the intraoral scan data. By having a representation of margin line splines, the processor may be configured to project the margin line spline for each of the teeth into the display of the graphical user interface to provide a visualization of the position of the determined gingival margin line.

[0050] In an example according to the methods described herein, the processor may be configured to compare the determined gingival margin line and e.g., the coronal margin output from the trained machine learning model through the health state labels using the representation of the previously described margin line splines and the coronal margin for each tooth. Accordingly, in an example according to the method, the processor is configured to acquiring the coronal margin and acquiring the determined gingival margin, wherein at least the determined gingival margin is represented as a collection of margin line splines. To determine the soft tissue recession measure, the processor may be configured to determining corresponding points between the estimated coronal margin and the determined gingival margin and calculating a distance between the identified corresponding points as previously described.

[0051] In case two scans, acquired at two different times (i.e., in a multi-scan embodiment) are used to determine the development and / or progression of soft tissue recession, the representation of the soft tissue recession in a user interface may comprise the step of snapping the past determined (i.e. at the first scan) soft tissue recession measure, gingival margin and / or coronal margin to the current (second visit) scan data. This aids in visualizing the progression of soft tissue recession to the dental practitioner and patient. If not being snapped to the current (second visit) scan data, or at least to the generated 3D model thereof, the risk of the determined gingival margin and / or coronal margin as stored in the system at an earlier visit not showing is present. Accordingly, in an embodiment, the processor may be configured to loading one of the coronal margin and / or determined gingival margin acquired from a first or second dental visit and calculating a transformation between the determined gingival margin or the coronal margin and the 3D intraoral scan data acquired at the first or second clinical visit. When the transformation has been calculated by the processor, the transformation is automatically applied to the coronal margin and / or determined gingival margin from the first or second clinical scan, to snap the coronal margin and / or the previously determined gingival margin to the 3D model of the intraoral scan data acquired at the second or first visit. The processor may then be configured to displaying the 3D model of the clinical visit together with the transformed coronal margin and / or the previously determined gingival margin, and the second clinical visit determined gingival margin.

[0052] The method(s) described herein may be configured to provide a classification of the soft tissue recession measure into one or more classes, wherein a class may represent a severity of the determined soft tissue recession measure for each tooth. Accordingly, the processor may in the method described herein be configured to classifying the soft tissue recession measure into two or more severity classes, wherein the two or more severity classes are found based on two or more pre-defined thresholds, wherein the two or more pre-defined thresholds are evaluated against the distance comparison between the coronal margin and the determined gingival margin. It should be noted that the coronal margin may be the preferred reference point when performing a multi-scan comparison to evaluate soft tissue recession. However, the apical margin determined from the output of the machine learning model may similarly be used.

[0053] The two or more pre-defined thresholds may describe the severity of soft tissue recession for a single tooth in the oral cavity and may be comprised of a first class representing a low soft tissue recession severity, a second class representing a medium soft tissue recession severity and a third class representing a high soft tissue recession severity. Each of the classes may be represented by a range of values, where the first class representing low severity may be determined if the soft tissue recession measure falls within the range of 0.5 to 2 mm. The second class representing medium severity may be determined if the soft tissue recession measure falls within the range of 2 to 4 mm, and the third class representing high severity may be determined if the soft tissue recession measure falls within the range of more than 4 mm.

[0054] One type of recession may be a Type 1 (RT1) which may be defined by no loss of interproximal attachment. Interproximal CEJ is clinically not detectable at both mesial and distal aspects of the tooth.

[0055] Another type of recession may be a Type 2 (RT2), which may be defined by loss of interproximal attachment. The amount of interproximal attachment loss (measured from the interproximal CEJ to the depth of the interproximal sulcus / pocket) is less than or equal to the buccal attachment loss (measured from the buccal CEJ to the apical end of the buccal sulcus / pocket).

[0056] Yet another type of recession may be a Type 3 (RT3), which may be defined by loss of interproximal attachment. The amount of interproximal attachment loss (measured from the interproximal CEJ to the apical end of the sulcus / pocket) is greater than the buccal attachment loss (measured from the buccal CEJ to the apical end of the buccal sulcus / pocket).

[0057] Accordingly, the processor, according to the method, may be configured to take as input the soft tissue recession measure calculated for each tooth and evaluating the soft tissue recession measure against one or more of the pre-set threshold classes. The threshold classes may be pre-defined in the software or may be set by the dental practitioner. In an example, the pre-set thresholds already stored in the software may, if needed, be adjusted by the dental practitioner. The pre-set thresholds for the low, medium, and high class, may be based on the annotation data, as will be described, as these data, used to train the machine learning model may be based on an extensive amount of data, which has been evaluated manually for training purposes, by clinical experts.

[0058] In an example, the classification may also be based on the retrieval of clinical measurements from a storage medium. The clinical measurements may be understood as clinical measurements which are not directly obtained from the intra-oral scan data but could be X- ray, CBCT, bitewing etc. data and or clinical data obtained previously from e.g., visual, and tactile examination methods that are stored in a storage medium, such as e.g., a dental chart or in connection with a previously obtained 3D intra-oral scan. The clinical measurements may be provided as output from a trained machine learning model having been trained on e.g., CBCT, X-ray, bitewing etc. Accordingly, in an example, the processor may be configured to classifying the soft tissue recession measure by retrieving a clinical measurement (i.e., a measurement stored in a storage medium and acquired directly from the patient by e.g., probing - example: clinical attachment loss) from the storage medium and retrieving the calculated soft tissue recession measure. Using the retrieved clinical measurement and the calculated soft tissue recession measure, the processor may be configured to classify the soft tissue recession measure into one or more classes. Using the clinical measurements in combination with the calculated soft tissue recession measure it may be possible to provide a more accurate and precise single-scan or multi-scan assessment of for example the pocket depth development and the clinical attachment loss associated with a tooth. These measurements are normally not assessable using intra-oral scan data and may only be seen in e.g., radiation images. However, combining the measure of the soft tissue recession measure found with the method described herein and previous clinical findings stored by a dental practitioner in a dental application software, it may be possible to combine the information and get an even more accurate, fast, and reliable estimate of the severity of the seen soft tissue recession measure.

[0059] In yet an example, a classification of the soft tissue recession measure may also be based on a shape classification (in addition or as a stand-alone to the above-described classifications). A shape classification as described herein may provide an automated detection of a local gingival retraction, which may be associated with different causes of recession.

[0060] For example, a V-shaped local retraction (i.e., soft tissue recession) may be associated with teeth subjected to occlusal trauma, especially in patients with bruxism and clenching habits. V-shaped recession is known as " Stillman's cleft .", and in cases with V-shaped recession it is common to find abfraction; while on the occlusal surface, wear facets caused by attrition are commonly found, as being part of a lesion caused by occlusal trauma. Accordingly, an automatic classification into a V-shaped recession may provide a source of association which with the methods proposed is identified automatically.

[0061] Another type of shape-classification may be a U-shaped retraction (i.e. soft tissue recession), which may be associated with chronic inflammatory periodontal disease, inadequate tooth brushing or inadequate frenulum insertion. Accordingly, an automatic classification into a V-shaped recession may provide a source of association which with the methods proposed is identified automatically.

[0062] In one example of shape classification, the processor may be configured to perform a shape classification of the soft tissue recession measure, wherein the shape classification comprises classifying at least a part of the soft tissue recession measure into a least one of a first or second shape of soft tissue recession measure, wherein the first and second shape of the soft tissue recession measure is classified based on a pre-determined set of rules.

[0063] In an example, the processor may be configured to classify the first shape and second shape by determining from the coronal margin and the determined gingival margin a curvature of the soft tissue recession measure. The curvature may be determined by approximating a behavior of the margin lines (i.e., the coronal, apical and / or gingival margin line) by determining an angle change of the normals along the curvature of the soft tissue recession measure, wherein an angle change below a set threshold represents a substantially flat curvature behavior and an angle change above the set threshold represent a substantially steep curvature. Alternatively, the curvature may be determined by approximating the behavior of the curvature using a circular representation along the curvature of the soft tissue recession measure, wherein a radius of the circular representation below a set threshold represents a large curvature, and wherein a radius of the circular representation above a set threshold represents a small curvature.

[0064] In any case, the approximated behavior determined may be processed in the processor in order to translating the approximated behavior into a first shape and a second shape by applying the determined set of rules.

[0065] The determined set of rules may be defining a U-shaped behavior as the first shape or a V- shaped behavior as the second shape of the curvature.

[0066] To be able to predict a health state of the soft tissue in intra-oral scan data, the method utilizes, as already described, a trained machine learning model, wherein the trained machine learning model may be trained based on a training dataset. The trained machine learning model may be developed based on a training process utilizing a computer- implemented method comprising loading from a storage media a training dataset of intra- oral scan data comprising a plurality of data samples. Each of the plurality of data samples may comprise a 3D model comprising one or more dental features having associated dental feature labels. The dental features representing teeth and gingiva in the data samples and wherein the associated dental features labels represent annotations of the data samples identifying the health state of the soft tissue.

[0067] From the loaded data samples, the method comprises generating a first input structure by performing a data transformation of the plurality of data samples, wherein the data transformation generates a plurality of descriptive representations of each of the dental features of the plurality of data samples. The descriptive representations may be configured as any suitable data format fitting a chosen neural network architecture which will be explained.

[0068] The method furthermore comprises assigning at least one descriptive label to the plurality of descriptive representations, wherein the at least one descriptive label is acquired from the associated dental feature labels. In this way each of the descriptive representations is provided with a label describing from the annotations, the health state of each of the descriptive representations. This ensures that each data point in the training data set is provided with a known ground truth which allows the neural network to learn and optimize the health state predictions.

[0069] To improve even further the performance of the machine learning model, the method may comprise generating a second input structure, wherein the second input structure comprises a feature vector representing an anatomical descriptor of each of the one or more dental features. The anatomical descriptor may comprise data information regarding tooth type (i.e., molar, incisor, canine etc.), semantic information (curvature, color etc.) which may aid the machine learning model in providing better predictions.

[0070] Upon having generated both input structures, the method comprises feeding the first and second input structures to a machine learning model, wherein the machine learning model is configured to process each of the first and second input structures to generate a global structure comprising a processed representation of the first and second input structures, wherein the gio bal structure is configured to be processed by the machine learning model to generate a health state prediction structure for each of the plurality of descriptive representations. This enables the model to predict accurately a health state of each of the descriptive representations as input to the machine learning model.

[0071] In an embodiment, the first input structure is configured to be fed to a first neural network architecture of the machine learning model configured to generate a first prediction structure. The first neural network architecture may be suitable for processing the descriptive representations as generated from the data transformations.

[0072] In an embodiment, the second input structure is configured to be fed to a second neural network architecture of the machine learning model configured to generate a second prediction structure. The second neural network architecture may be configured to process the additional information, such as semantic information, tooth kind etc. as an additional input to the machine learning model. Utilizing two different architectures ensures that different types of data information present in the input data can be processed individually to allow the machine learning model to optimize the probability of predicting accurately the health state of soft tissue.

[0073] In accordance herewith, in an embodiment the global structure comprising the first prediction vector and the second prediction vector.

[0074] As previously mentioned, the machine learning model is configured to output a health state prediction structure which may comprise a plurality of numerical values for each of the descriptive representations, wherein each of the plurality of numerical values represents two or more health state labels, wherein at least one label represents a healthy state, at least one label represents an initial state and / or at least one label represents an advanced state of soft tissue recession.

[0075] To asses which side of the tooth the health prediction structure has provided predictions for, the health state prediction structure may comprise a position label for each for the descriptive representations, wherein at least of part of each of the position labels represents at least one of a buccal, lingual, or inter-dental side identification of the descriptive representations.

[0076] As an additional input to the machine learning model, the method may comprise calculating a curvature representation of each of the descriptive representations and feeding the curvature representation into the machine learning model. The curvature representation may assist the machine learning model in providing better predictions as the soft tissue health may in 3D data be (and for that matter in real world) be reflected in an increased curve and / or rise in tissue at the transition between a recessed gingiva and the exposed part of the tooth.

[0077] To provide a better performance and accuracy estimate of the machine learning model, the training of the machine learning model furthermore may comprise providing a confidence label to the training data set. That is, the annotated data may comprise a label representing the confidence of the annotated descriptive representation being a e.g., a healthy or non- healthy soft tissue. Accordingly, the dental feature labels may comprise a confidence label. The method may therefore comprise generating a confidence label vector representing for each of the descriptive representations a confidence that the descriptive representation is associated with at least one of the two or more health state labels, wherein the values used to generate the confidence vector is acquired from the dental feature labels. The confidence label vector may be fed into the machine learning model as an additional input.

[0078] In an embodiment, the confidence label may be given as an annotation input, where each point is provided with a confidence label describing how certain the annotator is that the point is associated with gingival recession. In case a solving a segmentation task the addition of a confidence label to the dataset would generate four classes comprises tooth, ideal gingiva, actual gingiva and confidence label.

[0079] In accordance with above-described embodiments of the method of training the machine learning model, the associated dental feature labels may comprise annotations associated with each dental feature and loaded from the data storage, wherein the annotations represent at least one or more of the health state labels, the position label and / or the confidence label.

[0080] The position label which may describe a buccal, lingual and / or occlusal side may not be needed, as the position may be determined in a processing step comprising determining a side of a tooth by determining a tooth pose and from the tooth pose determination estimating a side (i.e. position) of a tooth, wherein the side may be a buccal, lingual or occlusal side.

[0081] When having trained the machine learning model on the basis of the previous described data, the method may comprise applying the machine learning model to a target intra-oral scan data sample to compute a health state prediction structure for the target intra-oral scan data sample, wherein the computed health state prediction structure comprises a plurality of a numerical values representing at least one or more of a health state label, a position label and / or confidence label for each of the descriptive representations in the target intra-oral scan data sample. From this health state prediction structure, suitable data for generating the previously described apical margin, coronal margin, confidence values, positions determinations etc. may be used to determine and represent in a single or multi-scan recession solution, a soft tissue recession in an intra-oral scan data not previously seen by the trained machine learning model. The training process of the machine learning model as just described may be performed by a system for training a machine learning model for predicting a health state of soft tissue in intra-oral scan data, wherein the system may comprise a storage media configured to store a training dataset of intra-oral scan data comprising a plurality of data samples. A processor of the system may be configured to selecting a plurality of data samples from the storage media, wherein each data sample comprises a 3D model comprising one or more dental features and associated dental feature labels. The processor may furthermore be configured to generating a first input structure by performing a data transformation of the plurality of data samples, wherein the data transformation generates a plurality of descriptive representations of each of the dental features of the plurality of data sample and assigning at least one descriptive label to the plurality of descriptive representations, wherein the at least one descriptive label is generated from the associated dental feature labels. In alignment with the previous described training process, the processor may be configured to generating a second input structure, wherein the second input structure comprises a feature vector representing an anatomical descriptor of each of the one or more dental features and feeding the first and second input structures to a machine learning model, wherein the machine learning model is configured to process each of the first and second input structures to generate a global structure comprising a processed representation of the first and second input structures, wherein the global structure is configured to be processed by the machine learning model to generate a dental soft tissue state prediction structure for each of the plurality of descriptive representations.

[0082] In general, the processor may be configured to perform any of the following processes:

[0083] - feeding the first input structure to a first neural network architecture of the machine learning model configured to generate a first prediction structure.

[0084] - feeding the second input structure to a second neural network architecture of the machine learning model configured to generate a second prediction structure.

[0085] - generate the global structure comprising the first prediction structure and the second prediction structure.

[0086] - processing of the first input structure by utilizing a first type architecture and processing the second input structure utilizing a second type architecture of the machine learning model. - calculating a curvature representation of each of the descriptive representations and feeding the curvature representations into the machine learning model.

[0087] - generating a confidence label vector representing for each of the descriptive representations, a confidence that the descriptive representation is associated with at least one of two or more health state labels, wherein the values used to generate the confidence vector is acquired from the dental feature and feeding the confidence label for each descriptive representation into the machine learning model.

[0088] As previously described, the methods for training a machine learning model to determine a health state prediction for intra-oral scan data may involve a data transformation to the input data allowing the machine learning model to process the data efficiently. Similarly, the input data to an already trained machine learning model, also described herein may undergo a corresponding data transformation allowing the trained model to process the data. Accordingly, in the following different examples of data processing allowing the generation of descriptive representation as defined herein will be described.

[0089] The input data may, as previously described, comprise intra-oral scan data. The intra-oral scan data may be used to generate a digital 3D model, which can be received or accessed by the processor.

[0090] The digital 3D model may be displayed on a display screen in form of a 3D mesh, representing surfaces of teeth and connected gingival tissue of the dental situation. The 3D mesh may be comprised of individual facets, for example triangular facets. Alternatively, the digital 3D model may be displayed as a point cloud comprising points, a graph comprising nodes and edges, a volumetric representation comprising voxels, or any other suitable 3D representation form. Accordingly, the 3D intra-oral scan data may be provided in the format of a mesh structure, point cloud structure, graph and node structure, volumetric representation, or any other suitable format.

[0091] To allow the training off or the already trained machine learning model to process any of these data formats, the data may be processed to undergo a data transformation prior to input to the machine learning model. The method described herein may comprise providing at least a part of the digital 3D model or the intra-oral scan data as the input to the machine learning model. In an embodiment only a part of the digital 3D model is input to the machine learning model, preferably after data transformation. The part of the 3D model input to the neural network may for the purpose of predicting a presence of a coronal margin (forming part of the health state prediction) comprise at least one tooth surface acquired from a buccal and / or lingual site, or at least one tooth surface together with the connected soft tissue of that tooth surface, preferably of a buccal and / or lingual site of a tooth. For the machine learning model to predict the health state of soft tissue, it is preferred that the data input to the machine learning model comprises both tooth and soft tissue representation in the data, as the clinical evaluation of soft tissue is focused on the area of a tooth situation covering substantially the transition from tooth to gingiva as previously explained. Having said that, also a single tooth data without connected soft tissue represented in the data may be presented to the machine learning model, which may be trained to detect both situations, i.e., tooth without connected soft tissue and tooth with connected soft tissue. It should be noted that the 3D model input to the neural network may comprise a representation of all surfaces of the tooth, such as the buccal, lingual and occlusal surface of the tooth, whereby the neural network predicts health states for each site of the entire tooth.

[0092] In an alternative, the entire 3D model may be used as input to the machine learning model preferably after data transformation. In this case, all the teeth and connected soft tissue are used as input to the machine learning model. In an example it may be that the entire set of teeth without connected soft tissue is presented as input to the machine learning model.

[0093] In an embodiment, each point cloud input to the 3D model may comprises as previously explained both tooth and gingiva parts, wherein the gingiva part forming part of the input is extended to provide more curvature content to the machine learning model. By using an extended gingiva for each input to the machine learning model, the learning process is optimized, as the machine learning model in this from for example curvature behavior associated with gingival recession learns that at a distance from the tooth to gingiva, an increasing gingiva reflects a transition between unhealthy and healthy gingiva. That is, a gingival recession behavior show a lack of gingiva close to a tooth when recession has arisen, and with distance from the tooth the gingiva increases in thickness causing a decreasing to increasing curvature behavior across a distance from tooth to actual gingiva. Therefore, by processing the data to ensure that an extended part of the gingiva forms part of the dataset input to the neural network, an optimized recognition of gingival recession is achieved. As previously mentioned, this increase and decrease of gingival behavior may be determined from the curvature information provided to the neural network.

[0094] In any of the above mentioned data inputs, i.e. the entire model of a plurality of teeth with or without connected soft tissue, the data transformation which will be described in the following may allow the machine learning model to process the digital 3D model input directly in its three-dimensional format or to e.g. processing a two-dimensional representation of the at least part of the digital 3D model, such as a 2D image or a plurality of 2D images.

[0095] The intra-oral scan data may in addition to the above-mentioned formats comprises data information consisting of white light data, fluorescence data and / or infrared data, where the data information may be collected directly from the intra-oral scan data in connection with the three-dimensional or two-dimensional structures contained therein.

[0096] Returning in more detail to the details about the data transformation, the method may comprise generating the descriptive representations by converting the intra-oral scan data, into a point cloud and / or mesh. The data transformation may generally be understood as converting the three-dimensional structures of the intra-oral scan data into a numerical representation (also denoted descriptive representation), before being input to the trained neural network. Numerical representation allows for efficient processing by the neural network of the data structures.

[0097] In an embodiment, the input may be in 3D format. An example of the input in 3D format may be a point cloud, the 3D mesh, the volumetric representation, or a graph representation of the digital 3D model.

[0098] The 3D model may comprise a 3D mesh format, wherein the data transformation comprises transforming the 3D mesh representation into a point cloud representation. Conversion of the 3D model representation into a point cloud representation may comprise creating a point cloud by classifying the loaded 3D model into teeth and gingiva, using e.g., facet classification techniques, converting each tooth from a mesh structure to a point cloud by sampling the mesh format with a uniform distribution, such as using a poission disk sampling of the data points in the mesh data. This ensures that a uniform distribution of the points is chosen allowing a reduction of data points in the dataset to be processed by the machine learning model. A chosen sampling with a uniform distribution ensures that the generated point cloud is not over-represented in some areas and under-represented in others.

[0099] The mentioned facet classification technique should be understood as a segmentation or classification of facets of the 3D model into teeth and gingiva allowing a clear distinguishing of teeth and gingiva in the 3D model.

[0100] Alternative sampling methods could be a random generation of data points from the mesh data, or the chosen sampling method could be to generate the points based on the area of facets represented in the mesh data. All above solutions would have been considered as a suitable option; however, the inventors have realized that a uniform sampling provides the preferred sampling method as the point cloud generated with the uniform sampling more precisely reflects the data structure of the original 3D model input.

[0101] Accordingly, providing the at least part of the digital 3D model as the input may comprise sampling the at least part of the digital 3D model or converting the at least part of the digital 3D model into a numerical form. For example, one or more points from the point cloud or mesh representation of the digital 3D model may be converted into numerical form, for example a matrix of input values describing a certain set of characteristics. The matrix may then form the input for the machine learning model.

[0102] The data transformation into a point cloud may be done on the entire 3D model at once or alternatively, the data transformation may be done on a single tooth represented in the 3D model at a time. In the latter case of single tooth representation, the data transformation may comprise identifying in the 3D model each single tooth using e.g., a segmentation of the data into tooth and soft tissue, such as the gingiva. A subsequent step may then be to convert each single tooth and preferably the connected gingiva into a point cloud. Accordingly, at least a part of the point cloud converted 3D model, e.g., in the form of one or more single teeth, may be input to the trained neural network, and / or the full converted 3D model may be input to machine learning model.

[0103] The resulting data transformation may comprise a matrix representation describing each point in the point cloud together with a tooth identification. This matrix representation may be input to the machine learning model to further process the matrix representation to generate a prediction of a health state.

[0104] The neural network architecture suitable for processing an input 3D format comprising point clouds may be a trained PointNet network, or a trained PointNet-type network, such as PCT, PointNET(++). These neural network architectures may be capable of processing the digital 3D model, or at least a part of the digital 3D model, in the point cloud format. Accordingly, if the digital 3D model input to the trained neural network is in the 3D mesh form, it may be possible to convert the digital 3D model into the point cloud form, by collapsing all vertices from the 3D mesh format so that only points remain, thus forming the point cloud. The remaining point cloud may then be processed directly by the PointNet neural network, or the PointNet-type neural network. In one embodiment, a pointnet++ part segmentation may be used for the PointNet-type neural network, wherein the pointnet++ part segmentation utilizes teeth type (i.e. type of tooth being molars or non-molars, UNN numbers etc.) as an input.

[0105] An alternative may comprise inputting the mesh structure of the 3D model directly as an input to the neural network, either provided as a full 3D model mesh structure and / or as a single tooth provided, as described above in relation to point clouds.

[0106] In yet a further embodiment, providing the at least part of the digital 3D model as the input for the trained neural network may comprise providing the voxel representation of the at least part of the digital 3D model to the trained neural network directly.

[0107] In another embodiment, the input may be in 2D format such as a 2D image or a plurality of 2D images of the at least one tooth surface and / or the connected soft tissue of the digital 3D model, machine learning model may thus be configured to process 2D input formats. In this case, the data transformation may comprise generating one or more of 2D images from the digital 3D model by a virtual camera, referred to also as the camera. The virtual camera may take snapshots of the individual teeth and / or corresponding connected soft tissue of the digital 3D model. For example, a tooth and / or corresponding connected soft tissue in the digital 3D model may be positioned such that a desired tooth surface of the tooth, potentially including the connected soft tissue is facing the camera. The tooth may be rotated and translated via a local coordinate system such that one or more of the tooth surfaces are photographed. This may be an automatic process where predetermined movements of the tooth have been set up to capture, via the virtual camera, the one or more of the tooth surfaces and potentially the connected soft tissue of each tooth.

[0108] Alternatively, the virtual camera may be moved to a plurality of predetermined positions to capture snapshots of the desired tooth surface and potentially the connected soft tissue. The snapshots of the individual teeth may alternatively be generated by manual movement of the virtual camera to desired positions.

[0109] The plurality of 2D images of the at least one tooth surface and potentially the connected soft tissue of the digital 3D model, besides being generated by taking snapshots of the digital 3D model, may be generated directly by the intraoral scanner recording the dental situation.

[0110] Providing at least a part of the digital 3D model as the input for the neural network may comprise generating one or more 2D images of the at least one tooth surface and potentially the connected soft tissue. The at least one tooth surface and potentially the connected soft tissue in the digital 3D model may have a directly corresponding tooth surface in the actual dental situation. Thus, the process of generating the one or more 2D images of the at least one tooth surface and potentially the connected soft tissue of the digital 3D model with the virtual camera may coincide to taking the corresponding photographs of the at least one tooth surface in the actual dental situation.

[0111] The generated one or more 2D images may form the input for the trained neural network. Alternatively, or additionally, the input may be at least one pixel belonging to the one or more 2D images. The one or more 2D images may be generated by means of the virtual camera. Accordingly, in an example according to the method described herein, the trained machine learning model may be based on a 2D image inputs, wherein the 2D image input is generated from the 3D intra-oral scan data and / or the generated 3D model. In correspondence with the preceding embodiment, the machine learning model may comprise as one of the architectures a convolutional neural network (CNN) suitable for processing 2D images. The CNN may comprise a plurality of convolutional layers capable of capturing low-level features of the input, i.e. obtaining convolved features. Moreover, one or more of pooling layers may be present, responsible for reducing the spatial size of convolved features. Convolutional neural networks may be particularly suitable for processing matrix information such as 2D images. Another example of the trained neural network, particularly suitable for performing classification tasks, is a trained Residual Network (ResNet) or a variation of it. Example of the trained neural network suitable for performing semantic segmentation tasks is a trained U-Net or a variant of it, for example U-Net++. In an alternative embodiment, the trained neural network may use a self-attention-mechanism to process data as in a transformer network.

[0112] As previously described the method may comprise generating a second input structure being a second feature vector. This feature vector may comprise natural color information, geometric information, semantic information and / or tooth kind information which will be described in the following in more detail.

[0113] The 3D model generated from the intra-oral scan data may comprise natural color information and / or geometric information associated with the at least one tooth surface and / or connected soft tissue of the dental situation. The tooth surface may be any one of an occlusal, a labial, a buccal, a mesial, a distal a cervical and / or an incisal surface.

[0114] With regards to natural color information this may be comprised in the scan data obtained via the intraoral scanner used to scan the dental situation.

[0115] Natural color information may relate to surface color of scanned teeth and / or gingiva (also denoted soft tissue throughout) and may be obtained by scanning the intraoral situation of the patient by means of the intraoral scanner. This information may be expressed through Red-Green-Blue (RGB) intensity values. The presence of natural color information may be beneficial as it may assist the neural network in identifying the coronal margin, because recessed tissue may be of different color compared to healthy soft tissue. In an example soft tissue recession may appear in a lighter red soft tissue structure than healthy soft tissue as the gum has shifted towards the root of the tooth, allowing the root of the tooth to appear through the gum. This increased transparency of the red color of the gum may indicate recession evolving. Thus, the natural color information may be of relevance for the trained neural network in a process of predicting soft tissue recession. In more detail, the natural color information may be used by the neural network to identify the coronal and apical margins discussed herein and as a result the soft tissue recession.

[0116] Accordingly, the data transformation described herein may additionally include natural color information suitable for predicting the presence and / or position of a coronal margin in the input data.

[0117] Geometric information, also referred to as topological information, forming part of the intraoral scan data used to generate the 3D model may be depth information relating to a camera position, an angle between a facet normal and a camera direction, the facet normal and / or curvature information. The term “camera” in this context relates to the virtual camera used to view the digital 3D model. The camera may be characterized by a virtual camera property, for example field of view (FOV) and / or focal length. Geometric information examples mentioned here may serve as descriptors of geometrical properties of the input for the trained neural network.

[0118] The depth information may comprise distance information of the at least part of the digital 3D model to the camera. Depth information may be in the form of a depth image. The depth image may display at least a part of the digital 3D model according to its distance from the camera. For example, parts of the digital 3D model more distant to the camera may be shaded darker than the parts closer to the camera. A distance threshold may be defined such that, when a distance between the camera and the part of the digital 3D model is greater than the distance threshold, the part of the digital 3D model is not shown in the depth information. The distance of the at least part of the digital 3D model from a given camera position indirectly describes a topology of the at least part of the digital 3D model, and consequently a smoothness of a surface of the at least part of the digital 3D model. The variations of surface smoothness may represent a relevant parameter in detecting soft tissue recession in the 3D model because soft tissue recession clinically show a smooth, polished surface in case of recession caused by abrasion. Other cases of recession show swollen, inflamed gingival tissue also affecting the smoothness behavior.

[0119] The facet normal may be a vector perpendicular to a facet of the digital 3D model. The facet can be seen as a “building block” of the digital 3D model in 3D mesh format and can, for example, be a triangle if the digital 3D model is in format of a triangular mesh. A triangular facet may be defined by three points (vertices) interconnected with three edges. Two neighboring facets that have a small variation in their respective facet normals may indicate a local area that is smooth. As facet normals are connected and used to estimate curvature behavior in the data being looked at, a significant variation in the facet normals of neighboring facets, if detected, may be an indication of swollen tissue which may be an indication of recession. The variation in two surface normals can be expressed in terms of an angle between the two surface normals.

[0120] The curvature information may be in the form of a per-facet value and may be obtained by measuring a change along facet normals of neighboring facets of the digital 3D model. This change may give insight into whether the associated section of the digital 3D model is flat or curved. Thereby, the curvature information may describe topology of teeth and / or connected soft tissue by revealing smooth and non-smooth surfaces. With regards to using curvature information for predicting a presence and / or position of a coronal margin and / or apical margin indication soft tissue recession, it has been observed that a curvature change may be present in the area where the tooth transitions into gingiva. Accordingly, it has been realized that a change in curvature may be a significant parameter for detecting the coronal margin and / or apical margin and / or the respective positions in intra-oral scan data.

[0121] Therefore, in an embodiment the data transformation may comprise calculating from the 3D model generated from the intra-oral scan data a curvature change in at least areas of tooth transitioning into gingiva for each tooth of the 3D model. Accordingly, for each vertex representing a tooth a curvature calculation may be performed by the processor. The curvature calculations may be provided as a matrix representation in accordance with the previously described data transformation and subsequently input to the trained machine learning model. Then use of such a curvature calculation has shown to significantly improve the detection of the coronal margin in the data used to generate this invention.

[0122] Both the geometric information and the natural color information may be expressed in numerical format, such as matrices. For example, natural color information may be expressed numerically as matrix of values between 0 and 255 in Red, Green, Blue (RGB) color space.

[0123] Optionally, the method may further comprise providing semantic information of the at least part of the digital 3D model to the trained neural network. Thus, the input may additionally comprise semantic information of at least a part of the digital 3D model. Semantic information of at least a part of the digital 3D model may comprise one of e.g., a tooth identifier, a tooth surface identifier, and / or a jaw identifier.

[0124] The tooth identifier may provide information on which tooth is comprised in the input (e.g., an incisor, a canine, a molar, or a pre-molar). The tooth surface identifier may provide information on which tooth surface is comprised in the input (e.g., a buccal, lingual, mesial, distal, or occlusal surface) while the jaw identifier may provide information to what jaw (upper or lower) the at least part of the digital 3D model relates to.

[0125] By incorporating semantic information into the input to the neural network, a semantic context may be given to data present in the input, thereby providing semantic context to the neural network which may adapt its behavior according to the provided semantic information. Semantic information may be converted to numerical format, as other data formats, before being processed by the neural network. Better performance of the neural network may be achieved by additionally incorporating this semantic information into the input. For example, for soft tissue recession it has been found that e.g., canine is more prone to soft tissue recession. Accordingly, information about a tooth being a canine tooth may increase performance of the neural network.

[0126] Further, the method may comprise obtaining an output from the trained or during training of the neural network based on the provided input, wherein the output comprises a prediction of presence and / or position of the coronal margin and / or apical margin associated with at least one tooth of the 3D intra-oral scan data.

[0127] The output may be in the form of a probability value representing the likelihood that at least the part of the digital 3D model comprises the coronal margin and / or the apical margin (i.e., forming part of the health state prediction). This probability value may relate to at least one facet, at least one point, at least one node, at least one pixel or at least one voxel, comprised in the at least part of the digital 3D model, depending on the format of the provided input. By obtaining the output in this manner, information about at least a coronal margin and / or apical margin in the dental situation may be obtained from a single digital 3D model of the dental situation.

[0128] The method according to the disclosure may comprise assigning the coronal margin and / or apical margin, as output from the trained machine learning model via the health state prediction structure to at least a part of the digital 3D model. In this way, the coronal margin and / or apical margin may be associated with the corresponding region in the digital 3D model and thereby with the corresponding region of the actual dental situation represented by the digital 3D model. Correspondence of at least the part of the digital 3D model and the coronal margin and / or apical margin may be established by means of an identifier, which may be in form of a number. For example, the coronal margin and / or apical margin may be detected for a facet, a point, a voxel, a node and / or a pixel of the input. The exact position of such facet, a point, a voxel, a node and / or a pixel, in at least the part of the digital 3D model may be defined through the identifier. The described assigning step allows for a correct marking and display of the coronal margin and / or apical margin on the digital 3D model.

[0129] The neural network may, in a manner described above, perform a per-element classification or segmentation task to classify or segment the input according to presence of the coronal margin and / or apical margin. That is, the neural network may as previously be described for example perform a binary classification providing an output indicating “presence” or “no presence” of the coronal margin and / or apical margin. Alternatively, a segmentation task may be performed by the trained neural network, wherein the output from the neural network comprises one or more labels assigned to each pixel, point, voxel, facet etc. depending on the type of input format. Each label assigned in accordance with the output from the trained neural network may represent e.g., a buccal presence of coronal margin and / or apical margin, a lingual presence of coronal margin and / or apical margin, central presence of coronal margin and / or apical margin and / or interdental presence of coronal margin and / or apical margin.

[0130] The output from the neural network may be in a vector format of fixed size which can be derived from a processed (using the trained neural network) representation of the input. Values in the output may comprise at least one probability value representing the likelihood that the input comprises a coronal margin and / or apical margin.

[0131] In an embodiment, obtaining the output from the trained neural network may comprise obtaining a probability matrix, wherein the probability matrix allows to transfer the information reg. a position of the coronal margin to the at least part of the digital 3D model. The probability matrix may comprise probability values associated with each element of the input.

[0132] For providing a suitable training dataset for training the machine learning model, definitions of soft tissue recession used in an annotation process are provided. Soft tissue recession as previously mentioned may be construed as an apical shift (i.e., towards the root of a tooth) of the gingival margin with respect to the cemento-enamel junction (CEJ). Such apical shift is illustrated as an example in Figure 2, where a part of teeth in an intra-oral cavity is shown. As may be seen in Figure 2, the apical shift 103 of the gingival margin (GM) from a position of the CEJ reflects that soft tissue recession has developed. Soft tissue recession may be identified at different sites of the tooth-gingival boundary in the mouth, such as at a centralsite as illustrated in Figure 3b and Figure 3c and / or inter-dental sites, as illustrated in Figure 3d. Figure 3a illustrates a healthy state of the teeth, where Figure 3b illustrate an first development state of soft tissue recession at a central-site marked by dotted circle 104, Figure 3c illustrates by dotted circle 105 an advanced soft tissue recession at a central-site and Figure 3d illustrates inter-dental site soft tissue recession by dotted triangles 106. Furthermore, soft tissue recession may appear around teeth and implants in addition to the central and interdental sites. From a clinical perspective, different anatomical sites may be used when assessing soft tissue recession, as the commonly used reference of CEJ may not always be visible and can therefore not always be used in a manual assessment of soft tissue recession. Accordingly, other suitable anatomical landmarks may therefore be used to evaluate manually via annotations of a 3D intra-oral scan data if soft tissue recession is present or not. The suitable anatomical landmarks described herein, including the CEJ if detectable, are commonly defined throughout as the coronal margin. The coronal margin is at least one anatomical landmark which may be determined by the trained machine learning model as described herein. Accordingly, for training the machine learning model to accurately determine a health prediction structure describing at least the coronal margin, manual annotations of soft tissue recession are used.

[0133] Accordingly, the training dataset used for training the machine learning model in detecting at least the coronal margin (represented in the health state prediction structure described herein), annotations covering the following areas have been applied to a training dataset comprises a plurality of samples with 3D models on intra-oral cavities.

[0134] The coronal margin annotated for training of the machine learning model may for the central-sites of a tooth comprise one or more of: in case of teeth having tooth wear, the edge of tooth wear defect or the neighboring teeth level, in case of teeth having a filling, the reconstructed CEJ or the neighboring teeth level, in case of teeth having a crown, the edge of the crown or the reconstructed CEJ may be used as coronal margin.

[0135] The coronal margin used herein for the inter-dental sites may comprise one or more of: the apical point of the contract surface and / or the CEJ.

[0136] The coronal margin described herein may be comprised of a point or a plurality of connected points in a 3D model representing the coronal margin.

[0137] 3D intraoral scan data used for training of the machine learning model, may be considered as data generated, preferably from an intra-oral scanner, which have already been mentioned. The 3D intra-oral scan data described herein is preferably obtained by an intra- oral scanner and stored in a memory being in communication with a piece of digital dental software. The digital dental software, executing the computer-implemented method described herein, is configured to load into a processor the 3D intra-oral scan data, either as a raw data file and / or as a generated 3D model representing the 3D intra-oral scan data. The 3D intra-oral scan generally comprises a scan of the oral cavity of a patient and contains data points for each anatomical structure, such as teeth, gingiva, palatal etc. contained in the oral cavity.

[0138] The training process may, in more detail, comprise the use of a training data set comprising the previously mentioned annotations. The annotations may have been generated manually or automatically. The training dataset may be in any format suitable for the chosen neural network architecture to process the data. That is, different neural network architectures may be chosen to solve a certain task, such as for example classification or segmentation. To train a neural network architecture for a specific purpose the training data, as previously described in relation the input data generated from the 3D model when the already trained neural network is used in an application, may undergo a data transformation creating a data format suitable for the neural network architecture chosen. As previously described the training data may be provided in a 2D format, 3D meshes or point cloud format.

[0139] To ensure that the neural network is capable of learning to detect the intended coronal margin, the training dataset may for each tooth and / or full 3D model comprise associated labels where the labels correspond to for example a manually annotated coronal margin and / or apical margin, or an area of the two combined for each tooth, part of a tooth and / or gingiva in the training dataset. The labeling may be performed for example by a qualified dental practitioner. The training dataset may be labeled according to identification of a coronal margin, for example with “Yes” or “No” labels indicating coronal margin and / or apical margin presence or absence, respectively. Additionally, or alternatively, the training dataset may be labeled with annotations marking an area of the coronal margin and / or apical margin on the intra-oral scan data, such as the plurality of 3D models used for generating the training dataset.

[0140] In an example, the training dataset may also be in 3D format in which the labels may be placed directly on the 3D format. To generate a training dataset with the annotations applied thereto, the data transformation may furthermore comprise the step of extracting the annotation information from the labelled data and label the point cloud format, mesh format with one or more classes reflecting the annotated information. To train the neural network in performing a binary classification, each tooth of the training dataset may be labelled with a class of “no recession” or “recession” present. To train the neural network in performing a segmentation, each tooth of the training dataset may e.g., be labelled with a class of “buccal recession”, “lingual recession”, “interdental recession”. Each of the mentioned classes may be assigned e.g., a color in the point clouds or meshes generated from the previously described data transformation allowing the neural network, in the training process, to distinguish between a “recession” case, a “no recession” case and / or a ’’buccal recession”, “lingual recession”, “interdental recession” case.

[0141] The labels, for example in the form of annotations, as described herein, may be regarded as a target or a ground truth for a given training data set. An advantage of such ground truth data applied in the training of the neural network is that it is clinically valid and accepted, due to the information originating from manual annotations of the training dataset performed by trained clinical professionals.

[0142] The training dataset may be fed into the neural network, which applies the training dataset and the ground truth data to learn how to determine, classify or segment the presence of a coronal margin. Accordingly, an output of the neural network may be a probability value, indicating which output class the data contained in the input belong to. An error may be determined between the ground truth and the output by means of a loss function. One example of such a loss function may be a mean-squared-error function. To train the neural network this loss function should be minimized. The minimization of the loss function may be achieved for example by using a stochastic gradient descent algorithm. Through this process of minimizing the loss function, weights of the neural network may be updated.

[0143] The loss function may in an embodiment comprise a weighted loss function, wherein one or more weights are added to the output of the machine learning model in the iterative training process. In an embodiment a first weight may be added to the output identified as ideal gingiva and a second weight may be added to the output identified as teeth, wherein the first weight is higher than the second weight. This ensures that the machine learning model during training is given a larger penalty on wrong estimates associated with ideal gingiva, whereby a more accurate machine learning model estimate is achieved. A third weight may be applied to the output identified as the actual gingiva of the current model, which is or at least can be found from a separate segmentation of the model into teeth and gingiva and which segmentation is used for annotating the points associated with the acutal gingiva. In such embodiment, the third weight is lower than both the first and second weight as the annotations associated with the actual gingiva is considered highly accurate. Provided these weights to the estimates output from the machine learning model during the training phase and adjusting the weights during training of the model ensures that the machine learning model provides an accurate estimate of the ideal gingiva, whereby a recession measure may be determined accurately.

[0144] The training process may be an iterative process in which the same training data set may be fed, iteratively, into the neural network until a stopping criterion is reached. The stopping criterion, regardless of the training process employed, may be a specific threshold value for the loss. The stopping criterion may, additionally or alternatively, be a number of training epochs.

[0145] Furthermore, in the training process, the training data may be modified randomly to generate an augmented dataset reducing overfitting. In other words, using and augmented dataset ensures that the neural network is presented with a plurality of slightly modified versions of parts of training dataset to ensure that the finally trained neural network may be able to accurately generate predictions on dataset which the neural network was not trained on.

[0146] In addition, to the already described parts of the training process, in an embodiment, the training process may also include to calculate the curvature of at least the area representing the transition between gingiva and tooth in each of the 3D intra-oral scan data used as the training data. This may ensure that the neural network learns better to predict the presence of a coronal margin, as the curvature change shows significant changes when a soft tissue is present.

[0147] Also, and as previously described in relation to the already trained neural network, the tooth identification, in the form of the tooth being an incisor, a canine, a molar or a pre-molar may be used as input in the training process. During the training process, a suitable network architecture may be chosen in view of solving a specific problem. The network architecture chosen may be updated to ensure better performance of the trained neural network.

[0148] In an example, according to the methods described herein, the training process comprises modifying the chosen neural network architecture, wherein the modification comprises generating a first classification and / or segmentation output from the neural network and subsequently updating the first classification and / or segmentation output with an additional network layer, wherein the additional network layer comprises semantic information, such as tooth identification for each tooth. Accordingly, two neural network architectures may be used. The architectures may be the same or different from each other. By providing a neural network architecture which provides an initial classification and / or segmentation output, and subsequently is given additional information from e.g., a second architecture it is ensures that final output from the network architecture comprises an increased confidence level as to the provided network predictions.

[0149] In an embodiment, the network architecture may comprise a first architecture providing a classification and / or segmentation vector and a second architecture providing a feature vector, wherein the global vector comprises the classification vector and the feature vector and is configured to be processed by the machine learning model to generate the final prediction vector.

[0150] The first layer architecture may be configured to receive as input e.g., a point cloud data generated from the data transformation, preferably comprising a 3D point cloud of the intraoral scan data and a curvature calculation as previously described for each point in the point cloud, wherein the first network architecture preferably is configured as a point cloud classifier.

[0151] The second layer architecture may be configured to receive semantic information, such as e.g., the tooth identification, which may be considered of high relevance to the prediction of a coronal margin as it has been realized that the soft tissue recession is more likely to occur on some teeth rather than others. The second layer may be configured as a deep neural network with linear layers, where the output may be comprised of a feature vector. According to the modified network architecture each of the outputs from the first and second layer may be combined into a global vector as previously described, wherein The global vector may be processed using a deep neural network with linear layers to generate the final prediction of presence of a coronal margin and / or apical margin and / or an area thereof.

[0152] The training of the neural network generally results in a final neural network model, suitable for solving the problem set out according to the description herein, i.e., to predict the presence and / or position of a coronal margin and / or apical margin as contained within the health state prediction structure, in intra-oral scan data generated from an intra-oral scanner.

[0153] In accordance with the description of the different embodiments provided herein, the solutions described may be implemented as part of a system comprising a soft tissue recession detection module (also denoted an application module), wherein the soft tissue recession detection module is configured to execute all or parts of the method(s) of determining and displaying a soft tissue recession measure as described herein. The system may preferably be configured as a piece of software working as a standalone processing of data obtained by e.g., an intra-oral scanner at a point in time corresponding to the direct use of the software during or right after scanning of an intra-oral cavity. Alternative the software may be configured to load into a processor, such as a processor of a soft tissue recession detection module, the intra-oral scan data to perform analysis of the data and at least aiding in diagnosing soft tissue recession at a later point in time than when the actual intra-oral scanning was performed. The software running the soft tissue recession application may also be configured to be run on a cloud base system, wherein the cloud is configured to communicate with a client server at e.g., a dental practice or a patient app.

[0154] In any case, the soft tissue recession detection module may be configured as a computer program forming part of the system, and comprising instructions which, when the program is executed by a computer, cause the computer to carry out any of the method steps described herein. The system may comprise a computer-readable medium comprising instructions, which when executed by a computer, cause the computer to carry out any of the method steps described herein.

[0155] DETAILED DESCRIPTION

[0156] The detailed description set forth below in connection with the appended drawings is intended as a description of various examples according to the disclosure. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts and examples covered throughout the description. However, it will be apparent to those skilled in the art that these concepts and examples may be practiced without the specific details mentioned or in combination with one or more examples described herein. Several examples of the devices, systems, mediums, programs, and methods are described by various modules, components, steps, processes, algorithms, etc. Depending upon application, design constraints or other reasons, these elements may be implemented using electronic hardware, computer program, or any combination thereof. In the following several examples of the methods and system described herein will be disclosed in more detail.

[0157] BRIEF DESCRIPTION OF THE FIGURES

[0158] The examples provided according to the disclosure may be best understood from the following detailed description taken in conjunction with the accompanying figures. The figures are schematic and simplified for clarity, and they just show details to improve the understanding of the claims, while other details are left out. Throughout, the same reference numerals are used for identical or corresponding parts. The individual features of each example given may each be combined with any or all features of the other examples. These and other examples, features and / or technical effects will be apparent from and elucidated with reference to the illustrations described hereinafter in which:

[0159] Figure 1 illustrates a healthy and unhealthy oral situation of a single tooth;

[0160] Figure 2 illustrates the apical shift of recessed gum in a tooth away from the CEJ;

[0161] Figure 3a-d illustrates in 3a) a healthy state of teeth, 3b) a mild central recession of a tooth, 3c) a severe central recession of a tooth, 3d) interdental recession of teeth; Figure 4 an example process of the method of determining soft tissue recession according to embodiments;

[0162] Figure 5 illustrates a soft tissue recession application in a dental software system according to embodiments;

[0163] Figure 6 illustrates example inputs to the machine learning model and example outputs therefrom;

[0164] Figure 7 illustrates a comparison of the apical and coronal margin as found using the machine learning model;

[0165] Figure 8 illustrates a graphical user interface of a dental system showing a determined soft tissue recession;

[0166] Figure 9 illustrates an example method of calculating a soft tissue recession measure;

[0167] Figure 10 illustrates an example method of calculating soft tissue recession measure using planes;

[0168] Figure 11 illustrates and example method of calculating soft tissue recession measure according to Figure 10;

[0169] Figure 12 illustrates an example result of determining soft tissue recession according to embodiments;

[0170] Figure 13 illustrates details of segmentation and determining margin and junction labels using segmented intra-oral scan data;

[0171] Figure 14 illustrates splines of gingival margin, apical and / or coronal margin according to embodiments;

[0172] Figure 15 illustrates two margin lines (such as gingival, coronal, or apical margin) to be aligned according to embodiments described;

[0173] Figure 16 illustrates a snapping of one margin line to a 3D model;

[0174] Figure 17a illustrates two margin lines before snapping according to Figure 16;

[0175] Figure 17b illustrates two margin lines after snapping according to Figure 16;

[0176] Figure 18 illustrates a flow of the training process of a machine learning model according to embodiments;

[0177] Figure 19 illustrates an example machine learning model architecture;

[0178] Figure 20 illustrates a second example machine learning model architecture;

[0179] Figure 21 illustrates an example output from a machine learning model according to embodiments; Figure 22 illustrate example processed binary outputs from a machine learning model according to embodiments;

[0180] Figure 23 illustrates example processed segmented outputs from a machine learning model according to embodiments;

[0181] Figure 24 illustrates an example of a scanning system according to the disclosure;

[0182] Figure 25 illustrates an example flow of a first and second dental visit by a patient;

[0183] Figure 26 illustrates an example of central recession measurement of a tooth;

[0184] Figure 27a illustrates and example of interdental recession measurement of on a full jaw scan;

[0185] Figure 27b illustrates a zoomed version of Figure 27a showing the interdental measurement points;

[0186] Figure 28 illustrates a dataset with three labels for use in the machine learning model; Figure 29 illustrates a dataset with two labels for use in the machine learning model; Figure 30 illustrates a non-molar tooth of a dental model with recession measure; and Figure 31 illustrates a molar tooth of a dental model with recession measure.

[0187] Referring initial to Figure 1 and Figure 2, the general dental terms used throughout this description to explain the disclosed method(s) are illustrated. Reference is made to previous explanations of Figure 1 and 2. In the context of the output from the machine learning model, the CEJ marking illustrated in Figures 1 and 2 may in embodiments described here be considered as the coronal margin, that may be determined from the output from the machine learning model.

[0188] The intraoral-scan data described herein may generally be obtained from an intra-oral scanner, as illustrated in Figure 24. Here, a dental scanning system 1 is shown. The dental scanning system 1 is suitable for scanning an intraoral object (such as a dental arch and the teeth, gum, and other intraoral object thereof) of a patient and / or to determining a healthcondition and / or a probability thereof based on scanning of the intraoral object. The scanning system 1 comprises a scanning device 2 (such as an intraoral scanner) configured for scanning of the intraoral object. The scanning device 2 comprises an illumination-unit 3 configured to illuminate the intraoral object with light; an image-sensor 4 configured to record images of light from the illuminated intraoral object; an illumination-controller 5a configured to operate the illumination-unit 3 in one or more modes of illumination depending on the intended use of scanning device 2. The scanning device 2 may further comprise an acquisition controller 5b configured to operate the image-sensor 4 in one or more acquisition modes, wherein the scanning device 2 may be configured to change between the one or more illumination modes, whereby the scanning device 2 forms one or more datasets of the intraoral object in dependence of the one or more illumination-modes. The scanning device 2 is further configured to change between the one or more acquisition modes, whereby the scanning device forms one or more dataset of the intraoral object in dependence on the one or more acquisition modes. A data processor 6 of the scanning system 2 may in dependence of the illumination and acquisition mode be configured to form, from a first dataset of the one or more datasets, a 3D-model (also denoted a 3D digital model generated from the intra-oral scan data) 7 of the intraoral object and to form, from e.g. a second dataset of the one or more datasets, a 2D-image of the intraoral object and / or further details of the 3D digital model. Further the scanning system 1 may be configured to apply, on the 2D-image and / or the 3D digital model 7, a diagnostic software application to identify a diagnostic feature (such as a diagnostic indication, oral health evaluation etc.) of the intraoral object and determine, based on the diagnostic feature of the intraoral object, a health-condition or the intra oral object.

[0189] As illustrated in Figure 24, the system 1 further comprises a display unit 8, a computer keyboard or touchpad 14, a computer mouse or touchscreen 15 for entering data and activating virtual buttons (user interaction elements) visualized on the visual display unit 8. The visual display unit 8 can be a computer screen, a touchpad screen or e.g., a smart phone screen comprising a graphical user interface and having a display unit 8, whereon the 3D digital model 7 and e.g., a health-condition are displayed. Further the dental scanning system 1 may comprise a storage media / medium 16 configured to store data, such as acquired scan data form the scanning device 2 in relation to specific patient records, analysis data, patient specific identification data, diagnostic data etc. The storage media / medium 16 may be configured as cloud storage or for example storage on multiple computer services, which are configured to communicate with each other over a network. Accordingly, the processing and storage of the data relevant for analysis may be performed in a cloud setup and loaded into a computer therefrom. As will be apparent in the following, this diagnostic feature may be the determination of a health state of soft tissue in the intra-oral scan data acquired by the dental scanner 2 of the dental scanning system 1.

[0190] Accordingly, the dental scanning system may be configured with or be in communication with a processor configured to perform any of the following method(s) described in the following with reference to Figures provided herein.

[0191] With reference to Figure 4, the dental scanning system comprises a processor configured to perform a computer-implemented method for determining and displaying a health state of soft tissue in an oral cavity of a patient. The processor, as illustrated in Figure 4, 201 is configured to retrieve intra-oral scan data of a patient, wherein the intra-oral scan data is obtained by an intraoral scanner (e.g., the dental scanner 2) and comprises a plurality of dental features. The processor is configured to generating an input structure by performing a data transformation of the intra-oral scan data from a 3D space to a plurality of descriptive representations of each of the dental features in the intra-oral scan data, as illustrated in Figure 4, 202. The input structure is feed into a trained machine learning model, configured to generate a health state prediction structure representing at least two or more health state(s) of each of the descriptive representation of the intraoral scan data, as illustrated in Figure 4, 203. The health state prediction structure is utilized to generate a 3D representation of the intra-oral scan data with assigned health state labels for each of the dental features, as illustrated in Figure 4, 204, and the processor is configured to determining a soft tissue recession measure by identifying in the 3D representation the health state labels associated with soft tissue recession and evaluating a measure of the health state labels associated with soft tissue recession, as illustrated in Figure 4, 205. To be able to visualize the resulting health state, the processor is configured to displaying in a graphical user interface (a representation of the determined soft tissue recession measure, Figure 4, 206.

[0192] With this solution, the trained machine learning model can predict a health state of soft tissue in intra-oral scan data without relying on a manual assessment from a dental practitioner and using only a single-scan data. A single scan data solution as described herein does not require any past saved data to assess in soft tissue recession is present and does not require any manual visual inspection by a dental practitioner. Accordingly, as will be explained, the trained machine learning model has been trained and built to allow an automatic detection of soft tissue recession based purely on a single intraoral scan data. That is, the trained machine learning model is trained based on a training dataset of intra-oral scan data comprising a plurality of data samples, wherein each of the plurality of data sample comprises a 3D model comprising one or more dental features having associated dental feature labels.

[0193] Turning to Figure 5, the intra-oral scan data is input to a processor of the dental scanning system, wherein the processor 23 is configured to generate a 3D model 21 of the intra-oral scan data 20. To allow the intra-oral scan data to be processed by a trained machine learning model, in accordance with embodiments described herein, the intra-oral scan data is processed by the processor to undergo a data transformation. The data transformation comprises to generate first input structure comprising a plurality of descriptive representations 22 of each of the dental features of the plurality of data samples. With reference to Figure 5, this can be explained as follows. The intra-oral scan data comprises a plurality of dental features, preferably construed as teeth 21a and associated gingiva 21b of the intra-oral scan data, as illustrated in Figure 5. The plurality of dental features, i.e., teeth 21a and gingiva 21b, is input to the processor 23, which is configured to generate the input structure. The data transformation performed by the processor 23 comprises generating a plurality of descriptive representations 22, illustrated in an example of Figure 5 as a pointcloud representation 22 of each dental feature. In more detail, in the example of pointclouds 22 representations, the processor 23 is configured to generate from a mesh structure 24 representing the 3D model input (only one tooth shown), a point cloud 22 defining the above-mentioned descriptive representations.

[0194] It should be noted that the descriptive representations 22 (also denoted point clouds above) illustrated in Figure 5 is merely one example. As is previously described, the descriptive representations may also be generated based on other transformations, such as using 2D images, graph nodes, mesh structures, etc.

[0195] As also illustrated in an example in Figure 5, the processor may also be configured to calculate curvatures 25 of the input intra-oral scan data. The curvatures as calculated from the intra-oral scan data by the processor may be represented as heat map 25 as illustrated in Figure 5.

[0196] To determine the health state of the soft tissue in the intra-oral scan data, the previously described descriptive representations 22 and / or curvature calculations 25 is transformed into a vector representing an input to the trained machine learning model 26, wherein the trained machine learning model is configured to output health state predictions 27 for each of the descriptive representations in the transformed intra-oral scan data.

[0197] In more detail, the processor is configured to take as an example input, at least one tooth of the intra-oral scan data, where each tooth undergoes a data transformation, generating a point cloud representation for each tooth. This point cloud representation is illustrated in Figure 6, as a number of points 1 to 27 for a tooth 21a. These points are input into the machine learning model which generates a health state prediction structure 28 representing, in the example illustrated, a vector of numerical values, where a numerical value represents two or more different health states for each of the points 1 to 27 in the tooth. As illustrated in Figure 6, the health state prediction structure 28 is utilized by the processor to generate a 3D representation of the tooth 29. As can be seen the prediction structure 28 is transformed back onto the tooth at the respective points 1 to 27, where a 0 in the case illustrated for example reflects a healthy tissue and / or tooth and a 1 in the case illustrated reflects unhealthy tissue of the gum (i.e., soft tissue). These 0 and l’s may be considered as health state labels for each point in the tooth data.

[0198] As also illustrated in Figure 6, the prediction structure may also be utilized to provide a color coding to each of the points in the tooth, such as for example red (r) for areas containing recession and (w) for areas being e.g., tooth. The color coding may also be considered as a health state label for each point in the tooth data. Any suitable color coding may be chosen.

[0199] As a result of this prediction structure, it is possible for the processor to process the data and determine a soft tissue recession measure by identifying in the 3D representation the health state labels associated with soft tissue recession and evaluating, such as quantifying a measure of the health state labels associated with soft tissue recession as illustrated in Figure 6. That is, the processor is configured to determining from the assigned health state labels (i.e., the 0 and l’s or (r) or (w) as illustrated in Figure 6) a pointset representing soft tissue recession points of each dental feature in the intra-oral scan data and quantifying the soft tissue recession measure for each dental feature based on the determined pointset.

[0200] As may be seen in Figure 6, each of the assigned health state labels comprises a pointset reflecting a position of the label on e.g., a tooth as illustrated in Figure 6. Accordingly, from the pointsets marked with health state labels 0 and 1, an apical margin and / or a coronal margin of the soft tissue for each of the dental features may be represented. That is, the apical margin 30 may be construed as the border between a recessed gingiva and the exposed tooth area, marked with 1 in Figure 6, where the gum has substantially disappeared. The coronal margin 31 may be defined by the border defining the transition from tooth to exposed tooth, as illustrated in Figure 6.

[0201] Each of these apical and coronal margins may be superimposed into the 3D model 21 of Figure 5 to visualize the predictions of the machine learning model. Such visualization is illustrated in Figure 7 and 8, where the apical margin 30 and the coronal margin as determined from the output of the machine learning model is visualized on a 3D model 21. Furthermore, the coronal margin may be configured to represent the cemento-enamel junction, i.e., the healthy transition border between gum and tooth.

[0202] Utilizing the determined apical 30 and coronal margin 31, as just described, the soft tissue recession measure may in an embodiment be determined by the processor being configured to measuring a distance between a most apical point of the apical margin and the most coronal point of the coronal margin for each dental feature. Such determination is illustrated in more detail in Figure 7, where it is seen how the processor is configured to determine for each of the apical 30 and coronal 31 margins, the most apical points 32a, 32b for a tooth (two teeth illustrated) and calculate a distance 33a, 33b for each of the tooth. The distances 33a, 33b provide a quantification of soft tissue recession in accordance with the methods presented herein.

[0203] As can be seen on Figures 7 and 8, a representation, such as the coronal 31 and apical 30 margin, or a distance representation for the determined soft tissue recession measure is configured as a projection of the soft tissue recession measure onto the 3D model, wherein the 3D model is generated from the intraoral scan data.

[0204] In another example, each of the assigned health state labels as illustrated in relation to Figure 6, may be used in processing step to mark the entire area between an apical and coronal margin as a recessed area of the soft tissue. That is, from the pointsets marked with health state labels 0 and 1, an apical margin and / or a coronal margin of the soft tissue for each of the dental features may be determined defining the borders of the soft tissue recession. That is, the apical margin 30 may be construed as the border between a recessed gingiva and the exposed tooth area, marked with 1 in Figure 6, where the gum has substantially disappeared. The coronal margin 31 may be defined by the border defining the transition from tooth to exposed tooth, as illustrated in Figure 6. The area between the apical and coronal border may be identified in a processing step and assigned a color value. A subsequent superimposing step configured to color parts of the 3D model associated with the area between the apical and coronal border ensures that the extend of gingival recession is visualized to a user.

[0205] The representation of the soft tissue recession may be configured as a representation in a window of the graphical user interface, which window is different from where the 3D model, generated from the intraoral scan data, is presented. This is especially illustrated in Figure 8 as an example, where box 34 in the graphical user interface illustrates a summary of the predictions as output from the trained machine learning model. That is, the box 34 may summaries a value of soft tissue recession per tooth, a score of soft tissue recession, a severity class, a confidence representing the certainty that the machine learning output is accurate and potentially also a shape classification of the recession for each tooth in the intra-oral scan data.

[0206] With regards to the confidence, the trained machine learning model is configured to output a confidence label vector for each of the dental features, wherein a confidence label falling under a first class represents a high degree of confidence that the health state predictions is accurately detected by the trained machine learning model, and wherein a confidence label falling under a second class represents a low degree of certainty that the healthy state predictions is accurately detected by the trained machine learning model, wherein the processor is configured to utilizing the confidence label to output a recession confidence measure for each dental feature of the intraoral scan data.

[0207] As seen in Figures 7 and 8, the computer-implemented method may comprise displaying the apical margin 30 and the coronal margin 31 of the health state prediction superimposed onto the 3D representation of the intra-oral scan data in the graphical user interface 8.

[0208] In another example, the representation of soft tissue recession may be a coloring to the apical margin, wherein the coloring defines the distance between the most apical point of the apical margin and the most coronal point of the coronal margin for each dental feature falling within one or more set threshold(s).

[0209] In yet an example, the methods described herein comprises displaying in the graphical user interface a continuous color map as a part of the graphical user interface, and projecting the apical margin and the coronal margin onto the 3D representation of the intra-oral scan data, wherein the continuous color map comprises a slider, which upon activation by a user, causes the processor to adjust the threshold defining the amount of soft tissue recession to visualize.

[0210] In embodiments, the method may accordingly, comprise projecting the apical margin 30 and the coronal margin 31 onto the 3D representation of the intra-oral scan data, and determining, by the processor, a maximum distance between the apical margin and the coronal margin for each tooth on the 3D model, and displaying the maximum distance at a local area of each tooth where the maximum distance is determined, as illustrated in Figures 7 and 8.

[0211] Turning now to Figures 18 to 29, the training process generating a trained machine learning model capable of predicting a health state of a soft tissue as utilized by the proposed method(s) described herein will be elaborated on in further detail.

[0212] Figure 18 illustrates a flow of the training process utilized to train the machine learning model used in accordance with the previously described application method. In Figure 18 the process of the computer-implemented method for training the machine learning model for predicting a health state of soft tissue in intra-oral scan data is illustrated, wherein the method comprises loading from a storage media a training dataset of intra-oral scan data comprising a plurality of data samples, wherein each of the plurality of data sample comprises a 3D model comprising one or more dental features having associated dental feature labels, 301. Generating a first input structure by performing a data transformation of the plurality of data samples, wherein the data transformation generates a plurality of descriptive representations of each of the dental features of the plurality of data samples,

[0213] 302. Assigning at least one descriptive label to the plurality of descriptive representations, wherein at least one descriptive label is acquired from the associated dental feature labels,

[0214] 303. Generating a second input structure, wherein the second input structure comprises a feature vector representing an anatomical descriptor of each of the one or more dental features, 304. Feeding the first and second input structures to a machine learning model, wherein the machine learning model is configured to process each of the first and second input structures to generate a global structure comprising a processed representation of the first and second input structures, wherein the global structure is configured to be processed by the machine learning model to generate a health state prediction structure for each of the plurality of descriptive representations, 305.

[0215] As may be apparent to the skilled person, it should be noted that the main difference between the training process and utilizing an already trained machine learning model in an application as previously described, is that the input structures for the training process utilizes ground truth data, such as the annotations (dental feature labels) described herein to allow optimization and quality in the training process of the machine learning model. The annotations are unknown in view of a real application (as the one described in relation to Figures 4 to 8) and the input data is unknown to the machine learning model.

[0216] With reference to Figures 20, in an example, the first input structure 400 is configured to be fed to a first neural network architecture 401 of the machine learning model configured to generate a first prediction structure 405. This is illustrated in more detail in Figure 20, where the input data structure 400 (as generated from the previously described transformation) may be configured as a point cloud. In the example shown in Figure 20, this point cloud may be input to a machine learning model utilizing a first neural network architecture 401, being a point cloud classifier, such as a PCT, PointNet and / or PointNet++ classifier).

[0217] Furthermore, as illustrated in Figure 20, the second input structure 402 is configured to be fed to a second neural network architecture 403 of the machine learning model configured to generate a second prediction structure 404. The first prediction structure may in the example shown in Figure 20 be configured as a classification vector, segmentation vector or any other suitable vector describing the classification suitable based on the input data. Similarly, the second prediction structure may be configured, as in the example illustrated in Figure 20, as a feature vector, wherein the feature vector may describe for example tooth kind information, tooth type information, semantic information etc. The two neural network architectures mentioned herein may be the same or different from each other. Furthermore it is noted that a single network architecture may be utilized such as for example when doing a segmentation task.

[0218] As illustrated in Figure 20, the global structure 406 comprising the first prediction vector 405 and the second prediction vector 404.

[0219] As previously explained, and illustrated in Figure 20 in light of training the machine learning model, the health state prediction structure 407 (also denoted 28 in Figure 6 in relation to applying the trained machine learning model in an application) comprises a plurality of numerical values for each of the descriptive representations, wherein each of the plurality of numerical values represents two or more health state labels, wherein at least one label represents a healthy state, at least one label represents an initial state and / or at least one label represents an advanced state of soft tissue recession.

[0220] As previously elaborated on in view of applying the trained machine learning model in an application using data unknown to the machine learning model, the health state prediction structure comprises a position label for each for the descriptive representations, wherein each of the position labels represents at least one of a buccal, lingual, or inter-dental side identification of the descriptive representations. Furthermore, to allow the machine learning model to estimate better the soft tissue recession, the machine learning models has also been trained on curvatures of the input structures, as these curvatures is believed to represent a change in soft tissue reflecting soft tissue recession. Accordingly, the method comprises calculating a curvature representation of each of the descriptive representations and feeding the curvature representation into the machine learning model.

[0221] In an embodiment the first input structure may comprise both teeth and gingiva, wherein the gingiva comprises an area of gingiva extending from the tooth to ensure the gingival behavior associated with recession is reflected in the dataset. The area of gingiva to include in the dataset may be set by a threshold in the system, whereby the set threshold is optimized to ensure that enough gingiva is represented to allow the behavioral transitions of the gingiva to be taken into account. An example of an extended gingiva part forming part of the data set is illustrated in Figure 28, where three labels are seen to represent the dataset. A first label 1280 representing the extended gingiva area, a second label 1281 representing the ideal gingiva area and a third label 1281 representing tooth. This labeled dataset may be input to the neural network during training to allow the machine learning model to learn how to segment or classify an unknow dataset into the three labels. The extended gingiva as shown in Figure 28 ensures that the behavioral nature of the gingiva associated with recession are reflected in the dataset used for training. In an alternative, the dataset is only provided with two labels, as illustrated in Figure 29, where it is seen that only the gingiva label 1281 (including both ideal and extended gingiva) is included in the dataset together with the teeth label 1282.

[0222] Preferably, the method of training the machine learning model also comprises generating a confidence label vector representing for each of the descriptive representations a confidence that the descriptive representation is associated with at least one of the two or more health state labels, wherein the values used to generate the confidence vector is acquired from the dental feature label and feeding the confidence label vector into the machine learning model. In accordance herewith, and as illustrated in Figure 21, the prediction structure 407 or structures 407 as output from the machine learning model (in a training phase and after the model has been trained) may include a representation of the intra-oral scan data input describing at least one of a pointset 502 of the teeth 501 represented in the input data, a health state label 503, a curvature representation 504 and / or a confidence label 505, which may as previously described be utilized to represent measures of soft tissue recession in intra-oral scan data.

[0223] For training purpose, the data samples comprise associated dental feature labels, which dental features labels, as previously described comprise annotations associated with each dental feature and loaded from the data storage, wherein the annotations represent at least one or more of the health state labels, the position label and / or the confidence label.

[0224] The previous described example mainly refers to the input data being transformed into point clouds. However, other suitable data transformations may be utilized by the proposed method, wherein at least one other example utilizes a machine learning model which is based on e.g., 2D images 601 as input data. Such an example is illustrated in Figure 19, where a machine learning architecture 600 using convolutional networks is illustrated. The neural network 600 in this case may comprise a plurality of convolution layers 602 capable of capturing low-level features of the input, i.e., obtaining convolved features. Moreover, one or more of pooling layers 603 may be present, responsible for reducing the spatial size of convolved features. Convolutional neural networks may be particularly suitable for processing matrix information such as 2D images 600. Accordingly, for the convolutional neural method to predict the health state on a plurality of teeth in intra-oral scan data, the intra-oral scan data may be processed by a processor to undergo a data transformation, wherein the data transformation results in a plurality of 2D snapshots showing each tooth and connected gum tissue for the respective snapshots.

[0225] Accordingly, the input 601 to the machine learning model may be in a 2D format and may comprise at the least part of the digital 3D model. Figure 19 illustrates the input 601 in the 2D format. The at least part of the digital 3D model may be a 2D image of a tooth and the associated gingiva which has its lingual or buccal surface at interdental or central sites affected by potential soft tissue recession. Providing the input in the 2D format may comprise converting these data structures into input tensors which may be processed by the trained neural network.

[0226] The output 605 may comprise at least one probability value representing health state predictions for each of the teeth in the intra-oral data.

[0227] As illustrated in Figure 19, the health state predictions may comprise a binary classification task classifying the input 601 into “Yes” or “No” output classes representing healthy tissue or nonhealthy tissue.

[0228] As should be apparent, the machine learning model may be trained on any suitable network architecture matching the input data and the problem to be solved. That is, the problem to be solved may be a binary classification, a segmentation, a two-or more class segmentation etc.

[0229] An example of an output from a binary classification using point clouds as an input is illustrated in Figure 22. Here it is seen how the health state prediction structure may be processed to describe the data with a binary representation. That is, Figure 22 represents values of the health state prediction structure belonging to recessed gingiva 702 in one color and values representing tooth and / or non-recessed gingiva 701 in another color.

[0230] Similarly, Figure 23 illustrates an example of a segmentation task, where values 80 representing tooth and non-recessed gingiva is provided in one color, values 805 representing a lingual recession in another color, value representing inter-dental recession in a third color 803 and values representing buccal recession in a fourth color 804.

[0231] From the output of the machine learning model, in an embodiment the most apical and most coronal point may (as previously described) be determined to determine a measure of soft tissue recession. That is, in an embodiment, the method may comprise determining using a tooth-pose algorithm the coronal-apical direction as illustrated in e.g. Figure 26, where the coordinates system x and y for determining the orientation of the tooth is illustrated. The tooth-pose estimation is configured to calculate a local coordinate system defining the X, Y, Z direction for each tooth in a jaw or in a single tooth sample as illustrated in Figure 26. From this coordinate system, the coronal-apical direction may be determined, and the most apical and coronal points may be determined. That is, in Figure 26 the most apical point may be determined as the point 1301 being the lowest point on the gingival margin and the most coronal point may be determined as the point 1302 being the lowest point on the ideal gingival margin - i.e. where the gingival margin should have been in case recession were not present. The soft tissue recession is after determining these two most apical and coronal points 1301, 1302 determined by measuring the distance between the most apical 1301 and most coronal 1302 point, as illustrated by arrow 1303 in Figure 26. Figure 26 show a situation where central recession may be determined on the basis of the output from the trained machine learning model.

[0232] In the case interdental recession (i.e. recession happening in between two neighboring teeth), a similar situation as explained in relation to Figure 26 is illustrated in Figure 27a and Figure 27b. Here it is seen in Figure 27a how from a full jawscan (also possible from a single tooth) interdental recession may be found using a trained machine learning model. In Figure 27a, the interdental recession is illustrated in an example within the area 1304 encircled by circle 1305. A measure of soft tissue for the interdental site 1304 may be determined similarly as previously described and illustrated in Figure 27b, which shows a zoomed version of the area 1305 of Figure 27a. In Figure 27b, it is illustrated how using the methods described herein, the most apical 1306 point of interdental recession may be determined and similarly that the most coronal point 1307 may be determined. The distance 1308 may then be calculated between the most apical 1306 and the most coronal point 1307, the distance corresponding to the determined soft tissue recession measure. In this situation, the method may also utilize a tooth pose algorithm to find the orientation of the jaw or teeth in the jaw and to determine an up-direction 1309 assisting in defining the apical and coronal points.

[0233] Returning now to the utilization of the trained machine learning model, an embodiment (also explained previously as multi-scan solution) where two intra-oral scan data acquired at different points in time will be elaborated on with reference to especially Figures 9 to 17b. Referring initially to Figure 25, a patient may visit a dental practitioner several times to get an evaluation and treatment of their oral health. To assist the dental practitioner in assessing the oral health of a patient at a first visit and / or over time by utilizing one or more scan data taken at different times a defined workflow may form part of the dental visit. In this workflow, illustrated in Figure 25, the dental practitioner may use one or more of the application modules of a dental software, wherein the recession determination methods described herein may be configured as such application module.

[0234] With reference to Figure 25, an example workflow comprises a first dental visit (step (1)). At this first visit the oral cavity of the patient may be assessed using an intra oral scanner as previously described. Thus, at step (1) in Figure 25, the patient is scanned using e.g., an intraoral scanner by a dental practitioner. As can be seen in step (2) of the workflow in Figure 25, while scanning, the dental practitioner may be able to see the scanning live on a display unit 8. This first scan may be considered as a baseline scan, for example representing a first scan taken at a first point in time as previously elaborated on. At step (3) of Figure 25, the scan data is analyzed automatically utilizing one or more software applications, such as the soft tissue health state prediction described herein. Accordingly, the processor of the dental software 1000 illustrated in Figure 25 may be configured to perform the method(s) of automatically determining a soft tissue health state and directly display on the graphical user interface the findings. Accordingly, from a single scan, the processor according to the method(s) described is capable of flagging to a dentist a soft tissue recession determination.

[0235] The soft tissue recession determination obtained using the method(s) described herein, may be stored in a storage media, and used for further processing at a later point in time, such as at a second visit, illustrated as point (5) in Figure 25.

[0236] At the second later visit (5), the patient’s oral cavity is scanned by an intra-oral scanner. In this way two intra-oral scan data sets are provided to the computer processor, which may be used to evaluate soft tissue recession over time as will be explained in the following with reference to Figures 9 to 17b.

[0237] Referring to Figure 9, a flow representing the method of determining soft tissue recession from two scans taken at different points in time is illustrated. In a multi-scan solution using for example the output from the machine learning model from the first scan taken at a past point in time, the method comprises: loading into a processor 700 the health state predictions 701 or at least the coronal margin from the first scan, and the second scan data 702 taken at a later point in time than the first scan. Determine 703 from the second scan data at least the gingival margin thereof, wherein the gingival margin represents in the second scan data, a transition between teeth and gingiva, potentially reflecting a soft tissue recession. Determining 704 a middle point of one of the coronal margin or the determined gingival margin for each tooth in the intra-oral scan data, determining a direction of the tooth and determining a center of the tooth, wherein these determinations are used to generating 705 a first plane using the determined middle point, direction, and center. In this way a first reference plane may be generated to assist in determining the soft tissue recession measure as will be explained in the following. From the first plane, the method comprises generating 706 by a processor at least one second plane parallel to the first plane, wherein the at least one second plane is positioned x-distance from the first plane. In other words, the method generates a new "second" plane which is positioned to lie on the next Nth point, from the coronal and / or apical margin from the older scan. This way it is possible to identify a point where the second plane intersects with the margin line from the gingival margin line. In other words, in this way it is possible to identify for the first and second plane a point on the coronal margin and / or apical margin and the gingival margin at which the first and second plane intersects 707. Accordingly, the method comprises determining from the first and second planes intersecting points on the coronal and apical margins. The determined points on the coronal and / or apical margin and the identified gingival margin intersection points, may define the points used for measuring the distance that in term defines the soft tissue recession. Accordingly, if applying multiple set of planes, by updating the second plane position by x-distance from its original position, then multiple set of points may be generated on the coronal and / or apical margin and multiple set of intersection points may be identified on the gingival margin, which in term may define the multiple set of distances with which the recession measure is calculated. In other words, if applying a multiple of planes by updating 708 the second plane position x-distance from the first or second plane generating a new second plane position a multiple of points may be generated on the coronal and / or apical margin and gingival margin line defining the points for which a recession measure is calculated. Accordingly, identifying 709 a point on the coronal margin and gingival margin at which the new second plane position intersects the distance 710 between the identified points may be calculated. Repeating 711 the steps with a plurality of generated planes along the coronal and apical margin for all teeth represented in the intra-oral scan data, a recession measure may be calculated across a plurality of teeth in the intra-oral scan data.

[0238] The distance x at which the second parallel plane is generated and subsequently moved may depend on the number of points which needs to represent the soft tissue recession measure and may be as previously defined. Alternatively, this number may be adjusted or manually set by a dental practitioner depending on how accurate the spacing between the pointcorrespondences is considered to be relevant.

[0239] The above-described process may similarly be used to determine a distance between apical and coronal margin using only a single-scan data.

[0240] The previously described process steps of generating planes of intersection between e.g., the determined gingival margin and the coronal margin acquired from a past scan data, is illustrated in more detail in Figure 10. Here it is seen how a plurality of planes 800a, 800b, 800c, 800d, 800e have been generated across a tooth on both a buccal and lingual side of the tooth. The gingival margin 802 as determined from the second scan and the coronal margin 801 as acquired from the previous first scan machine learning model prediction are also illustrated in Figure 10. As is seen each of the planes 800a, 800b, 800c, 800d, 800e intersects both margins 801, 802 (i.e., the gingival and coronal margin) at a plurality of points, which are determined according to the proposed method.

[0241] As can be seen in Figure 11, the intersecting points 81a, 81b, 82a, 82b, 83a, 83b, 84a, 84b, 85a, 85b may be used to form a distance measure 90 across a tooth, (a plurality of teeth shown in Figure 11), which represent a measure of soft tissue recession in a multi-scan or sing scan using the apical and coronal margin from the health state prediction structure.

[0242] Figure 12 illustrates how such as determination of soft tissue recession may be visualized with the two margins superimposed onto the 3D model.

[0243] The computer-implemented method steps concerned with determining the gingival margin to be able to use it in a multi-scan recession detection will be described in more detail in relation to Figure 13. Here it is illustrated how the intra-oral scan data 901 are loaded into a processor 900, wherein the processor is configured to segmenting 902 the intra-oral data into gingiva and teeth, wherein the segmentation represents the intraoral data as a plurality of connected teeth points assigned 903 a set of tooth values, and a plurality of connected gingiva points assigned 904 a set of gingiva values. In other words, the intra-oral data may be segmented into a plurality of facets (i.e., connected teeth points) assigned a tooth value and a plurality of facets (i.e., connected gingiva points) assigned a gingiva value. This representation may be preferred as the intra-oral scan data may be represented as a mesh. Other possibilities could include a point-cloud representation of the intra-oral scan data, where each point representing tooth may be assigned a tooth value and each point representing gingiva may be assigned a gingiva value.

[0244] To determine the gingival margin from the plurality of connected points of the segmentation, the processor may be configured to determine 905 if a point or which points in the plurality of connected teeth points and gingiva points, each represent (such as comprises) at least one point from the set of tooth values and at least one point from the set of gingiva values and assigning a margin label to the respective points (i.e. those teeth and gingiva points), thereby identifying the points in the intra-oral scan data forming the gingival margin line. At the places where each of the teeth in the oral cavity are neighboring each other, a junction may be identified by the processor by determining if a point in the plurality of connected teeth points and gingiva points represents at least one point from the set of gingiva values and at least one point from at least two different sets of tooth values and then assigning a junction label to the respective points. With this method it is possible to determine the gingival margin line for each tooth and the connection points with neighboring teeth in the 3D intra- oral scan data. In accordance herewith a collection of margin labels and junction labels for each tooth may be determined 906. An example of junction label 905a is illustrated in Figure 13 together with an example of a margin label 905b.

[0245] In an example, the processor may be configured to identify for each tooth in the intraoral scan data, the one or more margin labels 905b for the buccal and / or lingual side of the tooth. In this way, the margin labels may be marked as a buccal and / or lingual margin label ensuring that both sides of the tooth are covered and / or for presentation purposes. Furthermore, in an example, the processor may be configured to identify for each tooth in the intraoral scan data, the one or more junction labels 905a for the buccal and / or lingual side of the tooth. Also, the junction labels 905a may be marked for buccal and / or lingual side to ensure both sides are covered and / or for presentation purposes. Several junction and margin labels may be present, where only one is illustrated in Figure 13 for the sake of providing an example.

[0246] As the input data to the processor, being the intra-oral scan data during processing as previously described may appear in random order, the junction and margin labels may correspondingly be randomly organized. Accordingly, to sort the randomly ordered data points (in the form of margin and junction label) to accurately determine the gingival margin line in relation to the input data, the processor may be configured to sorting the margin and junction labels into a structured order by utilizing a random determined junction label from the junction labels as a starting point. Based on the determined starting point traversing each margin label to determine an edge orientation of the each of the margin labels, wherein the determined edge orientation of the margin label is used to determine the subsequent margin label in the collection of margin labels.

[0247] The sorted margin and junction labels may in an example be configured to represent a collection of margin line splines for each of the teeth in the intraoral scan data. Such a margin line representation is illustrated in Figure 14, where each spline for each tooth is exemplified by 910, 911, 912 as single splines. By having a representation of margin line splines, the processor may be configured to project the margin line spline for each of the teeth into the display of the graphical user interface to provide a visualization of the position of the determined gingival margin line.

[0248] The margin line splines previously described may be configured to be aligned to allow two different margin lines, such as e.g., a gingival margin acquired from one scan to be aligned with e.g., a coronal margin, as acquired using the machine learning model. Figure 15 illustrates how two acquired margin lines 801, 802 (such as gingival and / or coronal) may comprise their own respective coordinate system. To be able to compare the two margin lines, a proper alignment is made, wherein the approach of aligning comprising to transform at least one of the acquired margin lines, to be able to load the transformed margin line to the coordinate system of the none-transformed margin line. The non-transformed margin line could be e.g., the gingival margin line and the transformed margin line could be the coronal margin acquired using the machine learning model.

[0249] When the two margin lines as described above have been aligned, they may preferably be superimposed onto a 3D model of the scan data from which e.g., the gingival margin was acquired. To allow both the gingival margin 802 and coronal margin 801 to be visible on the 3D model from which e.g., the gingival margin 802 was acquired, a second transformation may be performed to the coronal margin 801. The situation is illustrated in more detail in Figure 16, where the two acquired margin lines 801, 802 are shown in a crosssection view 1200a, 1200b, 1200c for three different examples. Each of the cross-sectional views 1200a, 1200b, 1200c has been acquired according to the respective planes 1201a, 1201b, 1201c. In the first example (most left situation 1200a) it is seen how the margin line 801 has substantially hidden inside the tooth of the 3D model. Similarly in the second example (middle situation 1200b), the margin line 801 is hidden inside the tooth of the 3D model. In these situations, the margin line 801 may not show at all points on the 3D model, why the second transformation may be applied. Using the second transformation, the margin line 801 will be shown in the 3D model, as it is transformed into the correct coordinate system of the 3D model. A situation where no transformation is needed is shown in Figure 16, third example (most right situation 1200c). Here it is seen on the cross-sectional view 1200c that the margin lines 801, 802 are accurately aligned. The three situations 1200a, 1200b, 1200c may be taken for the same model (i.e., scan data) and thus the transformation may be performed to only parts of the margin lines or the entire margin line in dependence of an evaluation of each spline of each tooth in the 3D model.

[0250] Results of using a secondary transformation as just described are also shown in more illustrative details in Figure 17a and Figure 17b. Here it may be seen in Figure 17a, how the margin line 801 appears as missing in areas of a tooth 1202. Using the secondary transformation as just described, the resulting snapping of the margin line 801 to the 3D model to allow it to be visible is shown in Figure 17b. Here it is clearly seen that the areas 1202 where the margin line 801 was missing, now contains the margin line 801. Disclosed herein is also a computer program product embodied in a non-transitory computer readable medium comprising computer readable program code configured to be executed by a hardware processor to cause the hardware data processor to perform the methods described herein when the computer readable program code is executed by the hardware data processor.

[0251] Referring to Figure 30 and 31 an example of representing the estimated recession findings from the machine learning processing and subsequent processing steps is illustrated. Alternatives for determining an estimate of gingival recession have been described, where Figure 30 illustrates a representation of the most coronal to most apical point determination. From a clinical perspective, finding the most coronal 1401 and most apical 1402 point to determine a gingival recession measure 1403 is most suitable for non-molar teeth as the area of recession may be less expanded across the tooth width. On the other hand, for molars illustrated in Figure 31, the most relevant determination of a gingival recession measure may be to use at least two points 1404, 1405 on the gingiva to determine the extent of recession. Therefore, the method may in an embodiment be configured to determine if the healthy state output of recession from the machine learning model is associated with a molar type of tooth or a non-molar type of tooth, wherein if a non-molar type of tooth is identified the previously described determination of the most apical and most coronal point may be applied by the method to determine a measure of gingival recession.

[0252] On the other hand, if a molar type of tooth is identified, the method may comprise generating a bounding box of the tooth and associated gingiva and dividing the bounding box into at least three equally spaced sections 1406, 1407, 1408. The equally spaced sections 1406, 1407, 1408 defining at least two computational lines extending through intersection points associated with the apical border of points associated with ideal gingiva and points associated with actual gingiva, and the coronal border of points associated with teeth and ideal gingiva. From the determination of the border points intersecting the at least two computational lines at least two gingival recession measures 1404, 1405 may be determined as the distance between the intersecting points. This determination reflects a more clinically relevant approach of determining gingival recession for a molar tooth. As seen in Figures 30 and 31, the recession measures determined as previously described may be rendered on the 3D model as for example as a line encircling the area of recession, where the encircled area may for example define a recession lesion. In an alternative, only the distances between the intersection points and / or the apical-to-coronal distance may be rendered on the 3D model to reflect the recession measure.

[0253] In an embodiment, the method may comprise determining the bounding box and computational lines as previously described, wherein the bounding box is divided into more than three equally spaced sections. The more sections generated from the bounding box, the more intersection points may be achieved to ensure more local recession measure of a molar tooth.

[0254] As illustrated in Figure 31, the molar tooth may have a recession measure 1409 which is higher than the provided equally spaced intersection lines, why the method may comprise to also determine the most apical and coronal point and evaluate the recession measure for a section based on the most apical and coronal point of that section.

[0255] From the determination of the recession measure according to a distance between two points, a color, pattern or similar overlaying to the model may be generated, wherein a first color, pattern or similar overlay may indicate a first severity of recession, a second color, patter or similar may indicate a second severity of recession and / or a third color, pattern or similar may indicate a third severity of recession. The severity coloring is generated in accordance with a set threshold applied for each of the different severities used to indicate a measure of recession.

[0256] According to examples described herein, the electronic hardware may include microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure. Computer program shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0257] Although some embodiments have been described and shown in detail, the disclosure is not restricted to such details, but may also be embodied in other ways within the scope of the subject matter defined in the following claims. It is to be understood that other embodiments may be utilized, and structural and functional modifications may be made without departing from the scope of the present invention.

[0258] Benefits, other advantages, and solutions to problems have been described herein with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any component(s) / unit(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as critical, required, or essential features or components / elements of any or all the claims or the invention. The scope of the invention is accordingly to be limited by nothing other than the appended claims, in which reference to a component / unit / element in the singular is not intended to mean “one and only one” unless explicitly so stated, but rather “one or more.” A claim may refer to any of the preceding claims, and “any” is understood to mean “any one or more” of the preceding claims.

[0259] It is intended that the structural features of the devices described above, either in the detailed description and / or in the claims, may be combined with steps of the method, when appropriately substituted by a corresponding process.

[0260] As used, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well (i.e., to have the meaning “at least one”), unless expressly stated otherwise. It will be further understood that the terms “includes,” “comprises,” “including,” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It will also be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, but an intervening element may also be present, unless expressly stated otherwise. Furthermore, “connected” or “coupled” as used herein may include wirelessly connected or coupled. As used herein, the term “and / or" includes any and all combinations of one or more of the associated listed items. The steps of any disclosed method are not limited to the exact order stated herein, unless expressly stated otherwise.

[0261] It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" or “an aspect” or features included as “may” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. Furthermore, the features, structures or characteristics may be combined as suitable in one or more embodiments of the disclosure. The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects.

[0262] The claims are not intended to be limited to the aspects shown herein but is to be accorded the full scope consistent with the language of the claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more.

Claims

CLAIMS1. A computer-implemented method for determining and displaying a health state of soft tissue in an oral cavity of a patient, the method comprising: retrieving in a processor intra-oral scan data of a patient, wherein the intra-oral scan data is obtained by an intraoral scanner and comprises a plurality of dental features; generating an input structure by performing a data transformation of the intra-oral scan data from a 3D space to a plurality of descriptive representations of each of the dental features in the intra-oral scan data; feeding the input structure to a trained machine learning model, configured to generate a health state prediction structure representing at least two or more health state(s) of each of the descriptive representation of the intraoral scan data; utilizing the health state prediction structure to generate a 3D representation of the intra-oral scan data with assigned health state labels for each of the dental features; determining a soft tissue recession measure by identifying in the 3D representation the health state labels associated with soft tissue recession by determining from the assigned health state labels a pointset representing soft tissue recession points of each dental feature in the intra-oral scan data; evaluating a measure of the health state labels associated with soft tissue recession, by quantifying the soft tissue recession measure for each dental feature based on the determined pointset; and displaying in a graphical user interface a representation of the determined soft tissue recession measure.

2. The computer-implemented method according to claim 1, wherein the trained machine learning model is trained on the basis of a training dataset of intra-oral scan data comprising a plurality of data samples, wherein each of the plurality of data sample comprises: a 3D model comprising one or more dental features having associated dental feature labels.

3. The computer-implemented method according to claim 1, wherein the pointset represents one or more of an apical margin and / or a coronal margin of the soft tissue for each of the dental feature.

4. The computer-implemented method according to claim 3, wherein determining the soft tissue recession measure comprises measuring a distance between a most apical point of the apical margin and the most coronal point of the coronal margin for each dental feature.

5. The computer-implemented method according to any of claims 1 to 4, wherein the trained machine learning model is configured to output a confidence label vector for each of the dental features, wherein a confidence label falling under a first class represents a high degree of confidence that the health state predictions is accurately detected by the trained machine learning model, and wherein a confidence label falling under a second class represents a low degree of certainty that the healthy state predictions is accurately detected by the trained machine learning model, wherein the processor is configured to utilizing the confidence label to output a recession confidence measure for each dental feature of the intraoral scan data.

6. The computer-implemented method according to claim 3, wherein the method comprises displaying the apical margin and the coronal margin of the health state prediction superimposed onto the 3D representation of the intra-oral scan data in the graphical user interface.

7. The computer-implemented method according to claim 3, wherein the method comprises assigning a coloring to the apical margin, wherein the coloring defines the distance between the most apical point of the apical margin and the most coronal point of the coronal margin for each dental feature falling within one or more set threshold(s).

8. The computer-implemented method according to claim 3, wherein the method comprises projecting the apical margin and the coronal margin onto the 3D representation of the intra- oral scan data, and determining, by the processor, a maximum distance between the apical margin and the coronal margin for each tooth on the 3D model, anddisplaying the maximum distance at a local area of each tooth where the maximum distance is determined.

9. A computer-implemented method for training a machine learning model for predicting a health state of soft tissue in intra-oral scan data, comprising: loading from a storage media a training dataset of intra-oral scan data comprising a plurality of data samples, wherein each of the plurality of data sample comprises: a 3D model comprising one or more dental features having associated dental feature labels; generating a first input structure by performing a data transformation of the plurality of data samples, wherein the data transformation generates a plurality of descriptive representations of each of the dental features of the plurality of data samples; assigning at least one descriptive label to the plurality of descriptive representations, wherein the at least one descriptive label is acquired from the associated dental feature labels; generating a second input structure, wherein the second input structure comprises a feature vector representing an anatomical descriptor of each of the one or more dental features; feeding the first and second input structures to a machine learning model, wherein the machine learning model is configured to process each of the first and second input structures to generate a global structure comprising a processed representation of the first and second input structures, wherein the global structure is configured to be processed by the machine learning model to generate a health state prediction structure for each of the plurality of descriptive representations.

10. Computer-implemented method according to claim 9, wherein the first input structure is configured to be feed to a first neural network architecture of the machine learning model configured to generate a first prediction structure, and wherein the second inputstructure is configured to be feed to a second neural network architecture of the machine learning model configured to generate a second prediction structure.

11. Computer-implemented method according to any of the previous claims, wherein the health state prediction structure comprises a plurality of numerical values for each of the descriptive representations, wherein each of the plurality of numerical values represents two or more health state labels, wherein at least one label represents a healthy state, at least one label represents an initial state and / or at least one label represents an advanced state of soft tissue recession.

12. Computer-implemented method according to any of the previous claims, wherein the health state prediction structure comprises a position label for each for the descriptive representations, wherein each of the position labels represents at least one of a buccal, lingual or inter-dental side identification of the descriptive representations.

13. Computer-implemented method according to any claim 11, further comprising generating a confidence label vector representing for each of the descriptive representations a confidence that the descriptive representation is associated with at least one of the two or more health state labels, wherein the values used to generate the confidence vector is acquired from the dental feature labels; feeding the confidence label vector into the machine learning model.

14. Computer-implemented method according to any of the previous claims, further comprising: calculating a curvature representation of each of the descriptive representations; feeding the curvature representation into the machine learning model.

15. A system comprising a soft tissue recession detection module configured as a computer program forming part of the system, and comprising instructions which, when the program is executed by a computer, cause the computer to carry out any of the method steps. of any of claims 1 to 14.

Images