Caries detection in intra-oral scan data

Abstract

The presented method relates to a lesion detection method utilizing intra-oral scan data of 3D geometries of a mouth. The method presented is configured to detect lesions, such as caries lesions in a discretized 3D geometry generated from the intra-oral scan data, using a trained learning model. Furthermore, the method(s) presented is configured to perform post- processing of output from the learning model to generate accurate and precise lesions detections and to display the detected lesions in the input discretized 3D geometry.

Description

[0001] CARIES DETECTION IN INTRA-ORAL SCAN DATA

[0002] FIELD

[0003] The disclosure relates to methods for detecting caries in intra-oral scan data. Furthermore, the disclosure relates to methods of representing caries lesions detected in intra-oral scan data and to systems configured to perform the methods described herein.

[0004] BACKGROUND

[0005] Oral health conditions in general pose a great health problem across the globe, where most oral health conditions are largely preventable or treatable if observed in the early stages of their development. The prevalence of oral health conditions increases globally, where some of the most occurring conditions include dental caries (tooth decay), periodontal diseases, tooth loss and oral cancers.

[0006] Digital dentistry is becoming increasingly popular and offers several advantages over nondigital techniques. Within digital dentistry it is possible to obtain 3D digital representations of a dentition of a patient, from where changes of the dentition may be assessed over time by for example comparing two models acquired at two different points in time. Assessment of changes in a patient’s dentition over time may be done manually by a dental practitioner for example by assessing a 3D representation of the dentition obtained by a dental scanning system and methods using intraoral scanners or data therefrom. The data may be input to a variety of software solutions, such as patient monitoring systems, dental software systems or similarly developed systems configured to automatically track changes over time. Digital dentistry thus offers solutions for practitioners to easily assess changes in a patient’s dentition over time and to decide on any suitable treatment of the patient.

[0007] An increased focus on providing automated diagnostic solutions is trending within digital dentistry. The automatic diagnostic solutions aim at aiding in detection, classification and / or quantification of a degree, such as a severity of a dental condition present in the oral cavity of a patient. Solutions are similarly aiming at providing a risk assessment and / or scoring of the dental health of a patient. One of the dental conditions in focus is e.g. caries, which is also referred to as tooth decay or cavities. Caries is one of the most common and widespread persistent diseases within dentistry today and is also one of the most preventable.

[0008] The typical development and risk concerned with caries are described shortly in the following. The first sign of dental caries is the identification of occlusal caries, which form on the top-most part of the tooth where food particles repeatedly come in direct contact with the teeth. It’s in this location where bacteria fester and pose a risk to one’s oral hygiene. If the teeth and surrounding areas are not cared for properly, the bacteria will begin to digest the sugars left over from food in the mouth and convert it into acids which may be strong enough to demineralize the enamel on one’s teeth and form tiny holes - causing the first stage of dental caries. As the enamel begins to break down, the tooth loses the ability to reinforce the calcium and phosphate structures of the teeth naturally through saliva properties and, in time, acid penetrates the tooth and destroys it from the inside out. Despite the impact tooth decay may have on one’s teeth if left unattended, dental caries or cavities are largely preventable with a good oral hygiene regimen. Accordingly, to efficiently prevent dental caries development regular dental checkups are needed. The dentist typically looks at the teeth and may probe them with a tool called an explorer to look for pits or areas of damage.

[0009] Such visual inspection requiring a tool often fail to identify cavities when these cavities are just forming, i.e. in their initial stages. Occasionally, if too much force is used, an explorer may puncture the porous enamel. This could cause formation of irreversible cavity formation and allow the cavity-causing bacteria to spread to healthy teeth. Caries that has destroyed enamel cannot be reversed. Most caries will continue to get worse and go deeper. With time, the tooth may decay down to the root which will cause severe discomfort for the patient if not treated. How long this takes varies from person to person and the general level of oral hygiene. However, as previously mentioned caries caught in the very early stages, i.e. initial stages, can be reversed.

[0010] To prevent caries development, the dental practitioner manually from visual inspections looks for early signs of caries manifesting on teeth in the oral cavity. These early signs may comprise white spots indicating that early caries have already created a porous structure in the enamel. In the later stages of early caries development brown and / or black spots may be seen on the teeth in the oral cavity. If detected in due time, potential tooth decay may be stopped. In case the early stage of caries is not actioned upon, the porous tooth structure may collapse creating irreversible cavities in the enamel. Such cavities forming in the teeth can only be treated with fillings typically made of dental amalgam or composite resin. Sometimes bacteria may infect the pulp inside the tooth even if the part of the tooth one may see remains relatively intact. In this case, the tooth typically requires root canal treatment or even extraction of the damaged tooth.

[0011] Accordingly, the development of caries is a process where dental caries may be easily treated if detected early. If undetected and untreated, caries may progress through the outer enamel layer of a tooth into the softer dentin so far as to require extraction of the tooth or to cause inflammation of periodontal tissue surrounding the tooth. An example of the development of caries can be seen in Figure 21, where three points in time (1), (2), (3), is illustrated together with a development of caries 800 on a tooth.

[0012] Digital dentistry solutions to focusing on assessing early caries development have recently appeared to take on the challenges of traditionally, caries detection and monitoring based on visual examination of the teeth alone or aided by a probe (tactile assessment). The visual or visual-tactile assessment is often supplemented by radiographic examination to assess caries lesions on surfaces not directly visible (i.e. approximal surfaces), to investigate the presence of hidden caries lesions, or to assess the severity of caries lesions and their proximity to the pulp in cases where moderate-extensive lesions are identified. Thus, visual inspection and radiographic assessment are considered the common practice caries detection methods in dentistry. These conventional methods are based on qualitative estimation of caries lesions, without an objective quantification of the actual tissue demineralization level. However, the focus of dentistry on minimal and noninvasive treatments requires sensitive enough methods to detect initial enamel demineralization. The subjectivity involved in the conventional methods, together with the low sensitivity of dental radiographs, has led to the development of optical caries detection methods that can potentially contribute to the objective early detection and routine monitoring of caries lesions. Currently, optical caries detection methods have been investigated to support the caries status assessment and monitoring, without the use of ionizing radiation. These methods mainly assess the changes of optical properties in the hard dental tissues and changes of light scattering in the presence of caries lesions. Methods based on fluorescence and tissue transillumination / reflectance are some of the existing optical caries detection methods known. These methods suffer from inaccurate estimates of caries lesions resulting in mistaken detection of fillings, crowns and other non-teeth material as caries lesions, making these methods unreliable. Other methods focus on providing x-ray systems that perform an automatic assessment of caries on intraoral radiographs, which rely on the use of ionizing radiation exposure to the patient. For dentistry experts to trust and apply such objective nonhuman automatic assessments of caries detections as alternatives to visual examination, such digital methods need to provide accurate and trustworthy lesion detection.

[0013] Accordingly, all the existing digital automated solutions for caries detections suffers from inaccurate results causing false-positive detections of caries lesions in teeth and / or suffers from the drawback of using ionizing radiation methodologies which exposes a patient to unnecessary ionizing radiation doses. Furthermore, the existing methods require, due to the false positives, that an expert operating the system, assess the software’s outcome and identify the actual caries lesions among all the identified areas of potential caries, which is time consuming.

[0014] Therefore, new digital methods overcoming these drawbacks are considered in the following, which improves the diagnostic accuracy without using ionizing radiation, improve reproducibility, accuracy and optimize time spent on analyzing the data.

[0015] SUMMARY

[0016] The present disclosure addresses the above-mentioned challenges by providing computer- implemented methods ensuring fast, reproducible, accurate and reliable determination of at least caries lesions in intra-oral scan data. The method comprises obtaining intra-oral surface scan data representing a 3D geometry of at least teeth and gingiva from a patient using an intra-oral scanner. The 3D geometry may be considered as intra-oral features of the mouth comprising at least teeth and gingiva but also other features of the mouth, such as palate, tongue etc. which may be of interest in view of diagnosing a state of the hard and soft tissue of the mouth.

[0017] The method comprises in more detail the steps of:

[0018] - pre-processing the intra-oral scan data to generate a discretization of the 3D geometry, wherein the discretized 3D geometry comprises facet representations of teeth and gingiva;

[0019] - pre-processing the discretized 3D geometry to generate granularity elements representing the facets of teeth, and comprising one or more numerical features representing quantified local information of the facets;

[0020] - inputting the granularity elements to a trained model, wherein the trained model is configured to assigning score probabilities to each of the granularity elements;

[0021] - mapping the assigned score probabilities of the granularity elements to the corresponding facets of the discretized 3D geometry;

[0022] - determining for each facet a condition score based on at least one of the mapped score probabilities;

[0023] - merging facets with nonzero condition score into coherent lesions of the discretized 3D geometry, and

[0024] - assigning a lesion score to each of the coherent lesions according to at least one of the mapped score probabilities associated with facets of the coherent lesions;

[0025] - representing the discretized 3D geometry with the lesions and assigned lesion score in a visual display.

[0026] The condition score may in the context of the application be understood as a caries score and / or any other dental condition score, which may be detected using the method steps described herein.

[0027] The above-mentioned facets may be construed as the cells that the discretized 3D geometry consists of. In one example a facet may be a triangle, whereas in other examples a facet may be a polygon or any other geometrical shape suitable for representing a discretized 3D geometry. Examples of a facet may be triangles comprising vertices and edges, where a vertex connects two edges. A simple point, a hexagon or any other suitable representation that may be used to generate a discretized 3D geometry from intra-oral scan data. The discretized 3D geometry may comprise a plurality of facets, where sets of plurality of facets for example describes e.g. a tooth and / or a gingival part of the discretized 3D geometry.

[0028] To ensure correct and efficient processing of the 3D geometry when detecting lesions of e.g. caries in the intra-oral scan data, the method comprises pre-processing the intra-oral scan data to generate a discretization of the 3D geometry. The discretized 3D geometry may be considered as a 3D model generated of the 3D geometry of teeth and gingiva (and potentially other features of the mouth of a patient).

[0029] As the disclosed method may be based on utilizing a trained learning model, such as a trained machine learning model, to efficiently detect caries lesions in the discretized 3D geometry, the method provides pre-processing steps ensuring that the input data to the trained model is suitable for the trained model to process. Accordingly, the method comprises pre-processing the discretized 3D geometry to generate granularity elements representing the facets of teeth and comprising quantified local information of the facets. This ensures that the trained learning model is provided with discretized feature information suitable for the trained machine learning model allowing efficient processing of the granularity element input data. As will be explained, the trained learning model has been trained on training data comprising similar granularity elements comprises quantified local information explaining features of the granularity elements of the training data.

[0030] The granularity elements may be configured as the smallest elements that possibly can describe the discretized 3D model. In an example, the granularity elements may represent a plurality of points describing teeth and gingiva of the discretized 3D geometry. In another example, the granularity elements may represent pixels, such as pixels of 2D images generated and describing the teeth and gingiva of the discretized 3D geometry.

[0031] The quantified local information may comprise at least one of position (x,y,z) of the granularity element, RGB color information, fluorescence red, blue, green information, surface curvature information and facet normal information. This ensures that local feature information for each granularity element is provided to the trained learning model, allowing the trained learning model to analyze a plurality of feature information associated with a dental condition of e.g. caries to evaluate what features are more likely to be associated with the dental condition and from that analysis, made by the machine learning model, provide assign score probabilities to the granularity elements.

[0032] The quantified local information may be calculated and / or extracted from the discretized 3D geometry and may be stored in the granularity elements. In this way a data structure of granularity elements with associated quantified local information may be generated as an input to the trained model.

[0033] The assigned score probabilities may comprise a vector of probabilities for two or more scores for each granularity element, wherein the two or more scores represent a severity of the dental condition. In case of the dental condition being caries, the severity may be chosen from the group of no caries (N), initial caries (I), moderate caries (M) and / or severe caries (S). For other dental conditions, the severity may be defined in any other suitable manner describing the clinical aspects of that condition. This means that for other dental conditions, less or more severity groups may be defined. For the caries solution described herein, the severity has been chosen to comprise initial, moderate and severe as this follows clinical practice when using visual inspections.

[0034] As described, the trained learning model provides assigned scores to each of the granularity elements. These assigned scores as output may not be easily interpretable by e.g. a dentist using a software system configured to execute the method described herein, why the assigned score probabilities therefore may be mapped back to the facets of the discretized 3D geometry. The method therefore comprises mapping the assigned score probabilities of the granularity elements to the corresponding facets of the discretized 3D geometry. With this mapping it is made sure that facets, representing the discretized 3D geometry, are given the correct score probabilities as output from the trained learning model. It is noted that one facet may be represented by only one or more than one granularity element.

[0035] In the mapping of the score probabilities, one facet may be provided with score probabilities from two or more granularity elements, as a facet may be represented by one or two or more granularity elements. Generally, the mapping of the score probabilities to facets ensures that facets of the discretized 3D geometry are objectively assigned a condition score, such as caries or no-caries score, which is independent from human digital interaction with the data associated with the discretized 3D geometry.

[0036] In cases where two or more granularity elements represent the same facet, the method comprises determining if at least two granularity elements represent one single facet in the discretized 3D geometry. If so, averaging the assigned score probabilities of the at least two granularity elements representing the one single facet, and mapping the average score probabilities to the one single facet.

[0037] In an alternative of mapping the assigned score probabilities to facets, the method may comprise determining if at least two granularity elements represent one single facet in the discretized 3D geometry. If so, assigning a weight to each of the assigned score probabilities of the two granularity elements, and averaging the assigned score probabilities with the assigned weights. The weighted average score probabilities may be mapped to the one single facet.

[0038] The assigned weights to the score probabilities may be determined based on an angle of view from which the facet, represented by the at least two granularity elements, were originally seen. In an example from the view of a virtual camera used to capture 2D images of the 3D geometries. In an example, if the facet is seen from two different views, where one view is less prone to being exposed to a condition, e.g. caries, this one view may cause the score probabilities of the granularity element associated with that view to be given a lower weight than the other view. Alternatively, or in addition to the angle of view, the weights may be based on any of the previously mentioned quantified local information. In an example, the color information, e.g. quantified using R,G,B color information may be used to assign a weight to the granularity elements, or the fluorescence information may be used etc. An alternative to R, G, B color information may be to use hue (H), saturation (S) and value (V) values. Any information which may be more suitable for describing a caries score may be used for the weighting of the granularity elements. The quality information of the scan data used for generating the 3D discretized geometry may also be considered as a basis for weighting the granularity elements when doing the mapping.

[0039] The mapped assigned scores may not necessarily provide the dentist with an easy and clearly interpretable scoring scheme to interpret the output from the trained learning model. Therefore, the method may comprise further processing comprising determining for each facet a condition score based on at least one of the mapped score probabilities. The determining of a condition score may in more detail comprise determining from a vector of probabilities, the score with the highest probability. The vector of probabilities may as previously mentioned be construed as probabilities for two or more scores for each granularity element representing the facets. When the score of the vector of probabilities with the highest probability has been determined, the method may be configured to assign to the facet the score with the highest probability. The score with the highest probability may comprise one of no caries score (N), initial caries score (I), moderate caries score (M) or severe caries score (S). In other words, a condition score may be considered the score assigned to a facet having the highest probability, where the score may represent the dental condition, e.g. no, initial, moderate or severe scoring. As previously mentioned, another number of scores may be applied for other dental conditions than caries. In this way, the determined condition score for a facet represents the highest possible probability of whether the facet is associated with the dental condition or not.

[0040] In an alternative, the method may comprise determining for each facet a condition score by calculating from the vector of probabilities the sum of probabilities represented by the initial caries score (I) moderate caries score (M) and severe caries score (S), and assigning to the facet, the score with the highest probability of the sum of probabilities and the probability of no caries score (N). In this way, the scores representing a caries score may be summed to evaluate if the general probability of caries is higher or lower than the probability of no caries to be assigned to the facet as a condition score. As an additional step, the method may comprise evaluating, for facets associated with a condition score being caries, the probabilities of the initial, moderate and severe score and assign to the facet a condition score representing the highest probability. As an example, if a facet is considered caries from looking at the sum of probabilities of being caries and comparing to the probability of no caries, the score with the highest caries probability may be assigned to the facet. In this way each facet being considered as caries is given an actual score of initial, moderate or severe in accordance with the highest probability thereof.

[0041] In accordance herewith, if the dental condition evaluated by the method described herein is e.g. different from caries (e.g. instead cracks, plaque, wear etc.), the determining for each facet a condition score, may be calculated from the vector of probabilities, by calculating the sum of probabilities represented by the scores associated with a severity score, i.e. a nonzero score, of the vector of probabilities. Then assigning to the facet, the score with the highest probability of the sum of probabilities associated with a nonzero score and the probability of a zero score, indicating no presence of the dental condition.

[0042] As is apparent, the severity associated with a dental condition (e.g. caries, cracks, plaque, recession, wear etc.) may be considered to comprise a nonzero score in the vector of probabilities. On the contrary a zero score probability represents that the dental condition is not considered to be present for that facet.

[0043] The facets with determined condition scores represent in the discretized 3D geometry a plurality of single facets with assigned condition score. To evaluate if the assigned condition score(s) of the facets is inter-connected and provide meaningful interpretation for the dentist, which resembles a clinical visual inspection of a tooth, the method is configured to merge facets together to define coherent lesions. A lesion is the clinically accepted term for a e.g. a caries condition, which appears clinically in lesions (i.e. larger areas instead of merely being represented by single facets in a discretized 3D geometry). In the digital world, it is therefore relevant to evaluate and process the output from the trained model in a manner creating an interpretable output. The facet representation with condition scores provides a localized estimation of which facets is provided a nonzero condition score, but to improve the interpretability of the findings of the trained learning model, the method provides steps of merging facets of nonzero condition scores together to form the previous mentioned coherent lesions. Therefore, the merging of facets with nonzero condition scores into coherent lesions of the discretized 3D geometry may comprise determining groups of interconnected facets comprising nonzero condition scores. Further, adding to the groups, single facets comprising zero condition score that has at least two neighbors with nonzero condition score, and assigning to the single facets the highest condition score of the neighbors. These steps of merging single facets of zero-condition score to groups of interconnected faces may be repeated until reaching a set threshold.

[0044] In an alternative, adding to the groups, single facets comprising zero condition score that has at least two neighbors with nonzero condition score, may comprise assigning to the single facets a score of one of the neighboring scores. This means that the score does not need to be the highest score but may be any of the nonzero scores associated with neighbors to a single facet comprising zero condition score.

[0045] The set threshold may be based on evaluating if no more single facets with zero-condition score is present to add to the groups. Alternatively, the set threshold may be e.g. based on a stop for searching for more single facets with zero-condition score, when e.g. N number of facets may be present to add to the groups.

[0046] The method may be configured to removing small details such as islands and holes smaller than a specific length scale, e.g. 0.01mm, from nonzero condition scores using morphology filtering techniques, e.g. consisting of sequential application of a dilation- and erosion operator.

[0047] As a lesion from a clinical perspective is to have a certain size before being considered a lesion of clinical relevance, the method may ensure that groups of inter-connected facets that are smaller than a set threshold are not considered coherent lesions. Accordingly, the method may be configured to determine if groups of facets consist of less than N-number of inter-connected facets, and if so, removing the groups below the set N-number of interconnected facets. In an example, the N-number of facets of a group may be chosen from 1 to 5 number of facets, preferably less than or equal to 5 number of facets. Another example may be to use a minimal area covered by a coherent lesion as a threshold.

[0048] The coherent lesions as described above may according to the method described herein be assigned a nonzero or zero condition score to evaluate first which facets should be combined to generate the coherent lesions of facets. These nonzero or zero-condition scores assigned to each facet and merged into lesions may thus result in lesions of coherent facets comprising different nonzero condition scores within the lesions.

[0049] The coherent lesions may be given a single score which is representative for the lesion to be able to create an interpretable lesion detection method. Therefore, in the following a lesion score is considered a score of initial, moderate or severe which represents generally the severity associated with a coherent lesions.

[0050] Accordingly, to ensure that the lesions are assigned the most representative lesion score within the merged facets amounting to the coherent lesions, the method may be configured to assign a lesion score to each of the coherent lesions. The assigning according to the method may comprise for each coherent lesion determining the assigned condition scores represented in the coherent lesions and assigning the condition score with the highest severity to the coherent lesion. In other words, determining the facets associated with the coherent lesion and assigning the facet score with highest severity. This assigning of the condition score may be performed according to a set threshold. Accordingly, in one example the method for each coherent lesion determine the assigned condition scores represented in the coherent lesions, and assigns the condition score with the highest severity to the coherent lesion if it fulfills the set threshold.

[0051] In an example, if a coherent lesion is represented by a number of facets having assigned a condition score of initial caries, and another number of facets assigned a condition score of moderate caries, then the moderate condition score would be considered the highest severity for the coherent lesion, and the lesion score given to the lesion is moderate caries. One of the reasons for choosing moderate or higher score of a lesion (if represented in facets of the lesion) is that from a clinical perspective, the dentist may always consider performing a treatment if the lesion is considered to have a moderate score. Thus, the solution provided herein takes into account the clinical aspect and provides a well-interpretable estimate or aid in determining the scores of the lesions of a tooth, where if a lesion is represented by a set threshold of moderate scores (as provided by the facet score), the lesion may be given a moderate score. This ensures that determination of at least moderate or higher scores are always presented to the dentist, to aid the dentist in becoming aware of those lesions.

[0052] In other words, with this, the method may comprise computing lesion score from facet scores by looking at the distribution of facet scores inside the lesions. In an example, the distribution by facet count or distribution by facet area, and then accepting the most severe score above a certain threshold, e.g. 10%, to be assigned to the lesion. In the examples given above, the score probabilities as output from the trained learning model may be used to determining the condition score of a facet, where the further assigning of a lesion score to coherent lesions based on merged facets may rely on the single score given to each of the facet already at the condition score determination step. With this example alternative of the method described herein, the score probabilities may not be considered for any other step than for determining a single facet condition score. These examples provide at least a faster or cleaner processing, as the score probabilities are mainly used in the mapping step, with only a single representative condition score for each facet is brought forward in the method to finally assign a coherent lesion a lesion score.

[0053] In an alternative, the lesion score may be assigned on the basis of the score probabilities assigned to the facets in at least the mapping step of the method, where the score probabilities may be stored for each facet. These stored score probabilities may be retrieved from a memory as part of the assigning lesion score step and used to assign a lesion score to the coherent lesions. With this method, all score probabilities mapped to a facet may be brought forward to the subsequent steps of the method. This ensures that at the level of assigning a lesion score to the coherent lesions, each facet represented in the lesion may comprise more than one single score and the probabilities associated with the more than one single score. This alternative ensures that the score probabilities for a facet is kept and retrievable to process for as long as possible to use the score probabilities at the actual step of determining the lesion scores. The downsides may however be that the score probabilities require processing power or decision rules of assigning lesion scores to process than the alternative of using only single scores as assigned to facets at an earlier level of the processing. The decision rules with this alternative may be required to take into account several score and associated probabilities, whereas the alternative method described above more simplicity determines a lesion score based on the earlier single-valued given condition score of a facet.

[0054] As should be apparent from the examples given so far, the coherent lesions may be assigned a lesion score, which allows to describe the severity of the lesion and accordingly, the severity of the condition associated with the lesion. In the given examples, the lesions could describe dental conditions such as caries lesions, cracks, tooth wear, stains, plaque, cancerous tissue etc. even though detailed examples are mainly provided with respect to caries lesions.

[0055] To ensure that the method processing of the output of the trained learning model facilitates at least an aiding mean for the dentist to determine dental conditions from a discretized 3D geometry, the method may comprise representing the discretized 3D geometry in a visual manner on a computer implemented guided user interface and / or display of a computer. The method may for representing in a visual display comprising assigning a color to each of the coherent lesions according to the assigned lesion score, wherein an assigned lesion score of initial caries is provided in a first color, a lesion score of moderate caries is provided in a second color and a lesions score of severe caries is provided in a third color. With this, the solutions described herein provide not only an accurate estimate of dental caries (or any other suitable dental condition), but also an interpretable and visual representation of the findings from the trained learning model.

[0056] With all of the above identified steps of the method, it is therefore possible to generate a reliable, reproducible and objective caries lesion detection on discretized 3D geometries of intra-oral scan data. The method described herein is human-operator independent as it uses a trained learning model to score granularity elements of a discretized 3D geometry and process such scores to project it back to the discretized 3D geometry in a reproducible manner that does not require human-operator inputs. Furthermore, this method provides the possibility of objectively determining caries lesions in discretized 3D geometry representing the 3D geometry of a patient’s intra-oral cavity without exposing the patient to ionizing radiation. The method works mainly with 3D data and does not require tactile investigation in the mouth to determine caries lesions. The method steps described herein may be configured as instructions of a computer program. Accordingly, the disclosures also provide for a computer program having instructions which when executed by a computing device or system cause the computing device or system to perform the steps of

[0057] - obtaining intra-oral scan data representing a 3D geometry of at least teeth and gingiva from a patient using an intra-oral scanner;

[0058] - pre-processing the intra-oral scan data to generate a discretization of the 3D geometry wherein the discretized 3D geometry comprises facet representations of teeth and gingiva;

[0059] - pre-processing the discretized 3D geometry to generate granularity elements representing the facets of teeth, and comprising one or more numerical features representing quantified local information of the facets;

[0060] - inputting the granularity elements to a trained model, wherein the trained model is configured to assigning score probabilities to each of the granularity elements;

[0061] - mapping the assigned score probabilities of the granularity elements to the corresponding facets of the discretized 3D geometry;

[0062] - determining for each facet a condition score based on at least one of the mapped score probabilities;

[0063] - merging facets with nonzero condition score into coherent lesions of the discretized 3D geometry, and assigning a lesion score to each of the coherent lesions according to at least one of the mapped score probabilities associated with facets of the coherent lesions;

[0064] - representing the discretized 3D geometry with the lesions and assigned lesion score in a visual display. These steps are in accordance with the previous described method steps. Accordingly, the described effect and advantages for the steps described herein are generally the same for the method and the execution by the computer-program product.

[0065] As previously mentioned, the output from the trained learning model may be considered as a vector of score probabilities for each granularity element of the discretized 3D geometry. In an example, the output from the trained learning model may comprise vectors of probabilities provided in e.g. a matrix with probability scores given by e.g. :

[0066] G1 = [pi 1,P12,P13,P14] G2 = [P21,P22,P23,P24], where Gi and G2 represents that granularity elements, and the values pn, pn, pi3, pi4, represents the probabilities for each score associated with granularity element Gl, and values P21, P22, P23, P24 represents the probabilities for each score associated with a second granularity element G2. In other words, Gi represent the prediction of granularity element i and pij represents probability of granularity element i having score j, as given by:

[0067] Gi = [pij, Pij, Pij, Pij]

[0068] With the above given examples in mind, if a granularity element exemplified as Gi = [0.35, 0.3, 0.2, 0.15] is considered, the mapping of assigned scores as explained herein may in one example result in the facet represented by the granularity element G2 to comprise the score probabilities of G2. The determination of a condition score for that facet having G2 score probabilities may then result in a zero-condition score as the score of the score probabilities with the highest value is 0,35 associated with a no caries scores as described herein. In an alternative example, the determination of a condition score for that facet having G2 score probabilities, may comprise summing the score probabilities associated with a nonzero condition score. This would provide a summed score probability of 0,65 for nonzero condition score resulting in that facet being assigned a condition score of nonzero score of caries.

[0069] With these alternatives of determining a condition score of a facet, the method ensures that each facet of the discretized 3D model is given at least single score of caries or no-caries in a substantially binary manner.

[0070] According to the method described herein, a pre-processing step may comprise segmenting the discretized 3D geometry into teeth and gingiva; extracting each of the teeth from the segmentation; and determining and aligning principial axes of each tooth with a common coordinate system for all teeth from the segmentation. With this pre-processing step it is ensured that the granularity elements generated and input to the trained learning model represent each tooth and / or gingiva parts thereof, as each tooth and associated gingiva may be segmented from each other. Furthermore, providing each tooth at a time in the processing steps described herein may optimize processing power, as less data needs to be processed at the same time. With regards to representing the detected lesions in a visual display as previously mentioned, a few examples will be provided in the following. In an example, representing the discretized 3D geometry may comprise visualizing the lesions as filled areas, as outlines or as outlines with transparent filling. Visualizing the lesions as filled areas provides a clear visual overview of the areas that are exposed to e.g. the dental condition being caries. Visualizing the lesions as outlines ensures that the dentist and / or patient is observant of the area when visualized on the discretized 3D geometry while allowing a dentist to look at the actual lesion in its natural appearance as it was captured with the intra-oral scanner. This may allow the dentist to visually confirm the findings of the proposed method described herein. In an example, where the visualization comprises generating both an outline and a filled area, wherein the filled area is configured with a transparency allows the user to obtain a general understand and view of the area affected while allowing visual inspection of the actual tooth with the lesion. In this way, the dentist may be able to assert the lesion both using visual inspection of the discretized 3D geometry through the substantially transparent color covering the area of the lesion.

[0071] In an aspect of the disclosure, the use of a trained learning model to detect caries in intra- oral scan data is following the method steps described herein.

[0072] In accordance herewith, the disclosure also provides a computer-implemented method for training a learning model for use in detection of caries lesions in intra-oral scan data. The method for training a learning model comprises the step of inputting training data to the learning model to train the learning model through machine learning. The training data may comprise discretized 3D geometries having data representing caries lesions, where the data representing caries lesions comprises a score chosen from initial (I), moderate (M) or severe (S).

[0073] The data representing caries lesions may be configured as annotations provided to the training data during an annotation process, which will be explained. To allow training of the learning model in a training phase, the discretized 3D geometries with associated data representing caries lesions and corresponding scores, may be pre- processed in a similar manner as described previously with respect to using the trained model in a system configured to perform a method for detection caries lesions in intra-oral scan data.

[0074] Therefore, the method of training the trained model may comprise the steps of acquiring from a data storage the training data. Pre-processing the training data (i.e. pre-processing each of the discretized 3D geometries forming part of the training data) to generate granularity elements representing the facets of teeth for all of the training data. Each of the granularity elements comprising one or more numerical features representing quantified local information of the facets which the granularity elements represent. Accordingly, the training data is generally processed in the same manner as described in relation to use of the trained learning model.

[0075] The main difference between the data associated with a training phase and the data used in the trained learning model, lies with the training phase being configured to use annotated data to fit a model, where the “true” scores have been added by e.g. human experts. On the contrary, the data used outside of training does not have information on “true” scores. The annotation of data to generate the mentioned ground truth described herein, may be performed in a step prior to the training phase being initiated and subsequently used in the training phase. In accordance herewith the training data may comprise discretized 3D geometries in a predefined dataset along with manual markings (i.e. annotations) on the discretized 3D geometries, where indications of scores associated with the manual markings is applied by one or more human experts. As the discretized 3D geometry as described herein comprises facets, also the markings may be configured as facets.. The annotations in an example may mark an area on a discretized 3D geometry of the training data and associate the area with a score, such as initial caries, moderate caries or severe caries. The training data is used for minimizing the difference between the predictions of the model and the annotations by tuning the free parameters of the model (also referred to as model weights) using gradient based optimization.. To allow generation of a well-trained learning model the training data set may comprise between 500 to 5000 full jaw scans, which may result in 1000 to 10.000 single jaw scans if both jaws a represented. The more scans available for the training phase, the higher accuracy in score probabilities of the model may be expected.

[0076] Further details of the training of the learning model will be elaborated later on.

[0077] Generally, it is considered that the learning model may comprise a single machine learning model, but it may also comprise a plurality of models with the implementations described herein. If several models are used, the output from one model may be used as input into a second machine learning model.

[0078] In an example, the learning model may comprise a first machine learning model configured for classifying facets of the discretized 3D geometry into healthy facets or facets with caries. Healthy facets may have a no caries (N) condition score while the facets with caries may have a nonzero condition score, however unknown which of the initial caries score (I), moderate caries score (M) or severe caries score (S). The learning model may further comprise a second machine learning model configured for assigning one of the severity scores comprising initial caries score (I), moderate caries score (M) or severe caries score (S), to the facets with caries determined by the first machine learning model. The second machine learning model may thus disregard the healthy facets identified by the first machine learning model by processing only facets with caries identified by the first machine learning model. The second machine learning model may thus only keep severity labels on facets identified as having caries. During the training of the second machine learning model only facets manually labelled as having caries may contribute to the loss function.

[0079] A facet may be classified as a healthy facet by the first machine learning model if a probability for this is higher than 50%. For a facet with caries, a most probable severity label identified by the second machine learning model may be assigned.

[0080] An advantage of using the first machine learning model and the second machine learning model in this manner is that a higher focus can be placed on detecting caries by the first machine learning model and thereby a higher detection accuracy can be achieved. The first machine learning model may thus be specialized only to identify caries. Specialized task of classification of detected caries into a severity category is thereby left to the second machine learning model.

[0081] Both the first machine learning model and the second machine learning model may be of the same type and of the same architecture which is described throughout the disclosure. The two machine learning models may be trained for these different tasks using training data annotated manually. The training data is described further with respect to the trained model.

[0082] In an example, the previously mentioned segmentation to allow segmentation of teeth from gingiva may be based on a trained learning model, and the results of this segmentation trained model can be considered as an input for the lesion detection trained model.

[0083] As previously mentioned, the training data comprise score data (also explained as annotations), which have been collected by manual annotations or semi-automatically by an automatic collection of annotations. One main bottleneck within machine learning is associated with the desire to have a large amount of training data, and at least a semiautomated annotation of data may be considered herein.

[0084] An example of at least a semi-automated data annotation, a crowdsourced annotation process comprising a usage data collection or gamification setup may be considered. The learning or gamification setup may present automatically to the user of the learning or gamification setup a lesion detection in discretized 3D geometries in the form of e.g. marked or outlines areas of different lesion severities. The learning and gamification setup may be configured to allow a user to modify and / or edit the detected lesions in a visual display and to store the changes made to the automatically detected lesions. These gamified or learning results may be stored in a database, locally and / or in a cloud, which may be accessible by a training module, wherein the training module is configured to collect the data resulting from the gamification or learning setup. With such usage data collection or gamification module, for example forming part of a piece of software provided to dentist, it is possible to allow collection of annotated data in a fast and reliable way. With this method it is also possible to allow experts in dentistry to evaluate the finding of an automatic lesion detection and to identify falsely labeled lesions and or change the size of the lesions allowing the machine learning models to be re-trained or fine-tuned on new or updated data thereby potentially improving accuracy of prediction. The process may involve a continuous integration of new data and updating of the machine learning models. .

[0085] In addition, the gamification or usage data collection setup may also allow experts within dentistry to re-score wrongly scored lesions and provide another annotation, such as crowns, fillings, cracks or other types of e.g. restorations to the lesion instead. This may provide effective input to train a learning model on these types of condition or none-condition areas and generate a dataset representing potentially all dental features associated with an oral cavity. Such data collection would allow to build a large data set with different types of dental features annotated and would allow more accurate training processes, as the combining of annotations from more dental indications may lead to a larger dataset. In an example, this would result in 1 large dataset rather than for example 3 dataset for 3 different conditions. With such a method, it may be possible to fine tune an already trained model focusing on a single condition to re-focus the training to e.g. another dental condition. .

[0086] Accordingly, the annotated data, whether being collected for caries, cracks, crowns, fillings, plaque, tooth wear, recession or any other dental condition may with these smart annotation methods be collected in a faster and more efficient manner allowing development time optimization of new trained learning models corresponding to the ones described herein.

[0087] It should be noted that even though the method(s) described herein is mentioning mainly caries lesions, any other possible dental condition, such as cracks, plaque, tooth wear, recession, or any other dental restoration or condition may be detected with a similar method as described herein. That is, the findings of lesions described herein are also intended to cover at least cracks, tooth wear, recession, plaque etc.

[0088] BRIEF DESCRIPTION OF THE FIGURES Examples of the disclosure may be best understood from the following detailed description taken in conjunction with the accompanying figures. The figures are schematic and simplified for clarity, and they just show details to improve the understanding of the claims, while other details are left out. Throughout, the same reference numerals may be used for identical or corresponding parts. The individual features of each example may each be combined with any or all features of the other examples. Combinations, features and / or technical effects will be apparent from and elucidated with reference to the illustrations described hereinafter in which:

[0089] Figure 1 illustrates an example of a scanning system according to the disclosure;

[0090] Figure 2 illustrates a computing system comprising processing means configured to execute a program;

[0091] Figure 3 illustrates an example general workflow for lesion detection using a learning model according to the disclosure;

[0092] Figure 4 illustrates an exemplified illustration generation of granularity element to an exemplified post-processed output from the trained learning model;

[0093] Figure 4a illustrates exemplified facet representation of teeth and granularity element structure;

[0094] Figure 5a illustrates example of the output from the learning model;

[0095] Figure 5b illustrates an example of the output form the learning model;

[0096] Figure 5c illustrates an example post-processing of the output form the learning model; Figure 5d illustrates an example post-processing of the output form the learning model; Figure 5e illustrates an example post-processing of the output form the learning model; Figure 6a illustrates the merging process according to the method described;

[0097] Figure 6b illustrates an exemplified result of the merging process according to Figure 6a;

[0098] Figure 6c illustrates an output from the learning model, with different scores assigned to facets exemplified in two different gray tones;

[0099] Figure 6d illustrates a step of the merging process to create coherent lesions;

[0100] Figure 6e illustrates a step of the merging process to create coherent lesions;

[0101] Figure 6f illustrates coherent lesions generated as a result of the merging process;

[0102] Figure 7 illustrates an example of assigned condition scores of facets;

[0103] Figure 8 illustrates processing steps of the 2D snapshots approach; Figure 9 illustrates an example pre-processing according to a learning model utilizing a 2D snapshot approach;

[0104] Figure 10 illustrates processing steps associated with utilizing a learning model based on the 2D snapshot approach;

[0105] Figure 11 illustrates pre-processing steps according to the disclosure and in correspondence with the pre-processing illustration of Figures 9 and 10;

[0106] Figure 12 illustrates exemplified process steps associated with a learning model utilizing the 3D point cloud;

[0107] Figuerel3 illustrates exemplified post-processing results according to the learning model of Figure 14 illustrates a caries detection program in a visual display of a diagnostic module according to the disclosure and a result of a caries detection program;

[0108] Figure 15 illustrates and example of representation of the caries lesions detected in a visual display;

[0109] Figure 16 illustrates a comparison between lesions detected at a first point in time and a second point in time;

[0110] Figure 17 illustrates an example of the ground truth annotation according to the disclosure;

[0111] Figure 18 illustrates an example of a prior art caries detection;

[0112] Figure 19 illustrates an example of caries as detected in accordance with the method(s) described herein;

[0113] Figure 20 illustrates a workflow according to a clinical assessment of e.g. caries; and Figure 21 illustrates a caries development over time in a tooth;

[0114] Figure 22 illustrates the learning model utilizing the 3D point cloud, where the learning model comprises two separate machine learning models.

[0115] DETAILED DESCRIPTION

[0116] The detailed description set forth below in connection with the appended drawings is intended as a description of various examples according to the disclosure. The detailed description includes details for the purpose of providing a thorough understanding of various concepts and examples covered throughout the description. However, it will be apparent to those skilled in the art that these concepts and examples may be practiced without the specific details mentioned or in combination with one or more examples described herein. Several examples of the devices, systems, mediums, programs and methods are described by various modules, components, steps, processes, algorithms, etc. Depending upon particular application, design constraints or other reasons, these elements may be implemented using electronic hardware, computer program, or any combination thereof. In the following several examples of the methods and system described herein will be disclosed in more detail.

[0117] As previously mentioned, the solutions presented herein, generally aims at improving the automatic detection of a dental condition, such as caries, cracks, recession, wear, plaque or other relevant restorative elements forming part of an oral cavity of patient. These dental conditions may when detected be described as lesions. In the following descriptive examples of a method for detecting caries lesions in a dental cavity is described. However, the method may be expanded to include any other of the mentioned conditions.

[0118] With reference to Figure 3, and as previously mentioned, the caries lesion detection method

[0119] 200 described herein comprises obtaining intra-oral surface scan data 11 representing a 3D geometry of at least teeth and gingiva from a patient 12 using an intra-oral scanner 2. Preprocessing 210 the intra-oral scan data 11 to generate a discretization 207 of the 3D geometry, wherein the discretized 3D geometry 207 comprises facet representations of teeth

[0120] 201 and gingiva 202. Pre-processing 210 the discretized 3D geometry 207 to generate granularity elements representing the facets of teeth 201 and comprising one or more numerical features representing quantified local information of the facets. Inputting the granularity elements to a trained model 220, wherein the trained model 220 is configured to assign score probabilities to each of the granularity elements. Furthermore, the method comprises in one or more post processing steps 230, at least mapping the assigned score probabilities of the granularity elements to the corresponding facets of the discretized 3D geometry. A further post-processing step 230 may comprise determining for each facet a condition score based on at least one of the mapped score probabilities and merging facets with nonzero condition score into coherent lesions of the discretized 3D geometry. The lesions may in a further post-processing 230 be assigned a lesion score to each of the coherent lesions according to at least one of the mapped score probabilities associated with facets of the coherent lesions. Finally, the method comprises representing the discretized 3D geometry with the lesions and assigned lesion score in a visual display. As is noted, not all steps are presented in detail in Figure 3, where some steps in the following will be explained as part of the pre-processing 210 and some steps will be elaborated on as part of post-processing steps 230.

[0121] Intra-oral scanner

[0122] As is apparent, the method comprises obtaining intra-oral surface scan data 11, which is acquired using an intra-oral scanner 2. The intra-oral scanner may in an example be described as a dental scanning device 2, as illustrated for example in Figure 1. Here the dental scanning device 2 may be configured as a scanning device 2 for providing extra-oral scan data and / or intra-oral scan data. The scanning device may e.g. be an intraoral scanning device such as the TRIOS series scanners from 3 Shape A / S. The dental scanning device 2, may include a wireless capability as provided by a wireless network unit. Scanning device 2 may employ a scanning principle such as triangulation-based scanning, confocal scanning, focus scanning, ultrasound scanning, stereo vision, structure from motion, optical coherent tomography OCT, or any other scanning principle. During scanning, the scanning device 2 is generally moved and angled relative to the dentition (i.e the intra-oral cavity), such that at least sets of sub-scans overlap at least partially, in order to enable reconstruction of the digital dental 3D model by stitching overlapping subscans together in real-time and display the progress of the virtual 3D model on a display as a feedback to the user. The result of stitching is the digital 3D representation (also denoted the discretized 3D geometry) of a surface larger than that which can be captured by a single sub-scan, i.e. which is larger than the field of view of the 3D scanning device. Stitching, also known as registration and fusion, works by identifying overlapping regions of 3D surface in various sub-scans and transforming sub-scans to a common coordinate system such that the overlapping regions match, finally yielding the digital 3D model. An Iterative Closest Point (ICP) algorithm may be used for this purpose. Another example of a scanning device is a triangulation scanner, where a time varying pattern is projected onto the dental object and a sequence of images of the different pattern configurations are acquired by one or more cameras located at an angle relative to the projector unit.

[0123] The scanning device 2 may be a handheld intraoral scanner that is configured to be handled by the hand of a user during scanning of a patient. The ergonomic design of the scanning device, i.e. a housing of the scanner, is designed such that a single hand is able to hold the scanner during a scanning session.

[0124] Color texture of the dental object may be acquired by illuminating the object using different monochromatic colors such as individual red, green and blue colors or by illuminating the object using multi chromatic light such as white light. A 2D image may be acquired during a flash of white light.

[0125] More generally, scanning device 2 comprises one or more light projectors 3 configured to generate an illumination pattern to be projected on a three-dimensional dental object during a scanning session. The light projector(s) preferably comprises a light source, a mask having a spatial pattern, and one or more lenses such as collimation lenses or projection lenses. The light source may be configured to generate light of a single wavelength or a combination of wavelengths (mono- or polychromatic). The combination of wavelengths may be produced by using a light source configured to produce light (such as white light) comprising different wavelengths. Alternatively, the light projector(s) may comprise multiple light sources such as LEDs individually producing light of different wavelengths (such as red, green, and blue) that may be combined to form light comprising the different wavelengths. Thus, the light produced by the light source may be defined by a wavelength defining a specific color, or a range of different wavelengths defining a combination of colors such as white light. In an embodiment, the scanning device comprises a light source configured for exciting fluorescent material of the teeth to obtain fluorescence data from the dental object. Such a light source may be configured to produce a narrow range of wavelengths. In another embodiment, the light from the light source is infrared (IR) light, which is capable of penetrating dental tissue. The light projector(s) may be DLP projectors using a micro mirror array for generating a time varying pattern, or a diffractive optical element (DOF), or backlit mask projectors, wherein the light source is placed behind a mask having a spatial pattern, whereby the light projected on the surface of the dental object is patterned. The back-lit mask projector may comprise a collimation lens for collimating the light from the light source, said collimation lens being placed between the light source and the mask. The mask may have a checkerboard pattern, such that the generated illumination pattern is a checkerboard pattern. Alternatively, the mask may feature other patterns such as lines or dots, etc.

[0126] Scanning device 2 preferably further comprises optical components for directing the light from the light source to the surface of the dental object. The specific arrangement of the optical components depends on whether the scanning device is a focus scanning apparatus, a scanning device using triangulation, or any other type of scanning device. A focus scanning apparatus is further described in EP 2 442 720 Bl by the same applicant, which is incorporated herein in its entirety.

[0127] The light reflected from the dental object in response to the illumination of the dental object is directed, using optical components of the scanning device, towards the image sensor(s) 4. The image sensor(s) 4 are configured to generate a plurality of images based on the incoming light received from the illuminated dental object. The image sensor may be a highspeed image sensor such as an image sensor configured for acquiring images with exposures of less than 1 / 1000 second or frame rates in excess of 250 frames pr. second (fps). As an example, the image sensor may be a rolling shutter (CCD) or global shutter sensor (CMOS). The image sensor(s) may be a monochrome sensor including a color filter array such as a Bayer filter and / or additional filters that may be configured to substantially remove one or more color components from the reflected light and retain only the other non-removed components prior to conversion of the reflected light into an electrical signal. For example, such additional filters may be used to remove a certain part of a white light spectrum, such as a blue component, and retain only red and green components from a signal generated in response to exciting fluorescent material of the teeth.

[0128] The intra-oral scan data may be processed by a processor configured to generate scan data (such as extra-oral scan data and / or intra-oral scan data) by processing the data images acquired by the scanning device. The processor may be part of the scanning device. As an example, the processor may comprise a Field-programmable gate array (FPGA) and / or an Advanced RISC Machines (ARM) processor located on the scanning device. The scan data comprises information relating to the three-dimensional dental object. The scan data may comprise any of: 2D images, 3D point clouds, depth data, texture data, intensity data, color data, and / or combinations thereof. As an example, the scan data may comprise one or more point clouds, wherein each point cloud comprises a set of 3D points describing the three- dimensional dental object. As another example, the scan data may comprise images, each image comprising image data e.g. described by image coordinates and a timestamp (x, y, t), wherein depth information can be inferred from the timestamp. The image sensor(s) of the scanning device may acquire a plurality of raw 2D images of the dental object in response to illuminating said object using the one or more light projectors. The plurality of raw data images may also be referred to herein as a stack of 2D images. The 2D images may subsequently be provided as input to the processor, which processes the 2D images to generate scan data. The processing of the 2D images may comprise the step of determining which part of each of the 2D images are in focus in order to deduce / generate depth information from the images. The depth information may be used to generate 3D point clouds comprising a set of 3D points in space, e.g., described by cartesian coordinates (x, y, z). The 3D point clouds may be generated by the processor or by another processing unit. Each 2D / 3D point may furthermore comprise a timestamp that indicates when the 2D / 3D point was recorded, i.e., from which image in the stack of 2D images the point originates. The timestamp is correlated with the z-coordinate of the 3D points, i.e., the z-coordinate may be inferred from the timestamp. Accordingly, the output of the processor is the scan data, and the scan data may comprise image data and / or depth data, e.g. described by image coordinates and a timestamp (x, y, t) or alternatively described as (x, y, z). The scanning device may be configured to transmit other types of data in addition to the scan data. Examples of data include 3D information, texture information such as infra-red (IR) images, fluorescence images, reflectance color images, x-ray images, and / or combinations thereof.

[0129] Dental scanning system

[0130] The intra-oral scanner previously described may form part of a dental scanning system 1 as illustrated in Figure 1. The dental scanning system is configured for scanning an intraoral object (such as a dental arch and the teeth, gum and other intraoral object thereof) of a patient and / or determining a health-condition and / or a probability thereof based on scanning of the intraoral object. The dental scanning system 1, as illustrated in Figure 1, comprises the previously described intra-oral scanner 2 (i.e. the scanning device 2). A data processor 6 of the scanning device 2 or a computer 10 of the system 1 may be configured to generate a 3D model 7 (also denoted a discretized 3D geometry 207 in Figure 3) of the intraoral object, also defined as 3D geometry, of the patient. Further, the scanning system 1 may be configured to execute a diagnostic module comprising computer executable instruction(s) configured to identify a diagnostic feature (such as detecting a dental condition, providing an oral health evaluation etc.) based on the data information provided in the discretized 3D geometry representing the 3D geometry of teeth and gingiva of a patient. As seen in Figure 1, the dental scanning system comprises a computing system 10 configured with a visual display 8 allowing rendering and representation of the discretized 3D geometry 7, 207.

[0131] The computing system 10 of the dental scanning system 1, may as illustrated in Figure 2 comprise one or more processors 110 configured to process executable instructions e.g. provided by a computer program. Processor 110 may comprise a graphics processing unit 112 and / or be configured as a central processing unit (CPU). Computing system 10 further comprises a display device 108, a computer keyboard, touchpad, a computer mouse or touchscreen for entering data and activating virtual buttons (user interaction elements) visualized on the visual display unit 108. The visual display unit 108 may be a computer screen, a touchpad screen or e.g. a smart phone screen comprising a graphical user interface 109 and having a visual display 8, wherein the discretized 3D geometry 7, 207 and e.g. a health-condition detected is displayed. Further computing system 10 may comprise a storage media / medium 130 configured to store data 131, such as the scan data 11 acquired from the scanning device 2 during a scan session. Scan data 11 from a plurality of different scans may be stored in storage 130. Furthermore, scan data 11 that has been processed for using e.g. the diagnostic module and lesion detection method described herein may be stored in storage 130. Patient specific identification data, diagnostic data acquired from other scanning modalities than an intra-oral scanner, and other patient relevant information may be stored in storage 130. The storage media / medium 130 may be configured as cloud storage or for example storage on multiple computer services, which are configured to communicate with each other over a network 150. Processing and storage of data relevant for analysis by e.g. a diagnostic module may be performed in a cloud setup and loaded into a computer therefrom and / or performed locally. As seen in Figure 2, the storage 130 may also store 3D model representations (i.e. discretized 3D geometries 7, 207) generated historically for a patient, as well as 3D model representations 7, 207 generated during a clinical visit. Furthermore, Figure 2 illustrates an example of a diagnostic application module or program 140 configured as a caries detection program 140, which comprises executable instructions for detecting e.g. caries lesions in discretized 3D geometries 7, 207. As illustrated in Figure 2, the diagnostic caries detection program 140 may be configured with a 3D representation receipt module 141 configured to obtain discretized 3D geometries 207, 7 from image data 21 of an imaging device, such as the intra oral scanner 2. A learning model input generation module 142 is configured to pre-process the discretized 3D geometries to generate granularity elements representing the facets of teeth and comprising one or more numerical features representing quantified local information of the facets. The machine learning module 143 is configured to receive the granularity elements, wherein the trained model forming part of the machine learning module 143, is configured to assigning score probabilities to each of the granularity elements input thereto. A post-processing module 144, 230 is configured to perform one or more post-processing steps to the output from the trained learning model 220 configured to form part of the machine learning module 143. The learning module may be configured to mapping the assigned score probabilities to facets of the discretized 3D geometry, as will be elaborated on. The post-processing module may be configured with executable instructions generally performing one or more steps of determining for each facet a condition score and merging facets with nonzero condition score into coherent lesions, as well as assigning a final lesion score to each of the coherent lesions. The display module 145 may be configured to representing the discretized 3D geometry with the lesions and assigned lesions score in the graphical user interface 109 of the display device 108.

[0132] Referring to Figure 3, the caries detection program described in relation to Figure 2 and configured to execute the method described herein may be activated via graphical user interface 109 of the display device 108. The general workflow for using the lesion detection program (for example caries detection program) using a trained learning model is exemplified in Figure 3. Here, the general workflow comprises obtaining intra-oral surface scan data representing a 3D geometry of at least teeth and gingiva from a patient using an intra-oral scanner 2. Loading into processor(s) 210, 220, 230 the obtained scan data 11 and pre-processing the intra-oral scan data 11 to generate a discretized 3D geometry 207 of the scan data. Upon interaction with a user interface element 80, an activation of the previously described caries detection program 140 is achieved. This causes the caries detection program 140 to analyze the discretized 3D geometry 207 using the caries detection program 140. As an alternative to a human-activation of the caries detection program, using the user interaction element 80, the computer system described herein may automatically run the caries detection program on new scan data 11 input to the system, meaning that no active activation of the user interaction element 80 may be needed. To avoid unnecessary repetition, reference is made to the general method steps of the method as already described to form part of the method and the caries detection program 140.

[0133] It is noted that the trained learning model 220 may be stored locally on a computer in a clinical setup and / or may e.g. be stored in cloud service (i.e. network 150 of Figure 2), which is configured to receive the discretized 3D geometry 207 upon activation of the caries detection program or automatically upon receival of new image data 21 from the scanner 2. In the cloud situation, the trained learning model is configured process the discretized 3D geometry 207 in the cloud and transmit the results back to the local computer. The results may be considered one or more of the results as performed during post-processing. In this way, a lot of the heavy data processing may be performed on a cloud service rather than locally on the computer in a dental clinic. This may aid in providing a faster detection of lesions in the discretized 3D geometry and thereby more complex and potentially more accurate models.

[0134] From the output from the trained learning model, one or more post-processing steps ensure that the score probabilities assigned to a granularity element is processed so as to be visually represented in the graphical user interface 109 in an interpretable manner. In Figure 3, an example is provided as a caries score box 240, where the caries scores associated with a tooth 201a of the discretized 3D geometry 207 is provided in box 240. The caries score box 240 as illustrated may provide a representation of the score (e.g. initial, moderate severe) associated with a tooth, a confidence value representing information regarding how certain the trained model is that the score is the given score, and one or more of no. of lesions for that tooth and associated scores and / or area of lesions, distributions of lesions with certain scores etc. Turning now to Figure 4, an exemplified illustration generation of granularity element to an exemplified post-processed output from the trained learning model is illustrated. As seen in Figure 4, discretized 3D geometry 207 is pre-processed 210 to generate granularity elements 211 representing facets of at least teeth 201 of the discretized 3D geometry 207. The granularity elements 211 may be generated in different ways depending on the trained learning models used to process the discretized 3D geometries. The different ways of generating the granularity elements 211 will be elaborated on later. The granularity elements 211 are processed by the trained learning model 220 which is configured to assign score probabilities to each of the granularity elements. The assigned score probabilities are processed to project back to the teeth 201 of the discretized 3D geometry 207, the caries lesions 231a, 231b, 231c, 23 Id determined using the trained learning model 220 and processing 230 thereof.

[0135] Turning to Figure 4, the granularity element structure, exemplified by three granularity elements 211a, 211b, 211c will be described in more detail. As previously mentioned, and illustrated in Figure 4a a tooth 201a, 201b, 201c of the discretized 3D geometry 207, each tooth 201a, 201b, 201c, is represented by facets illustrated as triangles of the teeth 201a, 201b, 201c in the given example. It should be understood that other possible geometries suitable for representing a discretized 3D geometry may be considered a facet in the current description. To generate the granularity elements, each of the teeth of the discretized 3D geometry are pre-processed to allow a facet to be represented by at least one granularity element, provided as XIA, X2A, XSA, for tooth 201a, XIB, X2B, XSB, for tooth 201b, and Xie, X2B, Xsc, for tooth 201c in an exemplification. Each of these granularity elements comprises one or more numerical features 212p, 212c, 212f, 212s, 212n representing quantified local information of the facets. The quantified local information may comprises at least one of position (x,y,z), 212p of the granularity element, RGB color information 212c, fluorescence red, blue, green information 212f, surface curvature information 212s and facet normal information 212n. The quantified local information may be calculated from the discretized 3D geometry and / or read directly from the raw image data associated with the discretized 3D geometry. As is apparent from Figure 4a, the granularity element may be considered to represent, and quantified local information associated with that facet. Figure 4a only exemplifies the granularity element structure to be a single e.g. point or pixel per facet.

[0136] However, a facet may be represented by more than one granularity element.

[0137] As illustrated in Figure 4a, the granularity elements are configured to be input to the trained learning model 220.

[0138] Turning now to Figure 5a, an exemplified output from the trained model 220 is illustrated as score probabilities PXIA, PX2A and PXSA assigned to granularity elements XIA, X2A XSA, representing e.g. a tooth A. The assigned score probabilities PXIA, PX2A and PXSA comprises a vector of probabilities for two or more scores for each granularity element XIA, X2A X3A, wherein the two or more scores represent a severity of the dental condition. In case of the dental condition being caries, the severity may be chosen from the group of no caries (N), initial caries (I), moderate caries (M) and / or severe caries (S), as illustrated in Figure 5a.

[0139] The score probabilities exemplified in Figure 5a and Figure 5b is according to the method described herein mapped back to the facets of the discretized 3D geometry utilizing one or more processing steps which will be explained in the following.

[0140] The first processing steps applied to the output of the trained learning model 220 comprise mapping the assigned score probabilities PXIA, PX2A and PXSA of the granularity elements to the corresponding facets XIA, X2A, XSA of the discretized 3D geometry. This process of mapping ensures that each facet representing the discretized 3D geometry is given the correct score probabilities as output from the trained learning model. This is exemplified in for example Figure 5a. Here it is seen how the machine learning model 220 outputs score probabilities PXIA, PX2A and PXSA , which are considered to be mapped to facets XIA, X2A, XSA of tooth A in a mapping step 236. The mapping may be substantially a 1 :1 mapping between granularity elements and facets of a tooth (Figure 5a) or may take into count if two granularity elements represent the same facets as illustrated in Figure 5b. Here the granularity elements X2B and X3B both represent facet 213 of tooth B.

[0141] In the case illustrated in Figure 5b, where at least two granularity elements X2B and X3B represent the same facet 213, the method comprises determining if the at least two granularity elements X2B and X3B represent one single facet 213 in the discretized 3D geometry. If so, the method is configured to perform an averaging of the assigned score probabilities of the at least two granularity elements X2B and X3B representing the one single facet 213. Based on this averaging process, the method is configured to map the averaged score probabilities to the one single facet 213. In the context of the previous description, and taken into consideration the simplified example of Figure 5b, the averaged score probabilities mapped to the single facet 213 may be:

[0142] (PX2B + PX3B) / 2 = PX23B = [0.225 0.25 0.25 0.275]

[0143] In an alternative of mapping the assigned score probabilities to facets, the method comprises determining if at least two granularity elements represent one single facet in the discretized 3D geometry as just explained in relation to Figure 5b, In this alternative, a weight is assigned to each of the assigned score probabilities of the two granularity elements, and then averaging the assigned score probabilities. The weighted average score probabilities may then be mapped to the one single facet. As an example, the alternative provides the calculation of:

[0144] (W1 * PX2B +W2 * PX3B) / (Wl +W2)=PX23B where wi and W2 represents the assigned weights. As previously described, the weights may be determined based on an angle of view from which the facet, represented by at least two granularity elements, were originally seen. In an example, if the facet is seen from two different views, where one view is less prone to be exposed to a condition, e.g. caries, this one view may cause the score probabilities of the granularity element associated with that view to be given a lower weight than the other view. Alternatively, or in addition to the angle of view, the weights may be based on any of the previous mentioned quantified local information. In an example, the color information, R,G,B color information may be used to assign a weight to the granularity elements, or the fluorescence information may be used etc. Any information which may be more suitable for describing a caries score may be used for the weighting of the granularity elements. Quality information of the scan data used for generating the 3D discretized geometry may also be considered as a basis for weighting the granularity elements when doing the mapping.

[0145] As mentioned, the result of the just described step of mapping might not provide the most suitable data to present to a dentist interested in evaluating the lesions of a tooth, why further processing steps are performed according to the method described herein. Thus, to ensure proper representation of the lesions, the post-processing 230 of determining a condition score, finding coherent lesions and determining a lesions score will be described in more detail.

[0146] With reference to Figure 5c an example of, the determination of a condition score will be explained. As described, the score probabilities mapped 236 to facets representing teeth comprises a vector of probabilities for scores describing no caries (N), initial caries (I), moderate caries (M) or severe caries (S). To determining a condition score for a facet, the method in one example, with reference to Figure 5c, comprises determining 237 from the vector of probabilities, the score with the highest probability. With reference to Figure 5c, if looking at the vector of probabilities PXIA, PX2A and PXSA of tooth A, now mapped to facets as explained in relation to Figure 5a, the score with the highest probability represented by XIA, X2A and XSA would all be severe (S) caries. The method as described here would therefore in the example shown in Figure 5c assign to the facets represented by XIA, X2A and X3A, the score severe caries score (S) as this is the score of highest probability. This means that the mentioned facets of tooth A in Figure 5c, would in the determining step 237 be given a severe caries score as illustrated in tooth A.

[0147] In an alternative, the illustrated in Figure 5d, the determination 237 of a condition score may comprise calculating from the vector of probabilities PXIA, PX2A and PXSA the sum of probabilities represented by the initial caries score (I), moderate caries score (M) and severe caries score (S), and assigning to the facet the score with the highest probability of the sum of probabilities and the probability of no caries score. In this case the determination step 237 as an alternative to determination step of Figure 5c, would comprise calculating the sum of probabilities associated with a caries score which in the example given for e.g. facets represented by XIA, X2A and X3A as: PXIA = [0.1 sum(0.2 + 0.3 + 0.4)] = [0.1 0.9] which would result in the facets represented by XIA, X2A and XSA in tooth A of Figure 5d being given a general caries score (C) as the probability of caries independent of the severity is higher for caries than no caries. Accordingly in this example the actual severity associated with a facet may not need to be given, with a note to the method being able to store the score probabilities to be able to assign the severity at a later stage.

[0148] In another simplified example illustrated in Figure 5e, if using the probabilities of the example given in Figure 5b, where Px2B= [0.35 0.3 0.2 0.15], the summing approach just described would result in Px2B= [0.35 sum(0.3 + 0.2 + 0.15)] = [0.35 0.65] which would assign to facet 214 of tooth B a condition score of caries (C). However, in the other alternative described in relation to Figure 5c, the facet 214 of tooth B would be assigned none-caries (N) score. The difference between the two approaches is in Figure 5d represented with N / C, where N denoted the none-caries score and C denoted the caries score.

[0149] In yet another alternative, the method is to apply a set threshold to assign from the score probabilities the condition score of a facet. In this case the method may be configured to assigning most severe score which is associated with a probability of above for example 10%. Such thresholds may have different values for different scores, different teeth, different surfaces and may be tuned to achieve best performance for a given dataset. In the example given on Figure 5d, the score probabilities PX2B given would result in extensive caries condition score for facet 214 because the probability of severe caries is 0.15, which is above an example set threshold of 0.1.

[0150] In any of the just described examples, the condition score describes at least if caries is present or not and does not necessarily need to describe the actual score of severity. One can say that the condition score determination provides a substantial binary determination of caries or no caries for each facet of a tooth, without necessarily explaining the actual severity score of the facet. In any case, the score probabilities may however be stored according to the method to ensure that an exact severity can be assigned to a facet at later post-processing steps.

[0151] To create condition lesions, such as caries lesions, according to the method described herein a further processing step may be considered. With the just described condition score determination step, each facet is given at least one condition score representing whether the facet belongs to caries or no caries. In the step of creating lesions, the method is configured to merge the facets with nonzero condition scores into coherent lesions of the discretized 3D geometry. This is done by determining groups of inter-connected (for example via edges) facets comprising nonzero condition score. Then merging to the groups, single facets comprising zero condition score that has at least two neighbors with nonzero condition score and assigning to the single facets the highest condition score of the neighbors and repeating the merging to groups until reaching a set threshold of merging steps.

[0152] In an alternative, adding to the groups, single facets comprising zero condition score that has at least two neighbors with nonzero condition score, may comprise assigning to the single facets a score of one of the neighboring scores. This means that the score does not need to be the highest score but can be any of the nonzero scores associated with neighbors to a single facet comprising zero condition score.

[0153] This merging of facets to create coherent lesions may be best illustrated in Figure 6a to 6f, which will be explained in the following. Firstly, Figure 6a illustrates an example of the output from the trained learning model after it has gone through at least the mapping step to ensure mapping of score probabilities to correct facets and the further post-processing step of determining for each facet a condition score. As seen from Figure 6a, facets 233 with nonzero condition score are seen in groups 232 of facets. An example of the result of the merging process (to be explained) can be seen in Figure 6b, where it is clearly seen that most facets 233 of the tooth in Figure 6a, are included to define the final lesions 234a, 234b. Also, as a result of this process, the serrated behavior of the groups of inter-connected facets in Figure 6a is smoothed out as seen in Figure 6b. These groups have been found by determining inter-connected facets with nonzero condition scores. With reference to Figures 6c, 6d, 6e and 6f, the merging of facets into coherent lesions, will be explained in more detail. On a general level, to ensure that all facets which are possibly caries, as e.g. exemplified as facets 233 in Figure 6a, is included in groups of inter-connected facets, the process comprises merging to the groups, single facets 233 comprising zero condition score that has at least two neighbors with nonzero condition score and assigning to the single facets the highest condition score of the neighbors. Turning to Figure 6c, an example of the output from the machine learning model with the results of the steps of assigning a condition score to a facet is provided. Here a plurality of facets 233a, 233b, 233c, 233d, 233e, 233f, 233g, 233h, 233i, 233j , 233k, 2331, 233m, 233n, 233o, 233p is given a number as examples to explain the method in the following. To simplify the explanation going forward the facets 233a, 233b, 233c, 233d, 233e, 233f, 233g, 233h, 233i, 233j, 233k, 2331, 233m, 233n, 233o, 233p, 233q, 233r, 233s, 233t represented in Figures 6c to 6e will be referred to as facets 233a-t unless specific facts are being used as examples to describe the merging process, where then the specific facets will be referred to. Therefore, not all facet numbering may appear in all of Figures 6c to 6e, but merely the facets of relevance to explain the process may be illustrated. As seen in Figure 6c, the facets 233c, 233o comprises a first score (for example severe caries) and the facets 233a, 233n, 233p, 233b comprises a second score (for example moderate caries). The example shown in Figure 6c generally shows 7 groups of lesions as determined by a step of the merging process.

[0154] With reference to Figure 6d, the merging process further comprises to merging to the groups (i.e. 7 groups of Figure 6c), single facets, represented by 2331, 233m, 233j, 233i, 233h, comprising zero condition score that has at least two neighbors with nonzero condition score, and assigning to the single facets the highest condition score of the neighbors. In an example, if looking at facet 233j of Figure 6c and Figure 6d, it is seen that the facet 233j comprises a zero condition score and has neighbor facets 233c, 233o both comprising a nonzero condition score. In this case, therefore according to the method described herein facet 233j is merged to the group of facet 233c and 233 o creating the situation shown in Figure 6e. As is seen the example given in Figure 6d and Figure 6e, assigns to the single facet 233j , the highest score of the neighbors. As is seen this results in facet 233j receiving the same condition score as both facets 233c and 233o, in the example shown this results in facet 233j being assigned a severe condition score. If, however, e.g. facet 233c had been moderate, the result would have been the same as the condition score for facet 233o is higher.

[0155] Turning in more detail to Figure 6e, the process is generally repeated, as can be seen here. Moving to facet 233 q, which is considered the next single facet with zero condition score, the adding is continued. If looking at figure 6e it is clear that facet 233q also comprises two neighboring facets 233r, 233j, where facet 233r is provided with a condition score being moderate and the facet 233j is given a condition score of severe, as just described. With the method explained, this results in facet 233r being assigned a condition score of severe as the highest condition score of facets 233r, 233j neighboring facet 233q is severe, which is illustrated in Figure 6f.

[0156] The process of merging described in relation to Figures 6c to 6f is repeated until a set threshold is met. This threshold may be considered as to be when no more single facets may be added to groups inter-connected facets. For example, facet 233s of Figure 6f does not comprise two neighbors with nonzero condition score, and the process of adding here will be stopped. It is noted that Figures 6d illustrates at least 5 single facets 2331, 233m, 233j, 233i, 233h that according to the method described herein are added to groups of interconnected facets using the process described here. This results in Figure 6e, where and additional 3 single facets 233m, 233q, 233t with zero condition score is added to groups of inter-connected facets resulting in the two coherent lesions 234d, 234d illustrated in Figure 6f, as a broken line encircling lesions 234d and a dotted line encircling lesion 234d.

[0157] Accordingly, it is with these method steps ensured that the final lesions, represented as 234a, 234b in Figure 6b in one example and as a more detailed example lesions 234b, 234c in Figure 6f, are generated from the merging process. The process, as mainly seen in lesions 234a, 234b of Figure 6b also ensures smooth and confined lesions with substantially clear boundaries that defines the area of the lesions 234a, 234b. This improves interpretability when presented to user and creates a realistic picture of the clinical situation associated with the teeth exposed to the dental condition(s) described herein. It is noted that Figures 6a and 6b are provided to show mainly the input to this merging process and the output of the merging process, to reflect the optimization happening to lesion behavior due to the described creation of lesions process. The more detailed steps that may contribute to create coherent lesions are described in relation to Figures 6c to 6f. Furthermore a further steps, not illustrated in detail, may comprise a further processing step of removing facets which does not have a caries neighbor.

[0158] Another further processing step may comprise to remove lesions smaller than a set threshold. Accordingly, the method may comprising a determination of the size of number of facets contained in a lesion and apply a set threshold to evaluate if a lesions should be considered a clinical relevant lesions. As an example, if the set threshold was set to be number of facets contained in a lesions being less than three, then the lesion 234d of Figure 6f would be removed and not considered a clinical relevant lesion.

[0159] To provide the dentist with a clear interpretable lesion overview, the method comprises a step of assigning a lesion score to each of the coherent lesions 234a, 234b of Figure 6b and lesions 234c, 234d of Figure 6f. The assigning of the lesion score in more detail comprises for each coherent lesion determine the assigned condition scores represented in the coherent lesions and assigning the condition score with the highest severity to the coherent lesion. With reference to an example shown in Figure 7, and considering lesion 234a of Figure 6b, a simplified example could be that this lesion 234a comprises the score probabilities and assigned condition scores associated with Figure 7. The lesion would in this example be represented by facets of moderate and initial caries scores. According to at least one example of the method, the method may assign to the lesion 234a a score of moderate caries as this is the highest given score to any of the facets representing the lesion 234a. This means that the facet 235 originally being initial would be considered a moderate caries instead of a initial caries. In other words, the method looks at the distribution of conditions scores (of facets) within a coherent lesion by e.g. counting the distribution of facets associated with a condition score and based on that determining the lesion score. The count of distribution may be determined based on facet area. The most severe score accepted may be determined based on a set threshold and accepting the moist severe score above the set threshold. This method is considered the method that represents best the clinical standard approach of evaluating lesions of caries in the mouth of a patient and is considered the preferred method. An alternative method to assign a lesion score to a lesion could be to re-look at the full set of score probabilities associated with facets of a lesion and based on those score probabilities determine the lesion score. This is however considered a less accurate approach in view of clinical practice.

[0160] In any of the examples described herein, the score probabilities are preferably based on a clinically accepted standard, such the QLF method and / or ICDAS / ICCMS standard. In this way, a clinical understandable standard may be applied and used to interpret the results as output from the learning model. This allows the dental experts exposed to the proposed solution described herein to get an interpretable and understandable output which is easy to use in a clinical setup and which resembles the manual tactile visual inspections currently done today. The annotations as described herein are considered to be based on e.g. the ICDAS scores..

[0161] As elaborated on in the method steps provided herein, the processing may be done on a tooth-by-tooth basis. In such case, the method may include a pre-processing step of segmenting the discretized 3D geometry into teeth and gingiva; extracting each of the teeth from the segmentation; and determining and aligning a principial axes of each tooth with a common coordinate system for all teeth from the segmentation. With this pre-processing step it is ensured that the numerical features generated for input to the trained learning model represent each tooth and / or gingiva parts thereof, as each tooth and associated gingiva may be segmented from each other. Furthermore, providing each tooth at a time in the processing steps described herein may optimize processing power, as less data needs to be processed at the same time. . Such pre-processing step ensures that e.g. numerical features as represented in a simplified example in Figure 4, may be generated.

[0162] Referring back to Figure 2 and 3, the method may comprise representing the discretized 3D geometry 207 by assigning a color to each of the lesions according to the severity (i.e. score) assigned to the lesions. With reference to Figure 2, the computing system may comprise a display module 145 which is configured to perform the described steps of representing the output of the processing steps described herein on the discretized 3D geometry originally input to the caries detection program 140. Accordingly, the display module 145, may perform the visualization processing and output to a display device 108 comprising a graphical user interface 109, the discretized 3D geometry and the caries lesions with assigned scores. Referring to Figure 3, this may include updating the discretized 3D geometry 207 illustrated in Figure 3 with the findings of the processing steps described herein. Preferably, it may comprise assigning a color to the defined lesions according to the severity of the lesions. Furthermore, it may include to present in the graphical user interface of display 108 a caries lesion score overview (previously mentioned box 240), for example comprising tooth number and associated score, severity class and confidence etc. Other examples of visualization of the caries lesions have previously been described and reference is made thereto.

[0163] In alignment with previously described examples, the solution described herein also provides for a computer program having instructions which when executed by a computing device or system cause the computing device or system to perform the steps described herein. Accordingly, with reference to at least Figure 2 and generally the following figures, the computer program may be configured as a caries detection program 140, which upon activation is configured to perform the steps of:

[0164] - obtaining intra-oral scan data representing a 3D geometry of at least teeth and gingiva from a patient using an intra-oral scanner;

[0165] - pre-processing the intra-oral scan data to generate a discretization of the 3D geometry wherein the discretized 3D geometry comprises facet representations of teeth and gingiva;

[0166] - pre-processing the discretized 3D geometry to generate granularity elements representing the facets of teeth, and comprising one or more numerical features representing quantified local information of the facets;

[0167] - inputting the granularity elements to a trained model, wherein the trained model is configured to assigning score probabilities to each of the granularity elements;

[0168] - mapping the assigned score probabilities of the granularity elements to the corresponding facets of the discretized 3D geometry;

[0169] - determining for each facet a condition score based on at least one of the mapped score probabilities;

[0170] - merging facets with nonzero condition score into coherent lesions of the discretized 3D geometry, and assigning a lesion score to each of the coherent lesions according to at least one of the mapped score probabilities associated with facets of the coherent lesions;

[0171] - representing the discretized 3D geometry with the lesions and assigned lesion score in a visual display. Accordingly, the steps associated with the described method may be performed by the described computer product program.

[0172] As described throughout the method comprises generation of granularity elements by processing the discretized 3D geometry. As the generation of granularity elements is highly dependent on the choice of machine learning model used and corresponds to the generation of granularity elements performed during a training of the machine learning model, the explanation related thereto will be presented in connection with the details associated with the machine learning model and the training thereof in the following.

[0173] The learning model and training thereof

[0174] In accordance with the method(s) described herein, the caries detection method utilizes a learning model, preferably configured as a machine learning model applying a neural network architecture, to detect caries in discretized 3D geometries of intra-oral scan data. To make use of a learning model in an application (e.g. the caries detection program), the learning model may be trained on known data, from which the model may learn how to detect, in unknown data to the model, lesions of dental conditions (such as caries). The process of generating a trained learning model suitable for lesion detection in discretized 3D geometries acquired from intra-oral scan data requires a training phase for the model to learn how lesion features are represented in a discretized 3D geometry of intra-oral scan data.

[0175] A neural network type learning model generally comprises mapping an input (layer) to an output (layer) through a sequence of non-linear operations. The intermediate states in the sequence are referred to as hidden layers and the individual values neurons. Each non-linear operation is associated with a number of parameters referred to as the model weights. The learning model may be optimized with respect to the model weights minimizing a given loss function using gradient based techniques. The number of operations and the type of operations may depend on the problem and input and output data structure. For detecting caries or other dental conditions in discretized 3D geometries, at least two alternative approaches are considered. The first approach utilizes what will be denoted as a 2D snapshot approach, and a second approach utilizes 3D point clouds as input to the neural network input layer, which will be explained in more detail as follows. For each of the approaches the data used is the same and will be explained briefly in the following.

[0176] Another approach may comprise to perform a mesh flattening procedure, which ensures that teeth are flattened with minimal mesh distortion. The mesh flattening procedure may in an example be based on a free-boundary quasi-conformal parameterization. The flattened meshes may be projected onto appropriate 2D image planes, recording all relevant data. The projection may be carried out with a rasterization rendering routine. The image plane contains a predefined number of pixels, and each pixel's size is scaled for each tooth mesh such that the tooth object exactly fits into the image plane along its longest axis. Colors and fluorescence are sampled from the corresponding position on the flattened tooth mesh for each pixel in the image plane.

[0177] Training data and model fitting

[0178] In order to train the learning model to detect lesions of caries as an example (not ruling out other dental conditions) in accordance with the methods described herein, training data has been collected. The training data comprises a plurality of intra-oral scan data, which are acquired from for example a cloud repository. The scan data have been collected by performing intra-oral scans on different patients across the world, where intra-oral scanners may be used to acquire the scan data. To ensure gender and ethnicity variation in the dataset, the training data has been created with intra-oral scan data acquired from different countries, genders, and nationalities.

[0179] To verify the quality of the trained learning model, a part of the dataset collected is provided as test data. The test data is used only for validating the results during a training phase of the learning model. The test data is configured as scan data that is not used for training and can therefore be used to evaluate how well the trained model performs on data it was not directly optimized for. A training data set may comprise a bias towards some data samples rather than others. Clinically it is often observed that older patients have more dental restorations and stains (e.g. due to smoking), which are some parameters that can affect the performance of a caries detection algorithm by inducing false positive results. To reduce potential bias in the training data, it may be ensured that the training data population is representative for the population for which the model is intended to be used. That is the training data preferably represents statistical parameters which are important to consider, such as age, country, medical condition, habits, previous treatments etc. is evaluated on a statistically representative dataset..

[0180] The training data may comprise in vivo scans collected from a plurality of dental patients, wherein the training data comprises a wide age range. Furthermore, the training data may comprise full-mouth and / or partial mouth models generated from intra-oral scans. In addition, the training data preferably comprises intra-oral scan data where all permanent teeth are present. However, the training dataset may also comprise intra-oral scans where only permanent molars and pre-molars are present.

[0181] Each of the intra-oral scan data forming part of the training dataset comprises data associated with tooth color, fluorescence (red and green) and geometry information, such as depth values etc. This information corresponds to the previous mentioned quantified local information.

[0182] Annotations

[0183] To generate the ground truth data of the training data set, which is generally done during a training phase, the training data, generated using an intra-oral scanner, is pre-processed to generate a discretized 3D geometry for each 3D geometry represented in the training data.. Each of the discretized 3D geometries of the training data are annotated according to a predefined annotation protocol. The annotations on the discretized 3D geometries of the training data may be created manually by annotation experts or semi-automatically as described previously and in the following by e.g. a usage data collection and / or gamification setup. The annotations may be done automatically by a computer-implemented annotation protocol running in a processor. As one of the main challenges of developing trained learning models includes the time associated with generating the ground truth data, at least a semi-automated annotation process may be considered for optimizing the time spend on annotating the training dataset, for example by correcting the predictions of an a priori model.

[0184] One at least semi-automatic annotation process may comprise a crowd sourced annotation method. Such crowed sourced annotation method may comprise including intra-oral scan data in a learning system, where dental professionals are presented with intra-oral scans acquired from a database of intra-oral scans. The learning system may be configured to request a dentist input, wherein the request may be to tag, label and / or annotate areas of the intra-oral scan data comprising caries lesions according to visual inspection of the dentist. The learning system may be used in a digital dental system provided on a piece of computer software and forming part of, for example, a learning app connected with the software.

[0185] Another alternative annotation process may comprise to generate a computer program comprising a processor configured to perform a method of:

[0186] - reading into the processor a plurality of intra-oral scan data from a database of intra-oral scan data;

[0187] - utilizing a predefined annotation protocol to automatically annotate the intra-oral scan data according to the predefined annotation protocol, wherein the annotation protocol may be read by the processor in order to annotate the discretized 3D geometries of the intra-oral scan data.

[0188] Yet another alternative annotation process may comprise to generate for each discretized 3D geometry an automatically calculated caries lesion area for a tooth using a threshold for a green color channel of the fluorescence data and / or a caries score based on all color channels (natural RGB, and fluorescence RG) and a regression. A further step may comprise presenting to an annotating expert, in a piece of software having a user interface, islands of loss of fluorescence (or potential caries) and automatically assigning either histological scores or ICDAS / ICCMS score. In this way the annotating experts may save a lot of time used to annotate the training data, as they do not need to mark by e.g. painting in a visual display, the area themselves but may instead simply approve / edit the pre-defined marked areas, such as islands, and assign a score. With this method, the annotation of islands is more reliable and reproducible as it is based on a well-documented Quantitative light-induced fluorescence QLF method, from which the fluorescence loss may be calculated.

[0189] In addition to the ground truth data, further pre-processing of the discretized 3D geometries may be performed to allow sufficient training of the machine learning model. Details of this further processing depend highly on the approach taken, which will be explained in the following. An example of ground truth data is illustrated in Figure 16, where the annotated areas 160a, 160b, 160c, 160d as examples indicate different severities of caries annotated using one of the previous mentioned methods.

[0190] The learning model - 2D snapshot approach

[0191] The training of the neural network according to a 2D snapshot approach is configured to use 2D image snapshots to generate the granularity elements of the discretized 3D geometry. With reference to Figures 8, 9, 10 and 11, the 2D snapshot approach will be described as follows. In the following, it may be understood that pixels of a 2D snapshot may be considered as granularity elements.

[0192] The training data as previously described as being a plurality of discretized 3D geometries may in accordance with the 2D snapshot approach be pre-processed to generate granularity elements representing the facets of teeth and comprising one or more numerical features representing quantified local information of the facets. The high-level approach is illustrated in Figure 8, where it is seen how the 2D snapshot approach in an example may comprise generating a virtual camera(s) 180 in a virtual scene representing the discretized 3D geometry 1207 used for training and positioning the virtual camera 180 at one or more positions X, Y, Z to cover at least a part of the discretized 3D geometry 1207 used for training in the virtual scene. Using the positioned virtual camera 180, the method is configured to acquiring one or more 2D image snapshots 181, 182, 183 taken from position X, 181, taken form position Y, and 183 taken from position Z 182 of the discretized 3D geometry 1207 used for training.

[0193] As is apparent from Figure 8, the positioning of the virtual camera 180 in the virtual scene of the discretized 3D geometry 1207 used for training, allows acquisition of the 2D image snapshots of teeth taken from a buccal 204, lingual 205 and / or occlusal side 206 of the discretized geometry 1207. Potentially gingiva could be included in the snapshot. Preferably, the method comprises segmenting teeth from gingiva and another tooth before taking the 2D snapshots.

[0194] From the generated 2D snapshot images 181, 182, 183, the method may comprise a preprocessing step of generating a rendered representation of the discretized 3D geometry used for training in the form of sets of 2D image snapshots modalities, as illustrated in Figure 11. The 2D image shot modalities may be considered to comprise the previously mentioned quantified local information.

[0195] The rendered representations for each generated 2D snapshot view may comprises one or more of 2D snapshot modalities comprising a 2D mesh color map 184, a green and / or red fluorescence color map 185, calculation of a facet normal map 186 of the 2D snapshot with respect to the virtual camera position, calculation of a facet curvature map 187 of the 2D snapshot and / or calculation of a depth data map 188 of the 2D snapshot with respect to the virtual camera position. As seen in Figure 11, the 2D images snapshots may be generated from the virtual camera views X, Y, Z illustrated in Figure 8, and for each snapshot view one or more of the above defined color map 184, fluorescent map 185, facet normal map 186, curvature map 187 and depth map 188.

[0196] In addition to these rendered 2D snapshot modalities also a ground truth modality is generated which represents an annotated caries lesion in the 2D snapshot image. The ground truth modality is illustrated in Figure 11 as a mask 189, where the pixel values of the 2D snapshot associated with caries is given a color whereas pixels values associated with noncaries areas of the 2D snapshot is given a value of zero represented as a black background. The pre-processing of the discretized 3D geometries used for training may comprise modifying each of the 2D modalities by identifying in the 2D modalities pixels corresponding to facets that do not comprise teeth and assigning a value of zero to those pixels.

[0197] In view of training the learning model, at least one of the generated 2D snapshots and the ground truth in the form of the mask 189 may be input to a machine learning architecture, preferably optimized for processing 2D images. In an example the machine learning architecture for the 2D snapshots approach may be a convolution neural network, such as a U-Net or similar network architecture. In short the U-Net is a Convolutional Neural Network (CNN) which combines an encoding path that abstracts image features to create a feature map, followed by a decoding path that generates a segmentation mask. It is aided by skip connections, which preserve contextual feature information from the input data. A specific U-Net known as Attention U-Net incorporates attention mechanisms to focus on a specific area of an image.

[0198] The granularity element structure according to the 2D snapshot approach may be considered as comprising pixels of a 2D snapshot, where each pixel comprises, quantified local information represented by the corresponding numerical value of the associated modalities.

[0199] The input layer for the architecture of the 2D snapshot approach is configured to receive the granularity element structures which is feed into the machine learning model in the training phase. The machine learning model during training is then configured to process the granularity elements and the ground truth using a plurality of hidden layers so as to optimize the weights between neurons and finally assign to each of the granularity element score probabilities, when the weight optimization has converged.

[0200] The just described pre-processing of discretized 3D geometries may be done for all the discretized 3D geometries forming part of the training dataset. Accordingly, a large dataset of 2D modalities may be input to the machine learning model to allow learning thereof in the training phase. Training leads to an optimized model, which is stored for use in the software product. This finishes the training phase and allows use of the now trained learning model to be used on new unknown data, such as discretized 3D geometries not previously been seen by the trained learning model.

[0201] In summary, the proposed solution for training the machine learning model according to the 2D snapshots approach includes the following steps, which are illustrated in the flow chart of Figure 9. Here the steps are generally outlined as:

[0202] Receiving 300 as input a 3D jaw model (defined throughout as a discretized 3D geometry), as acquired form a training data set.

[0203] Performing 301a segmentation that identifies for each facet in the discretized 3D geometry, whether it is a specific tooth or gingiva.

[0204] Determining 302 a transformation matrix for each tooth to transform it to an origin of the virtual space of the discretized 3D geometry. With steps 301 and 302 it is ensured that each tooth of the discretized 3D geometry is identified and segmented from each other and the gingiva, as well as transformed to ensure that all teeth are aligned according to the same coordinate system, such as oriented at the origin of the virtual space, which ensures that for all teeth the occlusal surface is facing towards a positive direction of the Y axis of the virtual space. Further steps comprises:

[0205] Performing 303, 304 2D image snapshot generation of each tooth, wherein the 2D image snapshot generation comprises:

[0206] Extracting 305 a tooth from the segmented discretized 3D geometry (also denoted jaw scan) Positioning 306 virtual cameras in relation to the extracted tooth to generate 2D snapshots of each of the teeth in the 3D jaw model, wherein the placement of the camera ensures that information of clinical importance can be generated. The positions of the cameras include positions allowing views of the occlusal, buccal, and lingual surfaces of each tooth as previously described. The 2D snapshot generation may apply a forward rendering approach (as opposed to ray-tracing), which means that each facet of the surface is projected from 3D into the image plane and drawn there.

[0207] Rendering 307, for each of the generated images of an extracted tooth data modalities, where the data modalities may comprise one or more of: mesh color fluorescence color facet normal with regards to the camera position facet curvature

[0208] Depth with regards to camera position as previously described.

[0209] Furthermore, the ground truth previously described may also be rendered with the rendering 307 process. Generating the ground truth is based on the previously mentioned annotated information, and comprises the scores background, initial caries, moderate caries and / or extensive caries, wherein pixels belonging to the background is assigned a value of zero, pixels belonging to initial caries is assigned a value of blue, pixels belonging to moderate caries is assigned a value of yellow and pixels belonging to extensive caries is assigned the value of red. The scores of caries (initial, moderate, extensive) are acquired from the annotated data. As illustrated in Figure 9, each tooth of each of the training discretized 3D geometries used to train the neural network on is processed in the above-described manner, meaning that the algorithm runs until all teeth have been processed.

[0210] Not illustrated in more detail in Figure 9 and previously described is that the method may furthermore comprise the steps of modifying each image belonging to each of the rendered modalities, by adding pixels values of zero to pixels belonging to background, i.e. pixels that do not have a facet in that position.

[0211] Further steps of the learning phase associated with the 2D snapshot approach may include performing augmentation of the training data. Accordingly, the method may comprise performing an augmentation pipeline during the training process to increase the model's robustness and synthetically increase the size the dataset. The augmentation pipeline may comprise applying horizontal and vertical flips, elastic transformations, grid distortion, affine transformations, optical distortion, gaussian blurring, gaussian noise, multiplicative noise, pixel dropout, and / or channel dropout. All augmentations may be applied to the training 2D snapshot images during training with a set probability. In an example, the dropout rate is also given a set dropout threshold, in an example the drop out threshold may be 0.25. Applying the trained learning model - 2D snapshot approach

[0212] For the 2D snapshots approach, the use of the trained learning model may include similar steps as described in relation to the training of the model. The main difference between the training phase and the use of the trained machine learning model is that the ground truth data is not be generated for the new and unknow data input to the model. Accordingly, and with reference to Figure 10, the inference (i.e. use / application of the trained model) for the 2D snapshot approach is illustrated. As is seen when comparing Figure 9 and Figure 10, the pre-processing according to steps 400, 401, 402, 403, 404, 405, 406, represented by 406a and 406b of Figure 10 corresponds to the pre-processing steps 300, 301, 302, 303, 304, 305 and 306 of Figure 9, with the main difference being that the rendering 406b does not include rendering of a ground truth and that the 3D jaw model (i.e. the discretized 3D geometry) does not form part of the training dataset but may be acquired e.g. during a clinical scan of a patient. As these processing steps have already been described in relation to the training of the learning model, reference is made thereto. Accordingly, the descriptions provided in relation to Figure 8 and Figure 9 also applies for the pre-processing steps associated with the 2D snapshot approach using the already trained model.

[0213] The flow illustrated in Figure 10 generally represents at least some of the steps performed by the method upon activation of the caries detection program as previously described. Accordingly, the main aim with the method steps represented in the flow of Figure 10 is to detect lesions, such as caries in discretized 3D geometries using the trained learning model When the pre-processing steps 400, 401, 402, 403, 404 405, 406 (including 406a and 406b) has been performed, the pre-processed data is input to the trained learning model 407. From the learning model, the output is score probabilities assigned to granularity elements of the input data. It should be noted that even though both Figures 9 and 10 imply that all teeth are processed as described it may be the case that the algorithm does not process some teeth.

[0214] As previously mentioned, the score probabilities comprises probabilities of a pixel in the 2D image snapshot beings associated with a caries scores, such as no caries, initial caries, moderate caries and / or sever caries. The score probabilities as output form the machine learning model as given by the prediction 407 may in a post-processing step 408 be mapped to facets of the 3D model, as previously described. Accordingly, step 408 of Figure 10 may represent at least one step of the postprocessing performed by the method described herein. Further post-processing steps of determining 409 for each facet a condition score, merging 410 facets with nonzero condition score into coherent lesions and assigning 411 a lesion score to each of the coherent lesions is also illustrated in Figure 10. With these steps, output from the trained machine learning model 407 may be projected back to the discretized 3D geometry and visualized in a clear and clinical understandable manner. For details on these steps reference is made to previous sections describing these steps.

[0215] The learning model - 3D approach.

[0216] The training of the neural network according to a 3D approach is configured pre-process the discretized 3D geometry 1207 as considered to forming part of a training data set to generate granularity elements of points representing facets of teeth. In this approach the discretized 3D geometry 1207 is pre-processed to generate a 3D point cloud 120 as seen in Figure 12 and described in the following.

[0217] As described in relation to the 2D snapshot approach, the 3D approach similarly uses a data set of training data comprising discretized 3D geometries to train the machine learning model. The 3D approach is configured to pre-process the training discretized 3D geometries 1207 into a point cloud 120 for each tooth represented. An example of such a point cloud is illustrated in Figure 12, where the discretized 3D geometry 1207, or at least one tooth 120a thereof, may be processed as previously explained to generate a point cloud 120 of all the teeth of the discretized 3D geometry 1207. The point cloud 120 represents the tooth 120a in Figure 12.

[0218] Similarly to the 2D approach, the 3D approach comprises a step of reading into e.g. a caries detection program the discretized 3D geometry and performing a segmentation of the discretized 3D geometry. This allows separation of teeth and gingiva represented in the discretized 3D geometry. The 3D approach may comprise a step of aligning each tooth to the same coordinate system using a transformation matrix, as previously described. This ensures that for all teeth the occlusal surface is facing towards a positive direction of the Y axis of the virtual scene.

[0219] For each of the teeth from the segmentation, a 3D point cloud may be generated, as illustrated in an example of Figure 12. The point cloud of each tooth may be generated using a random sampling method, a Poisson distribution or similar method allowing a representation of the facets of the discretized 3D geometries in an optimized manner in view of processing power. The point cloud for a tooth may be generated by identifying a plurality of facet center points of each tooth and generating the point cloud based on these center points and further augmenting to the point cloud additional numerical features associated with the respective facets, where the numerical features represent the previous described quantified local information. One or more points associated with a facet represents in the 3D point cloud approach the granularity elements described as input the learning model.

[0220] In the 3D approach, for each of the teeth of the discretized 3D geometry, the granularity elements comprise numerical features representing quantified local information. The numerical information is associated with data information related to curvature information, RGB color information, red and green fluorescence information and depth information as also explained in relation to the 2D approach. This data information may be calculated from the discretized 3D geometry and / or extracted from the raw data of the discretized 3D geometry. For example, the curvature information may be computed from the discretized 3D geometry, whereas the RGB color information, red and green fluorescence information and depth information may be extracted from the raw data stored on the discretized 3D geometry. It is reminded that throughout discretized 3D geometry also includes the term 3D model.

[0221] All of this data, i.e. point cloud points and associated quantified local information may be concatenated into a common point cloud representation representing the granularity elements, which is feed to a machine learning model architecture. The machine learning architecture for the 3D approach comprises in one example a PointNet or PointNet++ architecture. Other alternatives may include PointNext, ASSANet, Octree-NN, pointtransformer or any variation thereof.

[0222] As with the 2D approach, the 3D approach comprises inputting the above-described concatenated point cloud representation is input into the learning model, which is configured to assigning score probabilities to granularity elements representing the point cloud.

[0223] Similarly to the 2D approach, the machine learning model in as training phase utilizes a ground truth annotation to optimize the weights of the model which upon convergence leads to an optimized model, which is stored for in the software product containing the caries detection program.

[0224] In the training phase, the described pre-processing of discretized 3D geometries may be done for all the discretized 3D geometries forming part of the training dataset. Accordingly, a large dataset of point clouds may be input to the machine learning model to allow learning thereof in the training phase. This finishes the training phase and allows use of the now trained learning model to be used on new unknown data, such as discretized 3D geometries not previously been seen by the trained learning model.

[0225] As is apparent, the main difference in the processing steps between the training phase and the use of the trained model in the 3D point cloud approach is that a ground truth is used to optimize the weights of the model during training. This ground truth is not part of the subsequent use of the trained model. Expect for use of the ground truth, the discretized 3D model 1207, and the processing thereof is however the same for both the training phase and when using the trained model in a software product which should be apparent for a skilled person and the description provided herein.

[0226] An example of a post-processed output from the machine learning model according to the 3D approach is illustrated in Figure 13. Here it is seen that a point cloud 120 of tooth 120a may be given an assigned score to each of the points in the point cloud 120. In the illustrated example, the areas 122, 123, 124 encircled may comprise points associated with a probability of caries as output from the machine learning model. All of the training data comprising a plurality of training discretized 3D geometries may undergo the above presented pre-processing, where all the pre-processed training data is configured to be input to the machine learning model to allow learning.

[0227] As it is mainly the pre-processing of the data that is different between the 2D and 3D snapshot approach described herein, it should be understood that the subsequent processing of the score probabilities of mapping, determining, merging and assigning score is substantially the same, why reference is made to previous descriptions thereof.

[0228] Applying the trained learning model - 3D approach

[0229] When the machine learning model has been trained as just described for the 3D approach, the use of the trained (and previously described optimized model) learning model may include similar steps as described in relation to the training of the model. For the 3D approach, the main difference between the training phase and the use of the trained machine learning model is that the ground truth data is not generated for the new and unknown data input to the model. Accordingly, the generation of point clouds and subsequent generation of granularity elements comprising numerical features representing quantified local information, as described for the training data may similar be applied for the unknown data when using the trained machine learning model. Therefore, these steps are not repeated, and reference is made to the previous section.

[0230] As mentioned generally for both approaches, the 3D approach, at least when using the machine learning model on data acquired of a 3D geometry during a clinical intra-oral scan, may also comprise a series of post-processing steps allowing detection of lesions, such as caries lesions in discretized 3D geometries using the output from trained learning model, and reference is made to previous sections describing these steps.

[0231] In the 3D approach the output comprises score probabilities representing a probability of a point in the point cloud being associated with one or more caries scores.

[0232] In the 3D point cloud approach the given input can be construed as points of the discretized 3D geometry representing facets of teeth, which results in the score probabilities as output form the machine learning model already being represented at a facet level. Accordingly, the score probabilities as output represents a mapping of the score probabilities to corresponding facets, which are subsequently processed in accordance with the previous description of determining a condition score, merging into coherent lesions and assigning lesion scores to coherent lesions.

[0233] Figure 22 illustrates the learning model 220 utilizing the 3D point cloud 120 as input, where the learning model 220 comprises two separate machine learning models, namely a first machine learning model 221 configured for classifying facets of the discretized 3D geometry into healthy facets or facets with caries 223. Healthy facets may have a no caries (N) condition score while the facets with caries 223 may have a nonzero condition score, however still unknown which of the initial caries score (I), moderate caries score (M) or severe caries score (S). The learning model 220 may further comprise a second machine learning model 222 configured for assigning one of the severity scores comprising initial caries score (I), moderate caries score (M) or severe caries score (S), to the facets with caries 223 determined by the first machine learning model 221. As seen on the figure, facets 224 are classified as having the moderate caries score (M) while the facet 225 is classified as having the severe caries score (S). The second machine learning model 222 may thus disregard the healthy facets identified by the first machine learning model 221 by processing only facets with caries 223 identified by the first machine learning model 221. The second machine learning model 222 may thus only keep severity labels on facets identified as having caries 223. During the training of the second machine learning model 222 only facets manually labelled as having caries may contribute to the loss function.

[0234] A facet may be classified as a healthy facet by the first machine learning model 221 if a probability for this is higher than 50%. For a facet with caries, a most probable severity label identified by the second machine learning model 222 may be assigned.

[0235] An advantage of using the first machine learning model 221 and the second machine learning model 222 in this manner is that a higher focus can be placed on detecting caries by the first machine learning model 221 and thereby a higher overall detection accuracy can be achieved. The first machine learning model 221 may thus be specialized only to identify caries. Specialized task of classification of detected caries into a severity category is thereby left to the second machine learning model 222.

[0236] Both the first machine learning model 221 and the second machine learning model 222 may be of the same type and of the same architecture as examples of the learning model 220 described throughout the disclosure. The two machine learning models 221, 222 may be trained for these different tasks using training data annotated manually. The training data is described further with respect to the trained model.

[0237] Displaying detected lesions

[0238] The display of the coherent lesions detected by the previously described pre-processing steps, learning model and post-processing steps may be performed by the display module 145 which communicates with a display device 108 configured with a graphical user interface 109 as seen in Figure 2. An example of displaying the lesions detected is illustrated in Figure 14. Here it is seen how the discretized 3D geometry 207 comprises a number of teeth 200a, 200b, 200c, 200d, 200e for which lesions, such as caries lesions 30, 31, 32, 33, 34, 35, have been detected according to the method described herein. Lesions 30, 31, 32, 33, 34, 35 may as seen in Figure 14 be represented as a colored area of the teeth, where the coloring may represent a severity of the lesions according to the assigned lesion score.

[0239] Furthermore, as illustrated in Figure 14, the raw data of the lesions may be extracted by the display module to allow representation of the raw data 200eRl, 200eR2, 200eR3 in the graphical user interface. The raw data 200eRl, 200eR2, 200eR3 may comprise the raw scan data as acquired form the intra-oral scanner in the form of 2D images. Accordingly, in an example, the display module may be configured to receive as input a tooth of interest as indicated by a user e.g. hovering over a specific tooth. From the received tooth of interest, the display module may be configured to search through a database of stored raw image data associated with the intra-oral scan data of the discretized 3D geometry and identify the 2D images associated with a specific tooth. The identified 2D image data associated with the tooth of interest may be presented in a part of the user interface different from the discretized 3D geometry representation. The display module may be configured to represent in the graphical interface the 2D images identified to visualize to the dentist the raw image data associated with a specific tooth allowing visual inspection of the raw data in comparison to the discretized 3D geometry. This method aids in detection the caries lesions and flagging the lesions to the dentist and allows an automatic generation of the raw image data when receiving by a processor, an interaction activation from e.g. a hovering over a tooth to visualize the raw data.

[0240] In another example, illustrated in Figure 15 the display module may be configured to generate in the graphical user interface severity score descriptors 60, which indicates with colors, the severity of the lesions detected by the method described herein. As seen in Figure 15, a caries lesion with a severe score is indicated in a first color 61, a moderate score is provided in a second color 62, and an initial score is provided in a third color 63. Each of these colors according to the lesions detected in the discretized 3D geometry is assigned to the lesions according to their respective scores of initial, moderate or severe. The severity score descriptors 60 may be considered a user interface descriptor aiding in a fast and interpretable understanding of the findings according to the method described herein. The descriptors 60 may e.g. be generated by the display module to using the information given by the output from the trained machine learning model.

[0241] In an example, the severity descriptors 60 may include a representation of areas of lesions detected, a percentage of lesions representing initial, moderate and / or severe caries, no of teeth affected by caries or any other suitable information representing the findings of the caries detection program.

[0242] As illustrated in Figure 15, and previously described, the lesions may be presented as filled areas, as outlines 64 or as outlines 64 with transparent filling 65. Visualizing the lesions as filled areas provides a clear visual overview of the areas that are exposed to caries. Visualizing the lesions as outlines ensures that the dentist and / or patient is observant of the area when visualized on the discretized 3D geometry while allows a dentist to look at the actual lesion in its natural appearance as it was captured with the intra-oral scanner. This may allow the dentist to visually confirm the findings of the proposed method described herein. In an example, where the visualization comprises generating both an outline and a filled area, wherein the filled area is configured with a transparency allows the user to obtain a general understand and view of the area affected while allowing visual inspection of the actual tooth with the lesion. In this way, the dentist may be able to assert the lesion both using visual inspection of the discretized 3D geometry through the substantially transparent color covering the area of the lesion.

[0243] As also illustrated in Figure 15, data generated from previous clinical visits indicated by bar 66 may be used to evaluate a potential development of the dental condition.

[0244] An example of assessing the development of caries on a tooth over time is illustrated in Figure 16. Here it is seen how two data sets represented as discretized 3D geometries 70, 71 are loaded into the graphical user interface by the display module. The display module is accordingly configured to receive an input from the user instructing loading of data with a certain time stamp indicating a previously acquired scan data. The display module may be configured to ensure that the visualized parts of the discretized 3D geometries taken at a first point in time 70 and taken at a second point in time 71 are acquired from the same views. Thus, the display module may be configured to read the viewpoint associated with a currently visualized discretized 3D geometry, e.g. 71, and search through the chosen date for a second discretized geometry 70 to identify the same viewpoints. When the viewpoints are identified for the second discretized geometry, the parts corresponding to the view may be represented in the graphical user interface. In this way it is ensured that two discretized 3D geometries may be visualized at the same time to compare the caries lesions area to see the development thereof.

[0245] In the example illustrated in Figure 16 a comparison slider 90 may be generated in the graphical user interface allowing shifting between the discretized 3D geometries. In this way the development of e.g. caries may easily be visualized to the patient and / or the dentist may be given a quick indication of the progression of caries from a visual perspective. The slider may be generated in the display module as soon as a request on loading two models is received from a user interaction with the user interface. As seen in Figure 16, the comparison between two scans using the slider, may show a tooth 130a at a first time and the same tooth 130b at a second time separated by a slider 90. Upon activation of the slider 90, a visualization of the first tooth 130a and the second tooth 130b may be processed, as when the slider is moved toward the first tooth 130a, the visualization ensures proper visualization of the full tooth 130b with all details represented, such as the caries lesion 63. When the slider is moved towards the second tooth 130b, the display module processes the data to ensure a full visualization of the first tooth 130a. In this way, a movement of the slider triggers the display module to update the visualization of the data associated with the first tooth 130a and the second tooth 130b allowing an interpretable aid understanding the amount of change between the scans over time.

[0246] In yet another embodiment, a caries progression may also be presented in visual manner by the processors of the computing system described herein being configured to compare lesions detected in a first scan taken at a first point in time with lesions of a second scan taken at a second point in time. The comparison may be configured to assert an area difference allowing the processor to calculate a difference in area of the detected caries lesions. In an example this may be done by the processor executing e.g. a comparison program, which may be configured to loading previous detected caries lesions for a discretized 3D geometries (acquired at a first point in time) and loading current detected caries lesions for a discretized 3D geometry acquired at a second dental visit. The previously and current detected caries lesions may be detected according to the methods described herein and may be compared in view of the area that the lesions are covering on a tooth. The size of the area may be determined for both the current and previous detected lesions, and the comparison may show a progression if the area of the current has increased with respect to the area of the previous detected lesion. This method may be performed for all teeth in discretized 3D geometries where a caries lesion has been detected. This comparison may be visualized as just described.

[0247] Turning now to Figures 18 and 19, the output of a lesion detection method using a learning model as described herein is illustrated on a discretized 3D geometry in Figure 18, and an example prior art method result is illustrated in Figure 18. As may be seen when comparing Figures 18 and 19, it is clear that the method described herein more accurately and precisely estimate caries lesions. Looking especially at tooth 160x and 160y it is clear that the prior art method of Figure 18, is insufficient in clearly identifying the correct lesion, as the tooth 160x in the prior art (Figure 18) is wrongly classified as caries and actually is a filling or similar restoration. The method described herein for which the output is seen in Figure 19 clearly estimates correctly the caries lesion on tooth 160x, in accordance with the ground truth 160a shown in Figure 17. Similar observation is seen with respect to tooth 160y. Accordingly, the method described herein improves the diagnostic accuracy without using ionizing radiation, improves reproducibility and optimize time spend on analyzing the data as well as at least aids in providing a diagnostic caries detection on discretized 3D geometries.

[0248] In a further example, the methods described herein may also comprise recording the detected lesions. Accordingly, in an example, the derived lesion detection may automatically be transferred to a dental chart using e.g. a recordal module of the computing system. Such automatic dental charting or other types of recordal system may provide fewer manual steps and accordingly are less prone to human errors. The digital tools can e.g. be a computer system, a computer program product, or a digital environment.

[0249] Dental conditions

[0250] As previously mentioned, different dental conditions that are relevant to asses using the same method or at least similar methods as described herein may comprise one or more of the following conditions.

[0251] Dental plaque, which is a bacterial biofilm that accumulates on the tooth surface and is associated with and a significant risk factor for the most prevalent oral diseases worldwide affecting people of all ages. When dental plaque accumulates on the crowns of teeth, the natural, smooth, shiny appearance of the enamel is lost, and a dull and matt effect is produced. As it builds up, masses of dental plaque become more readily visible to the naked eye. After a few days of accumulation of dental plaque on the tooth surface, the biofilm matures and creates risk for development of dental caries, gingivitis, and periodontal diseases.

[0252] Tooth wear is the gradual but persistent reduction of tooth substances. Tooth wear is generally not caused by dental decay (caries) or diseases but is a gradual and consistent process that may cause increased tooth sensitivity, reduction of the vertical dimension and compromised aesthetics.

[0253] Gingiva recession is a periodontal condition where the gum (gingiva) around the tooth will recede, exposing the root of the tooth. In more detail, gingiva recession is the displacement of the gingival margin apical to the Cemento-Enamel Junction (CEJ) of a tooth or the platform of a dental implant first the neck and later also the root of a tooth gets exposed. In a healthy state of the oral cavity, the CEJ is hidden / covered with gingiva (the gingival margin of the attached gingiva) and is therefore not visible. Different types / classes of gingiva recession exist, including:

[0254] -General / horizontal recession where the gingival margin has generally / horizontally shifted apical on several consecutive / all teeth within the dental arch.

[0255] - Local recession, where the gingival margin has shifted apical only on an individual or some none-consecutive teeth.

[0256] Malocclusion is the incorrect correlation between the upper and lower teeth to each other.

[0257] Bruxism is excessive teeth grinding or jaw clenching. It is an oral parafunctional activity, i.e., it is unrelated to normal functions such as eating or talking.

[0258] A cracked tooth is an incomplete fracture originating from the chewing surface of the tooth and extends vertically toward the root of the tooth. A cracked tooth can result from chewing on hard foods, grinding your teeth at night, and can even occur naturally as you age. Cracks in teeth vary in severity. Some are mild and invisible, while others are significant and cause a lot of pain. It’s a common condition and the leading cause of tooth loss in industrialized nations.

[0259] Gingivitis is an inflammatory condition of the gingival tissue, most caused by bacterial infection. Gingivitis is characterized by swelling, redness, exudate, a change of normal contours, bleeding, and, occasionally, discomfort. Gingivitis affects over 90% of the world population to some degree and is prevalent at all ages (Coventry et al., ABC of oral health: periodontal disease, BMJ, 2000). The manifestations of gingival inflammation are vascular changes consisting essentially of increased volume of crevicular fluid and increased blood flow at the marginal gingival region, and clinically, gingiva will appear with edema, less stippled, and more red than healthy gingiva. Diagnosis of gingivitis by the clinician relies on the identification of signs and symptoms of inflammation resulting from the disease process in the gingival tissues, and is based on inspection of the color, texture, edema of gingiva, and bleeding on probing. The assessment is either noninvasive using a visual technique, or invasive using instrumentation. Bleeding is an early sign of gingivitis and clinical assessment is often based on invasive use of a periodontal probe, and the suitability of bleeding provocation in the gingival margin is dependent on the probing pressure. In general, measurement of the gingival inflammation is subjective in nature and requires expert examiner training and knowledge to examine a patient at multiple sites to arrive at an appropriate diagnosis and management strategy. The assessment can be time-consuming and uncomfortable for the patient. Gingivitis is reversible with professional treatment, patient motivation, and good oral hygiene instruction, which is central for regenerating healthy gingiva without irreversible damage. However, patients are often unaware that they are suffering from gingivitis before it is diagnosed and being demonstrated to them when they attend a dental appointment (Blicher et al., Validation of self-reported periodontal disease: a systematic review, J Dent Res, 2005). Gingivitis is the first and mildest stage of progression of periodontal disease, and if left untreated it can lead to presence of an abnormal depth of the gingival sulcus (periodontal pocket), loss of jawbone surrounding the teeth, and eventually tooth loss. Therefore, an early diagnosis is crucial for preventing periodontal diseases. Development of non-invasive techniques, such as methods mentioned in the disclosure, which could predict the changes in microcirculation and morphology related to the condition would rather have high significance in the diagnosis or monitoring of gingival / periodontal diseases.

[0260] Periodontal disease is a set of inflammatory conditions affecting the tissues surrounding the teeth, starting with gingivitis in its early stage. In its more serious form (periodontitis), gingiva can pull away from the tooth and a space between the tooth and the surrounding gingiva is formed (periodontal pocket). Periodontal pockets provide an ideal environment for bacteria to grow and may spread infection to the structures that keep teeth anchored in the mouth, and the underlying bone is destroyed (bone loss). As periodontal disease advances leading to more bone loss, the teeth may loosen or fall out. The prevalence of periodontitis is high and a report from US showed that 47.2% of adults aged 30 years and older have some form of periodontal disease, and the prevalence increases with age were 70.1% of adults 65 years and older have periodontal disease (Eke et al., Prevalence of Periodontitis in Adults in the United States: 2009 and 2010. J Dent Res. 2012). The most frequently used diagnostic tool for assessing the health status and attachment level of the tissues surrounding the teeth is a periodontal probe, which is placed between the gingiva and the tooth. This clinical assessment is very time-consuming and thus also expensive, is unpleasant for the patient, and also lacks reproducibility (Shayeb et al., 2014). To fill out a detailed periodontal chart, typically, 6 sites per tooth are probed, and for full-mouth pocket depth measurement, the dentist or the hygienist uses up to 20 min for probing and measuring the pocket depth of a patient. Furthermore, it is well known that the traditional periodontal probing shows insufficient reproducibility and accuracy, due to significant differences in individual techniques, probe instrument, and variation in probing depth force, even among different operators or even the same operator at different times (Theil et al., 1991), (Andrade et al., 2012). The resulting errors can impact clinical decision-making, especially during longitudinal monitoring of the periodontal status.

[0261] All of the above-mentioned dental diseases are often detected manually by a dental practitioner, and therefore one or more applications, such as a program similar to caries detection program described herein aims at detecting such diseases in automated manner to assist the dental practitioner in assessing efficiently and quickly the oral health of a patient.

[0262] Workflow utilizing intraoral diagnostic software

[0263] As previously elaborated, a patient may visit a dental practitioner several times to get an evaluation and treatment of their oral health. To assist the dental practitioner in assessing the oral health of a patient at a first visit and / or over time by utilizing one or more scan data taken at different times a defined workflow may form part of the dental visit. In this work flow the dental practitioner may use one or more of the application modules described herein. Thus, each of the application modules may form part of the computer system used in the workflow. In an example a workflow could look as follows with reference to Figure 20, where in a first visit (step (1)) a patient may enter into a dental clinic for a first patient visit. At this first visit the oral cavity of the patient may be assessed by the use of an intra oral scanner as previously described. Thus, at step (1) in Figure 20, the patient is scanned using e.g. an intraoral scanner by a dental practitioner. As can be seen in step (2) of the workflow in Figure 20, while scanning, the dental practitioner may be able to see the scanning live on display 8. This first scan may be considered as a baseline scan, for example representing a first scan taken at a first point in time as previously elaborated on. At step (3) of Figure 20, the scan data is further analyzed utilizing one or more software applications, such as the applications modules described herein. Accordingly, at least in step (3) of the illustrated workflow, the dental practitioner may be able to utilize a provided software to analyze the oral state and health of the oral cavity of a patient by applying any of the application modules (forming part of the software) as described herein.

[0264] The step (3) of the workflow may utilize software applications (i.e. application modules) that are configured to detect, classify, monitor, predict, prevent, visualize and / or record any dental condition that may be present in the patient’s oral cavity. Examples of such applications are described throughout the disclosure and each of the applications may be triggered by an application module and may be configured automated procedure embedded in a computer implemented method.

[0265] The recording of the resulting analysis data may be provided by software ensuring the possibility of storing the data and analysis results in for example a dental chart, such as for example a dental chart forming a direct part of the software application or alternatively connecting directly with a patient management system.

[0266] Furthermore, as illustrated in Figure 20, it is also possible that the dental software (including e.g. oral health assessment software) is configured to connect with for example a smart phone application 700 or a cloud service 600, whereby engagement with the patient or any other entity using the analyzed scan data beyond the dental clinic may be enabled.

[0267] All the scan data and the corresponding analysis related thereto of a patient in the first visit, represented by points (1) to (3) in Figure 20 may be considered as baseline scan data and analysis and may be used for further dental health and state tracking when a patient undergoes a second visit. Accordingly, after the first visit to the dental clinic, the patient may be engaging in a second visit (steps (5) to (7)), during which visit the patient undergoes a second scan of the oral health utilizing for example an intraoral scanner. This second visit provides the dental practitioner with a second scan data taken at a second point in time in comparison to the scan data that was taken at the first visit.

[0268] During the second visit the dental practitioner may again utilize analysis software applications to assess, for example, potential changes in the oral health of the patient in comparison to the first visit. Thus, when utilizing one or more of the application modules as described herein, the dental practitioner is enabling the possibility of analyzing the second scan data up against the first scan data thereby allowing a detection of a change in the oral health. That is the software configured as one or more of the application modules described herein may be configured to automatically or by active engagement with the software by the dental practitioner to detect a change in dental condition, providing a classification and / or quantification of a dental condition, monitor for example a development of the dental condition, provide predictive measurements, provide preventive measurements etc.

[0269] Furthermore, in relation to the first visit, on the second visit any finding may automatically be recorded in a dental chart or patient management system.

[0270] With the workflow described in Figure 20, it is easy for the dental practitioner to track over time the development of the oral health of a patient by utilizing any one or the one or more application modules described herein.

[0271] According to examples described herein, the electronic hardware may include microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure. Computer program shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0272] Although some embodiments have been described and shown in detail, the disclosure is not restricted to such details, but may also be embodied in other ways within the scope of the subject matter defined in the following claims. In particular, it is to be understood that other embodiments may be utilized, and structural and functional modifications may be made without departing from the scope of the present invention.

[0273] Benefits, other advantages, and solutions to problems have been described herein with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any component(s) / unit(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as critical, required, or essential features or components / elements of any or all the claims or the invention. The scope of the invention is accordingly to be limited by nothing other than the appended claims, in which reference to a component / unit / element in the singular is not intended to mean “one and only one” unless explicitly so stated, but rather “one or more.” A claim may refer to any of the preceding claims, and “any” is understood to mean “any one or more” of the preceding claims.

[0274] It is intended that the structural features of the devices described above, either in the detailed description and / or in the claims, may be combined with steps of the method, when appropriately substituted by a corresponding process.

[0275] As used, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well (i.e. to have the meaning “at least one”), unless expressly stated otherwise. It will be further understood that the terms “includes,” “comprises,” “including,” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It will also be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, but an intervening element may also be present, unless expressly stated otherwise. Furthermore, “connected” or “coupled” as used herein may include wirelessly connected or coupled. As used herein, the term “and / or" includes any and all combinations of one or more of the associated listed items. The step of any disclosed method is not limited to the exact order stated herein, unless expressly stated otherwise.

[0276] It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" or “an aspect” or features included as “may” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the disclosure. The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects.

[0277] The claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language of the claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more.

Claims

CLAIMS1. A caries lesion detection method comprising the steps of:- obtaining intra-oral surface scan data representing a 3D geometry of at least teeth and gingiva from a patient using an intra-oral scanner;- pre-processing the intra-oral scan data to generate a discretization of the 3D geometry, wherein the discretized 3D geometry comprises facet representations of teeth and gingiva;- pre-processing the discretized 3D geometry to generate granularity elements representing the facets of teeth, and comprising one or more numerical features representing quantified local information of the facets;- inputting the granularity elements to a trained model, wherein the trained model is configured to assigning score probabilities to each of the granularity elements;- mapping the assigned score probabilities of the granularity elements to the corresponding facets of the discretized 3D geometry;- determining for each facet a condition score based on at least one of the mapped score probabilities;- merging facets with nonzero condition score into coherent lesions of the discretized 3D geometry, and- assigning a lesion score to each of the coherent lesions according to at least one of the mapped score probabilities associated with facets of the coherent lesions;- representing the discretized 3D geometry with the lesions and assigned lesion score in a visual display.

2. The caries lesion detection method according to claim 1, wherein the quantified local information comprises at least one of position (x,y,z) of the granularity element, RGB color information, fluorescence red, blue, green information, surface curvature information and facet normal information.

3. The caries lesion detection method according to claim 1, wherein the quantified local information is calculated and / or extracted from the discretized 3D geometry, and is stored in the granularity elements.

4. The caries lesion detection method according to claim 1, wherein the assigned score probabilities comprise a vector of probabilities for two or more scores for each granularity element, wherein the two or more scores represents a severity of caries chosen from the group of no caries (N), initial caries (I), moderate caries (M) and / or severe caries (S).

5. The caries lesion detection method according to claim 1, wherein the mapping comprises:- determining if at least two granularity elements represent one single facet in the discretized 3D geometry; and if so,- averaging the assigned score probabilities of the at least two granularity elements representing the one single facet; and- mapping the average score probabilities to the one single facet.

6. The caries lesion detection method according to claim 1, wherein the mapping comprises:- determining if at least two granularity elements represent one single facet in the discretized 3D geometry; and if so,- assigning a weight to each of the assigned score probabilities of the two granularity elements; and- averaging the assigned score probabilities with the assigned weights; and- mapping the average score probabilities to the one single facet.

7. The caries lesion detection method according to claim 4, wherein determining for each facet a condition score based on at least one of the mapped score probabilities comprises- determining from the vector of probabilities, the score with the highest probability; and- assigning to the facet the score with the highest probability, wherein the score with the highest probability may comprise one of no caries score (N), initial caries score (I), moderate caries score (M) or severe caries score (S).

8. The caries lesion detection method according to claim 4, wherein determining for each facet a condition score based on at least one of the mapped score probabilities comprises- calculating from the vector of probabilities the sum of probabilities represented by the initial caries score (I), moderate caries score (M) and severe caries score (S); and- assigning to the facet the score with the highest probability of the sum of probabilities and the probability of no caries score.

9. The caries lesion detection method according to any of the previous claims, wherein merging facets with nonzero condition scores into coherent lesions of the discretized 3D geometry comprises:- determining groups of inter-connected facets comprising nonzero condition score;- merging to the groups, single facets comprising zero condition score that has at least two neighbors with nonzero condition score, and assigning to the single facets the highest condition score of the neighbors, and- repeating the merging to groups until reaching a set threshold.

10. The caries lesion detection method according to any of the previous claims, wherein assigning a lesion score to each of the coherent lesions comprises:- for each coherent lesion determine the assigned condition scores represented in the coherent lesions, and assigning the condition score with the highest severity to the coherent lesion.

11. The caries lesion detection method according to any of the previous claims, wherein representing the discretized 3D geometry comprises assigning a color to each of the coherent lesions according to the assigned lesion score, wherein an assigned lesion score of initial caries is provided in a first color, a lesion score of moderate caries is provided in a second color and a lesions score of severe caries is provided in a third color.

12. The caries lesion detection method according to any of the previous claims, further comprising computing the lesion score from the facet scores by looking at a distribution of the facet scores inside the lesions.

13. The caries lesion detection method according to any of the previous claims, wherein the learning model comprises a first machine learning model configured for classifying facets of the discretized 3D geometry into healthy facets or facets with caries, and a secondmachine learning model configured for assigning a severity score to the facets with caries determined by the first machine learning model.

14. The caries lesion detection method according to the previous claim 13, wherein the severity score is one the initial caries score (I), moderate caries score (M) or severe caries score (S).

15. The caries lesion detection method according to any of the previous claims 13 or 14, wherein the second machine learning model is disregarding the healthy facets identified by the first machine learning model by processing only facets with caries identified by the first machine learning model.

16. A computer program having instructions which when executed by a computing device or system cause the computing device or system to perform the steps of- obtaining intra-oral scan data representing a 3D geometry of at least teeth and gingiva from a patient using an intra-oral scanner;- pre-processing the intra-oral scan data to generate a discretization of the 3D geometry wherein the discretized 3D geometry comprises facet representations of teeth and gingiva;- pre-processing the discretized 3D geometry to generate granularity elements representing the facets of teeth, and comprising one or more numerical features representing quantified local information of the facets;- inputting the granularity elements to a trained model, wherein the trained model is configured to assigning score probabilities to each of the granularity elements;- mapping the assigned score probabilities of the granularity elements to the corresponding facets of the discretized 3D geometry;- determining for each facet a condition score based on at least one of the mapped score probabilities;- merging facets with nonzero condition score into coherent lesions of the discretized 3D geometry, and assigning a lesion score to each of the coherent lesions according to at least one of the mapped score probabilities associated with facets of the coherent lesions;- representing the discretized 3D geometry with the lesions and assigned lesion score in a visual display.

Images