Graphical representation of internal volume of dental object

Abstract

The present disclosure relates to an intraoral scanning system configured to determine a 3D graphical representation of a dental object. The system may include a handheld intraoral scanner configured to provide an invisible 2D image and a visible sub-scan of a dental subject; and one or more processors configured to determine a point cloud of the invisible 2D image and the visible sub-scans, determine a 3D finite element mesh, and align the point cloud with the 3D finite element mesh, where the 3D finite element mesh includes a plurality of finite elements, and where each finite element of the plurality of finite elements includes a plurality of vertices, determining one or more optical coefficients for each vertex of the plurality of vertices, and wherein the one or more optical coefficients correspond to a dental feature represented by the aligned point clouds; determining a triangular mesh representation of the aligned point clouds corresponding to the visible sub-scans, and wherein the triangular mesh representation comprises triangularly arranged vertices of the plurality of vertices; determining a tetrahedral mesh representation of the aligned point clouds corresponding to the invisible 2D image, and wherein the tetrahedral mesh representation comprises tetrahedral arranged vertices of the plurality of vertices; and wherein the 3D graphical representation is determined based on the triangular mesh representation and the tetrahedral mesh representation.

Description

TECHNICAL FIELD

[0001] The present invention relates to an intraoral scanning system. More specifically, the present invention relates to a system configured to determine a three-dimensional graphical representation based on a tetrahedral mesh representation, and wherein the system comprises a handheld intraoral scanner. BACKGROUND

[0002] In the field of dentistry, caries is one of the most common types of dental conditions, which, if not treated in time, can lead to severe tooth decay and eventually to the loss of one or more infected teeth. If the condition is identified early, it can be easily treated and reversible results can be achieved for the infected tooth area. There are many types of caries, but the most difficult to identify early is the type of interproximal caries, which extends from the enamel towards the dentin of the tooth in the area between adjacent teeth.

[0003] A common method of identifying caries is to assess the formation of a carious cavity due to the loss of dental material during the development of caries through 2D bite-wing X-ray images. However, non-ionizing IR light has proven to be able to perform an equivalent analysis of the internal tooth structure, since this light is able to penetrate the dental material. This makes it possible to image the internal areas of the tooth and even to identify demineralization at an early stage, before it leads to irreversible cavity formation or extends into the dentin or pulp.

[0004] In order to accurately identify interproximal caries and decide whether preventive or minor restorative measures are necessary, it is crucial to be able to identify, quantify and monitor the development of potential early interproximal demineralization lesions. The use of an intraoral scanner in combination with the ability to perform infrared imaging of the tooth makes it possible to accurately reconstruct a detailed 3D surface model of the tooth as well as a volumetric representation of the internal areas of the tooth under the tooth surface.

[0005] The digitization of the surface and internal tooth structure makes it possible to develop and apply the use of sophisticated automated machine learning algorithms to perform a variety of different automated tasks, such as segmenting, identifying and labeling different internal areas of the tooth structure, such as the enamel layer, the dentin, the dentin-enamel junction, demineralized areas related to caries, cracks, etc.

[0006] However, to implement advanced and computationally intensive automated machine learning methods, a proper digital representation of the tooth structure is required, including surface and internal information.

[0007] To date, much work has been done on 2D semantic segmentation (i.e., segmenting an image into specific, labeled components). Work in 3D has been significantly less due to the enormous memory requirements (e.g., a medium resolution image can contain 512x512 pixels, so to achieve the same resolution in voxels, a system can require more than 256 times more memory capacity). It would be highly helpful to provide one or more tools that can assist in analysis and / or guide treatment, to automatically and accurately segment teeth and identify internal components, especially directly on 3D volumetric representations. SUMMARY

[0008] One aspect of the present disclosure is to provide an intraoral scanning system configured to provide a graphical representation of a patient's teeth (i.e., a dental object) that is able to efficiently and with less computational power automatically segment the teeth and dental features. The graphical representation can be a volumetric 3D representation of the dental object that includes a volumetric segmentation of the dental object.

[0009] The system can acquire a volumetric 3D model of a patient's teeth (i.e., a dental object) and convert at least a subset of the volumetric 3D model (e.g., a specific number of points representing each tooth) into a graphical representation that preserves connected structures from the volumetric 3D model in an optimized manner, thereby reducing computational cost.

[0010] The embodiments meet the need to provide an intraoral scanning system configured to automatically, efficiently, and accurately segment individual teeth and dental features from a 3D volumetric model of a patient's dentition by utilizing a graphical representation that results in high accuracy.

[0011] The optimized graphical representation of the 3D volumetric model enables the ability to efficiently and automatically segment teeth and dental features with less computational power utilizing machine learning neural networks (e.g., a set of 3D convolutional neural networks that use the graphical representation of the volumetric 3D representation).

[0012] According to aspects, an intraoral scanning system is disclosed. The intraoral scanning system can be configured to determine a three-dimensional graphical representation of a dental object, the three-dimensional graphical representation comprising a volumetric segmentation of the dental object. The system can include a handheld intraoral scanner configured to provide a non-visible 2D image and a visible sub-scan of the dental object. The non-visible 2D image and the visible sub-scan can be provided by one or more light sources of the handheld intraoral scanner that emit light that scatters from the dental object and is acquired by one or more image sensors of the handheld intraoral scanner. The visible sub-scan can correspond to emitted light comprising wavelengths between, for example, 350 nm to 750 nm, while the non-visible 2D image can correspond to emitted light comprising wavelengths between, for example, 800 nm to 1100 nm.

[0013] The system can include one or more processors configured to determine a point cloud of a visible sub-scan and determine a three-dimensional (3D) finite element mesh and align the point cloud to the 3D finite element mesh. Each point cloud can include a set of three-dimensional coordinates corresponding to a portion of a dental object. The point cloud can then be aligned to the 3D finite element mesh for purposes of determining a volumetric representation of the dental object.

[0014] The point cloud can represent a 3D model of the dental object in the 3D finite element mesh, where the point cloud includes a plurality of points having a set of three-dimensional coordinates (e.g., three-dimensional Cartesian coordinates).

[0015] The 3D finite element mesh can include a plurality of finite elements that are a combination of triangles and tetrahedrons that form a skewed voxel. The tetrahedrons correspond to a volume of a portion of the dental object and the triangles correspond to a surface of the volume of the portion of the dental object.

[0016] The 3D finite element mesh includes a plurality of finite elements, and where each of the plurality of finite elements includes a plurality of vertices. Each of the plurality of finite elements can include one or more of: a tetrahedron, a combination of a tetrahedron and an octahedron, and a parallelepiped (skewed voxel). With the 3D finite element mesh, the one or more processors evaluate mesh points at the plurality of vertices. The evaluation by the one or more processors can include determining one or more optical coefficients for each of the plurality of vertices, and where the one or more optical coefficients correspond to a dental feature represented by the aligned point cloud. The one or more optical coefficients can be used to extract a dental feature of a volumetric segmentation represented by the non-visible 2D image. The dental feature can be an anatomical feature (e.g., enamel, dentin, or pulp), a disease feature (e.g., a crack or a caries), and a mechanical feature (e.g., a filling and / or a composite restoration).

[0017] The one or more optical coefficients can include one or more of the following optical indices: a refractive index, a light absorption index, and a light scattering index, and where the one or more optical coefficients correspond to the non-visible 2D image or the visible sub-scan. The one or more optical coefficients can be used to extract a dental feature or a volumetric segmentation represented by the non-visible 2D image.

[0018] The one or more processors can be configured to determine a triangular mesh representation of the aligned point cloud corresponding to the visible sub-scan, and where the triangular mesh representation includes a triangular arrangement of vertices of a plurality of vertices. The triangular mesh representation corresponds to a three-dimensional graphical representation of an outer surface of the dental object. The outer surface does not contain information about an interior region of the dental object.

[0019] The one or more processors can be configured to determine a tetrahedral mesh representation of the aligned point cloud corresponding to the non-visible 2D image, and wherein the tetrahedral mesh representation comprises a tetrahedral arrangement of vertices of a plurality of vertices. The tetrahedral mesh representation corresponds to a three-dimensional graphical representation of an interior region of the dental object.

[0020] The one or more processors can be configured to provide a three-dimensional graphical representation of the dental object by the triangular mesh representation and the tetrahedral mesh representation. Since the non-visible 2D image and the point cloud of the visible sub-scan have been aligned in the 3D finite element mesh, the triangular mesh representation and the tetrahedral mesh representation are then also aligned, which can then be combined to define a complete 3D graphical representation of the dental object.

[0021] The plurality of finite elements can be slanted voxels. With the 3D graphical representation of the dental object employing slanted voxels and a corresponding regular tessellation of tetrahedral elements, a uniform and simple 3D graphical representation of the dental object is provided, wherein each of the plurality of vertices can have at most four neighbours, whereas a voxel can have six neighbours. The lower number of neighbours provides a simpler 3D graphical representation, which will result in faster computation of a 3D volumetric representation based on the 3D graphical representation.

[0022] Tetrahedrons are the simplest piecewise linear topological elements that can be used to tessellate Euclidean space. Any other finite element, such as a voxel, will have more vertices or facets than a tetrahedron. The slanted voxel technique in combination with a tetrahedral-octahedral subdivision scheme is a very convenient way to obtain a regular and well-balanced tessellation of tetrahedrons of Euclidean space.

[0023] To improve the accuracy of a region of the dental object, the one or more processors are configured to reduce the size of a group of the plurality of finite elements corresponding to the region by an adaptive refinement technique. The one or more processors are configured to perform a subdivision on the grouped plurality of finite elements to reduce the size of the group of finite elements. By using large slanted voxels in low accuracy regions and smaller slanted voxels on high accuracy regions, the memory footprint of the 3D graphical representation can be reduced. The advantage of slanted voxels over regular voxels is that subdividing a slanted voxel into tetrahedrons provides a convenient way to obtain a tessellation of tetrahedrons of Euclidean space with different sized tetrahedrons adapted to the needs of the problem at hand.

[0024] To determine the level of subdivision, i.e. the size of the set of finite elements, an octree data structure is used. The octree data structure comprises a plurality of nodes comprising a set of nodes at different levels of the data structure. Each node corresponds to a finite element, e.g. a slanted voxel. For example, a first level of the data structure comprises non-subdivided finite elements and a second level comprises a subdivision of the finite elements of the first level, and so on for subsequent levels in the data structure. The subdivision can comprise halving of the nodes from the previous level. The one or more processors can be configured to perform a subdivision of the set of finite elements to a final level, wherein substantially the same or the same final level corresponds to the one or more optical coefficients of the nodes that will be removed, while the one or more processors will retain the previous level with respect to the final level. The previous level is considered to be the optimal level of subdivision of the finite elements.

[0025] This slanted voxel representation of the 3D graphical representation has the significant benefit that a search structure, e.g. an octree, can be used on the subdivided slanted voxels. Another benefit is that the implementation of the marching tetrahedra algorithm is much easier than the marching cubes algorithm, as there are in fact only two different cases with a single tetrahedron having a vertex with a positive or negative optical coefficient, while marching cubes has 12 different cases. By using a signed distance field representation and running the marching tetrahedra algorithm, both a triangular mesh representation representing the surface of the dental object and a tetrahedral mesh representation representing the interior region of the dental object can be obtained.

[0026] The one or more processors can be configured to subdivide the plurality of finite elements into a plurality of tetrahedrons and a plurality of triangles, and wherein the plurality of tetrahedrons corresponds to a tetrahedral mesh representation, and wherein the plurality of triangles corresponds to a triangular mesh representation.

[0027] The one or more optical coefficients can be determined by an inverse photo scattering algorithm configured to receive the non-visible 2D image and a measured reflection, absorption or refraction of the corresponding point cloud of the visible sub-scan.

[0028] The one or more processors can be configured to determine the 3D graphical representation via a marching tetrahedra algorithm by connecting the closest point cloud to each of the plurality of vertices. The marching finite element algorithm can be configured to select a set of one or more optical coefficients of a set of the plurality of vertices closest to construct a shape function representing a boundary of a first dental feature. The algorithm can be configured to construct a plurality of shape functions of a plurality of dental feature boundaries, e.g. the first dental feature boundary and a second dental feature boundary. The marching finite element algorithm can be a three-dimensional finite element algorithm, more accurately a three-dimensional nearest node finite element algorithm.

[0029] If the first set and the second set cover the same size of the dental object area, the first set can contain smaller and more tetrahedrons of the plurality of tetrahedrons and smaller and more triangles of the plurality of triangles than the second set. The first set has an increased resolution compared to the second set, which will result in an increased resolution of the dental object area covered by the first set of tetrahedrons and triangles. This is beneficial if a high resolution is needed, e.g. of a caries within the dental object, while a low or normal resolution of dental object areas of less interest to the system user.

[0030] The first set of the plurality of finite elements can correspond to a disease feature, while the second set of the plurality of finite elements can correspond to other dental features than the disease feature, e.g. dental features of less interest to the system user.

[0031] Each of the plurality of triangles represents a portion of a two-dimensional surface of a dental feature. For example, the two-dimensional surface can be an outer surface of a 3D graphical representation of a dental object or a dental arch. Further, the two-dimensional surface can be a surface of a caries arranged within the dental object. The one or more processors can be configured to determine the surface of the dental object in the graphical representation based on the triangulated mesh representation of the aligned point cloud.

[0032] Each of the plurality of tetrahedrons represents a portion of a three-dimensional volume of a dental feature. For example, the three-dimensional volume can be a volume of a 3D graphical representation of a dental feature, such as dentin, a caries, etc. The one or more processors can be configured to determine an internal region of the dental object in the graphical representation based on the tetrahedral mesh representation of the aligned point cloud.

[0033] An advantage of the 3D graphical representation of the dental object is that a faster and simpler way for determining a volumetric representation of the dental object is achieved. The volumetric representation of the dental object comprises a three-dimensional model of the outer surface and the internal region of the dental object. The 3D model of the dental object is displayed on a graphical user interface and the user is able to zoom in on the dental object at a higher level of detail to study the caries and navigate within the dental object for the purpose of studying other features of the dental object. The one or more processors can be configured to determine the volumetric representation of the 3D graphical representation by a graph neural network, and wherein the 3D graphical representation is received by the graph neural network and outputs a volumetric representation containing segmented teeth and gums of the dental object. The one or more processors can be configured to train the graph neural network by a volumetric representation determined based on a CBCT scan and / or an intraoral scan acquired by a handheld intraoral scanner.

[0034] The alignment of the point cloud of the non-visible 2D image with the visible sub-scan can be provided by a position and orientation of the handheld intraoral scanner during the non-visible sub-scan and the visible sub-scan.

[0035] The volumetric representation can be determined differently. The one or more processors are configured to determine one or more volume optical coefficients for each of the plurality of vertices of the graphical representation based on the corresponding one or more optical coefficients and a neural radiance field algorithm, and determine the volumetric representation of the surface and internal regions of the dental object based on the one or more volume optical coefficients.

[0036] To improve the speed of the neural radiance field algorithm, a lower resolution 3D finite element mesh is fed into the neural radiance field algorithm, and this means that the algorithm only has to process fewer optical coefficients to generate the voxel representation of the dental object. The resolution of the 3D finite element mesh can be further reduced since many adjacent optical coefficients can have the same or approximately the same value representing the same dental feature. If a group of optical coefficients has the same or approximately the same value, then the group of optical coefficients can be merged into one optical coefficient. The one or more optical coefficients can comprise a plurality of groups of optical coefficients, which would result in a reduction in the resolution of the 3D finite element mesh. The one or more processors can be configured to determine a resolution-reduced 3D finite element mesh of the 3D finite element mesh, and wherein the resolution-reduced 3D finite element mesh comprises a plurality of resolution-reduced vertices, and wherein the resolution-reduced 3D finite element mesh comprises a plurality of resolution-reduced finite elements. The one or more processors can be configured to align the point cloud with the resolution-reduced 3D finite element mesh, and wherein the alignment of the non-visible 2D image and the point cloud of the visible sub-scan is provided by the position and orientation of the handheld intraoral scanner during the non-visible sub-scan and the visible sub-scan. The one or more processors can be configured to determine one or more resolution-reduced optical coefficients for each of the plurality of resolution-reduced vertices by the inverse photo scattering algorithm, and determine one or more intermediate optical coefficients for each of the plurality of vertices of the graphical representation by interpolating between the one or more resolution-reduced optical coefficients of the plurality of resolution-reduced vertices of the resolution-reduced 3D finite element mesh. The one or more processors can be configured to determine one or more volume optical coefficients for each of the plurality of vertices of the graphical representation based on the corresponding one or more intermediate optical coefficients and the neural radiance field algorithm, and determine the volumetric representation of the surface and internal regions of the dental object based on the one or more volume optical coefficients.

[0037] The neural radiance field algorithm can be configured to process the estimated position and orientation of the handheld intraoral scanner, determine a ray or cone cast object based on the estimated position and orientation for each pixel of the handheld intraoral scanner image sensor unit, determine one or more volume optical coefficients for each of a plurality of vertices at which the ray or cone cast object intersects based on a loss function and the one or more optical coefficients, and wherein the neural radiance field algorithm is configured to be trained using the non-visible 2D image.

[0038] The one or more processors can be configured to train the neural radiance field algorithm by determining a 2D invisible image in the invisible 2D image, receiving estimated positions and directions of the handheld intraoral scanner corresponding to a ray or cone projection object of each pixel of the 2D invisible image. The ray or cone projection object reflects an observation axis or observation volume, respectively, of each pixel. Further, the one or more processors can be configured to train the neural radiance field algorithm by determining one or more volume optical coefficients of each of a plurality of vertices of intersection of the ray or cone projection objects based on the neural radiance field algorithm using the estimated positions and directions of the handheld intraoral scanner and the one or more optical coefficients; and determining a synthetic pixel value of each of the ray or cone projection objects based on the corresponding determined one or more volume optical coefficients of each of the plurality of vertices of intersection of the ray or cone projection objects; and minimizing a loss function between the synthetic pixel value and a corresponding true pixel value of each pixel of the 2D invisible image.

[0039] The invisible 2D image contains infrared information or near-infrared information. The infrared information or near-infrared information contains wavelengths between 750 nm and 1250 nm.

[0040] The system can be configured to display the graphical representation and / or a volume representation determined based on the graphical representation. The system can comprise a display unit configured to display a 3D model of the dental object, the 3D model containing the 3D graphical representation and / or the volume representation.

[0041] The display unit can be configured to display a 3D model of the dental object, the 3D model containing a volume representation of a surface and an interior region of the dental object.

[0042] The one or more processors can be configured to determine a first dental feature boundary of a first dental feature based on the determined one or more optical coefficients or one or more volume optical coefficients, and wherein the first dental feature boundary corresponds to the invisible 2D image, and wherein the one or more processors are configured to determine an interior measurement of the first dental feature boundary. The first dental feature boundary can be represented within the 3D finite element mesh or on a volume model of the dental object determined based on the 3D finite element mesh. The first dental feature boundary corresponds to a circumference of the dental feature represented within the 3D finite element mesh or the volume model of the dental object.

[0043] The interior measurement of the interior region within the 3D finite element mesh results in an improved accuracy of the interior region measurement. Thus, the volume of the dental feature can be determined with high accuracy, thereby more accurately determining the influence of the dental feature within the dental object.

[0044] By determining a dental feature boundary, an analytical approach is provided to determine a measure of a dental feature that results in a more accurate and faster measurement of the dental feature compared to a manual measurement performed by a dentist or orthodontist. The dental feature boundary determines an outer contour or shape of the dental feature within a dental object, and the outer contour or shape allows one or more processors to determine a size of the dental feature, where the size can be a distance between two points on the outer contour or shape. The size can correspond to a volume of the dental feature. The dental feature boundary can be a three-dimensional dental feature boundary determined within a 3D finite element mesh.

[0045] The dental feature boundary can include a plurality of vertices that contain optical coefficients corresponding to the dental feature. The plurality of vertices that contain optical coefficients corresponding to the dental feature are connected by marching finite element algorithms or spline functions, and an output of the marching finite element algorithms or spline functions is the dental feature boundary that contains the outer contour or shape of the corresponding dental feature.

[0046] An internal measure can be determined between a first portion of the first dental feature boundary and a second portion of the first dental feature boundary. The internal measure can be a distance between the first portion and the second portion, and the distance can be a minimum distance or a maximum distance. For example, the dental feature can be dental enamel, and the first portion can correspond to an upper portion of the dental enamel, and the second portion can correspond to a lower portion of the dental enamel, and where the upper portion is opposite the lower portion. In this example, the minimum distance corresponds to a thinnest portion of the dental enamel, and where the maximum distance corresponds to a thickest portion of the dental enamel. In a more general context, by knowing the minimum distance of all dental objects (e.g., teeth) of a dental arch (e.g., upper and lower jaw), a dentist or orthodontist can know which teeth have a critical enamel thickness that has a critical short propagation distance of potential caries from a surface to a dentin of the dental object.

[0047] The one or more processors can be configured to determine a plurality of internal measures of the first dental feature boundary or a shape object that fits to a geometry of the first dental feature boundary. The one or more processors can be configured to determine a minimum and / or a maximum internal measure of the plurality of internal measures. The minimum internal measure can correspond to a minimum distance determined between two portions (i.e., the first portion and the second portion) of the dental feature boundary. The maximum internal measure can correspond to a maximum distance determined between two portions (i.e., the first portion and the second portion) of the dental feature boundary.

[0048] The internal measure can be a distance along a normal vector of the first portion of the first dental feature boundary. The normal vector can extend between the first portion and the second portion of the first dental feature boundary.

[0049] In another example, the dental feature can be a caries and the internal measure can be a distance measure along a longitudinal axis extending between the occlusal and the furcation of the dental object that is indicative of the size of the caries. The distance measure can be indicative of the size of the caries towards the enamel or the pulp cavity of the dental object.

[0050] The one or more processors can be configured to perform a fitting of a shape object to a geometry of the first dental feature boundary, and wherein the internal measure is determined based on the fitted shape object. The geometry can be an outline or shape of the dental feature boundary. The shape object can be an ellipse, a sphere, an ellipsoid, or a bounding box. The fitting can be based on a principal component analysis. The fitting of the geometry to the dental feature boundary (e.g. the first dental feature boundary) will result in an implicit function that corresponds to the geometry of the dental feature boundary. The implicit function will improve the resolution of the dental feature boundary geometry. Using such an implicit function and refinement of the finite element volume mesh, using the same marching finite element method, a segment boundary with higher resolution can be generated. The improved resolution will necessarily result in a more accurate internal measure of the dental feature corresponding to the dental feature boundary.

[0051] The one or more processors can be configured to determine a plurality of internal measures of the first dental feature boundary or a shape object fitted to a geometry of the first dental feature boundary, and wherein the internal measure comprises a volume measure that can be determined based on the plurality of internal measures. The volume measure can comprise a volume size of the dental feature corresponding to the first dental feature boundary. For example, the dental feature can be a caries and it can be beneficial for a dentist or orthodontist to measure the volume of the caries as this can better indicate how the size of the caries develops over time. By monitoring the distance between two parts of the caries can provide a measure of the size in a single dimension, whereas by monitoring the volume of the caries can provide a measure of the size in multiple dimensions. Monitoring in multiple dimensions will necessarily result in an improved monitoring of the caries. The caries can not develop in the single dimension in which you monitor the size, but in other dimensions as well. Regardless of which dimension the caries changes in, the size volume will change.

[0052] The internal measure can be a sectional curvature measure of a portion of the first dental feature boundary or a shape object fitted to a geometry of the first dental feature boundary. The sectional curvature measure can be based on a mean or Gaussian curvature measure. Knowing the curvature of a section of the dental feature boundary helps to better understand the overall shape of the dental feature.

[0053] It can be beneficial for a dentist or orthodontist to know different internal measures of an internal portion of a dental object. It can be beneficial for a dentist or orthodontist to know the following internal measures: • Actual depth / extension of caries / cracks in mm or cubic mm, or actual depth / extension of caries / cracks relative to enamel, dentin and pulp cavity.

[0054] • For example, a dentist or orthodontist can need to know whether a caries / crack is only in the enamel, or whether it extends into the dentin or even reaches the pulp cavity.

[0055] • Ratio of affected enamel and / or dentin. For example, a caries can affect the percentage of enamel thickness or dentin thickness.

[0056] • Distance between caries and pulp.

[0057] • Depth of cracks into the coronal part (enamel-dentin) or into the root (cementum and dentin).

[0058] • Volume measure of defects developed within the dental object, and of tooth wear lesions, in enamel, dentin or pulp cavity.

[0059] • Volume or extension of restorations on teeth, and combinations of restorations with e.g. caries and cracks. For example, when scanning a tooth with a composite restoration, it is important to know the “depth” of the restoration relative to enamel, dentin and pulp cavity, and whether there are underlying caries or cracks at the edge / beneath the restoration.

[0060] Some of the above advantages have been achieved by the internal measurement between the first dental feature boundary, but the advantages of applying internal measurements between two different dental features will be explained below.

[0061] The one or more processors can be configured to determine a plurality of dental feature boundaries, and wherein the plurality of dental feature boundaries comprises the first dental feature boundary and at least a second dental feature boundary, and wherein the at least second dental feature boundary corresponds to a second dental feature; and wherein the internal measurement can be determined between the first dental feature boundary and the at least second dental feature boundary. For example, the first dental feature can be a disease feature, such as a crack or caries; and the second dental feature can be an anatomical feature, such as enamel, dentin or pulp cavity.

[0062] In one example, the first dental feature boundary can represent a disease feature, and the second dental feature boundary can represent an anatomical feature.

[0063] The one or more processors can be configured to group the plurality of dental feature boundaries into a plurality of dental feature boundary groups, and wherein the plurality of dental feature boundaries comprises the first dental feature boundary and at least a second dental feature boundary, and wherein each of the plurality of dental feature boundary groups corresponds to a different dental feature. In the case of a scan that implicitly scans the upper and lower jaw of a dental arch, this will result in a plurality of dental feature boundaries.

[0064] The internal measure can be determined between a first portion of the first dental feature boundary and a second portion of the second dental feature boundary. The internal measure can be a distance between the first portion and the second portion, and the distance can be a minimum distance or a maximum distance. For example, the first dental feature can be a caries, and the first portion can correspond to a lower portion of the enamel, and the second portion can correspond to a lower portion of the caries, and wherein the lower portion of the enamel is opposite the lower portion of the caries. In this example, the sign (positive or negative) of the internal measure will determine whether the caries extends through the enamel and into the dentin of the dental object. Further, the level of the internal measure corresponds to the depth of the caries into the enamel.

[0065] The lower portion of the caries and the lower portion of the enamel can be determined along a longitudinal axis of the dental object, the longitudinal axis extending between the occlusion and the furcation of the dental object. The system can be configured to provide an alert signal when the distance between the lower portions is below a certain minimum distance threshold, and wherein the alert signal indicates that the caries is about to enter the dentin region of the dental object. In another case, the system can be configured to monitor a region of the dental object where the distance between the two lower portions is below a maximum distance threshold and above a minimum distance threshold. When the distance between the two lower portions is above the maximum distance, the system can be configured to not monitor the region of the dental object.

[0066] The internal measure can be a distance between the first dental feature boundary and at least a second dental feature boundary, or can be an overlap rate between the first dental feature boundary and the second dental feature boundary. For example, the overlap rate can be a percentage of how much a caries or a crack affects the dentin, enamel, or pulp cavity. The overlap rate can be determined as follows: • determining a second volume of the second dental feature; • determining a first volume of the first dental feature that is overlapped by the second volume, and wherein the overlap rate is determined between the first volume and the second volume. For example, if the first volume is equal to the second volume, then 100% of the second volume is occupied by the first volume. Typically, the first volume will be less than the second volume.

[0067] In another example, the overlap rate can be determined as follows: • determining a second distance of the second dental feature, and wherein the second distance is determined between two portions of the second dental feature; • determining a first distance between the two portions of the first dental feature that is overlapped by the second distance, and wherein the overlap rate is determined between the first distance and the second distance. For example, if the first distance is equal to the second distance, then 100% of the second distance is occupied by the first distance. Typically, the first distance will be less than the second distance.

[0068] The one or more processors can be configured to determine a plurality of sub-internal measurements, including a first sub-internal measurement, a second sub-internal measurement, and a third sub-internal measurement. The first sub-internal measurement can correspond to a distance or volume measurement of the first dental feature, which can correspond to the first dental feature boundary. The second sub-internal measurement can correspond to a distance or volume measurement of the second dental feature, which can correspond to the second dental feature boundary. The third sub-internal measurement can be a distance or volume measure of overlap between the first dental feature boundary and at least the second dental feature boundary, and the internal measurement can be a ratio of the third sub-internal measurement to the first sub-internal measurement or the second sub-internal measurement. The ratio informs a size ratio of how much the first dental feature or the second dental feature is overlapped by the other dental feature. The ratio can be expressed in percentage, and in this example, the dentist or orthodontist would know how much percentage of the dentin or the enamel is affected by the caries or the crack, for example.

[0069] The one or more processors can be configured to determine a progression time for the first dental feature boundary to progress into or partially overlap the second dental feature boundary. The progression time can be determined based on a distance between the first dental feature boundary and at least the second dental feature boundary. Alternatively, the progression time can be determined based on a distance between the first dental feature boundary and at least the second dental feature boundary and a progression time algorithm trained to output a progression time based on a distance between two dental feature boundaries. The training of the progression time algorithm can be based on invisible 2D images taken of different patients, where a progression of dental features identified in the invisible 2D images is monitored over a period of time and registered in the progression time algorithm. By inputting the distance between the first dental feature boundary and the second dental feature boundary into the trained progression time algorithm, the trained progression time algorithm can be configured to determine the progression time by relating the distance to a monitored progression of a similar dental feature (e.g., caries) corresponding to the first dental feature boundary.

[0070] The plurality of dental feature boundaries can be determined by a marching finite element algorithm. The marching finite element algorithm can be configured to select a set of optical coefficients of a set of vertices closest to the set of optical coefficients to construct a shape function representing the first dental feature boundary. The algorithm can be configured to construct a plurality of shape functions of a plurality of dental feature boundaries (e.g., the first dental feature boundary and the second dental feature boundary). The marching finite element algorithm can be a three-dimensional finite element algorithm, or more accurately, a marching cubic or tetrahedral algorithm.

[0071] The determined point clouds of the invisible 2D images and the visible sub-scans can include a first set of point clouds corresponding to the invisible 2D images and a second set of point clouds corresponding to the visible sub-scans. Both the first set of point clouds and the second set of point clouds are aligned to the 3D finite element mesh.

[0072] The one or more processors can be configured to assign each point cloud to an optical coefficient of the one or more optical coefficients. The marching finite element algorithm can then select a set of point clouds closest to a set of the plurality of vertices for constructing shape functions representing a boundary of the first dental feature. The algorithm can be configured to construct a plurality of shape functions for a plurality of dental feature boundaries (e.g., the first dental feature boundary and a second dental feature boundary).

[0073] The one or more optical coefficients can be determined based on a neural radiance field model, wherein the neural radiance field model is configured to determine the one or more optical coefficients based on the spatial position information and the viewing angle information of the handheld intraoral scanner when capturing the non-visible sub-scans and the visible sub-scans.

[0074] The one or more processors can be configured to determine, for each of the plurality of vertices, a relative position between the handheld intraoral scanner and the dental object, wherein the relative position comprises the spatial position information and the viewing angle information of the corresponding projection object. The one or more processors can be further configured to determine the one or more optical coefficients by a neural radiance field model, wherein the neural radiance field model is configured to determine the one or more optical coefficients based on the spatial position information and the viewing angle information.

[0075] The neural radiance field model can be configured to be trained using a plurality of non-visible 2D images, and wherein the projection object can be a ray or a cone. The ray corresponds to a view along a pixel ray, while the cone corresponds to a viewing volume of a pixel.

[0076] Each pixel of the image sensor is configured to receive a non-visible sub-scan, which is then converted by the one or more processors into a composite pixel value for the corresponding pixel of the image sensor. The received non-visible sub-scan contains internal scatter from within the interior region of the dental object, and this means that each composite pixel value contains an average value of internal scatter from different planes of the distribution of the projection object along the corresponding pixel. Each composite pixel value can correspond to each of a plurality of vertices of a host 3D finite element mesh, and wherein the one or more processors are configured to determine one or more optical coefficients for each of the plurality of vertices of the host 3D finite element mesh. The internal scatter of the received non-visible sub-scan can be represented by one or more optical coefficients of a plurality of vertices of a sub-3D finite element mesh, and since the received non-visible sub-scan contains internal scatter captured from different planes, the neural radiance field model is configured to determine a sub-3D finite element mesh for each different plane. The sub-3D finite element meshes are arranged along the projection object such that a projection object starting from a vertex of the host 3D finite element mesh can intersect a corresponding vertex of each sub-3D finite element mesh. As such, the one or more optical coefficients of the vertex of the host 3D finite element mesh can be an average of the one or more optical coefficients of the corresponding vertices of the sub-3D finite element meshes that the projection object intersects.

[0077] The neural radiance field model can be trained by generating, based on the set of input parameters, an optical coefficient for each of the one or more coordinates, the optical coefficient representing a density value of light absorption, light scattering, or refractive index. The neural radiance field model can be trained by determining, based on the corresponding determined optical coefficient for each sub-3D finite element mesh representing internal scattering from within the dental object, a synthesized pixel value of the projected object. Further, the neural radiance field model can be trained by minimizing a loss function between the synthesized pixel value and a corresponding true pixel value of the plurality of pixels of the invisible subscan.

[0078] The invisible subscan comprises 2D infrared image data or 3D infrared image data.

[0079] The one or more processors can be further configured to determine a plurality of mesh points within the 3D surface model and determine the 3D internal geometry by arranging at least one of the synthesized pixel value or the one or more optical coefficients at each of the plurality of mesh points.

[0080] Since most teeth (i.e., dental objects) have similar internal structures, the variation of the one or more optical coefficients for different tooth samples will be limited, and there is a large covariance between the values of the one or more coefficients for the plurality of vertices. The one or more processors can be configured to perform a covariance analysis on the one or more optical coefficients for the plurality of vertices, determine the plurality of descriptive parameters based on the covariance analysis, and wherein the number of the plurality of descriptive parameters is less than the number of the one or more optical coefficients, and wherein the first dental feature boundary can be determined by providing the plurality of descriptive parameters into a marching finite element algorithm of the intraoral scanning system. In this example, the one or more processors will be faster in determining the first dental feature boundary compared to other cases where the marching finite element algorithm receives the one or more optical coefficients for each of the plurality of vertices.

[0081] The covariance analysis can be a principal component analysis or a self-encoder deep learning algorithm, wherein the self-encoder deep learning algorithm comprises a neural network. The neural network can be configured to receive the invisible 2D image and determine the plurality of descriptive parameters, e.g., the one or more optical coefficients for the plurality of vertices. The one or more processors can be configured to train the neural network by receiving one or more CBCT scans of one or more dental objects; determining one or more training optical coefficients for each of the one or more CBCT scans; receiving one or more invisible 2D images of the one or more dental objects from a handheld intraoral scanner; and training the neural network by mapping the one or more training optical coefficients onto a three-dimensional finite element mesh of the one or more invisible subscan.

[0082] The trainable neural network can be configured to classify a particular dental feature based on one or more training optical coefficients, and wherein the neural network can be configured to receive one or more optical coefficients of a plurality of vertices corresponding to an invisible sub-scan and determine a plurality of descriptive parameters of the particular dental feature based on the trained optical coefficients and the received one or more optical coefficients.

[0083] The invisible 2D images can include a plurality of 2D infrared images, wherein a position and an orientation of each of the plurality of 2D infrared images are known. The plurality of 2D infrared images can be supported by a color 2D image and a fluorescence 2D image. The system can receive a CBCT scan of the dental object, thereby creating an accurate voxel model of the dental object. The accurate voxel model can be segmented into a plurality of layers, which are used to create a parameterized voxel or hexahedral mesh. Each of the plurality of 2D infrared images of the dental object is mapped onto the accurate voxel model of the dental object based on the position and the orientation of each of the plurality of 2D infrared images. The accurate voxel model containing the mapped plurality of 2D infrared images is fed into a neural network to train the neural network. With the trained neural network, a parameterized model can be generated, and the model describes the dental object based on the 2D infrared images. The parameterized model contains a plurality of vertices and corresponding one or more optical coefficients.

[0084] It would be advantageous for a dental practitioner to have a measure of the extent of progression of a disease feature, such as a second dental feature boundary, into, for example, an anatomical feature, such as a first dental feature boundary. One or more processors can be configured to determine, in a 3D finite element mesh, a plurality of normal lengths from the second dental feature in a direction toward the first dental feature boundary and along a longitudinal axis of the dental object, and determine a maximum normal length of the plurality of normal lengths. The one or more processors can be further configured to determine a penetration depth of the maximum normal length that penetrates the first dental feature, and wherein the internal measure comprises the penetration depth. The penetration depth can be a distance measured in millimeters or microns. The one or more processors can determine a penetration depth ratio, and wherein the penetration depth ratio is a ratio between the penetration depth and the maximum normal length of the first dental feature boundary along the longitudinal axis.

[0085] A sign of each normal length of the plurality of normal lengths determines whether the normal length of the plurality of normal lengths is located within the first dental feature. For example, if the sign of the normal length is negative, it indicates that the normal length is located within the first dental feature; and if positive, it indicates that the normal length is located outside the first dental feature. One or more processors can be configured to select a set of normal lengths from the plurality of normal lengths, wherein each normal length of the set has a first sign indicating that the set of normal lengths is located within the first dental feature, and determine a volume measure based on the set of normal lengths, and wherein the internal measure is the volume measure.

[0086] The one or more processors can be configured to determine a notification signal when the internal measurement is above a measurement threshold, and wherein the notification signal is displayed on a user interface of the system. In this way, the dentist will be alerted if the system provides any critical internal measurements.

[0087] It is important that the internal measurements are visualized in such a way that the dentist can easily interpret and understand the internal measurements and ultimately interpret and understand whether the internal measurements are assessed by the system as being a critical risk for one or more dental diseases. The intraoral scanning system can comprise a user interface which can be configured to display a 3D model of the dental object and a 2D cross-section of the dental object, and wherein the internal measurements are displayed on the 2D cross-section of the dental object. With the 3D model, the user of the system can be enabled to select which parts of the 3D model the user wants to display as a 2D cross-section. The one or more internal measurements can be displayed on the 2D cross-section with or without the dental feature boundaries. The internal measurements between dental features or dental feature can be displayed with an arrow or a line which indicates where the internal measurement was performed. In case the internal measurement comprises a volume measure, the dental feature to which the volume measure corresponds can be colored to enhance the contrast of the dental feature and the internal measurement is displayed on or next to the dental feature.

[0088] The dentist can be able to add two or more measurement points on the 3D model, a 2D image of a cross-section of the 3D model or a 2D cross-section image and the one or more processors are configured to perform an internal measurement between the two or more measurement points. The two or more measurement points can be added by a cursor on the user interface, wherein the cursor can be moved around by a motion sensor of a handheld intraoral scanner or by a mouse. In this case, the one or more processors are configured to assign each of the two or more measurement points to the nearest vertex of a plurality of vertices, whereby the one or more processors can be configured to determine the internal measurement between the two or more measurement points. A first measurement point of the two or more measurement points can correspond to a first part of a dental feature boundary and a second measurement point of the two or more measurement points can correspond to a second part of the dental feature boundary.

[0089] The dentist can be able to add a contour line on the 3D model that encloses a dental feature depicted on the 3D model, a 2D image of a cross-section of the 3D model, or a 2D cross-sectional image, and the one or more processors can be configured to perform an internal measurement on the enclosed dental feature. In this example, the internal measurement can be a volume measurement of the enclosed dental feature. The contour line on the 3D model can enclose an overlap between the first and second dental feature boundaries, i.e., an overlap between the first and second dental features. The contour line can be added by a cursor on the user interface, where the cursor can be moved around by a motion sensor of the handheld intraoral scanner or by a mouse. In this example, the one or more processors are configured to assign the contour line to the nearest vertex of the plurality of vertices, whereby the one or more processors can be configured to determine the internal measurement of the enclosed overlap or dental feature.

[0090] A user of the system can select a dental object in the displayed 2D cross-section by a marker on the displayed three-dimensional model of the dental object. The marker can be a window that makes it easier for the user to see which dental object on the 3D model is selected to become the view of the 2D cross-section.

[0091] The first dental feature boundary can be displayed on the 3D model of the dental object and / or the 2D cross-sectional image of the dental object. By displaying the dental feature boundary on the 3D model and / or the 2D cross-sectional image, a better interpretation of the internal measurement performed by the one or more processors can be provided. The dental feature boundary can be displayed on the dental object as its corresponding actual dental feature, such as enamel, dentin, pulp chamber, crack, caries, filling, or composite restoration, etc.

[0092] The one or more processors can be configured to change the viewing angle of the 2D cross-section on the location of a marker on the 3D model. The marker can be a line that indicates the cross-section displayed on the 2D cross-section. The user can rotate the marker, and the cross-section displayed on the 2D cross-section will also rotate symmetrically with the rotation of the marker.

[0093] The one or more processors can be configured to change the viewing depth into the dental object of the 3D model. By increasing the viewing depth, the user can view the dental object more deeply; and when decreasing the viewing depth, the user can view the dental object less deeply. The viewing depth can be adjusted by a slider on the user interface, or can be adjusted by pressing a button on the handheld intraoral scanner and the movement of the handheld intraoral scanner detected by a motion sensor disposed within the handheld intraoral scanner.

[0094] The user interface can include a transparency unit, which can be configured to change the transparency of the first dental feature boundary. In certain examples, it can be beneficial for the user to not see one or more dental features on the dental object, and thus, the user can be able to change the transparency of the selected dental feature boundary. For example, the user can only want to view the anatomical features and have no interest in viewing any disease features, and in this example, the user can increase the transparency of the disease features to the extent that they are not visible to the user on the user interface. In yet another example, the user can be able to remove unwanted dental features on the dental object by selecting the unwanted dental features and then pressing a remove button on the keyboard or a graphical remove button on the user interface. By removing any dental feature or reducing the transparency of any dental feature, the transparency of the corresponding internal measurements applied to the same dental object is also removed or reduced.

[0095] The user interface can include a selector, which can be configured to select one or more of a plurality of dental feature boundaries to be displayed, and wherein the plurality of dental feature boundaries includes the first dental feature boundary and at least a second dental feature boundary. The transparency unit can be configured to change the transparency of the selected one or more of the plurality of dental feature boundaries.

[0096] The user interface can be configured to display a plurality of dental feature boundaries, and wherein the plurality of dental feature boundaries includes the first dental feature boundary and at least a second dental feature boundary, and wherein the plurality of dental feature boundaries can have different colors with the purpose of increasing the contrast between dental features on the dental object, thereby improving the visualization of the dental features.

[0097] The one or more processors can be configured to determine a first dental feature corresponding to the first dental feature boundary, and wherein the first dental feature is determined based on the one or more optical coefficients corresponding to the first dental feature boundary and a dental feature algorithm. The dental feature algorithm includes one or more of an anatomical feature determiner, a disease feature determiner, and a mechanical feature determiner. The anatomical feature determiner can be configured to determine that the one or more optical coefficients correspond to an anatomical feature when the one or more optical coefficients are within an anatomical feature coefficient range. The disease feature determiner can be configured to determine that the one or more optical coefficients correspond to a disease feature when the one or more optical coefficients are within a disease feature coefficient range. The mechanical feature determiner can be configured to determine that the one or more optical coefficients correspond to a mechanical feature when the one or more optical coefficients are within a mechanical feature coefficient range. The anatomical feature coefficient range, the disease feature coefficient range, and the mechanical feature coefficient range can be pre-determined and stored in a memory of the system. In another example, the ranges can be occasionally updated according to manual inputs by users. The users can identify dental features on a graphical user interface and classify the identified dental features as anatomical features, disease features, or mechanical features. The manual inputs can be provided by different users of the system, and the memory can be a cloud server or a server connected to an intraoral scanning system, which can include multiple handheld intraoral scanners and user interfaces connected through a wireless network.

[0098] The one or more processors can be configured to align the non-visible sub-scan with the visible sub-scan and determine a point cloud of the aligned sub-scans. The alignment can be based on a position and a viewing angle of the handheld intraoral scanner when capturing the sub-scans. The position and the viewing angle can be determined by motion sensors disposed within the handheld intraoral scanner. Alternatively, the one or more processors can be configured to align the non-visible sub-scan with the visible sub-scan based on common dental features between the non-visible sub-scan and the visible sub-scan. For example, the position of the handheld intraoral scanner can be determined by recognizing dental objects in the visible sub-scan and the same dental objects in the non-visible sub-scan based on the common dental features, such as geometry and / or shade colors of the dental objects, or one or more of the dental features. The one or more processors can be configured to determine the viewing angle of the dental objects of the visible sub-scan and the non-visible sub-scan based on triangulation between one or more light sources of the handheld intraoral scanner, one or more image sensors of the handheld intraoral scanner, and the dental objects or one or more dental features.

[0099] To enhance contrast between different dental features of the dental object, the one or more processors can be configured to determine a point cloud of the visible sub-scan and a point cloud of a composition of the visible sub-scan and the non-visible sub-scan. The one or more processors can be configured to determine surface information from the visible sub-scan in real-time for generating or updating a three-dimensional (3D) model of the dental object. The one or more processors can also be configured to determine a plurality of dental feature boundaries, including the first dental feature boundary and / or the second dental feature boundary, based on the composition or the non-visible sub-scan. Through the composition of the non-visible sub-scan and the visible sub-scan, the contrast between the dental features of the dental object becomes further enhanced relative to a case where the non-visible sub-scan is not composed.

[0100] The non-visible sub-scan mainly includes scattering from inside the dental object (i.e., the interior region of the dental object) and significantly less surface reflection of the teeth. The visible sub-scan mainly includes surface reflection of the teeth and significantly less reflection from inside the teeth (i.e., the interior region of the dental object).

[0101] The composed scan information does not contain a superposition of two 2D images, where each of the two 2D images is related to a different emission wavelength from the projector unit. In this example, no enhancement of the internal structure information is provided. Rather, the composed scan information can be a combination of intensity levels of each pixel of the image sensor unit related to different wavelengths. For example, during a first time period, the image sensor unit can capture light information related to a visible wavelength; and during a second time period, the image sensor unit can capture light information related to an infrared or near-infrared wavelength, and the intensity levels of both time periods are recorded and combined to enhance the dental features. The combination of intensity levels can be done digitally through subtraction and / or addition of the intensity levels. In another example, the intensity levels can be captured and recorded during at least three time periods for at least three different wavelengths (e.g., a white wavelength, a blue wavelength, and a near-infrared wavelength).

[0102] The one or more processors can be configured to display the composed scan information and the 3D model on a display unit of the system, and wherein the dental feature boundaries can be added to the composed scan information and / or the 3D model. BRIEF DESCRIPTION OF DRAWINGS

[0103] Various aspects of the disclosure can be best understood from the following detailed description taken in conjunction with the accompanying drawings. The drawings are schematic and simplified for clarity and focus, and they only show details considered to be relevant for improving understanding of the claims. Throughout the drawings, like reference numerals are used for like or corresponding parts. Individual features of each aspect can each be combined with any or all of the features of the other aspects. These and other aspects, features, and / or technical effects will be apparent from and elucidated with reference to the drawings described below, in which: Figure 1 An example of a graphical representation of a dental object is shown; Figures 2A to 2E Different examples of a system are shown; Figure 3 An example of one or more processors is shown; Figure 4 Another example of one or more processors is shown; Figure 5 An example of a graph neural network is shown; Figure 6A And 6B Different examples of a neural radiance field algorithm are shown; Figure 8 Another example of one or more processors is shown; Figure 9 An intraoral scanning system is shown; Figure 10 A dental object defined as a 3D finite element mesh is shown; Figure 11A And 11B An example of one or more processors is shown; Figure 12 An example of an internal measurement is shown; Figure 13A , 13B And 13C different examples of an internal measurement are shown; Figure 14A , 14B And 14C different examples of determining boundaries of a plurality of dental features are shown; Figure 16A And 16B An example of one or more processors configured to determine one or more optical coefficients is shown; Figure 17 An example of one or more processors configured to perform a covariance analysis is shown; Figure 18A And 18B An example of how a neural network can be trained to determine a plurality of descriptive parameters is shown; Figure 19 An example of a user interface of a system is shown; and Figure 20A And 20B An example of one or more processors performing finite element tessellation is shown. DETAILED DESCRIPTION

[0104] The detailed description set forth below, in connection with the appended drawings, is intended as a description of various configurations and is not intended to limit the concepts. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts can be practiced without these specific details. Numerous aspects of devices, systems, media, programs, and methods are described by various blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as “elements”). Such elements can be realized by electronic hardware, computer programs, or any combination thereof, depending on the particular application or

[0105] Electronic hardware can include microprocessors, microcontrollers, digital signal processors (DSPs), field-programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure. Computer programs, whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise, should be interpreted broadly to encompass instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc.

[0106] Scans for providing intraoral scan data can be performed by a dental scanning system that can comprise a handheld intraoral scanning device such as the TRIOS series scanners from 3Shape A / S. The dental scanning system can comprise wireless capabilities provided by a wireless network unit. The scanning device can employ a scanning principle such as triangulation-based scanning, confocal scanning, focused scanning, ultrasound scanning, x-ray scanning, stereo vision, structure from motion, optical coherence tomography, OCT, or any other scanning principle. In an embodiment, the scanning device is capable of obtaining surface information by projecting a pattern and translating the focal plane along the optical axis of the scanning device and capturing a plurality of 2D images at different focal plane positions, such that each series of captured 2D images corresponding to each focal plane forms a stack of 2D images. The acquired 2D images are also referred to herein as raw 2D images, where raw in the present context means that the images are not subjected to image processing. The focal plane positions are preferably moved along the optical axis of the scanning system, such that for a given view of the object, i.e. for a given arrangement of the scanning system relative to the object, the 2D images captured at a plurality of focal plane positions along the optical axis form a stack of said 2D images (also referred to herein as a sub-scan). After moving the scanning device relative to the object or imaging the object at different views, a new stack of 2D images for this view can be captured. The focal plane positions can be changed by at least one focusing element, e.g. a moving focusing lens. During a scanning session, the scanning device is generally moved relative to the dentition and angled such that at least some groups of sub-scans at least partially overlap, in order to enable reconstruction of a digital dental 3D model by stitching the overlapping sub-scans together in real time, and to display the progress of the virtual 3D model on a display as feedback to the user. The result of the stitching is a digital 3D representation of the surface that is larger than the surface that can be captured by a single sub-scan, i.e. larger than the field of view of the 3D scanning device. The stitching, also referred to as registration and fusion, works by identifying overlapping regions of the 3D surface in the various sub-scans and transforming the sub-scans to a common coordinate system such that the overlapping regions match, ultimately resulting in a digital 3D model. Iterative Closest Point (ICP) algorithms can be used for this purpose. Another example of a scanning device is a triangulation scanner, where a time-varying pattern is projected onto the dental arch and a sequence of images of different pattern configurations is acquired by one or more cameras positioned at an angle relative to the projector unit.

[0107] A handheld intraoral scanner can include one or more light projectors configured to generate an illumination pattern to be projected onto a three-dimensional dental object during a scanning session. The light projector(s) can include a light source, a mask having a spatial pattern, and one or more lenses, such as a collimating lens or a projection lens. The light source can be configured to generate light of a single wavelength or a combination of wavelengths (monochromatic or polychromatic). The combination of wavelengths can be produced by using a light source configured to produce light including different wavelengths, such as white light. Alternatively, the light projector(s) can include multiple light sources, such as LEDs, that individually produce different wavelengths of light that can be combined to form light including different wavelengths, such as red, green, and blue. Thereby, the light produced by the light source can be defined by a wavelength that defines a specific color, or by a range of different wavelengths that define a combination of colors, such as white light. In an embodiment, the scanning device includes a light source configured for exciting fluorescent materials of the teeth to obtain fluorescence data from the dental object. Such a light source can be configured to produce a narrow range of wavelengths. In another embodiment, the light from the light source is infrared (IR) light that is capable of penetrating dental tissue. The light projector(s) can be a DLP projector using a micro-mirror array for generating a time-varying pattern, or a diffractive optical element (DOF), or a backlit mask projector, where the light source is placed behind a mask having a spatial pattern, such that the light projected on the surface of the dental object is patterned. The backlit mask projector can include a collimating lens for collimating the light from the light source, placed between the light source and the mask. The mask can have a checkerboard pattern, such that the generated illumination pattern is a checkerboard pattern. Alternatively, the mask can feature other patterns, such as lines or dots.

[0108] Color texture of the dental arch can be obtained by illuminating the object using different monochromatic colors, such as separate red, green, and blue, or using polychromatic light, such as white light. The 2D images can be acquired during a flash of white light.

[0109] Generally, the process of obtaining surface information of a dental arch in real-time requires the scanning device to illuminate the surface and obtain a large number of 2D images. Typically, high-speed cameras with frame rates of 300-2000 2D frames per second are used depending on the technology and 2D image resolution. The scanning device needs to process a large amount of image data, either to forward the raw image data stream directly to an external processing device, or to perform some image processing before transmitting the data to an external device or display. This process requires multiple electronic components inside the scanner to operate with high workloads, thereby requiring high current demands.

[0110] The scanning device comprises one or more light projectors configured to generate an illumination pattern to be projected onto the three-dimensional dental arch during a scanning session. The light projector(s) preferably comprise a light source, a mask with a spatial pattern, and one or more lenses, such as collimating lenses or projection lenses. The light source can be configured to generate light of a single wavelength or a combination of wavelengths (monochromatic or polychromatic). The combination of wavelengths can be produced by using a light source configured to produce light comprising different wavelengths, such as white light. Alternatively, the light projector(s) can comprise multiple light sources, such as LEDs, that individually produce different wavelengths of light that can be combined to form light comprising different wavelengths, such as red, green, and blue. Thereby, the light produced by the light source can be defined by a wavelength defining a specific color, or by a range of different wavelengths defining a combination of colors, such as white light. In an embodiment, the scanning device comprises a light source configured for exciting fluorescent material of the teeth to obtain fluorescence data from the dental arch. Such a light source can be configured to produce a narrow range of wavelengths. In another embodiment, the light from the light source is infrared (IR) light that is able to penetrate dental tissue. The light projector(s) can be DLP projectors using a micro-mirror array for generating a time-varying pattern, or a diffractive optical element (DOF), or a backlit mask projector, wherein the light source is placed behind a mask with a spatial pattern, so that the light projected onto the surface of the dental arch is patterned. The backlit mask projector can comprise a collimating lens for collimating the light from the light source, placed between the light source and the mask. The mask can have a checkerboard pattern, so that the generated illumination pattern is a checkerboard pattern. Alternatively, the mask can feature other patterns, such as lines or dots.

[0111] The scanning device preferably further comprises optical components for guiding the light from the light source to the surface of the dental arch. The specific arrangement of the optical components depends on whether the scanning device is a focus scanning apparatus, a scanning device using triangulation, or any other type of scanning device. The same applicant further describes a focus scanning apparatus in EP 2 442 720 B1, the entire content of which is incorporated herein.

[0112] Using optical components of the scanning device, light reflected from the dental arch in response to the illumination of the dental arch is directed toward the image sensor(s). The image sensor(s) are configured to generate a plurality of images based on the incident light received from the illuminated dental arch. The image sensor unit can be a high-speed image sensor, such as an image sensor configured to acquire images with an exposure of less than 1 / 1000 of a second or a frame rate of more than 250 frames per second (fps). As an example, the image sensor can be a rolling shutter (CCD) or a global shutter sensor (CMOS). The image sensor(s) can be a monochrome sensor containing a color filter array such as a Bayer filter and / or a further filter that can be configured to substantially remove one or more color components from the reflected light before the reflected light is converted into an electrical signal and only retain the other, non-removed components. For example, such a further filter can be used to remove a certain portion of the white light spectrum, such as the blue component, and only retain the red and green components from the signal generated in response to the excited fluorescent material of the teeth.

[0113] The network unit can be configured to connect the dental scanning system to a network comprising a plurality of network elements including at least one network element configured to receive the processed data. The network unit can comprise a wireless network unit or a wired network unit. The wireless network unit is configured to wirelessly connect the dental scanning system to a network comprising a plurality of network elements including at least one network element configured to receive the processed data. The wired network unit is configured to establish a wired connection between the dental scanning system and a network comprising a plurality of network elements including at least one network element configured to receive the processed data.

[0114] The dental scanning system preferably further comprises a processor configured to generate scanning data (such as extra-oral scanning data and / or intra-oral scanning data) by processing two-dimensional (2D) images acquired by the scanning device. The processor can be part of the scanning device. As an example, the processor can comprise a field-programmable gate array (FPGA) and / or an advanced RISC machine (ARM) processor located on the scanning device. The scanning data comprises information related to a three-dimensional dental arch. The scanning data can comprise any of the following: 2D images, 3D point clouds, depth data, texture data, intensity data, color data, and / or combinations thereof. As an example, the scanning data can comprise one or more point clouds, wherein each point cloud comprises a set of 3D points describing a three-dimensional dental arch. As another example, the scanning data can comprise images, each image comprising image data described by, for example, image coordinates and a timestamp (x, y, t), wherein depth information can be inferred from the timestamp. The image sensor(s) of the scanning device can acquire a plurality of raw 2D images of the dental arch in response to illuminating the object using one or more light projectors. The plurality of raw 2D images can also be referred to herein as a stack of 2D images. The 2D images can subsequently be provided as input to the processor, which processes the 2D images to generate the scanning data. The processing of the 2D images can comprise a step of determining which part of each of the 2D images is in focus in order to infer / generate depth information from the images. The internal depth information can be used to generate a 3D point cloud comprising a set of 3D points described by, for example, Cartesian coordinates (x, y, z) in space. The 3D point cloud can be generated by the processor or another processing unit. Each 2D / 3D point can also comprise a timestamp indicating when the 2D / 3D point was recorded, i.e., which image from the stack of 2D images the point originates from. The timestamp is related to the z-coordinate of the 3D point, i.e., the z-coordinate can be inferred from the timestamp. Thus, the output of the processor is the scanning data, and the scanning data can comprise image data and / or depth data described by, for example, image coordinates and a timestamp (x, y, t), or alternatively (x, y, z). The scanning device can be configured to transmit other types of data in addition to the scanning data. Examples of data include 3D information, texture information, such as infrared (IR) images, fluorescence images, reflective color images, x-ray images, and / or combinations thereof.

[0115] Figure 1An example of a graphical representation 50 of a dental object 2 is shown, which graphical representation comprises a 3D finite element mesh 4 defined in a three-dimensional coordinate system 6. The 3D finite element mesh 4 comprises a point cloud determined based on an invisible 2D image and a visible sub-scan, and the 3D finite element mesh 4 comprises a plurality of finite elements (8, 9, 10), which in this example comprises a combination of triangles (8A, 8B, 8C, 8D) and tetrahedrons (9A, 9B, 9C, 9D) forming a slanted voxel 10. The tetrahedrons 9 correspond to a volume of a portion of the dental object 2, and the triangles 8 correspond to a surface of the volume of the portion of the dental object.

[0116] Figure 2A 、 2B , 2C, 2D and 2E show an intraoral scanning system 1 configured to determine a three-dimensional graphical representation 50 of a dental object 2. The system 1 comprises a handheld intraoral scanner 20 configured to provide an invisible 2D image and a visible sub-scan of the dental object 2. The invisible 2D image can contain infrared information or near-infrared information. The system 1 comprises one or more processors 13 configured to determine 21 a point cloud of the invisible 2D image and the visible sub-scan, and to determine 22 a three-dimensional finite element mesh 4, and to align the point cloud with the 3D finite element mesh 4, wherein the 3D finite element mesh 4 comprises a plurality of finite elements (8, 9, 10). Each of the plurality of finite elements comprises a plurality of vertices. The one or more processors 13 are further configured to determine 23 one or more optical coefficients for each of the plurality of vertices, and wherein the one or more optical coefficients correspond to a dental feature represented by the aligned point cloud. The one or more processors 13 are configured to determine 5 a triangular mesh representation of the aligned point cloud corresponding to the visible sub-scan, and wherein the triangular mesh representation comprises triangularly arranged vertices of the plurality of vertices. The one or more processors 13 are further configured to determine 25 a tetrahedral mesh representation of the aligned point cloud corresponding to the invisible 2D image, and wherein the tetrahedral mesh representation comprises tetrahedrally arranged vertices of the plurality of vertices, and wherein the three-dimensional graphical representation 50 is determined 26 based on the triangular mesh representation and the tetrahedral mesh representation. Figure 2B More particularly, the 3D finite element mesh 4 is shown, which mesh comprises a plurality of slanted voxels (10A, 10B, 10C, 10D, 10E) having an aligned point cloud 27. Figure 2CA portion 28 of the 3D finite element mesh 4 is shown, and wherein the one or more processors 13 determine one or more optical coefficients 32 for each of a plurality of vertices 33. For example, the one or more optical coefficients 32 are determined by an inverse photo scattering algorithm configured to receive the measured reflectance, absorption, or refraction of the invisible 2D image and the corresponding point cloud of the visible sub-scan. The values of the one or more optical coefficients 32 correspond to a dental feature 40, such as an anatomical feature, a disease feature, or a mechanical feature. In this example, the dental feature 40 is the enamel of the dental object 2. Further, the plurality of vertices 33 containing the optical coefficients corresponding to the dental feature are connected by a marching finite element algorithm or a spline function, and the output of the marching finite element algorithm or the spline function is a dental feature boundary 31 containing the shape or contour of the corresponding dental feature 40. The connected point cloud is the closest point cloud to each of the plurality of vertices that has an optical coefficient corresponding to the same dental feature 40. Further, the enamel is represented by one or more optical coefficients with a value of 5, while 0 corresponds to a non-dental object, such as air, water, gum, etc. As shown, the one or more processors 13 are configured to subdivide the plurality of finite elements into a plurality of tetrahedrons 9 and a plurality of triangles 8, and wherein the plurality of tetrahedrons 9 correspond to a tetrahedral mesh representation 9 and the plurality of triangles 8 correspond to a triangular mesh representation 8. The plurality of triangles 8 and tetrahedrons 9 form a plurality of slanted voxels 10. The plurality of triangles 8 correspond to a triangular mesh representation 8 and the plurality of tetrahedrons correspond to a tetrahedral mesh representation 9, wherein the triangular mesh representation 8 contains surface information for a volume covered by the tetrahedral mesh representation 9. As such, the one or more processors 13 are configured to determine a graphical surface representation of the dental object 2 based on the triangular mesh representation 8 of the aligned point cloud 27. Further, the one or more processors 13 are configured to determine a graphical volume representation of the dental object 2, and the one or more processors 13 are configured to combine the graphical surface representation and the graphical volume representation to determine a complete 3D graphical representation 50 of the dental object 2. Figure 2D

[0117] Figure 3 ​An example is shown in which the one or more processors 13 identify an anatomical feature 40B (e.g., tooth enamel) and a disease feature 40A (e.g., a cavity), and in this particular example, the one or more processors 13 are configured to increase the resolution of the disease feature 40A by performing tessellation on a plurality of slanted voxels 10A containing one or more optical coefficients 32 corresponding to the disease feature 40A. The remaining slanted voxels 10B contain one or more optical coefficients 32 corresponding to the anatomical feature 40B. The tessellated slanted voxels 10 are grouped into a first group, and the non-tessellated slanted voxels 10B are grouped into a second group. In this example, the first group of slanted voxels 10A includes a plurality of tetrahedrons and a plurality of triangles with higher resolution relative to the second group. In other words, when the first and second groups cover the same size of area, the first group 10A contains smaller and more tetrahedrons 9 and smaller and more triangles 8 than the second group.

[0118] Figure 4 An example is shown in which the one or more processors 13 are configured to determine a volumetric representation 60 of the 3D graph representation 50 by a graph neural network 61, and in which the 3D graph representation is received by the graph neural network, and in which the graph neural network 61 is configured to determine the volumetric representation 60 such that it contains segmented teeth 62 and gums 63 of the dental object. The segmented teeth 62 and gums 63 result in a complete volumetric 3D model of the dental object 2, in this example, a complete volumetric 3D model of the upper and lower jaws in a bite configuration. The graph neural network 61 can be trained by the one or more processors 13 based on CBCT scans and / or intraoral scans acquired by the handheld intraoral scanner 20.

[0119] Figure 5 An example is shown in which the graph neural network 61 is based on a neural radiance field algorithm 62. The one or more processors 13 are configured to determine one or more volumetric optical coefficients 32’ for each of a plurality of vertices of the 3D graph representation 50 based on the corresponding one or more optical coefficients 32 and the neural radiance field algorithm. The one or more volumetric optical coefficients 32’ are related to the one or more optical coefficients 32 by a projection object 62 of the algorithm 62. The position and angle of the projection object 62 are determined by the one or more processors 13. The volumetric representation 65 of the surface and internal regions of the dental object is determined based on the one or more volumetric optical coefficients 32’.

[0120] To improve the speed of the neural radiance field algorithm, the lower resolution 3D finite element mesh 4 is fed into the neural radiance field algorithm and this means that the algorithm only has to process fewer one or more optical coefficients 32 to generate the volumetric representation 65 of the dental object 2. Since many neighboring optical coefficients 32 can have the same or approximately the same value, representing the same dental feature 40, the resolution of the 3D finite element mesh can be reduced. If a group of optical coefficients has the same or approximately the same value, they can be merged into one optical coefficient. The one or more optical coefficients 32 comprise a plurality of groups of optical coefficients, which results in a reduced resolution of the 3D finite element mesh 4. In Figure 6A In the method, the one or more processors 13 are configured to determine 66A a resolution-reduced 3D finite element mesh of the 3D finite element mesh 4, and wherein the resolution-reduced 3D finite element mesh comprises a plurality of resolution-reduced vertices, and wherein the resolution-reduced 3D finite element mesh comprises a plurality of resolution-reduced finite elements. The one or more processors 13 are configured to align 66B the point cloud with the resolution-reduced 3D finite element mesh, and wherein the alignment of the non-visible 2D image and the point cloud of the visible sub-scan is provided by the position and orientation of the handheld intraoral scanner during the non-visible sub-scan and the visible sub-scan. The one or more processors 13 are configured to determine 66C one or more resolution-reduced optical coefficients for each of the plurality of resolution-reduced vertices by the inverse photo scattering algorithm, and to determine 66D one or more intermediate optical coefficients 32 for each of the plurality of vertices of the graphical representation 50 by interpolating between the one or more resolution-reduced optical coefficients of the plurality of resolution-reduced vertices of the resolution-reduced 3D finite element mesh. The one or more processors 13 can be configured to determine 66E one or more volume optical coefficients for each of the plurality of vertices of the graphical representation 50 based on the corresponding one or more intermediate optical coefficients 32 and the neural radiance field algorithm, and to determine the volumetric representation 65 of the surface and internal regions of the dental object 2 based on the one or more volume optical coefficients.

[0121] In the method, Figure 6B In the method, the neural radiance field algorithm can be configured to process 66E’ the estimated position and orientation of the handheld intraoral scanner 20, to determine 66E’’ a ray or cone cast object 62 for each pixel of the image sensor unit of the handheld intraoral scanner 20 based on the estimated position and orientation, to determine 66E one or more volume optical coefficients 32’ for each of the plurality of vertices where the ray or cone cast object 62 intersects based on the loss function and the one or more optical coefficients 32, and wherein the neural radiance field algorithm is configured to be trained using the non-visible 2D image.

[0122] Figure 8An example of one or more processors 13 configured to train a neural radiance field algorithm is shown. The one or more processors 13 can be configured to train the neural radiance field algorithm by determining 80 a 2D invisible image in a 2D visible image, receiving 81 an estimated position and orientation of a handheld intraoral scanner that corresponds to a ray or cone projection object for each pixel of the 2D invisible image. The ray or cone projection object reflects an observation axis or observation volume, respectively, of each pixel. Further, the one or more processors can be configured to train the neural radiance field algorithm by determining 82 one or more volume optical coefficients 32’ for each of a plurality of vertices of intersection of the ray or cone projection objects 62 based on the neural radiance field algorithm, the estimated position and orientation of the handheld intraoral scanner, and the one or more optical coefficients 32; and determining 83 a composite pixel value for each ray or cone projection object 62 based on the corresponding determined one or more volume optical coefficients for each of the plurality of vertices of intersection of the ray or cone projection objects 62. The one or more processors 13 are further configured to minimize a loss function between the composite pixel value and a corresponding true pixel value of each pixel of the 2D invisible image.

[0123] Figure 9 An intraoral scanning system 1 is shown that is configured to determine 8 an internal measurement of an internal region of a dental object 2. The system comprises a handheld intraoral scanner 20 that is configured to provide 12 a 2D invisible image and a visible sub-scan of the dental object. Further, the system 1 comprises one or more processors 13 that are configured to determine 21 a point cloud of the invisible 2D image and the visible sub-scan, determine 22 a three-dimensional (3D) finite element mesh and align the point cloud with the 3D finite element mesh, and wherein the 3D finite element mesh comprises a plurality of finite elements, and wherein each of the plurality of finite elements comprises a plurality of vertices, determine 23 one or more optical coefficients for each of the plurality of vertices, and wherein the one or more optical coefficients correspond to a dental feature represented by the aligned point cloud; determine 27 a first dental feature boundary of a first dental feature based on the determined one or more optical coefficients, and wherein the first dental feature boundary corresponds to the invisible 2D image; and determine 28 an internal measurement of the first dental feature boundary.

[0124] Figure 10 A dental object 2 is shown that comprises information about a surface 5 and an internal region 3 of the dental object 2. In this example, the dental object 2 is defined as a 3D finite element mesh 4, wherein each of a plurality of vertices has a three-dimensional coordinate 6. In this example, the dental object 2 comprises dental features, such as a pulp cavity 100, a dentin 101, and an enamel 102. The surface of the dental object comprising the dental features is represented by triangles, and the volume 8 of the dental object comprising the dental features is represented by a plurality of finite elements, which in this example comprise tetrahedrons 9.

[0125] Figure 11A and 11B An example is shown in which the one or more processors 13 determine one or more optical coefficients 32 for each of the plurality of vertices 33 and determine dental feature boundaries (31A, 31B) based on the one or more optical coefficients 32. In this example, the dental object 2 is still represented by the 3D finite element mesh 4, but one or more optical coefficients 32 are shown for a portion 4A of the 3D finite element mesh 4. In this example, the one or more processors 13 determine a disease feature, represented by a value 5, and apply a first dental feature boundary 31A to the mesh 4A that encloses this disease feature; and apply a second dental feature boundary 31B to the mesh 4A that represents the boundary between enamel (represented by a value 3) and dentin (represented by a value 2).

[0126] In Figure 11B , the one or more processors 13 are configured to determine a first dental feature corresponding to the first dental feature boundary 31A, and wherein the first dental feature is determined based on the one or more optical coefficients 32 corresponding to the first dental feature boundary 31A and a dental feature algorithm 35. The dental feature algorithm 35 comprises one or more of: an anatomical feature determiner 35A configured to determine that the one or more optical coefficients 32 correspond to an anatomical feature when the one or more optical coefficients are within an anatomical feature coefficient range; a disease feature determiner 35B configured to determine that the one or more optical coefficients 32 correspond to a disease feature when the one or more optical coefficients 32 are within a disease feature coefficient range; and a mechanical feature determiner 35C configured to determine that the one or more optical coefficients 32 correspond to a mechanical feature when the one or more optical coefficients 32 are within a mechanical feature coefficient range.

[0127] Figure 12 A dental object 2 is shown that includes a first dental feature boundary 31A representing a caries 40A, a second dental feature boundary 31B representing the boundary between enamel and dentin, and a third dental feature boundary 31C representing a pulp cavity. In a direct measurement 41 example, the one or more processors 13 are configured to determine an internal measurement 44 between a first portion 41A of the first dental feature boundary 31A and a second portion 41B of the first dental feature boundary 31A. In a fit example 42, the one or more processors 13 are configured to perform a fit of a shape object 43 to the geometry of the first dental feature boundary 31A, and wherein the internal measurement 44 is determined based on the fitted shape object 43.

[0128] Figure 13A 、 13B and 13C show different examples of internal measurements.Figure 13A and 13B One or more processors 13 are shown configured to determine a plurality of internal measurements 43 of a first dental feature boundary (31A, 41) or a shape object (43, 42) fitted to the geometry of the first dental feature boundary 31A, and wherein the one or more processors 13 are configured to determine a minimum and / or maximum internal measurement 44 of the plurality of internal measurements 44A. The one or more processors 13 are configured to determine a volume measurement of a dental feature corresponding to the dental feature boundary (31A), and the determined volume measurement is based on the plurality of internal measurements 44A. Figure 13C One example is shown where the one or more processors 13 are configured to determine a plurality of normal lengths 50A from a second dental feature 31B to the first dental feature boundary 31A and along a longitudinal axis 51 of the dental object 2 in the 3D finite element mesh. The one or more processors 13 are then configured to determine a maximum or minimum normal length of the plurality of normal lengths 50A, in this example, the maximum and minimum normal lengths indicate the maximum and minimum thickness of the enamel of the dental object 2, respectively.

[0129] Figure 14A , 14B and 14C show one or more processors 13 configured to determine a plurality of dental feature boundaries (31A, 31B, 31C, 31D), where the plurality of dental feature boundaries (31A, 31B, 31C, 31D) includes a first dental feature boundary 31A, a second dental feature boundary 31B, a third dental feature boundary 31C, and a fourth dental feature boundary 31D. In this example, the first dental feature boundary corresponds to a disease feature 40A, the second dental feature boundary 31B corresponds to enamel 40B, the third dental feature boundary corresponds to dentin 40C, and the fourth dental feature boundary corresponds to a pulp cavity 40D. In this example, the one or more processors 13 are configured to determine a plurality of internal measurements 44A between the second dental feature boundary 31B and the third dental feature boundary 31C, and wherein the internal measurements 44A correspond to a thickness of the enamel. The one or more processors 13 are also configured to determine an internal measurement 44B between the second dental feature boundary 31B and the first dental feature boundary 31A. The one or more processors 13 are configured to determine a thickness, i.e., a distance, of the enamel 40B at the maximum internal measurement 44B between the second dental feature boundary 31B and the first dental feature boundary 31A. If the maximum internal measurement is greater than the thickness of the enamel 40B at the same region of the dental object 2, the disease feature 40A penetrates the dentin 40C. In this case, the one or more processors 13 are configured to generate a notification signal to the user interface to notify the user of the system 1 about the penetration of the disease feature 44B into the dentin. In Figure 14A In the example shown, the one or more processors 13 are configured to determine an internal measurement 44A between the second dental feature boundary 31B and the third dental feature boundary 31C, and wherein the internal measurement 44A corresponds to a thickness of the enamel. The one or more processors 13 are also configured to determine an internal measurement 44B between the second dental feature boundary 31B and the first dental feature boundary 31A. The one or more processors 13 are configured to determine a thickness, i.e., a distance, of the enamel 40B at the maximum internal measurement 44B between the second dental feature boundary 31B and the first dental feature boundary 31A. If the maximum internal measurement is greater than the thickness of the enamel 40B at the same region of the dental object 2, the disease feature 40A penetrates the dentin 40C. In this case, the one or more processors 13 are configured to generate a notification signal to the user interface to notify the user of the system 1 about the penetration of the disease feature 44B into the dentin. In Figure 14BIn the illustrated example, the one or more processors 13 are configured to determine a first internal measure 44A between the third dental feature boundary 31B and the first dental feature boundary 31C in a first direction along the longitudinal axis 51 of the dental object, and a second internal measure 44B between the third dental feature boundary 31C and the first dental feature boundary 31A in a second direction along the longitudinal axis 51, and wherein the second direction is opposite the first direction. The sign of the internal measures (44A, 44B) determines whether the internal measures (44A, 44B) contain a measure of a penetration depth of the first dental feature 40A into the third dental feature 40C. In the present example, the second internal measure 44B corresponds to a penetration depth of the first dental feature 40A into the third dental feature 40C. In the present example, the notification signal contains information about the penetration depth, and the penetration depth is into the dentin 40C. In Figure 14C In the illustrated example, the one or more processors 13 are configured to determine a plurality of normal lengths (44A, 44B) from the second dental feature boundary 31B in a direction towards the first dental feature boundary 31A, and along the longitudinal axis 51 of the dental object 2 in the 3D finite element mesh 2. In the present example, the plurality of normal lengths (44A, 44B) have the same sign. The sign of the plurality of normal lengths (44A, 44B) indicates that the first dental feature 40A penetrates the second dental feature 40B. The one or more processors 13 are configured to determine a maximum normal length 44B of the plurality of normal lengths (44A, 44B), and determine a penetration depth 44B equal to the maximum normal length 44B. The one or more processors are configured to determine a notification signal when the internal measure 44B (i.e. the penetration depth) is above a measure threshold, and wherein the notification signal is displayed on a user interface of the system 1.

[0130] Each of the plurality of vertices can correspond to a pixel of an image sensor of the handheld intraoral scanner 10, or alternatively, a group of the plurality of vertices can correspond to a pixel of an image sensor of the handheld intraoral scanner 10. Figure 16A And 16B An example of the one or more processors configured to determine the one or more optical coefficients 32 is shown. In Figure 16AIn some embodiments, the image sensor comprises at least five pixels (87A, 87B, 87C, 87D, 87E), wherein each pixel comprises a projection object (86A, 86B, 86C, 86D, 86E) that intersects the primary 3D finite element mesh 4A and a plurality of sub 3D finite element meshes (4B, 4C, 4D, 4E). The primary 3D finite element mesh 4A contains one or more optical coefficients 32 for each of a plurality of vertices, and the one or more optical coefficients 32 correspond to what is observed by the pixel (87A, 87B, 87C, 87D, 87E). The sub 3D finite element meshes (4B, 4C, 4D, 4E) correspond to internal reflections, and in this example, the projection objects (86A, 86B, 86C, 86D, 86E) are rays that correspond to the field of view of the pixels (87A, 87B, 87C, 87D, 87E). Each pixel (87A, 87B, 87C, 87D, 87E) is configured to receive non-visible 2D images and then convert these images into a composite pixel value. The received non-visible 2D images contain internal reflections from within the interior region of the dental object, and this means that each composite pixel value contains an average from internal reflections distributed along different planes of the projection objects (86A, 86B, 86C, 86D, 86E) of the corresponding pixel (87A, 87B, 87C, 87D, 87E). Each composite pixel value can correspond to each of a plurality of vertices 33 of the primary 3D finite element mesh 4A, and wherein one or more processors are configured to determine one or more optical coefficients 32 for each of the plurality of vertices of the primary 3D finite element mesh based on one or more optical coefficients 32A of the sub 3D finite element meshes (4B, 4C, 4D, 4E). The internal reflections of the received non-visible 2D images can be represented by one or more optical coefficients 32A of a plurality of vertices of the sub 3D finite element meshes (4B, 4C, 4D, 4E), and since the received non-visible 2D images contain internal reflections captured from different planes, the neural radiance field model is configured to determine a sub 3D finite element mesh (4B, 4C, 4D, 4E) for each different plane. The sub 3D finite element meshes (4B, 4C, 4D, 4E) are arranged along the projection objects (86A, 86B, 86C, 86D, 86E) such that the projection objects (86A, 86B, 86C, 86D, 86E) starting at the vertices of the primary 3D finite element mesh 4A can intersect a corresponding vertex of each sub 3D finite element mesh (4B, 4C, 4D, 4E). As such, the one or more optical coefficients 32 of a vertex of the primary 3D finite element mesh 4A is an average of the one or more optical coefficients 32A of the corresponding vertex of the sub 3D finite element mesh (4B, 4C, 4D, 4E) that the projection object (86A, 86B, 86C, 86D, 86E) intersects.

[0131] Since most teeth (i.e. dental objects) have a similar internal structure, the variation of one or more optical coefficients 32 of different tooth samples will be limited, and there is a large covariance between the values of one or more coefficients 32 of the plurality of vertices 33. Figure 17 An example is shown in which the one or more processors 13 are configured to perform a covariance analysis on the one or more optical coefficients 32 with the aim to reduce the number of one or more optical coefficients 32 to be processed in order to determine the dental feature boundary (31A, 31B). The one or more processors 13 are configured to perform a covariance analysis 90 on the one or more optical coefficients 32 of the plurality of vertices and determine a plurality of descriptive parameters 32B based on the covariance analysis 90, and wherein the number of the plurality of descriptive parameters 32B is less than the number of the one or more optical coefficients 32. The first and / or second dental feature boundary (31A, 31B) is determined by providing the plurality of descriptive parameters 32B into a marching finite element algorithm 91 of the intraoral scanning system 1.

[0132] The covariance analysis 90 is a principal component analysis or a self-encoder deep learning algorithm, wherein the self-encoder deep learning algorithm comprises a neural network. The neural network is configured to receive the one or more optical coefficients 32 of the plurality of vertices corresponding to the non-visible 2D image and determine the plurality of descriptive parameters 32B.

[0133] Figure 18A And 18B An example is shown of how to train the neural network to determine the plurality of descriptive parameters 32B. In Figure 18A the one or more processors 13 are configured to train the neural network by receiving one or more CBCT scans 200A of one or more dental objects, determining 200C one or more training optical coefficients of each of the one or more CBCT scans, receiving 200B one or more non-visible sub-scans of the one or more dental objects from a handheld intraoral scanner, and training the neural network by mapping 200D the one or more training optical coefficients onto the one or more non-visible sub-scans. In Figure 18B the neural network is trained to classify 200E a specific dental feature 40 based on the one or more training optical coefficients, and wherein the neural network is configured to receive the one or more optical coefficients 32 of the plurality of vertices corresponding to the non-visible 2D image and determine the plurality of descriptive parameters 32B of the specific dental feature 40 based on the trained optical coefficients and the received one or more optical coefficients.

[0134] Figure 19One example of a user interface 210 of the system 1 is shown, which displays a three-dimensional (3D) model 211 of the dental object 2 and a two-dimensional (2D) cross-section 212 of the dental object 2, and wherein the internal measurement 44 is displayed on the 2D cross-section 212 of the dental object 2. In the present example, the dental object 2 to be displayed on the 2D cross-section 212 is selected by a marker 213, and the dental feature boundaries (31A, 31B, 31C) are displayed on the 2D cross-section 212.

[0135] Figure 20A and 20B One example is shown in which the one or more processors 13 are configured to perform a subdivision of the finite element 10. For simplicity of explanation of the subdivision, the finite element is a tetrahedron 9. In Figure 20A The finite element 9 is subdivided into a plurality of finite elements (9A-9F) and then subdivided once more (III). The subdivision is represented by an octree data structure 200, which in the present example contains four levels of subdivision (I, II, III, IV). The octree data structure 200 contains a plurality of nodes, which include groups of nodes located on different levels of the data structure 200. Each node corresponds to a finite element, e.g. a slanted voxel. For example, the first level (I) of the data structure 200 contains the un-subdivided finite element, and the second level (II) contains the subdivision of the finite element of the first level, and so on for the subsequent levels (III, IV) in the data structure 200. The subdivision can include halving of nodes from a previous level. The one or more processors 13 can be configured to perform a subdivision of a group of the plurality of finite elements (8, 9, 10) to a final level (IV), wherein the final level (IV) corresponds to nodes having approximately the same or the same one or more optical coefficients 32 to be removed, and a previous level (in the present example, the third level (III) relative to the final level (IV)) is to be retained by the one or more processors 13. The previous level (III) is considered to be the optimal level of subdivision of the finite element (8, 9, 10).

[0136] While certain embodiments have been detailed and illustrated, the disclosure is not limited to such details but can be practiced with the scope of the subject matter defined in the appended claims. In particular, it is understood that other embodiments can be utilized and structural and functional modifications can be made without departing from the scope of the present disclosure.

[0137] Benefits, other advantages and solutions to problems have been described herein with regard to specific embodiments. However, the benefits, advantages, solutions to problems and any one or more of the features (s) / elements described can be implemented, alone or in combination with others, without the incurrence of the claimed application. Therefore, the scope of the application is limited only by the appended claims, wherein the use of the singular includes the plural (i.e., "one", "the" and "said" means "one or more" unless explicitly stated otherwise. The claims can refer to "any" preceding claim and "any" is understood to mean "any one or more of the preceding claims".

[0138] The structural features of the apparatus described above in the detailed description and / or claims, when suitably replaced by corresponding processes, can be combined with the steps of the method.

[0139] Unless specifically stated otherwise, as used herein, the singular forms "a", "an" and "the" include plural form (i.e. "at least one" has the meaning of "one or more"). It will be further understood that the terms "include", "comprise", "including" and / or "comprising", when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It will be further understood that, when an element is referred to as being "connected" or "coupled" to another element, it can be directly connected or coupled to the other element or intervening elements can be present. In addition, as used herein the term "connected" or "coupled" can include wirelessly connected or coupled. As used herein the term "and / or" includes any and all combinations of one or more of the associated listed items. The steps of any disclosed methods can not be limited to the exact order stated herein unless expressly stated otherwise.

[0140] It is to be understood that the singular forms "a", "an", and "the" include plural referents unless expressly stated otherwise. It should be understood that like numerals refer to like elements unless otherwise indicated in the figures and discussion. It is to be understood that other variations and modifications of the aspects described and illustrated herein are possible in light of the foregoing disclosure, and it is to be further understood that one or more aspects of this disclosure can be used alone or in any combination of one or more aspects where the context permits. It is intended that the specification and examples be considered as exemplary only, with the true scope of the disclosure being indicated by the following claims.

[0141] The claims are not intended to be limited to the aspects shown herein, but are to be accorded the full scope consistent with the language of the claims where appropriate, and to encompass all equivalent aspects, unless otherwise stated. Reference to an element in the singular is not intended to mean “one and only one” unless specifically stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more.

[0142] Item 1. An intraoral scanning system configured to determine a three-dimensional (3D) graphical representation of a dental object, the system comprising: • a handheld intraoral scanner configured to provide an invisible 2D image and a visible sub-scan of the dental object; and • one or more processors configured to: o determine a point cloud of the visible sub-scan; o determine a 3D finite element mesh and align the point cloud with the 3D finite element mesh, wherein the 3D finite element mesh comprises a plurality of finite elements, and wherein each of the plurality of finite elements comprises a plurality of vertices; o determine one or more optical coefficients for each of the plurality of vertices, and wherein the one or more optical coefficients correspond to a dental feature represented by the aligned point cloud; o determine a triangular mesh representation of the aligned point cloud corresponding to the visible sub-scan, and wherein the triangular mesh representation comprises triangularly arranged vertices of the plurality of vertices; o determine a tetrahedral mesh representation of the aligned point cloud corresponding to the invisible 2D image, and wherein the tetrahedral mesh representation comprises tetrahedrally arranged vertices of the plurality of vertices; and wherein the 3D graphical representation is determined based on the triangular mesh representation and the tetrahedral mesh representation.

[0143] 2. The intraoral scanning system of item 1, wherein the plurality of finite elements are slanted voxels.

[0144] 3. The intraoral scanning system of any of the preceding items, wherein the one or more optical coefficients are determined by an inverse photo scattering algorithm configured to receive the 2D invisible image and the 3D visible sub-scan and estimate a reflection, an absorption, or a refraction of a corresponding vertex of the finite element mesh.

[0145] 4. The intraoral scanning system of any of the preceding items, wherein the one or more processors are configured to subdivide the plurality of finite elements into a plurality of tetrahedrons and a plurality of triangles, and wherein the plurality of tetrahedrons correspond to the tetrahedral mesh representation and the plurality of triangles correspond to the triangular mesh representation.

[0146] 5. The intraoral scanning system according to any of the preceding items, wherein the one or more processors are configured to perform one or more tessellations of the plurality of finite elements into a first set of a plurality of tetrahedrons and a plurality of triangles and a second set of a plurality of tetrahedrons and a plurality of triangles, and wherein the first set and the second set have different resolutions of the plurality of finite elements.

[0147] 6. The intraoral scanning system according to item 5, wherein the first set comprises a higher resolution of the plurality of tetrahedrons and the plurality of triangles relative to the second set, and wherein the first set of the plurality of finite elements corresponds to a first portion of the dental object and the second set of the plurality of finite elements corresponds to a second portion of the dental object.

[0148] 7. The intraoral scanning system according to any of the preceding items, wherein the one or more processors are configured to determine the 3D graphical representation via a marching tetrahedra algorithm by connecting the point cloud aligned closest to each of the plurality of vertices.

[0149] 8. The intraoral scanning system according to any of the preceding items, wherein if the first set and the second set cover the same sized area of the dental object, the first set contains smaller and more tetrahedrons of the plurality of tetrahedrons and smaller and more triangles of the plurality of triangles than the second set.

[0150] 9. The intraoral scanning system according to items 5 to 8, wherein the first set of the plurality of finite elements corresponds to a disease feature and the second set of the plurality of finite elements corresponds to a dental feature other than the disease feature.

[0151] 10. The intraoral scanning system according to any of the preceding items, wherein the one or more processors are configured to determine a surface of the dental object in the graphical representation based on the triangular mesh representation of the aligned point cloud.

[0152] 11. The intraoral scanning system according to any of the preceding items, wherein the one or more processors are configured to determine an interior region of the dental object in the graphical representation based on the tetrahedral mesh representation of the aligned point cloud.

[0153] 12. The intraoral scanning system according to any of the preceding items, wherein the one or more processors are configured to determine a volumetric representation of the 3D graphical representation by a graph neural network, and wherein the 3D graphical representation is received by the graph neural network, and wherein the graph neural network is configured to determine a volumetric representation containing a segmentation of teeth and gums of the dental object.

[0154] 13. The intraoral scanning system according to item 12, wherein the one or more processors are configured to train the graph neural network by a volumetric representation determined by a CBCT scan and / or an intraoral scan acquired by a handheld intraoral scanner.

[0155] 14. The intraoral scanning system according to any of the preceding items, wherein the alignment of the point clouds of the invisible 2D images and the visible sub-scans is provided by the position and orientation of the handheld intraoral scanner during the invisible sub-scans and the visible sub-scans, and wherein the one or more processors are configured to: • determine one or more volume optical coefficients for each of the plurality of vertices of the graphical representation based on the corresponding one or more optical coefficients and a neural radiance field algorithm; and • determine a volume representation of the surface and internal regions of the dental object based on the one or more volume optical coefficients.

[0156] 15. The intraoral scanning system according to any of items 1 to 13, wherein the one or more processors are configured to: • determine a resolution-reduced 3D finite element mesh of the 3D finite element mesh, and wherein the resolution-reduced 3D finite element mesh comprises a plurality of resolution-reduced vertices, and wherein the resolution-reduced 3D finite element mesh comprises a plurality of resolution-reduced finite elements; • align the point clouds to the resolution-reduced 3D finite element mesh, wherein the alignment of the point clouds of the invisible 2D images and the visible sub-scans is provided by the position and orientation of the handheld intraoral scanner during the invisible sub-scans and the visible sub-scans; • determine one or more resolution-reduced optical coefficients for each of the plurality of resolution-reduced vertices by an inverse photo scattering algorithm; • determine one or more intermediate optical coefficients for each of the plurality of vertices of the graphical representation by interpolating between the one or more resolution-reduced optical coefficients of the plurality of resolution-reduced vertices of the resolution-reduced 3D finite element mesh; • determine one or more volume optical coefficients for each of the plurality of vertices of the graphical representation based on the corresponding one or more intermediate optical coefficients and a neural radiance field algorithm; and • determine a volume representation of the surface and internal regions of the dental object based on the one or more volume optical coefficients.

[0157] 16. The intraoral scanning system according to item 14 or 15, wherein the neural radiance field algorithm is configured to: • process the estimated position and orientation of the handheld intraoral scanner; • determine, for each pixel of the image sensor unit of the handheld intraoral scanner, a ray or cone projection object based on the estimated position and orientation; • determine one or more volume optical coefficients for each of the plurality of vertices of the ray or cone projection object intersection based on the loss function and the one or more optical coefficients; and wherein the neural radiance field algorithm is configured to be trained using the invisible 2D images.

[0158] 17. The intraoral scanning system of item 16, wherein the one or more processors are configured to train the neural radiance field algorithm by: • determining a 2D invisible image of the invisible 2D image; • receiving an estimated position and orientation of the handheld intraoral scanner that corresponds to a ray or cone cast object for each pixel of the 2D invisible image; • determining, based on the neural radiance field algorithm, one or more volume optical coefficients for each of a plurality of vertices of intersection of the ray or cone cast objects using the estimated position and orientation of the handheld intraoral scanner and the one or more optical coefficients; and • determining a composite pixel value for each ray or cone cast object based on the corresponding determined one or more volume optical coefficients for each of the plurality of vertices of intersection of the ray or cone cast objects; and • minimizing a loss function between the composite pixel value and a corresponding true pixel value for each pixel of the 2D invisible image.

[0159] 18. The intraoral scanning system of any of the preceding items, wherein the invisible 2D image contains infrared information or near-infrared information.

[0160] 19. The intraoral scanning system of any of the preceding items, comprising a display unit configured to display a 3D model of the dental object, the 3D model containing a 3D graphical representation.

[0161] 20. The intraoral scanning system of any of the preceding items, comprising a display unit configured to display a 3D model of the dental object, the 3D model containing a volumetric representation of a surface and internal regions of the dental object.

[0162] 21. The intraoral scanning system of any of the preceding items, wherein the one or more optical coefficients correspond to a dental feature, and wherein the one or more processors are configured to: • determine a first dental feature boundary of a first dental feature based on the determined one or more optical coefficients or one or more volume optical coefficients, and wherein the first dental feature boundary corresponds to the invisible 2D image, and • determine an internal measurement of the first dental feature boundary.

[0163] 22. The intraoral scanning system of item 21, wherein the internal measurement is determined between a first portion of the first dental feature boundary and a second portion of the first dental feature boundary.

[0164] 23. The intraoral scanning system of any of items 21 and 22, wherein the one or more processors are configured to perform a fit of a shape object to a geometry of the first dental feature boundary, and wherein the internal measure is determined based on the fitted shape object.

[0165] 24. The intraoral scanning system of any of items 21 to 23, wherein the one or more processors are configured to determine a plurality of internal measures of the first dental feature boundary or a shape object fitted to a geometry of the first dental feature boundary, and wherein the one or more processors are configured to determine a minimum internal measure of the plurality of internal measures.

[0166] 25. The intraoral scanning system of any of items 21 to 24, wherein the one or more processors are configured to determine a plurality of internal measures of the first dental feature boundary or a shape object fitted to a geometry of the first dental feature boundary, and wherein the one or more processors are configured to determine a volume measure based on the plurality of internal measures.

[0167] 26. The intraoral scanning system of any of items 21 to 25, wherein the internal measure is a sectional curvature measure of a portion of the first dental feature boundary or a shape object fitted to a geometry of the first dental feature boundary.

[0168] 27. The intraoral scanning system of any of items 21 to 26, wherein the one or more processors are configured to determine a plurality of dental feature boundaries, and wherein the plurality of dental feature boundaries comprises the first dental feature boundary and at least a second dental feature boundary, and wherein the at least second dental feature boundary corresponds to a second dental feature; and wherein the internal measure is determined between the first dental feature boundary and the at least second dental feature boundary.

[0169] 28. The intraoral scanning system of item 28, wherein the internal measure is a distance between the first dental feature boundary and the at least second dental feature boundary, or an overlap between the first dental feature boundary and the second dental feature boundary.

[0170] 29. The intraoral scanning system of item 27 or 28, wherein the one or more processors are configured to determine a plurality of sub-internal measures, the plurality of sub-internal measures comprising a first sub-internal measure, a second sub-internal measure, and a third sub-internal measure, and wherein: • the first sub-internal measure corresponds to a size measure of the first dental feature corresponding to the first dental feature boundary; • the second sub-internal measure corresponds to a size measure of the second dental feature corresponding to the second dental feature boundary; • the third sub-internal measure is a size measure of an overlap between the first dental feature boundary and the at least second dental feature boundary; and • the internal measurement is a ratio between the first sub-internal measurement or the second sub-internal measurement and the third sub-internal measurement.

[0171] 30. The intraoral scanning system of any of items 27 to 29, wherein the plurality of dental feature boundaries are determined by a marching finite element algorithm.

[0172] 31. The intraoral scanning system of any of items 21 to 30, wherein the one or more processors are configured to: • perform a covariance analysis on the one or more optical coefficients or the one or more volumetric optical coefficients of the plurality of vertices; • determine the plurality of descriptive parameters based on the covariance analysis, and wherein a number of the plurality of descriptive parameters is less than a number of the one or more optical coefficients or the one or more volumetric optical coefficients; and wherein the plurality of dental feature boundaries are determined by providing the plurality of descriptive parameters into a marching finite element method of the intraoral scanning system.

[0173] 32. The intraoral scanning system of item 31, wherein the covariance analysis is a principal component analysis or an autoencoder deep learning algorithm, wherein the autoencoder deep learning algorithm comprises a neural network, and wherein the neural network is configured to receive the one or more optical coefficients or the one or more volumetric optical coefficients of the plurality of vertices corresponding to the non-visible 2D image and determine the plurality of descriptive parameters.

[0174] 33. The intraoral scanning system of item 32, wherein the one or more processors are configured to train the neural network by: • receiving one or more CBCT scans of the one or more dental objects; • determining one or more training optical coefficients for each of the one or more CBCT scans; • receiving one or more non-visible sub-scans of the one or more dental objects from the handheld intraoral scanner; and • training the neural network by mapping the one or more training optical coefficients onto the one or more non-visible sub-scans.

[0175] 34. The intraoral scanning system of any of items 32 and 33, wherein the neural network is trained to classify a particular dental feature based on the one or more training optical coefficients; and wherein the neural network is configured to receive the one or more optical coefficients or the one or more volumetric optical coefficients of the plurality of vertices corresponding to the non-visible 2D image and determine the plurality of descriptive parameters of the particular dental feature.

[0176] 35. The intraoral scanning system of any of items 21 to 34, wherein the first dental feature or the second dental feature corresponds to one of: • an anatomical feature, such as enamel, dentin, or pulp; • a disease feature, such as a crack or a cavity; and • a mechanical feature, such as a filling and / or a composite restoration.

[0177] 36. The intraoral scanning system of any of items 21 to 34, wherein the one or more processors are configured to: • determine a plurality of normal lengths from the second dental feature in a direction towards the first dental feature boundary and along a longitudinal axis of the point cloud; • determine a maximum normal length of the plurality of normal lengths; • determine that the first dental feature boundary extends a depth along the maximum normal length within the second dental feature boundary, and wherein the internal measure is the depth extension.

[0178] 37. The intraoral scanning system of item 36, wherein the one or more processors are configured to determine a volume measure by performing a volume measurement based on the plurality of normal lengths, and wherein the volume measure comprises a volume of the first dental feature boundary disposed within the second dental feature boundary, and wherein the internal measure is the volume measure.

[0179] 38. The intraoral scanning system of any of items 21 to 37, wherein the one or more processors are configured to determine a notification signal when the internal measure is above a measurement threshold, and wherein the notification signal is displayed on a user interface of the system.

[0180] 39. The intraoral scanning system of any of the preceding items, wherein the one or more processors are configured to align the point cloud with the 3D finite element mesh by determining a shortest distance between a 3D point in the 3D point cloud and a vertex of the plurality of vertices.

Claims

1. An intraoral scanning system configured to determine a three-dimensional (3D) graphical representation of a dental object, the system comprising: • a handheld intraoral scanner configured to provide a non-visible 2D image and a visible sub-scan of a dental object; and • one or more processors configured to: o determine a point cloud of the non-visible 2D image and the visible sub-scan; o determine a 3D finite element mesh and align the point cloud to the 3D finite element mesh, wherein the 3D finite element mesh contains a plurality of finite elements, and wherein each of the plurality of finite elements contains a plurality of vertices; o determine one or more optical coefficients for each of the plurality of vertices, and wherein the one or more optical coefficients correspond to a dental feature represented by the aligned point cloud; o determine a triangular mesh representation of the aligned point cloud corresponding to the visible sub-scan, and wherein the triangular mesh representation includes triangularly arranged vertices of the plurality of vertices; o determine a tetrahedral mesh representation of the aligned point cloud corresponding to the non-visible 2D image, and wherein the tetrahedral mesh representation includes tetrahedrally arranged vertices of the plurality of vertices; and wherein the 3D graphical representation is determined based on the triangular mesh representation and the tetrahedral mesh representation. The plurality of finite elements are slanted voxels.

2. The intraoral scanning system of claim 1, wherein, The one or more optical coefficients are determined by an inverse photo scattering algorithm configured to receive measured reflectance, absorption, or refraction of the non-visible 2D image and the corresponding point cloud of the visible sub-scan.

3. The intraoral scanning system of any of the preceding claims, wherein, The one or more processors are configured to subdivide the plurality of finite elements into a plurality of tetrahedrons and a plurality of triangles, and wherein the plurality of tetrahedrons correspond to the tetrahedral mesh representation and the plurality of triangles correspond to the triangular mesh representation.

4. The intraoral scanning system of any of the preceding claims, wherein, The one or more processors are configured to perform one or more subdivisions of the plurality of finite elements into a first set of the plurality of tetrahedrons and triangles and a second set of the plurality of tetrahedrons and triangles, and wherein the first set and the second set have different resolutions of the plurality of finite elements.

5. The intraoral scanning system of any of the preceding claims, wherein, The first set includes a higher resolution of the plurality of tetrahedrons and the plurality of triangles relative to the second set, and wherein the first set of the plurality of finite elements corresponds to a first portion of the dental object and the second set of the plurality of finite elements corresponds to a second portion of the dental object.

6. The intraoral scanning system of claim 5, wherein, The one or more processors are configured to determine the 3D graphical representation via a marching tetrahedra algorithm by connecting the point cloud aligned closest to each of the plurality of vertices.

7. The intraoral scanning system of any of the preceding claims, wherein, If the first set and the second set cover the same size area of the dental object, the first set contains smaller and more tetrahedrons of the plurality of tetrahedrons and smaller and more triangles of the plurality of triangles than the second set.

8. The intraoral scanning system of any of the preceding claims, wherein, A first set of the plurality of finite elements corresponds to a disease feature and a second set of the plurality of finite elements corresponds to other dental features other than the disease feature.

9. The intraoral scanning system of any one of claims 5 to 8, wherein, ​ 10. The intraoral scanning system of any of the preceding claims, wherein, The one or more processors are configured to determine a surface of the dental object in the graphical representation based on a triangular mesh representation of the aligned point cloud.

11. The intraoral scanning system of any of the preceding claims, wherein, The one or more processors are configured to determine an internal region of the dental object in the graphical representation based on a tetrahedral mesh representation of the aligned point cloud.

12. The intraoral scanning system of any of the preceding claims, wherein, The one or more processors are configured to determine a volumetric representation of the 3D graphical representation by a graph neural network, and wherein the 3D graphical representation is received by the graph neural network, and wherein the graph neural network is configured to determine a volumetric representation of a segmented tooth and gingiva containing the dental object.

13. The intraoral scanning system of claim 12, wherein, The one or more processors are configured to train the graph neural network by a volumetric representation determined by a CBCT scan and / or an intraoral scan acquired by a handheld intraoral scanner.

14. The intraoral scanning system of any of the preceding claims, wherein, The alignment of the invisible 2D image and the point cloud of the visible sub-scan is provided by a position and orientation of the handheld intraoral scanner during the invisible sub-scan and the visible sub-scan, and wherein the one or more processors are configured to: • determine one or more volumetric optical coefficients for each of a plurality of vertices of the graphical representation based on a corresponding one or more optical coefficients and a neural radiance field algorithm; and • determine a volumetric representation of a surface and an internal region of the dental object based on the one or more volumetric optical coefficients.

15. The intraoral scanning system of any one of claims 1 to 13, wherein, The one or more processors are configured to: • determine a resolution-reduced 3D finite element mesh of the 3D finite element mesh, and wherein the resolution-reduced 3D finite element mesh comprises a plurality of resolution-reduced vertices, and wherein the resolution-reduced 3D finite element mesh comprises a plurality of resolution-reduced finite elements; • align the point cloud to the resolution-reduced 3D finite element mesh, wherein the alignment of the invisible 2D image and the point cloud of the visible sub-scan is provided by a position and orientation of the handheld intraoral scanner during the invisible sub-scan and the visible sub-scan; • determine one or more resolution-reduced optical coefficients for each of the plurality of resolution-reduced vertices by an inverse photo scattering algorithm; • determine an intermediate one or more optical coefficients for each of a plurality of vertices of the graphical representation by interpolating between the one or more resolution-reduced optical coefficients of the plurality of resolution-reduced vertices of the resolution-reduced 3D finite element mesh; • determine one or more volumetric optical coefficients for each of a plurality of vertices of the graphical representation based on a corresponding intermediate one or more optical coefficients and a neural radiance field algorithm; and • determine a volumetric representation of a surface and an internal region of the dental object based on the one or more volumetric optical coefficients.

16. The intraoral scanning system of claim 14 or 15, wherein, The neural radiance field algorithm is configured to: • process an estimated position and orientation of the handheld intraoral scanner; • determine a ray or a cone projection object based on the estimated position and orientation for each pixel of an image sensor unit of the handheld intraoral scanner; • determine, based on the loss function and the one or more optical coefficients, one or more volume optical coefficients for each of a plurality of vertices of intersection of the ray or cone cast object; and wherein the neural radiance field algorithm is configured to be trained using the non-visible 2D images.

Images