A graphical representation of an inner volume of a dental object
Patent Information
- Authority / Receiving Office
- EP · EP
- Patent Type
- Applications
- Current Assignee / Owner
- 3SHAPE AS
- Filing Date
- 2024-06-20
- Publication Date
- 2026-04-29
Smart Images
Figure EP2024067293_26122024_PF_FP_ABST
Abstract
Description
[0001] A GRAPHICAL REPRESENTATION OF AN INNER VOLUME OF A DENTAL
[0002] OBJECT
[0003] FIELD
[0004] The disclosure relates to an intraoral scanning system. More specifically, the disclosure relates to a system that is configured to determine a three-dimensional graphical representation based on a tetrahedral mesh representation, and wherein the system includes a handheld intraoral scanner.
[0005] BACKGROUND
[0006] In the field of dentistry one of the most common types of dental conditions is dental caries which, if left untreated, can lead to severe tooth decay and evidently the loos of the one or more infected teeth. If the condition is identified early, it may be possible to treat very easily and achieve a reversable result of the infected tooth area. Many types of dental caries exist, but the most difficult type to identify in an early stage is interproximal caries which extends from the enamel towards the dentine of the tooth in the area in between adjacent teeth.
[0007] A common way of identifying caries is by means of 2D bitewing x-ray images to assess cavity formation because of loss of dental material during the development of a caries. However, non-ionizing IR light has been proven to enable an equivalent analysis of internal tooth structures as the light is able to penetrate the tooth material. This makes is possible to image internal regions of the tooth and even identify demineralization in a very early stage before it leads to irreversible cavity formation or extend into the dentine or pulp.
[0008] To confidently identify interproximal caries and decide if preventive or minor restorative measures should be taken it is crucial to be able to identify, quantify and monitor the development of potential early lesions of interproximal demineralization. With the use of intraoral scanners in combination with the ability to do infrared imaging of the teeth it is possible to accurately reconstruct a detailed 3D surface model of the tooth as well as reconstructing a volumetric representation of the internal regions of the teeth below the tooth surface.
[0009] The digitalization of the surface and internal tooth structure enables the use of sophisticated automated machine learning algorithms to be developed and applied to perform multiple different automated tasks such a segmenting, recognizing, and labelling different internal regions of the tooth structure like the enamel layer, the dentine, the dentine-enamel, demineralization regions associated with dental caries, cracks etc. However, the enablement of advanced and computationally heavy automated machine learning methods requires a suitable digital representation of the tooth structure including both surface and internal information.
[0010] To date, there has been a great deal of work performed on 2D semantic segmentation (i.e., the segmentation of images into specific, labelled components). There has been significantly less work in 3D due to the enormous memory requirements (e.g., a mediumresolution image might contain 512x512 pixels, so to achieve the same resolution in voxels, a system may require more than 256 times more memory capacity. It would be particularly helpful to provide one or more tools that may aid in analyzing and or guiding treatments that may automatically and accurately segment teeth and identify internal components, particularly directly on 3D volumetric representations.
[0011] SUMMARY
[0012] It is an aspect of the present disclosure to provide an intraoral scanning system that is configured to provide a graphical representation of a patient’s teeth, i.e. dental objects, that is able to efficiently and with less computational power to automatically segment teeth and dental features. The graphical representation may be of a volumetric 3D representation of a dental object that includes volumetric segments of the dental object.
[0013] Volumetric 3D models of a patient’s teeth, i.e. dental objects, may be acquired by the system, and at least a subset of the volumetric 3D models (e.g. a specific number of points representing each tooth) may be converted into graphical representations that preserve the structure of connections from the volumetric 3D model in an optimized way to reduce computational costs.
[0014] The implementations address the need to provide an intraoral scanning system that is configured to segment individual teeth and dental features automatically, effectively, and accurately from a 3D volumetric model of a patient’s detention by the use of the graphical representation that results in a high degree of accuracy.
[0015] The optimized graphical representation of the 3D volumetric model enables the ability to efficiently and with less computational power to automatically segment teeth and dental features using machine learning neural networks such as a set of 3D convolutional neural networks that uses the graphical representation of the volumetric 3D representation.
[0016] According to the aspects, an intraoral scanning system is disclosed. The intraoral scanning system may be configured to determine a three-dimensional graphical representation of a dental object that includes volumetric segments of the dental object. The system may include a handheld intraoral scanner that is configured to provide non-visible 2D images and visible sub-scans of a dental object. The non-visible 2D images and the visible subscans may be provided by one or more light sources of the handheld intraoral scanner emits light that scatters off the dental object, and the scattering is acquired by one or more image sensors of the handheld intraoral scanner. The visible sub-scans may correspond to emitted light that includes wavelengths, such as between 350 nm and 750 nm, and the non-visible 2D images may correspond to emitted light that includes wavelengths, such as 800 nm to 1100 nm.
[0017] The system may include one or more processors that is configured to determine point clouds of the visible sub-scans, and to determine a three-dimensional (3D) finite element mesh and align the point clouds to the 3D finite element mesh. Each of the point clouds may include a set of three-dimensional coordinates that corresponds to a part of the dental object. The point clouds may then be aligned to the 3D finite element mesh for the purpose of determining a volumetric representation of the dental object. The point clouds may represent a 3D model of the dental object in the 3D finite element mesh, wherein the point clouds include multiple points with a set of three-dimensional coordinates, such as three-dimensional cartesian coordinates.
[0018] The 3D finite element mesh may include a plurality of finite elements which is a combination of triangles and tetrahedrons that forms a skewed voxel. A tetrahedral corresponds to a volume of a part of the dental object, and a triangle corresponds to a surface of the volume of the part of the dental object.
[0019] The 3D finite element mesh includes a plurality of finite elements, and wherein each of the plurality of finite elements includes multiple vertices. Each of the plurality of finite element may include one or more of following: tetrahedral, tetrahedral combined with octahedral, and parallelepiped (skewed voxels). With 3D finite element mesh the grid points at the multiple vertices are evaluated by the one or more processors. The evaluation by the one or more processors may include determine one or more optical coefficients for each of the multiple vertices, and wherein the one or more optical coefficients correspond to a dental feature represented by the aligned point clouds. The one or more optical coefficients can be used to extract a dental feature of volumetric segments represented by the non-visible 2D images. The dental feature may be an anatomy feature being an enamel, a dentine, or a pulp, a disease feature being a crack or a caries, and a mechanical feature being a filling and / or a composite restoration.
[0020] The one or more optical coefficients may include one or more of following optical indices: refraction index, light absorption index, and light scattering index, and wherein the one or more optical coefficients correspond to either the non-visible 2D images or the visible sub-scans. The one or more optical coefficients can be used to extract a dental feature or volumetric segment represented by the non-visible 2d images, non-visible 2D image
[0021] The one or more processors may be configured to determine a triangle mesh representation of the aligned point clouds that correspond to the visible sub-scans, and wherein the triangle mesh representation includes triangle arranged vertices of the plurality of vertices. The triangle mesh representation corresponds to a three- dimensional graphical representation of an external surface of the dental object. The external surface does not include information about the inner region of the dental object.
[0022] The one or more processors may be configured to determine a tetrahedral mesh representation of the aligned point clouds that correspond to the non- visible 2D images, and wherein the tetrahedral mesh representation includes tetrahedral arranged vertices of the plurality of vertices. The tetrahedral mesh representation corresponds to a three- dimensional graphical representation of the inner region of the dental object.
[0023] The one or more processors may be configured to provide a three-dimensional graphical representation of the dental object by the triangle mesh representation and the tetrahedral mesh representation. Since the point clouds of the non-visible 2D images and the visible sub-scans are already aligned in the 3D finite element mesh, the triangle mesh representation and the tetrahedral mesh representation are also then aligned and can then just be combined in order to define a full 3D graphical representation of the dental object.
[0024] The plurality of finite elements may be skewed voxels. The 3D graphical representation of the dental object with skewed voxels and the corresponding regular tessellation by tetrahedral elements provides a uniform and simple 3D graphical representation of the dental object where each of the multiple vertices may have a maximum of four neighbours, wherein a voxel may have six neighbours. The lower number of neighbours provides a more simpler 3D graphical representation that would result in a faster computation of a 3D volumetric representation based on the 3D graphical representation.
[0025] The tetrahedron is the simplest piecewise linear topological element that can be used to tessellate Euclidean space. Any other finite element, such as a voxel, would have more vertices or facets than the tetrahedron. The skewed voxels technique together with the tetrahedral-octahedral subdivision scheme is a very convenient method to obtain a regular and well-balanced tetrahedral tessellation of the Euclidean space. For improving the precision of an area of a dental object, the one or more processors is configured to reduce the size of a group of the plurality of finite elements that corresponds to the area by the technique of adaptive refinement. The one or more processors is configured to perform a subdivision of the grouped plurality of finite elements for reducing the size of the finite elements of the group. By using large skew voxels in areas of low precision and smaller ones on areas of high precision, one can reduce the memory footprint for the 3D graphical representation. The advantage of a skewed voxel in view of a normal voxel is that the subdivision of a skewed voxel into tetrahedra would provide an easy method to obtain a tetrahedral tessellation of the Euclidean space with tetrahedra of different sizes that is adapted to the requirements of the problem at hand.
[0026] To determine the level of subdivisions, i.e. the size of the group of the plurality of finite elements, an octree data structure is used. The octree data structure includes a plurality of nodes that includes groups of nodes on different levels of the data structure. Each node corresponds to a finite element, such as a skewed voxel. For example, a first level of the data structure includes a none subdivided finite element and a second level includes a subdivision of the finite element of the first level and soon for the next levels in the data structure. The subdivision may include halving the nodes from the previous level. The one or more processors may be configured to perform the subdivision of a group of the plurality of finite elements to a final level where the one or more optical coefficients of the corresponding nodes to the final level that are about the same or the same will be removed, and the previous level relative to the final level will be kept by the one or more processors. The previous level is considered to be the optimal level of subdivision of a finite element.
[0027] This skewed voxel representation of the 3D graphical representation has the great benefit, that one can use a search structure, like an octree on the subdivided skewed voxels. Another benefit is that a marching tetrahedra algorithm would be much simpler to implement than the marching cube algorithm, since we only have essentially two distinct cases where the vertices of a single tetrahedra have either a positive or a negative optical coefficient, in opposition to marching cubes, which has 12 distinct cases. By using a signed distance field representation and run the marching tetrahedra algorithm, then both the triangle mesh representation which represents the surface of the dental object and a tetrahedral mesh representation which represents the inner region of the dental object are obtained.
[0028] The one or more processors may be configured to subdivide the plurality of finite elements into a plurality of tetrahedrons and a plurality of triangles, and wherein the plurality of tetrahedrons corresponds to the tetrahedral mesh representation, and wherein the plurality of triangles corresponds to the triangle mesh representation.
[0029] The one or more optical coefficients may be determined by an inverse photon scattering algorithm that is configured to receive measured reflections, absorptions, or refractions of the corresponding point clouds of the non-visible 2D images and the visible subscans.
[0030] The one or more processors may be configured to determine the 3D graphical representation by connecting the point clouds that are aligned closest to each of the plurality of vertices via a marching tetrahedral algorithm. The marching finite element algorithm may be configured to select a set of the one or more optical coefficients that are nearest a set of the multiple vertices for constructing a shape function that represents the first dental feature boundary. The algorithm may be configured to construct multiple shape function for multiple dental feature boundaries, such as the first dental feature boundary and the second dental feature boundary. The marching finite element algorithm may be a three-dimensional finite element algorithm, or more precisely, a three dimensional nearest-node finite element algorithm.
[0031] If the first group and the second group covers the same size of area of the dental object, the first group may include smaller and more tetrahedrons of the plurality of tetrahedrons and smaller and more triangles of the plurality of triangles than the second group. The first group has an improved resolution than the second group which would result in an improved resolution of an area of the dental object that is covered by the first group of tetrahedrons and triangles. This is beneficial if wanting a high resolution of for example a caries within the dental object and low or normal resolution of an area of the dental object with less of interest for the user of the system.
[0032] The first group of the plurality of finite elements may correspond to a disease feature and the second group of the plurality of finite elements may correspond to a dental feature other than the disease feature, for example, a dental feature with less interest for the user of the system.
[0033] Each of the plurality of triangles represents a part of a two-dimensional surface of a dental feature. For example, the two-dimensional surface may be the outer surface of the 3D graphical representation of the dental object or dentition. Furthermore, the two- dimension surface may be a surface of a caries that is arranged within the dental object. The one or more processors may be configured to determine a surface of the dental object in the graphical representation based on the triangle mesh representation of the aligned point clouds.
[0034] Each of the plurality of tetrahedral represents a part of a three-dimensional volume of a dental feature. For example, the three-dimensional volume may be the volume of the 3D graphical representation of a dental feature, such as a dentine, a caries etc. The one or more processors may be configured to determine an inner region of the dental object in the graphical representation based on the tetrahedral mesh representation of the aligned point clouds.
[0035] The advantage of the 3D graphical representation of a dental object is that a faster and simpler way for determining a volumetric representation of the dental object is achieved. The volumetric representation of the dental object includes a three-dimensional model of an outer surface and the inner region of the dental object. The 3D model of the dental object is displayed on a graphical user interface, and the user is able to zoom into the dental object to investigate the caries with a higher level of details and to navigate within the dental object for the purpose of investigating other dental features of the dental object. The one or more processors may be configured to determine the volumetric representation of the 3D graphical representation by a graph neural network, and wherein the 3D graphical representation is received by the graph neural network and output the volumetric representation that includes segmented teeth and gingiva of the dental object. The one or more processors may be configured to train the graph neural network by volumetric representations determined based on CBCT scans and / or intraoral scans acquired by a handheld intraoral scanner.
[0036] The alignment of the point clouds of the non-visible 2D images to the visible sub-scans may be provided by positions and directions of the handheld intraoral scanner during the non-visible sub scans and the visible sub-scans,
[0037] The volumetric representation may be determined differently. The one or more processors is configured to determine one or more volumetric optical coefficients for each of the plurality of vertices of the graphical representation based on the corresponding one or more optical coefficients and a neural radiance field algorithm and determine the volumetric representation of a surface and an inner region of the dental object based on the one or more volumetric optical coefficients.
[0038] To improve the speed of the neural radiance field algorithm then a lower resolution of the 3D finite element mesh is fed into the neural radiance field algorithm, and which means that the algorithm must process less one or more optical coefficients to generate a volumetric representation of the dental object. It is possible to further reduce the resolution of the 3D finite element mesh as many of the neighbouring optical coefficients may have the same or about the same value representing the same dental feature. A group of optical coefficients may be merged into one optical coefficient if they have the same value or about the same value. The one or more optical coefficients may include multiple groups of optical coefficients which would then result in a reduced resolution of the 3D finite element mesh. The one or more processors may be configured to determine a reduced-resolution 3D finite element mesh of the 3D finite element mesh, and wherein the reduced-resolution 3D finite element mesh includes a plurality of reduced-resolution vertices, and wherein the reduced-resolution 3D finite element mesh includes a plurality of reduced-resolution finite elements. The one or more processors may be configured to align the point clouds to the reduced-resolution 3D finite element mesh, and wherein the alignment of the point clouds of the non-visible 2D images and the visible sub-scans is provided by positions and directions of the handheld intraoral scanner during the non-visible sub scans and the visible sub-scans. The one or more processors may be configured to determine one or more reduced-resolution optical coefficients for each of the plurality of reduced-resolution vertices by an inverse photon scattering algorithm and to determine intermediate one or more optical coefficients for each of the plurality of vertices of the graphical representation by interpolating between the one or more reduced-resolution optical coefficients of the plurality of reduced- resolution vertices of the reduced-resolution 3D finite element mesh. The one or more processors may be configured to determine one or more volumetric optical coefficients for each of the plurality of vertices of the graphical representation based on the corresponding intermediate one or more optical coefficients and a neural radiance field algorithm, and to determine a volumetric representation of a surface and an inner region of the dental object based on the one or more volumetric optical coefficients.
[0039] The neural radiance field algorithm may be configured to process the estimated positions and directions of the handheld intraoral scanner, determine ray or cone casting objects based on the estimated positions and directions for each pixels of an image sensor unit of the handheld intraoral scanner, determine, based on a loss function and one or more optical coefficients, the one or more volumetric optical coefficients for each of the plurality of vertices which the ray or cone casting objects intersects, and wherein the neural radiance field algorithm is configured to be trained using the non-visible 2D images.
[0040] The one or more processors may be configured to train the neural radiance field algorithm by determining 2D non-visible images of the non-visible 2D images, receiving the estimated positions and directions of the handheld intraoral scanner that corresponds to ray or cone casting objects for each pixel of the 2D non-visible images. The ray or cone casting objects reflects the viewing axis or viewing volume, respectively, of each pixel. Furthermore, the one or more processors may be configured to train the neural radiance field algorithm by determining, based on the neural radiance field algorithm using the estimated positions and directions of the handheld intraoral scanner and one or more optical coefficients, the one or more volumetric optical coefficients for each of the plurality of vertices which the ray or cone casting objects intersects, and determining a synthetic pixel value for each of the ray or cone casting objects based on the corresponding determined one or more volumetric optical coefficients for each of the plurality of vertices which the ray or cone casting objects intersects; and minimizing a loss function between the synthetic pixel value and a corresponding true pixel value for each pixel of the 2D non-visible images.
[0041] The non-visible 2D images include infrared information or near-infrared information. The infrared information or the near-infrared information includes wavelengths of between 750 nm and 1250 nm
[0042] The system may be configured to display the graphical representation and / or the volumetric representation that is determined based on the graphical representation. The system may include a displaying unit configured to display a 3D model of the dental object that includes the 3D graphical representation and / or the volumetric representation.
[0043] The displaying unit may be configured to display a 3D model of the dental object that includes the volumetric representation of a surface and an inner region of the dental object.
[0044] The one or more processors may be configured to determine a first dental feature boundary of a first dental feature based on the determined one or more optical coefficients or one or more volumetric optical coefficients, and wherein the first dental feature boundary corresponds to the non-visible 2D images, and wherein the one or more processors is configured to determine an internal measurement of the first dental feature boundary. The first dental feature boundary may be represented within the 3D finite element mesh or on a volumetric model of the dental object that is determined based on the 3D finite element mesh. The first dental feature boundary corresponds to a circumference of a dental feature that is represented within the 3D finite element mesh or the volumetric model of the dental object. The internal measurement of an inner region within a 3D finite element mesh results in an improved precision of the measurement of an inner region. Thereby, it is possible to determine with a high precision the volume of the dental feature, and thereby, the impact of the dental feature within the dental object is more precisely determined.
[0045] By determining a dental feature boundary provides an analytical approach in determining a measure of a dental feature that results in a more precise and faster measurement of the dental feature when comparing to a manual measurement performed by a dentist or orthodontic. The dental feature boundary determines an outer contour or shape of a dental feature within the dental object, and the outer contour or shape allows the one or more processors to determine the size of the dental feature, wherein the size may be a distance between two points on the outer contour or shape. The size may correspond to a volume of the dental feature. The dental feature boundary may be a three dimensional dental feature boundary determined within the 3D finite element mesh.
[0046] The dental feature boundary may include multiple vertices that includes an optical coefficient that corresponds to a dental feature. The multiple vertices that includes optical coefficients that correspond to a dental feature are connected by a marching finite element algorithm or a spline function, and the output of marching finite element algorithm or the spline function is the dental feature boundary that includes the outer shape or contour of the corresponding dental feature.
[0047] The internal measurement may be determined between a first part of the first dental feature boundary and a second part of the first dental feature boundary. The internal measurement may be a distance between the first par and the second part, and the distance may be a minimum or a maximum distance. For example, the dental feature may be the enamel, and the first part may correspond to an upper part of the enamel and the second part may correspond to a bottom part of the enamel, and wherein the upper part is opposite to the bottom part. In this example, the minimum distance corresponds to the thinnest part of the enamel, and wherein the maximum distance corresponds to the thickest part of the enamel. In a broader content, by knowing the minimum distance for all dental objects, such as teeth, for a dentition, such as upper and lower jaws, the dentist or the orthodontic would be able to know which teeth have a critical thickness of enamel that would have a critical short travel distance for a potential caries from a surface to the dentin of the dental object.
[0048] The one or more processors may be configured to determine multiple internal measurements of the first dental feature boundary or of a shape object fitted to the geometry of the first dental feature boundary. The one or more processors may be configured to determine a minimum and / or a maximum internal measurement of the multiple internal measurements. The minimum internal measurement may correspond to the minimum distance determined between the two parts, i.e. the first part and the second part, of the dental feature boundary. The maximum internal measurement may correspond to the maximum distance determined between the two parts, i.e. the first part and the second part, of the dental feature boundary.
[0049] The internal measurement may be a distance along a normal vector of the first part of the first dental feature boundary. The normal vector may extend between the first part and the second part of the first dental feature boundary.
[0050] In another example, the dental feature may be a caries, and the internal measurement may be a distance measure that indicates the size of the caries along a longitudinal axis that extends between the occlusal and the furcation of the dental object. The distance measure may indicate the size of the caries towards the enamel or the pulp chamber of the dental object.
[0051] The one or more processors may be configured to perform a fitting of a shape object to a geometry of the first dental feature boundary, and wherein the internal measurement is determined based on the fitted shape object. The geometry may be an outer contour or shape of the dental feature boundary. The shape object may be an ellipse, a sphere, an ellipsoid, or a bounding box. The fitting may be based on a principal component analysis. The fitting to the geometry of the dental feature boundary, such as the first dental feature boundary, would result in an implicit function that corresponds to the geometry of the dental feature boundary. The implicit function would improve the resolution of the geometry of the dental feature boundary. Using such an implicit function and a refinement of the finite element volume mesh, using the same marching finite elements method, one can generate a segment boundary with a higher resolution. The improved resolution would inevitably result in a more precise internal measurement of the dental feature that corresponds to the dental feature boundary.
[0052] The one or more processors may be configured to determine multiple internal measurements of the first dental feature boundary or to a shape object fitted to the geometry of the first dental feature boundary, and wherein the internal measurement includes a volumetric measurement that may be determined based on the multiple internal measurements. The volumetric measurement may include a volume size of the dental feature that corresponds to the first dental feature boundary. For example, the dental feature may be a caries and it would be of an advantage for the dentist or the orthodontic to measure the volume of the caries as that would give an improved indication of how the size of the caries evolves over a period of time. By monitoring a distance between two parts of the caries would provide a measure of the size in a single dimension, wherein by monitoring the volume of the caries would provide a measure of the size in multiple dimensions. By monitoring in multiple dimensions would inevitably results in an improved monitoring of the caries. It may be that the caries does not evolve in the single dimension you are monitoring the size but instead in another dimension. The volume of the size would change no matter which of the dimensions the caries is changing within.
[0053] The internal measurement may be a sectional curvature measure of a part of the first dental feature boundary or to a shape object fitted to the geometry of the first dental feature boundary. The sectional curvature measure may be based on a mean or a gaussian curvature measure. By knowing the curvature of a section of the dental feature boundary would be of an advantage in getting a better understanding of the overall shape of the dental feature.
[0054] For a dentist or an orthodontic it would be of a benefit to know different internal measurements of the inside part of a dental object. Following internal measurements would be of an advantage to the dentist or the orthodontic to know: • The actual depth / extension of cracks / caries in millimetres or cubic millimetres, or caries / cracks in relation to the enamel, dentin and pulp chamber. o For example, the dentist or the orthodontic may need to know if a crack / caries is only in enamel or it extends to the dentin or it even reaches the pulp chamber.
[0055] • The ratio of the enamel and / or the dentin that is affected. For example, caries may affect a percentage of the enamel thickness or the dentin thickness.
[0056] • The distance between the caries and the pulp.
[0057] • The depth of the crack into the tooth coronal part (enamel-dentin) or into the root (cementum and dentin).
[0058] • A volumetric measure of the enamel, dentin or pulp chamber, developed defects within the dental object, and tooth wear lesions.
[0059] • The volume or extension of restorations on a tooth and combination of those with for example caries and cracks. For example, when scanning a tooth with a composite restoration it is good to know how “deep” that restoration is in relation to the enamel, dentin and pulp chamber and the potential presence of caries or cracks on the borders / under those restorations.
[0060] Some of the above-mentioned advantages are already achieved by the internal measurements of the first dental feature boundary, however, the advantages that applies an internal measurement between two different dental features would be explained below.
[0061] The one or more processors may be configured to determine a plurality of dental feature boundaries, and wherein the plurality of dental feature boundaries includes the first dental feature boundary and at least a second dental feature boundary, and wherein the at least second dental feature boundary corresponds to a second dental feature; and wherein the internal measurement may be determined between the first dental feature boundary and the at least second dental feature boundary. For example, the first dental feature may be a disease feature, such as a crack or a caries, and the second dental feature may be an anatomy feature, such as an enamel, a dentine or a pulp chamber. In one example, the first dental feature boundary may represent a disease feature, and the second dental feature boundary may represent an anatomy feature.
[0062] The one or more processors may be configured to group a plurality of dental feature boundaries into multiple dental feature boundary groups, and wherein the plurality of dental feature boundaries includes the first dental feature boundary and at least a second dental feature boundary, and wherein each of the multiple dental feature boundary groups corresponds to different dental features. In a scanning situation which implies scanning both a lower and an upper jaw of a dentition would result in the plurality of dental features boundaries for
[0063] The internal measurement may be determined between a first part of the first dental feature boundary and a second part of the second dental feature boundary. The internal measurement may be a distance between the first par and the second part, and the distance may be a minimum or a maximum distance. For example, the first dental feature may be a caries, and the first part may correspond to a bottom part of the enamel, and the second part may correspond to a bottom part of the caries, and wherein the bottom part of the enamel is opposite to the bottom part of the caries. In this example, the sign of the internal measurement, whether it is positive or negative, would determine whether the caries is extending through the enamel and into the dentine of the dental object. Furthermore, the level of the internal measurement corresponds to the depth of the caries into the enamel. The bottom part of the caries and the bottom part of the enamel may be determined along a longitudinal axis of the dental object that extends between occlusal and the furcation of the dental object. The system may be configured to provide an alert signal if the distance between the bottom parts is below a certain minimum distance threshold, and wherein the alert signal indicates that the caries is close to enter the dentin area of the dental object. In another situation, the system may be configured to monitor an area of the dental object that has a distance between the two bottom parts that are below a maximum distance threshold and above a minimum distance threshold. The system may be configured not to monitor the area of the dental object when the distance between the two bottom parts is above the maximum distance. The internal measurement may be a distance between the first dental feature boundary and the at least second dental feature boundary or an overlap ratio between the first dental feature boundary and the second dental feature boundary. For example, the overlap ratio may be a percentage of how much of the dentin, enamel or the pulp chamber is affected by a caries or a crack. The overlap ratio may be determined as following:
[0064] • determine a second volume of the second dental feature,
[0065] • determine a first volume of the first dental feature that is overlapped by the second volume, and wherein the overlap ratio is determined between the first volume and the second volume. For example, if the first volume equals the second volume, then 100 % of the second volume is occupied by the first volume. Normally, the first volume would be smaller than the second volume.
[0066] In another example, the overlap ratio may be determined as following:
[0067] • determine a second distance of the second dental feature, and wherein the second distance is determined between two parts of the second dental feature,
[0068] • determine a first distance between two parts of the first dental feature that is overlapped by the second distance, and wherein the overlap ratio is determined between the first distance and the second distance. For example, if the first distance equals the second distance, then 100 % of the second distance is occupied by the first distance. Normally, the first distance would be smaller than the second distance.
[0069] The one or more processors may be configured to determine multiple sub-internal measurements that include a first sub-internal measurement, a second sub-internal measurement, and a third sub-internal measurement. The first sub-internal measurement may correspond to a distance or volumetric measurement of the first dental feature that may correspond to the first dental feature boundary. The second sub-internal measurement may correspond to a distance or volumetric measurement of the second dental feature that may corresponds to the second dental feature boundary. The third sub-internal measurement may be a distance measure or a volumetric measure of an overlap between the first dental feature boundary and the at least second dental feature boundary, and the internal measurement may be a ratio between the third sub-internal measurement and the first sub-internal measurement or the second sub-internal measurement. The ratio informs a size ratio of how much of the first dental feature or the second dental feature is overlapped by the other dental feature. The ratio may be in percentage, and in this example, the dentist or the orthodontic would know how much of for example the dentin or the enamel is affected by caries or cracks in percentage.
[0070] The one or more processors may be configured to determine a progress time for the first dental feature boundary to progress into or overlap partly the second dental feature boundary. The progress time may be determined based on the distance between the first dental feature boundary and the at least second dental feature boundary. Alternatively, the progress time may be determined based on the distance between the first dental feature boundary and the at least second dental feature boundary and a progress time algorithm that has been trained to output a progress time based on the distance between two dental feature boundaries. The training of the progress time algorithm may be based on non- visible 2D images performed on different patients, wherein the progression of dental features identified in the non-visible 2D images have been monitored over a period of time and registered within the progress time algorithm. By inputting the distance between the first and the second dental feature boundary into the trained progress time algorithm, the trained progress time algorithm may be configured to determine the progress time by correlating the distance to the monitored progression of a similar dental feature that corresponds to the first dental feature boundary, such as a caries.
[0071] The plurality of dental feature boundaries may be determined by a marching finite element algorithm. The marching finite element algorithm may be configured to select a set of the one or more optical coefficients that are nearest a set of the multiple vertices for constructing a shape function that represents the first dental feature boundary. The algorithm may be configured to construct multiple shape function for multiple dental feature boundaries, such as the first dental feature boundary and the second dental feature boundary. The marching finite element algorithm may be a three-dimensional finite element algorithm, or more precisely, a marching cubes or tetrahedra algorithm. The determined point clouds of the non-visible 2D images and the visible sub-scans may include a first set of point clouds that corresponds to the non-visible 2D images and a second set of point clouds that corresponds to the visible sub scans. Both the first set of point clouds and the second set of point clouds are aligned to the 3D finite element mesh. The one or more processors may be configured to assign each of the point clouds to an optical coefficient of the one or more optical coefficients. The marching finite element algorithm may then select a set of point clouds that are nearest a set of the multiple vertices for constructing a shape function that represents the first dental feature boundary. The algorithm may be configured to construct multiple shape function for multiple dental feature boundaries, such as the first dental feature boundary and the second dental feature boundary.
[0072] The one or more optical coefficients may be determined based on a neural radiance field model, wherein the neural radiance field model is configured to determine the one or more optical coefficients based on spatial location information and a viewing angle information of the handheld intraoral scanner while capturing the non-visible sub scans and the visible sub scans.
[0073] The one or more processors may be configured to determine for each of the multiple vertices a relative position between the handheld intraoral scanner and the dental object, wherein the relative position includes spatial location information and viewing angle information for a corresponding casting object. The one or more processors may be further configured to determine the one or more optical coefficients by a neural radiance field model, wherein the neural radiance field model is configured to determine the one or more optical coefficients based on the spatial location information and the viewing angle information.
[0074] The neural radiance field model may be configured to be trained using a plurality of non- visible 2d images, and wherein the casting object may be a ray or a cone. The ray corresponds to the viewing along the ray of the pixel and the cone corresponds to the viewing volume of the pixel. Each pixel of the image sensor is configured to receive non-visible sub-scans which are then turned into synthetic pixel values for the corresponding pixels of the image sensor by the one or more processors. The received non-visible sub-scans includes internal scatterings from within an inner region of the dental object, and which means that each of the synthetic pixel values includes an average of internal scatterings from different planes that are distributed along a casting object of the corresponding pixel. Each of the synthetic pixel values may correspond to each of the multiple vertices of a primary 3D finite element mesh, and wherein the one or more processors is configured to determine the one or more optical coefficients for each of the multiple vertices of the primary 3D finite element mesh. The internal scatterings of the received non-visible sub-scans may be represented by one or more optical coefficients of multiple vertices of a sub 3D finite element mesh, and since the received non-visible sub-scans include internal scatterings captured from different planes, the neural radiance field model is configured to determine a sub 3D finite element mesh for each of the different planes. The sub 3D finite element meshes are arranged along the casting object, such that a casting object starting from a vertex of the primary 3D finite element mesh may intersects the corresponding vertex of each of the sub 3D finite element meshes. Thereby, the one or more optical coefficients of the vertex of the primary 3D finite element mesh may be an average of the one or more optical coefficients of the corresponding vertex of the sub 3D finite element meshes which the casting object intersects.
[0075] The neural radiance field model may be trained by generating, based on the set of input parameters, an optical coefficient for each of the one or more coordinates which represents a density value for absorption of light, scattering of light, or refractive index. The neural radiance field model may be trained by determining the synthetic pixel value for the casting object based on the corresponding determined optical coefficient for each of the sub 3D finite element meshes that represent the internal scatterings from within the dental object. Further, neural radiance field model may be trained by minimizing a loss function between the synthetic pixel value and a corresponding true pixel value of the plurality of pixels of the non-visible sub-scans.
[0076] The non-visible sub-scans include 2D infrared image data or 3D infrared image data. The one or more processors may be further configured to determine a plurality of grid points within the 3D surface model and determine the 3D inner geometry by arranging at least one of the synthetic pixel value the one or more optical coefficients at each of the plurality of grid points.
[0077] Since most of the teeth, i.e. dental objects, have similar interior structure, the variation in the one or more optical coefficients for different tooth samples will be limited and there is a lot of covariance between the values of the one or more coefficients of the multiple vertices. The one or more processors may be configured to perform a covariance analysis of the one or more optical coefficients for the multiple vertices, determine a plurality of descriptive parameters based on the covariance analysis, and wherein a number of the plurality of descriptive parameters is smaller than a number of the one or more optical coefficients, and wherein the first dental feature boundary may be determined by providing the plurality of descriptive parameters into a marching finite element algorithm of the intraoral scanning system. In this example, the one or more processors would be faster in determining the first dental feature boundary when comparing to the other situation where the marching finite element algorithm receives the one or more optical coefficients for each of the multiple vertices.
[0078] The covariance analysis may be a principal component analysis or an autoencoder deep learning algorithm, wherein the autoencoder deep learning algorithm includes a neural network. The neural network may be configured to receive the non-visible 2d images and determine the plurality of descriptive parameters like one or more optical coefficients of the multiple vertices. The one or more processors may be configured to train the neural network by receiving one or more CBCT scans of one or more dental objects, determining one or more training optical coefficients for each of the one or more CBCT scans, receiving one or more non-visible 2d images from a handheld intraoral scanner of the one or more dental objects, and training the neural network by mapping the one or more training optical coefficients onto a three-dimensional finite element mesh of the one or more non-visible sub scans. The neural network may be trained to classify specific dental features based on the one or more training optical coefficients, and wherein the neural network may be configured to receive the one or more optical coefficients of the multiple vertices that corresponds to the non-visible sub-scans and determine a plurality of descriptive parameters for a specific dental feature based on the trained optical coefficients and the received one or more optical coefficients.
[0079] The non-visible 2d images may include a plurality of 2D infrared images where the position and orientation of each of the plurality of 2D infrared images is known. The plurality of 2D infrared images may be supported by colour 2D images and fluorescent 2D images. The system may receive a CBCT scan of dental objects creating an accurate voxel model of the dental objects. The accurate voxel model may be segmented into layers, which are used to create a parametric voxel or a hexahedral mesh. Each of the plurality of 2D infrared images of the dental objects are mapped onto the accurate voxel model of the dental objects based on the position and orientation of each of the plurality of 2D infrared images. The accurate voxel model including the mapped plurality of 2D infrared images are feed into the neural network for training the neural network. With the trained neural network, a parametric model can be generated, and which describes a dental object based on 2D IR images. The parametric model includes the multiple vertices and the corresponding one or more optical coefficients.
[0080] It would be of an advantage to the dental practitioner to have a measure of how far a disease feature, e.g. a second dental feature boundary, is progressing into for example an anatomy feature, e.g. a first dental feature boundary. The one or more processors may be configured to determine a plurality of normal lengths from a second dental feature in a direction towards the first dental feature boundary and along a longitudinal axis of the dental object in the 3D finite element mesh and determine a maximum normal length of the plurality of normal lengths. The one or more processors may further be configured to determine a penetration depth of the maximum normal length that penetrates the first dental feature, and wherein the internal measurement includes the penetration depth. The penetration depth may be a distance measured in millimetres or micrometres. A penetration depth ratio may be determined by the one or more processors, and wherein the penetration depth ratio is the ratio between the penetration depth and a maximum normal length of the first dental feature boundary along the longitudinal axis.
[0081] A sign of each of the plurality of normal lengths determines whether a normal length of the plurality of normal lengths is within the first dental feature. For example, if the sign of the normal length is negative then it indicates that the normal length is within the first dental feature, and if positive then it indicates that the normal length is outside the first dental feature. The one or more processors may be configured to select a group of normal lengths of the plurality of normal lengths, wherein each of the normal lengths of the group has a first sign that indicates that the group of normal lengths is within the first dental feature and determine a volume measurement based on the group of normal lengths, and wherein the internal measurement is the volume measurement.
[0082] The one or more processors may be configured to determine a notification signal when the internal measurement is above a measurement threshold, and wherein the notification signal is displayed on a user interface of the system. Thereby, the dental would be warned if any critical internal measurements are provided by the system.
[0083] It is of importance that the internal measurement is being visualized in a manner that the dentist can easily interpret and understand that the internal measurement and the eventually risk if the internal measurement is evaluated by the system as being critical for one or more dental diseases. The intraoral scanning system may include a user interface that may be configured to display a 3D model of the dental object and a 2D cross-section of the dental object, and wherein the internal measurement is displayed on the 2D crosssection of the dental object. Via the 3D model it may be possible for the user of the system to select which parts of the 3D model the user wants to have displayed as a 2D cross section. The internal measurement or a plurality of internal measurements may be displayed on the 2D cross section together with or without the dental feature boundaries. The internal measurement of a dental feature or between dental features may be displayed together with an arrow or a line that indicates where the internal measurement has been performed. In a situation where the internal measurement includes a volume measure, the dental feature which the volume measure corresponds to may be coloured for enhancing the contrast of the dental feature, and the internal measurement may be displayed on the dental feature or next to.
[0084] The dentist may be able to add two or more measurement points on the 3D model, a 2D image of a section of the 3D model, or the 2D cross section image, and the one or more processors is configured to perform the internal measurement between the two or more measurement points. The two or more measurement points may be added via a cursor on the user interface, wherein the cursor may be moved around via a motion sensor of the handheld intraoral scanner or by a mouse. In this example, the one or more processors is configured to assign each of the two or more measurement points to the closest vertex of the multiple vertices, and thereby, the one or more processors may be configured to determine the internal measurement between the two or more measurement points. A first measurement point of the two or more measurement points may correspond to the first part of the dental feature boundary, and a second measurement point of the two or more measurement points may correspond to the second part of the dental feature boundary.
[0085] The dentist may be able to add a contour line on the 3D model that encompass a dental feature depicted on the 3D model, a 2D image of a section of the 3D model, or the 2D cross section image, and the one or more processors may be configured to perform the internal measurement on the encompassed dental feature. In this example, the internal measurement may be a volume measurement of the encompassed dental feature. The contour line on the 3D model may encompass an overlap between the first and the second dental feature boundary, i.e. between the first and the second dental feature. The contour line may be added via a cursor on the user interface, wherein the cursor may be moved around via a motion sensor of the handheld intraoral scanner or by a mouse. In this example, the one or more processors is configured to assign the contour line to the closest vertices of the multiple vertices, and thereby, the one or more processors may be configured to determine the internal measurement of the encompassed overlap or dental feature.
[0086] The dental object in the displayed 2D cross-section may be selected by a user of the system via a marker on the displayed three-dimensional model of the dental object. The marker may be a window which makes it easier for the user to see which dental object is being selected on the 3D model to be view as a 2D cross-section.
[0087] The first dental feature boundary may be displayed on the 3D model of the dental object and / or the 2D cross-section of the dental object. By displaying the dental feature boundaries on the 3D model and / or the 2D cross-section would provide a better interpretation of the internal measurements being performed by the one or more processors. The dental feature boundaries may be displayed on the dental object as the actual dental feature that it corresponds to, such as an enamel, a dentine, a pulp chamber, crack, caries, filling or a compose restoration etc.
[0088] The one or more processors may be configured to change a viewing angle of the 2D crosssection on a position of a marker on the 3D model. The marker may be a line that indicates the cross-section that is displayed on the 2D cross-section. The user can rotate the marker, and the cross-section displayed on the 2D cross-section is also rotated symmetrically with the rotation of the marker.
[0089] The one or more processors may be configured to change a viewing depth into a dental object of the 3D model. By increasing the viewing depth, the user can see further into the dental object, and when decreasing the viewing depth, the user is able to see less into the dental object. The viewing depth may be adjusted by a slider on the user interface or by a button press on the handheld intraoral scanner and a movement of the handheld intraoral scanner that is detected by a motion sensor that is arranged within the handheld intraoral scanner.
[0090] The user interface may include a transparency unit that may be configured to change the transparency of the first dental feature boundary. In some examples it is beneficial for the user not to visualize one or more dental features on the dental object, and thereby, the user may be able to change the transparency of selected dental feature boundaries. For example, the user may not be interested to see the any disease features but only the anatomy features, and in this example, the user would increase the transparency of the disease features to an extend they can’t be seen by the user on the user interface. In yet another example, the user may be able to remove the unwanted dental features on the dental object by selecting these and press a remove button on a keyboard or a graphical remove button on the user interface. By removing or reducing the transparency of any dental features would also remove or reduce the transparency of corresponding internal measurements that may be applied on the same dental object.
[0091] The user interface may include a selector that may be configured to select one or more of a plurality of dental feature boundaries to be displayed, and wherein the plurality of dental feature boundaries includes the first dental feature boundary and at least a second dental feature boundary. The transparency unit may be configured to change the transparency of the selected one or more of the plurality of dental feature boundaries.
[0092] The user interface may be configured to display a plurality of dental feature boundaries, and wherein the plurality of dental feature boundaries includes the first dental feature boundary and at least a second dental feature boundary, and wherein the plurality of dental feature boundaries may have different colors for the purpose of improving the contrast between the dental features on the dental object, and thereby, improving the visualization of the dental features.
[0093] The one or more processors may be configured to determine the first dental feature that corresponds to the first dental feature boundary, and wherein the first dental feature may be determined based on the one or more optical coefficients that correspond to the first dental feature boundary and a dental feature algorithm. The dental feature algorithm includes one or more of following an anatomy feature determiner, a disease feature determiner, and a mechanical feature determiner. The anatomy feature determiner may be configured to determine that the one or more optical coefficients corresponds to an anatomy feature when the one or more optical coefficients is within an anatomy feature coefficient range. The disease feature determiner may be configured to determine that the one or more optical coefficients corresponds to a disease feature when the one or more optical coefficients is within a disease feature coefficient range. The mechanical feature determiner may be configured to determine that the one or more optical coefficients corresponds to a mechanical feature when the one or more optical coefficients is within a mechanical feature coefficient range. The anatomy feature coefficient range, the disease feature coefficient range and the mechanical feature coefficient range may be predetermined and stored in a memory of the system. In another example, the ranges may be occasionally updated based on manual input by a user. The user may identify dental features on a graphical user interface and categorize the identified dental features into an anatomy feature, a disease feature or a mechanical feature. The manual input may be provided by different users of the system, and the memory may be a cloud server or a server that are connected to the intraoral scanning system that may include multiple handheld intraoral scanners and user interfaces connected via a wireless network.
[0094] The one or more processors may be configured to align the non-visible sub-scans to the visible sub-scans and determine the point clouds of the aligned sub-scans. The alignment may be based on positions and viewing angles of the handheld intraoral scanner while capturing the sub-scans. The positions and the viewing angles may be determined by a motion sensor that is arranged within the handheld intraoral scanner. Alternatively, the one or more processors may be configured to align the non-visible sub-scans to the visible subscans based on common dental features between the non-visible sub-scans and the visible sub-scans. For example, the position of the handheld intraoral scanner may be determined by identifying a dental object in the visible sub-scans and the same dental object in the non-visible sub-scans based on common dental features, such as the geometry and / or a shade color of the dental object or one or more of the dental features. The one or more processors may be configured to determine the viewing angle to the dental object of the visible sub-scans and the non-visible sub-scans based on triangulation between one or more light sources of the handheld intraoral scanner, one or more image sensors of the handheld intraoral scanner, and the dental object or the one or more dental features.
[0095] For enhancing the contrast between the different dental features of a dental object the one or more processors may be configured to determine point clouds of the visible sub-scans and a composition of the non-visible sub-scans and the visible sub-scans. The one or more processors may be configured to determine, in real time, surface information from the visible sub-scans for generating or updating a three-dimensional (3D) model of the dental object. The one or more processors may be further configured to_determine a plurality of dental feature boundaries, including the first dental feature boundary and / or the second dental feature boundary, based on the composition or on the non-visible sub-scans. By the composition of the non-visible sub-scans and the visible sub-scans the contrast between the dental features of the dental object become more enhanced in relation to a noncomposition of the non-visible sub-scans.
[0096] The non-visible sub-scans include mainly scattering from inside a dental object, i.e. internal region of a dental object, and significantly less of surface reflection of a tooth. The visible sub-scans include mainly surface reflection of a tooth and significantly less reflection from inside a tooth, i.e. internal region of a dental object.
[0097] A composed scan information does not include an overlay of two 2D images where each of the two 2D images relates to different emitted wavelengths from the projector unit. In this example, no enhancement of internal structure information is provided. Instead, the composed scan information may be a combination of intensity levels of each pixel of the image sensor unit that relates to different wavelengths. For example, at a first time period, the image sensor unit may capture light information that relates to visible wavelength, and at a second time period, the image sensor unit may capture light information that relates to a infrared or near-infrared wavelength, and the intensity levels of the two time periods are recorded and combined for enhancing a dental feature. The combination of the intensity levels may be done digitally by subtraction and / or addition of the intensity levels. In another example, the intensity levels may be captured and recorded during at least three time periods for at least three different wavelengths, such as white-coloured wavelength, blue-coloured wavelength and near-infrared wavelength.
[0098] The one or more processors may be configured to display the composed scan information and the 3D model on a displaying unit of the system, and wherein the dental feature boundaries may be added to the composed scan information and / or the 3D model.
[0099] BRIEF DESCRIPTION OF THE FIGURES Aspects of the disclosure may be best understood from the following detailed description taken in conjunction with the accompanying figures. The figures are schematic and simplified for clarity, and they just show details to improve the understanding of the claims, while other details are left out. Throughout, the same reference numerals are used for identical or corresponding parts. The individual features of each aspect may each be combined with any or all features of the other aspects. These and other aspects, features and / or technical effect will be apparent from and elucidated with reference to the illustrations described hereinafter in which:
[0100] FIG. 1 illustrates an example a graphical representation of a dental object;
[0101] FIGS. 2A to 2E illustrate different examples of the system;
[0102] FIG. 3 illustrates an example of one or more processors;
[0103] FIG. 4 illustrates another example of one or more processors;
[0104] FIG. 5 illustrates an example of a graph neural network;
[0105] FIGS. 6 A and 6B illustrate different examples of a neural radiance field algorithm
[0106] FIG. 8 illustrates another example of one or more processors;
[0107] FIG. 9 illustrates an intraoral scanning system;
[0108] FIG. 10 illustrates the dental object defined as a 3D finite element mesh;
[0109] FIGS. 11 A and 1 IB illustrates an example of one or more processors;;
[0110] FIG. 12 illustrates an example of an internal measurement;
[0111] FIGs. 13A, 13B and 13C illustrate different examples of the internal measurement;
[0112] FIGs. 14A, 14B and 14C illustrate different examples on determining a plurality of dental 4feature boundaries;
[0113] FIGS. 16A and 16B illustrate examples of the one or more processors that is configured to determine one or more optical coefficients;
[0114] FIG. 17 illustrates an example of the one or more processors that is configured to perform a covariance analysis;
[0115] FIGs. 18A and 18B illustrate examples on how to train a neural network for determining a plurality of descriptive parameters;
[0116] FIG. 19 illustrates an example of a user interface of the system; and
[0117] FIGs. 20A and 20B illustrate an example of the one or more processors performing a subdivision of a finite element. DETAILED DESCRIPTION
[0118] The detailed description set forth below in connection with the appended drawings is intended as a description of various configurations. The detailed description includes specific details for the purpose of providing a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. Several aspects of the devices, systems, mediums, programs and methods are described by various blocks, functional units, modules, components, circuits, steps, processes, algorithms, etc. (collectively referred to as “elements”). Depending upon particular application, design constraints or other reasons, these elements may be implemented using electronic hardware, computer program, or any combination thereof.
[0119] The electronic hardware may include microprocessors, microcontrollers, digital signal processors (DSPs), field programmable gate arrays (FPGAs), programmable logic devices (PLDs), gated logic, discrete hardware circuits, and other suitable hardware configured to perform the various functionality described throughout this disclosure. Computer program shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executables, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.
[0120] A scanning for providing intra-oral scan data may be performed by a dental scanning system that may include a handheld intraoral scanning device such as the TRIOS series scanners from 3 Shape A / S. The dental scanning system may include a wireless capability as provided by a wireless network unit. The scanning device may employ a scanning principle such as triangulation-based scanning, confocal scanning, focus scanning, ultrasound scanning, x-ray scanning, stereo vision, structure from motion, optical coherent tomography OCT, or any other scanning principle. In an embodiment, the scanning device is capable of obtaining surface information by operated by projecting a pattern and translating a focus plane along an optical axis of the scanning device and capturing a plurality of 2D images at different focus plane positions such that each series of captured 2D images corresponding to each focus plane forms a stack of 2D images. The acquired 2D images are also referred to herein as raw 2D images, wherein raw in this context means that the images have not been subject to image processing. The focus plane position is preferably shifted along the optical axis of the scanning system, such that 2D images captured at a number of focus plane positions along the optical axis form said stack of 2D images (also referred to herein as a sub-scan) for a given view of the object, i.e. for a given arrangement of the scanning system relative to the object. After moving the scanning device relative to the object or imaging the object at a different view, a new stack of 2D images for that view may be captured. The focus plane position may be varied by means of at least one focus element, e.g., a moving focus lens. The scanning device is generally moved and angled relative to the dentition during a scanning session, such that at least some sets of sub-scans overlap at least partially, in order to enable reconstruction of the digital dental 3D model by stitching overlapping subscans together in real-time and display the progress of the virtual 3D model on a display as a feedback to the user. The result of stitching is the digital 3D representation of a surface larger than that which can be captured by a single sub-scan, i.e. which is larger than the field of view of the 3D scanning device. Stitching, also known as registration and fusion, works by identifying overlapping regions of 3D surface in various sub-scans and transforming sub-scans to a common coordinate system such that the overlapping regions match, finally yielding the digital 3D model. An Iterative Closest Point (ICP) algorithm may be used for this purpose. Another example of a scanning device is a triangulation scanner, where a time varying pattern is projected onto the dental arch and a sequence of images of the different pattern configurations are acquired by one or more cameras located at an angle relative to the projector unit.
[0121] The handheld intraoral scanner may comprise one or more light projectors configured to generate an illumination pattern to be projected on a three-dimensional dental object during a scanning session. The light projector(s) may comprise a light source, a mask having a spatial pattern, and one or more lenses such as collimation lenses or projection lenses. The light source may be configured to generate light of a single wavelength or a combination of wavelengths (mono- or polychromatic). The combination of wavelengths may be produced by using a light source configured to produce light (such as white light) comprising different wavelengths. Alternatively, the light projector(s) may comprise multiple light sources such as LEDs individually producing light of different wavelengths (such as red, green, and blue) that may be combined to form light comprising the different wavelengths. Thus, the light produced by the light source may be defined by a wavelength defining a specific colour, or a range of different wavelengths defining a combination of colours such as white light. In an embodiment, the scanning device comprises a light source configured for exciting fluorescent material of the teeth to obtain fluorescence data from the dental object. Such a light source may be configured to produce a narrow range of wavelengths. In another embodiment, the light from the light source is infrared (IR) light, which is capable of penetrating dental tissue. The light projector(s) may be DLP projectors using a micro mirror array for generating a time varying pattern, or a diffractive optical element (DOF), or back-lit mask projectors, wherein the light source is placed behind a mask having a spatial pattern, whereby the light projected on the surface of the dental object is patterned. The back-lit mask projector may comprise a collimation lens for collimating the light from the light source, said collimation lens being placed between the light source and the mask. The mask may have a checkerboard pattern, such that the generated illumination pattern is a checkerboard pattern. Alternatively, the mask may feature other patterns such as lines or dots, etc.
[0122] Color texture of the dental arch may be acquired by illuminating the object using different monochromatic colors such as individual red, green and blue colors or my illuminating the object using multichromatic light such as white light. A 2D image may be acquired during a flash of white light.
[0123] Generally the process of obtaining surface information in real time of a dental arch to be scanned requires the scanning device to illuminate the surface and acquire high number of 2D images. Typically a high speed camera is used with a framerate of 300-2000 2D frames pr second dependent on the technology and 2D image resolution. The high amount of image data needed to be handled by the scanning device to eighter directly forward the raw image data stream to an external processing device or performing some image processing before transmitting the data to an external device or display. This process requires that multiple electronic components inside the scanner is operating with a high workload thus requiring a high demand of current.
[0124] The scanning device comprises one or more light projectors configured to generate an illumination pattern to be projected on a three-dimensional dental arch during a scanning session. The light projector(s) preferably comprises a light source, a mask having a spatial pattern, and one or more lenses such as collimation lenses or projection lenses. The light source may be configured to generate light of a single wavelength or a combination of wavelengths (mono- or polychromatic). The combination of wavelengths may be produced by using a light source configured to produce light (such as white light) comprising different wavelengths. Alternatively, the light projector(s) may comprise multiple light sources such as LEDs individually producing light of different wavelengths (such as red, green, and blue) that may be combined to form light comprising the different wavelengths. Thus, the light produced by the light source may be defined by a wavelength defining a specific color, or a range of different wavelengths defining a combination of colors such as white light. In an embodiment, the scanning device comprises a light source configured for exciting fluorescent material of the teeth to obtain fluorescence data from the dental arch. Such a light source may be configured to produce a narrow range of wavelengths. In another embodiment, the light from the light source is infrared (IR) light, which is capable of penetrating dental tissue. The light projector(s) may be DLP projectors using a micro mirror array for generating a time varying pattern, or a diffractive optical element (DOF), or back-lit mask projectors, wherein the light source is placed behind a mask having a spatial pattern, whereby the light projected on the surface of the dental arch is patterned. The back-lit mask projector may comprise a collimation lens for collimating the light from the light source, said collimation lens being placed between the light source and the mask. The mask may have a checkerboard pattern, such that the generated illumination pattern is a checkerboard pattern. Alternatively, the mask may feature other patterns such as lines or dots, etc.
[0125] The scanning device preferably further comprises optical components for directing the light from the light source to the surface of the dental arch. The specific arrangement of the optical components depends on whether the scanning device is a focus scanning apparatus, a scanning device using triangulation, or any other type of scanning device. A focus scanning apparatus is further described in EP 2 442 720 Bl by the same applicant, which is incorporated herein in its entirety.
[0126] The light reflected from the dental arch in response to the illumination of the dental arch is directed, using optical components of the scanning device, towards the image sensor(s). The image sensor(s) are configured to generate a plurality of images based on the incoming light received from the illuminated dental arch. The image sensor unit may be a high-speed image sensor such as an image sensor configured for acquiring images with exposures of less than 1 / 1000 second or frame rates in excess of 250 frames pr. second (fps). As an example, the image sensor may be a rolling shutter (CCD) or global shutter sensor (CMOS). The image sensor(s) may be a monochrome sensor including a color filter array such as a Bayer filter and / or additional filters that may be configured to substantially remove one or more color components from the reflected light and retain only the other non-removed components prior to conversion of the reflected light into an electrical signal. For example, such additional filters may be used to remove a certain part of a white light spectrum, such as a blue component, and retain only red and green components from a signal generated in response to exciting fluorescent material of the teeth.
[0127] The network unit may be configured to connect the dental scanning system to a network comprising a plurality of network elements including at least one network element configured to receive the processed data. The network unit may include a wireless network unit or a wired network unit. The wireless network unit is configured to wirelessly connect the dental scanning system to the network comprising the plurality of network elements including the at least one network element configured to receive the processed data. The wired network unit is configured to establish a wired connection between the dental scanning system and the network comprising the plurality of network elements including the at least one network element configured to receive the processed data.
[0128] The dental scanning system preferably further comprises a processor configured to generate scan data (such as extra-oral scan data and / or intra-oral scan data) by processing the two-dimensional (2D) images acquired by the scanning device. The processor may be part of the scanning device. As an example, the processor may comprise a Field- programmable gate array (FPGA) and / or an Advanced RISC Machines (ARM) processor located on the scanning device. The scan data comprises information relating to the three- dimensional dental arch. The scan data may comprise any of: 2D images, 3D point clouds, depth data, texture data, intensity data, color data, and / or combinations thereof. As an example, the scan data may comprise one or more point clouds, wherein each point cloud comprises a set of 3D points describing the three-dimensional dental arch. As another example, the scan data may comprise images, each image comprising image data e.g. described by image coordinates and a timestamp (x, y, t), wherein depth information can be inferred from the timestamp. The image sensor(s) of the scanning device may acquire a plurality of raw 2D images of the dental arch in response to illuminating said object using the one or more light projectors. The plurality of raw 2D images may also be referred to herein as a stack of 2D images. The 2D images may subsequently be provided as input to the processor, which processes the 2D images to generate scan data. The processing of the 2D images may comprise the step of determining which part of each of the 2D images are in focus in order to deduce / generate depth information from the images. The internal depth information may be used to generate 3D point clouds comprising a set of 3D points in space, e.g., described by cartesian coordinates (x, y, z). The 3D point clouds may be generated by the processor or by another processing unit. Each 2D / 3D point may furthermore comprise a timestamp that indicates when the 2D / 3D point was recorded, i.e., from which image in the stack of 2D images the point originates. The timestamp is correlated with the z-coordinate of the 3D points, i.e., the z-coordinate may be inferred from the timestamp. Accordingly, the output of the processor is the scan data, and the scan data may comprise image data and / or depth data, e.g. described by image coordinates and a timestamp (x, y, t) or alternatively described as (x, y, z). The scanning device may be configured to transmit other types of data in addition to the scan data. Examples of data include 3D information, texture information such as infra-red (IR) images, fluorescence images, reflectance color images, x-ray images, and / or combinations thereof.
[0129] FIG. 1 illustrates an example of a graphical representation 50 of a dental object 2 that includes a 3D finite element mesh 4 defined in a three-dimensional coordinate system 6. The 3D finite element mesh 4 includes point clouds that are determined based on non- visible 2D images and visible sub-scans, and the 3D finite element mesh 4 includes a plurality of finite elements (8,9,10) which in this example includes combinations of triangles (8A,8B,8C,8D) and tetrahedrons (9A,9B,9C,9D) that forms a skewed voxel 10. A tetrahedral 9 corresponds to a volume of a part of the dental object 2, and a triangle 8 corresponds to a surface of the volume of the part of the dental object.
[0130] FIGs. 2A, 2B, 2C, 2D and 2E illustrate an intraoral scanning system 1 that is configured to determine a three-dimensional graphical representation 50 of a dental object 2. The system 1 includes a handheld intraoral scanner 20 that is configured to provide non-visible 2D images and visible sub-scans of the dental object 2. The non-visible 2D images may include infrared information or near-infrared information. The system 1 includes one or more processors 13 that is configured to determine 21 point clouds of the non-visible 2D images and the visible sub-scans and to determine 22 a 3D finite element mesh 4 and align the point clouds to the 3D finite element mesh 4, wherein the 3D finite element mesh 4 includes a plurality of finite elements (8, 9, 10). Each of the plurality of finite elements includes multiple vertices. The one or more processors 13 is further configured to determine 23 one or more optical coefficients for each of the plurality of vertices, and wherein the one or more optical coefficients correspond to a dental feature represented by the aligned point clouds. The one or more processors 13 is configured to determine 5 a triangle mesh representation of the aligned point clouds that correspond to the visible sub-scans, and wherein the triangle mesh representation includes triangle 8 arranged vertices of the plurality of vertices. The one or more processors 13 is further configured to determine 25 a tetrahedral mesh representation of the aligned point clouds that correspond to the non-visible 2D images, and wherein the tetrahedral mesh representation includes tetrahedral arranged vertices of the plurality of vertices, and wherein the 3D graphical representation 50 is determined 26 based on the triangle mesh representation and the tetrahedral mesh representation. FIG. 2B illustrates more specifically, the 3D finite element mesh 4 that includes a plurality of skewed voxels (10A,10B,10C,10D,10E) with aligned point clouds 27. FIG. 2C illustrates a part 28 of the 3D finite element mesh 4, and wherein the one or more processors 13 has determine one or more optical coefficients 32 for each of the multiple vertices 33. For example, the one or more optical coefficients 32 is determined by an inverse photon scattering algorithm that is configured to receive measured reflections, absorptions or refractions of the corresponding point clouds of the non-visible 2D images and the visible sub-scans. The value of the one or more optical coefficients 32 corresponds to a dental feature 40, such as an anatomy feature, a disease feature or a mechanical feature. In this example, the dental feature 40 is the enamel of the dental object 2. Furthermore, the multiple vertices 33 that includes optical coefficients that correspond to the dental feature are connected by a marching finite element algorithm or a spline function, and the output of the marching finite element algorithm or the spline function is the dental feature boundary 31 that includes the outer shape or contour of the corresponding dental feature 40. The point clouds that are connected are the once that are aligned closest to each of the plurality of vertices with an optical coefficient that corresponds the same dental feature 40. Additionally, the enamel is represented by the one or more optical coefficients that has a value of 5, and 0 corresponds to a none-dental object, such as air, water, gingival ect. In FIG. 2D the one or more processors 13 is configured to subdivide the plurality of finite elements into a plurality of tetrahedrons 9 and a plurality of triangles 8, and wherein the plurality of tetrahedrons 9 corresponds to the tetrahedral mesh representation 9, and the plurality of triangles 8 corresponds to the triangle mesh representation 8. The plurality of triangles 8 and tetrahedrons 9 forms multiple skewed voxels 10. The plurality of triangles 8 corresponds to the triangle mesh representation 8 and the plurality of tetrahedrons correspond to the tetrahedral mesh representation 9, wherein the triangle mesh representation 8 includes surface information of the volume that is covered by the tetrahedral mesh representation9. Thereby, the one or more processors 13 is configured to determine a graphical surface representation of the dental object 2 based on the triangle mesh representation 8 of the aligned point clouds 27. Furthermore, the one or more processors 13 is configured to determine a graphical volumetrically representation of the dental object 2, and the one or more processors 13 is configured to combine the graphical surface representation and the graphical volumetrically representation to determine a full 3D graphical representation 50 of the dental object 2.
[0131] FIG. 3 illustrates an example wherein the one or more processors 13 has identified an anatomy feature 40B, such as an enamel, and a disease feature 40A, such as a caries, and in this specific example, the one or more processors 13 is configured to increase the resolution of the disease feature 40A by performing subdivisions of the plurality of skewed voxels 10A that includes one or more optical coefficients 32 that correspond to the disease feature 40A. The remaining skewed voxels 10B include one or more optical coefficients 32 that correspond to the anatomy feature 40B. The subdivided skewed voxels 10 are grouped into a first group and the none- sub divided skewed voxels 10B are grouped into a second group. In this example, the first group of skewed voxels 10A includes a higher resolution of the plurality tetrahedrons and the plurality of triangles than in relation to the second group. In other words, the first group 10A includes smaller and more tetrahedrons 9 and smaller and more triangles 8 than the second group when the first group and the second group covers the same size of area.
[0132] Fig. 4 illustrates an example where the one or more processors 13 is configured to determine a volumetric representation 60 of the 3D graphical representation 50 by a graph neural network 61, and wherein the 3D graphical representation is received by the graph neural network, and wherein the graph neural network 61 is configured to determine the volumetric representation 60 such that it includes segmented teeth 62 and gingival 63 of the dental object. The segmented teeth 62 and gingival 63 results in a full 3D volumetric model of the dental object 2, in this example, of a lower and upper jaw in a bite configuration. The graphical neural network 61 may be trained by the one or more processors 13 based on CBCT scans and / or intraoral scans acquired by a handheld intraoral scanner 20.
[0133] FIG. 5 illustrates an example where the graph neural network 61 is based on a neural radiance field algorithm 62. The one or more processors 13 is configured to determine one or more volumetric optical coefficients 32’ for each of the plurality of vertices of the 3D graphical representation 50 based on the corresponding one or more optical coefficients 32 and a neural radiance field algorithm. The one or more optical coefficients 32’ are correlated to the one or more optical coefficients 32 via a casting object 62 of the algorithm 62. The position and the angle of the casting object 62 is determined by the one or more processors 13. The volumetric representation 65 of a surface and an inner region of the dental object are determined based on the one or more volumetric optical coefficients 32’.
[0134] To improve the speed of the neural radiance field algorithm then a lower resolution of the 3D finite element mesh 4 is fed into the neural radiance field algorithm, and which means that the algorithm must process less one or more optical coefficients 32 to generate a volumetric representation 65of the dental object 2. It is possible to reduce the resolution of the 3D finite element mesh as many of the neighbouring optical coefficients 32 may have the same or about the same value representing the same dental feature 40. A group of optical coefficients may be merged into one optical coefficient if they have the same value or about the same value. The one or more optical coefficients 32 includes multiple groups of optical coefficients which would then result in a reduced resolution of the 3D finite element mesh 4. In FIG. 6A the one or more processors 13 is configured to determine 66A a reduced- resolution 3D finite element mesh of the 3D finite element mesh 4, and wherein the reduced- resolution 3D finite element mesh includes a plurality of reduced-resolution vertices, and wherein the reduced-resolution 3D finite element mesh includes a plurality of reduced- resolution finite elements. The one or more processors 13 is configured to align 66B the point clouds to the reduced-resolution 3D finite element mesh, and wherein the alignment of the point clouds of the non-visible 2D images and the visible sub-scans is provided by positions and directions of the handheld intraoral scanner during the non-visible sub scans and the visible sub-scans. The one or more processors 13 is configured to determine 66C one or more reduced-resolution optical coefficients for each of the plurality of reduced- resolution vertices by an inverse photon scattering algorithm and to determine 66D intermediate one or more optical coefficients 32 for each of the plurality of vertices of the graphical representation 50 by interpolating between the one or more reduced-resolution optical coefficients of the plurality of reduced-resolution vertices of the reduced-resolution 3D finite element mesh. The one or more processors 13 may be configured to determine 66E one or more volumetric optical coefficients for each of the plurality of vertices of the graphical representation 50 based on the corresponding intermediate one or more optical coefficients 32 and a neural radiance field algorithm, and to determine a volumetric representation 65 of a surface and an inner region of the dental object 2 based on the one or more volumetric optical coefficients.
[0135] In FIG. 6B the neural radiance field algorithm may be configured to process 66E’ the estimated positions and directions of the handheld intraoral scanner 20, determine 66E” ray or cone casting objects 62 based on the estimated positions and directions for each pixels of an image sensor unit of the handheld intraoral scanner 20, and based on a loss function and the one or more optical coefficients 32, the one or more volumetric optical coefficients 32’ is determined 66E for each of the plurality of vertices which the ray or cone casting objects 62 intersects, and wherein the neural radiance field algorithm is configured to be trained using the non-visible 2D images.
[0136] FIG. 8 illustrates an example the one or more processors 13 that is configured to train the neural radiance field algorithm. The one or more processors 13 may be configured to train the neural radiance field algorithm by determining 80 2D non-visible images of the non- visible 2D images, receiving 81 the estimated positions and directions of the handheld intraoral scanner that corresponds to ray or cone casting objects for each pixel of the 2D non-visible images. The ray or cone casting objects reflects the viewing axis or viewing volume, respectively, of each pixel. Furthermore, the one or more processors may be configured to train the neural radiance field algorithm by determining 82, based on the neural radiance field algorithm using the estimated positions and directions of the handheld intraoral scanner and one or more optical coefficients 32, the one or more volumetric optical coefficients 32’ for each of the plurality of vertices which the ray or cone casting objects 62 intersects, and determining 83 a synthetic pixel value for each of the ray or cone casting objects 62 based on the corresponding determined one or more volumetric optical coefficients for each of the plurality of vertices which the ray or cone casting objects 62 intersects. The one or more processors 13 is further configured to minimize a loss function between the synthetic pixel value and a corresponding true pixel value for each pixel of the 2D non-visible images.
[0137] FIG. 9 illustrates an intraoral scanning system 1 that is configured to determine 8 an internal measurement of an inner region of a dental object 2. The system comprising a handheld intraoral scanner 20 that is configured to provide 12 non-visible 2D images and visible sub-scans of a dental object. Furthermore, the system 1 includes one or more processors 13 configured to determine 21 point clouds of the non-visible 2D images and the visible sub-scans, determine 22 a three-dimensional (3D) finite element mesh and align the point clouds to the 3D finite element mesh, and wherein the 3D finite element mesh includes a plurality of finite elements, and wherein each of the plurality of finite elements includes multiple vertices, determine 23 one or more optical coefficients for each of the multiple vertices, and wherein the one or more optical coefficients correspond to a dental feature represented by the aligned point clouds; determine 27 a first dental feature boundary of a first dental feature based on the determined one or more optical coefficients, and wherein the first dental feature boundary corresponds to the non-visible 2D images, and determine 28 an internal measurement of the first dental feature boundary.
[0138] FIG.10 illustrates the dental object 2 that includes information about the surface 5 and the inner region 3 of the dental object 2. In this example, the dental object 2 is defined as a 3D finite element mesh 4 wherein each of the multiple vertices has three-dimensional coordinates 6. In this example, the dental object 2 includes dental features, such as a pulp chamber 100, dentine 101 and enamel 102. The surfaces of the dental object, including the dental features, are represented by triangles and the volume 8 of the dental object, including the dental features, are represented by the plurality of finite elements which in this example, includes tetrahedral 9.
[0139] FIG. 11 A and 1 IB illustrate an example wherein the one or more processors 13 determines the one or more optical coefficients 32 for each of the multiple vertices 33 and determines dental feature boundaries (31 A, 3 IB) based on the one or more optical coefficients 32. In this example, the dental object 2 is yet again represented by the 3D finite element mesh 4, but the one or more optical coefficients 32 are illustrated for a part 4A of the 3D finite element mesh 4. In this example, the one or more processors 13 has determined a disease feature represented by the value of 5, and a first dental feature boundary 31 A is applied to the mesh 4A enclosing the disease feature, and a second dental feature boundary 3 IB is applied to the mesh 4A representing a boundary between the enamel, represented by the value of 3, and the dentine, represented by the value of 2.
[0140] In FIG. 1 IB, the one or more processors 13 is configured to determine the first dental feature that corresponds to the first dental feature boundary 31 A, and wherein the first dental feature is determined based on the one or more optical coefficients 32 that correspond to the first dental feature boundary 31 A and a dental feature algorithm 35. The dental feature algorithm 35 includes one or more of following an anatomy feature determiner 35 A that is configured to determine that the one or more optical coefficients 32 corresponds to an anatomy feature when the one or more optical coefficients is within an anatomy feature coefficient range; a disease feature determiner 35B that is configured to determine that the one or more optical coefficients 32 corresponds to a disease feature when the one or more optical coefficients 32 is within a disease feature coefficient range; and a mechanical feature determiner 35C that is configured to determine that the one or more optical coefficients 32 corresponds to a mechanical feature when the one or more optical coefficients 32 is within a mechanical feature coefficient range.
[0141] FIG. 12 illustrates the dental object 2 that includes a first dental feature boundary 31 A representing a caries 40A, a second dental feature boundary 3 IB representing the boundary between enamel and dentine, and a third dental feature boundary 31C representing the pulp chamber. In the direct measurement 41 example, the one or more processors 13 is configured to determine the internal measurement 44 between a first part 41 A of the first dental feature boundary 31 A and a second part 41B of the first dental feature boundary 31 A. In the fitting example 42, the one or more processors 13 is configured to perform a fitting of a shape object 43 to a geometry of the first dental feature boundary 31 A, and wherein the internal measurement 44 is determined based on the fitted shape object 43.
[0142] FIGs. 13A, 13B, and 13C illustrate different examples of the internal measurement. FIGs. 13 A and 13B illustrate the one or more processors 13 that is configured to determine multiple internal measurements 43 of the first dental feature boundary (31 A,41) or of a shape object (43,42) fitted to the geometry of the first dental feature boundary 31 A, and wherein the one or more processors 13 is configured to determine a minimum and / or a maximum internal measurement 44 of the multiple internal measurements 44A. The one or more processors 13 is configured to determine a volumetric measurement of the dental feature that corresponds to the dental feature boundary (31 A), and the determined volumetric measurement is based on the multiple internal measurements 44 A. FIG. 13C illustrates an example wherein the one or more processors 13 is configured to determine a plurality of normal lengths 50 A from a second dental feature 3 IB and to the first dental feature boundary 31 A and along a longitudinal axis 51 of the dental object 2 in the 3D finite element mesh. The one or more processor 13 is then configured to determine a maximum or a minimum normal length of the plurality of normal lengths 50A, and which in this example, the maximum and minimum normal length indicates, respectively, the maximum and minimum thickness of enamel of the dental object 2.
[0143] FIGs. 14 A, 14B and 14C illustrate the one or more processors 13 configured to determine a plurality of dental feature boundaries (31 A, 3 IB, 31C, 3 ID), wherein the plurality of dental feature boundaries (31 A, 3 IB, 31C, 3 ID) includes the first dental feature boundary 31 A, a second dental feature boundary 3 IB, a third dental feature boundary 31C and a fourth dental feature boundary 3 ID. In this example, the first dental feature boundary corresponds to a disease feature 40A, the second dental feature boundary 3 IB corresponds to enamel 40B, the third dental feature boundary corresponds to dentine 40C and the fourth dental feature boundary corresponds to the pulp chamber 40D. In the example illustrated in FIG. 14A, the one or more processors 13 is configured to determine an internal measurement 44A between the second dental feature boundary 3 IB and the third dental feature boundary 31C, and wherein the internal measurement 44 A corresponds to the thickness of the enamel. The one or more processors 13 is further configured to determine an internal measurement 44B between the second dental boundary 3 IB and the first dental feature boundary 31 A. The one or more processor 13 is configured to determine the thickness of the enamel 40B at the maximum internal measurement 44B, i.e. distance, between the second dental feature boundary 3 IB and the first dental feature boundary 31 A. If the maximum internal measurement is larger than the thickness of the enamel 40B at the same area of the dental object 2, then the disease feature 40A is penetrating dentine 40C. In this situation, the one or more processors 13 is configured to generate a notification signal to a user interface for notifying the user of the system 1 about the disease feature 44B penetrating into the dentine. In the example illustrated in FIG. 14B, the one or more processors 13 is configured to determine a first internal measurement 44A between the third dental feature boundary 3 IB and the first dental feature boundary 31C in a first direction along the longitudinal axis 51 of the dental object, and a second internal measurement 44B between the third dental feature boundary 31C and the first dental feature boundary 31 A in a second direction along the longitudinal axis 51, and wherein the second direction is opposite to the first direction. The sign of the internal measurement (44A,44B) determines whether the internal measurement (44A,44B) includes a measure of a penetration depth of the first dental feature 40A into the third dental feature 40C. In this example, the second internal measurement 44B corresponds to the penetration depth of the first dental feature 40 A into the third dental feature 40C. In this example, the notification signal includes an information about the penetration depth, and that the penetration depth is into the dentine 40C. In the example illustrated in FIG 14C, the one or more processors 13 is configured to determine a plurality of normal lengths (44A,44B) from a second dental feature boundary 3 IB in a direction towards the first dental feature boundary 31 A and along a longitudinal axis 51 of the dental object 2 in the 3D finite element mesh 2. In this example, the sign of the plurality of normal lengths (44A,44B) are the same. The sign of the plurality of normal lengths (44A,44B) indicates that the first dental feature 40 A is penetrating the second dental feature 40B. The one or more processors 13 is configured to determine a maximum normal length 44B of the plurality of normal lengths (44A,44B), and determine a penetration depth 44B that is the maximum normal length 44B. The one or more processors is configured to determine a notification signal when the internal measurement 44B, i.e. the penetration depth, is above a measurement threshold, and wherein the notification signal is displayed on a user interface of the system 1.
[0144] Each of the multiple vertices may correspond to a pixel of an image sensor of the handheld intraoral scanner 10, or, a group of the multiple vertices may correspond to a pixel of an image sensor of the handheld intraoral scanner 10. FIGs. 16A and 16B illustrate an example of the one or more processors that is configured to determine the one or more optical coefficients 32. In FIG. 16A, the image sensor includes at least five pixels (87A,87B,87C,87D,87E), wherein each of the pixels includes a casting object (86A,86B,86C,86D, 86E) that intersects a primary 3D finite element mesh 4A and multiple sub 3D finite element meshes (4B,4C,4D,4E). The primary 3D finite element mesh 4A includes the one or more optical coefficients 32 for each of the multiple vertices, and the one or more optical coefficients 32 corresponds to what a pixel (87A,87B,87C,87D,87E) is viewing. The sub 3D finite element meshes (4B,4C,4D,4E) corresponds to the internal reflections that in this specific example, the casting objects (86A,86B,86C,86D,86E) is a ray that corresponds to the viewing field of the pixel (87A,87B,87C,87D,87E). Each pixels (87A,87B,87C,87D,87E) is configured to receive non-visible 2D images which are then turned into synthetic pixel values. The received non-visible 2D images includes internal reflections from within an inner region of the dental object, and which means that each of the synthetic pixel values includes an average of internal reflections from different planes that are distributed along a casting object (86A,86B,86C,86D,86E) of the corresponding pixel (87A,87B,87C,87D,87E). Each of the synthetic pixel values may correspond to each of the multiple vertices 33 of a primary 3D finite element mesh 4 A, and wherein the one or more processors is configured to determine the one or more optical coefficients 32 for each of the multiple vertices of the primary 3D finite element mesh based on the one or more optical coefficients 32A of the sub 3D finite element meshes (4B,4C,4D,4E). The internal reflections of the received non- visible 2D images may be represented by one or more optical coefficients 32A of multiple vertices of a sub 3D finite element mesh (4B,4C,4D,4E), and since the received non- visible 2D images include internal reflections captured from different planes, the neural radiance field model is configured to determine a sub 3D finite element mesh (4B,4C,4D,4E) for each of the different planes. The sub 3D finite element meshes (4B,4C,4D,4E) are arranged along the casting object (86A,86B,86C,86D,86E), such that a casting object (86A,86B,86C,86D,86E) that starts at a vertex of the primary 3D finite element mesh 4A may intersects the corresponding vertex of each of the sub 3D finite (4B,4C,4D,4E). Thereby, the one or more optical coefficients 32 of the vertex of the primary 3D finite element mesh 4A is an average of the one or more optical coefficients 32A of the corresponding vertex of the sub 3D finite element meshes (4B,4C,4D,4E) which the casting object (86A,86B,86C,86D,86E) intersects.
[0145] Since most of the teeth, i.e. dental objects, have similar interior structure, the variation in the one or more optical coefficients 32 for different tooth samples will be limited and there is a lot of covariance between the values of the one or more coefficients 32 of the multiple vertices 33. FIG. 17 illustrates an example wherein the one or more processors 13 is configured to perform a covariance analysis of the one or more optical coefficients 32 for the purpose of reducing the number of one or more optical coefficients 32 to be processed in order to determine the dental feature boundaries (31 A, 3 IB). The one or more processors 13 is configured to perform a covariance analysis 90 of the one or more optical coefficients 32 for the multiple vertices and to determine a plurality of descriptive parameters 32B based on the covariance analysis 90, and wherein a number of the plurality of descriptive parameters 32B is smaller than a number of the one or more optical coefficients 32. The first and / or the second dental feature boundary (31 A, 3 IB) is determined by providing the plurality of descriptive parameters 32B into a marching finite element algorithm 91 of the intraoral scanning system 1.
[0146] The covariance analysis 90 is a principal component analysis or an autoencoder deep learning algorithm, wherein the autoencoder deep learning algorithm includes a neural network. The neural network is configured to receive the one or more optical coefficients 32 of the multiple vertices that corresponds to the non-visible 2D images and determine the plurality of descriptive parameters 32B.
[0147] FIGs. 18A and 18B illustrate examples on how to train the neural network for determining the plurality of descriptive parameters 32B. In FIG. 18A the one or more processors 13 is configured to train the neural network by receiving one or more CBCT scans 200 A of one or more dental objects, determining 200C one or more training optical coefficients for each of the one or more CBCT scans, receiving 200B one or more non-visible sub scans from a handheld intraoral scanner of the one or more dental objects, and training the neural network by mapping 200D the one or more training optical coefficients onto the one or more non-visible sub scans. In FIG. 18B the neural network is trained to classify 200E specific dental features 40 based on the one or more training optical coefficients, and wherein the neural network is configured to receive the one or more optical coefficients 32 of the multiple vertices that corresponds to the non-visible 2D images, and determine a plurality of descriptive parameters 32B for a specific dental feature 40 based on the trained optical coefficients and the received one or more optical coefficients.
[0148] FIG. 19 illustrates an example of a user interface 210 of the system 1 that displays display a three-dimensional (3D) model 211 of the dental object 2 and a two-dimensional (2D) cross-section 212 of the dental object 2, and wherein the internal measurement 44 is displayed on the 2D cross-section 212 of the dental object 2. In this example, the dental object 2 to be displayed on the 2D cross-section 212 is selected by a marker 213, and the dental feature boundaries (31A,31B,31C) are displayed on the 2D cross-section 212. FIGs. 20 A and 20B illustrate an example where the one or more processors 13 is configured to perform a subdivision of a finite element 10. To simplify the explanation of the subdivision, the finite element is a tetrahedral 9. In FIG. 20A, the finite element 9 is subdivided into multiple finite elements (9A - 9F) and then one more time (III). The subdivision is represented by an octree data structure 200 that in this example includes four levels of subdivisions (I, II, III, IV). The octree data structure 200 includes a plurality of nodes that includes groups of nodes on different levels of the data structure 200. Each node corresponds to a finite element, such as a skewed voxel. For example, a first level
[0149] (I) of the data structure 200 includes a none subdivided finite element and a second level
[0150] (II) includes a subdivision of the finite element of the first level and soon for the next levels (III, IV) in the data structure 200. The subdivision may include halving the nodes from the previous level. The one or more processors 13 may be configured to perform the subdivision of a group of the plurality of finite elements (8,9, 10) to a final level (IV) where the one or more optical coefficients 32 of the corresponding nodes to the final level (IV) that are about the same or the same will be removed, and the previous level, which in this example is the third level (III) relative to the final level (IV) will be kept by the one or more processors 13. The previous level (III) is considered to be the optimal level of subdivision of a finite element (8,9,10).
[0151] Although some embodiments have been described and shown in detail, the disclosure is not restricted to such details, but may also be embodied in other ways within the scope of the subject matter defined in the following claims. In particular, it is to be understood that other embodiments may be utilized, and structural and functional modifications may be made without departing from the scope of the present invention.
[0152] Benefits, other advantages, and solutions to problems have been described herein with regard to specific embodiments. However, the benefits, advantages, solutions to problems, and any component(s) / unit(s) that may cause any benefit, advantage, or solution to occur or become more pronounced are not to be construed as critical, required, or essential features or components / elements of any or all the claims or the invention. The scope of the invention is accordingly to be limited by nothing other than the appended claims, in which reference to an component / unit / element in the singular is not intended to mean “one and only one” unless explicitly so stated, but rather “one or more.” A claim may refer to any of the preceding claims, and “any” is understood to mean “any one or more” of the preceding claims.
[0153] It is intended that the structural features of the devices described above, either in the detailed description and / or in the claims, may be combined with steps of the method, when appropriately substituted by a corresponding process.
[0154] As used, the singular forms “a,” “an,” and “the” are intended to include the plural forms as well (i.e. to have the meaning “at least one”), unless expressly stated otherwise. It will be further understood that the terms “includes,” “comprises,” “including,” and / or “comprising,” when used in this specification, specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It will also be understood that when an element is referred to as being “connected” or “coupled” to another element, it can be directly connected or coupled to the other element but an intervening elements may also be present, unless expressly stated otherwise. Furthermore, “connected” or “coupled” as used herein may include wirelessly connected or coupled. As used herein, the term “and / or" includes any and all combinations of one or more of the associated listed items. The steps of any disclosed method is not limited to the exact order stated herein, unless expressly stated otherwise.
[0155] It should be appreciated that reference throughout this specification to "one embodiment" or "an embodiment" or “an aspect” or features included as “may” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the disclosure. Furthermore, the particular features, structures or characteristics may be combined as suitable in one or more embodiments of the disclosure. The previous description is provided to enable any person skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the generic principles defined herein may be applied to other aspects.
[0156] The claims are not intended to be limited to the aspects shown herein, but is to be accorded the full scope consistent with the language of the claims, wherein reference to an element in the singular is not intended to mean “one and only one” unless specifically so stated, but rather “one or more.” Unless specifically stated otherwise, the term “some” refers to one or more.
[0157] ITEMS
[0158] 1. An intraoral scanning system configured to determine a three-dimensional (3D) graphical representation of a dental object, the system comprising:
[0159] • a handheld intraoral scanner configured to provide non-visible 2D images and visible sub-scans of a dental object; and
[0160] • one or more processors configured to; o determine point clouds of the visible sub-scans, o determine a 3D finite element mesh and align the point clouds to the 3D finite element mesh, wherein the 3D finite element mesh includes a plurality of finite elements, and wherein each of the plurality of finite elements includes multiple vertices, o determine one or more optical coefficients for each of the plurality of vertices, and wherein the one or more optical coefficients correspond to a dental feature represented by the aligned point clouds; o determine a triangle mesh representation of the aligned point clouds that correspond to the visible sub-scans, and wherein the triangle mesh representation includes triangle arranged vertices of the plurality of vertices, o determine a tetrahedral mesh representation of the aligned point clouds that correspond to the non-visible 2D images, and wherein the tetrahedral mesh representation includes tetrahedral arranged vertices of the plurality of vertices, and wherein the 3D graphical representation is determined based on the triangle mesh representation and the tetrahedral mesh representation.
[0161] 2. The intraoral scanning system according to item 1, wherein the plurality of finite elements is skewed voxels.
[0162] 3. The intraoral scanning system according to any of the previous items, wherein the one or more optical coefficients is determined by an inverse photon scattering algorithm that is configured to receive 2D non-visible image and 3d visible sub-scan and estimate reflections, absorptions or refractions of the corresponding vertices of the finite element mesh.
[0163] 4. The intraoral scanning system according to any of the previous items, wherein the one or more processors is configured to subdivide the plurality of finite elements into a plurality of tetrahedrons and a plurality of triangles, and wherein the plurality of tetrahedrons corresponds to the tetrahedral mesh representation, and the plurality of triangles corresponds to the triangle mesh representation.
[0164] 5. The intraoral scanning system according to any of the previous items, wherein the one or more processors is configured to perform one or more subdivisions of the plurality of finite elements into a first group and a second group of a plurality of tetrahedrons and a plurality of triangles, and wherein the first group and the second group have different resolutions of the plurality of finite elements.
[0165] 6. The intraoral scanning system according to item 5, wherein the first group includes a higher resolution of the plurality of tetrahedrons and the plurality of triangles than in relation to the second group, and wherein the first group of the plurality of finite elements correspond to a first part of the dental object, and wherein the second group of the plurality of finite elements correspond to a second part of the dental object.
[0166] 7. The intraoral scanning system according to any of the previous items, wherein the one or more processors is configured to determine the 3D graphical representation by connecting the point clouds that are aligned closest to each of the plurality of vertices via a marching tetrahedral algorithm.
[0167] 8. The intraoral scanning system according to any of the previous items, wherein if the first group and the second group covers a same size of area of the dental object, the first group includes smaller and more tetrahedrons of the plurality of tetrahedrons and smaller and more triangles of the plurality of triangles than the second group.
[0168] 9. The intraoral scanning system according to any of items 5 to 8, wherein the first group of the plurality of finite elements corresponds to a disease feature and the second group of the plurality of finite elements corresponds to a dental feature other than the disease feature.
[0169] 10. The intraoral scanning system according to any of the previous items, wherein the one or more processors is configured to determine a surface of the dental object in the graphical representation based on the triangle mesh representation of the aligned point clouds.
[0170] 11. The intraoral scanning according to any of the previous items, wherein the one or more processors is configured to determine an inner region of the dental object in the graphical representation based on the tetrahedral mesh representation of the aligned point clouds.
[0171] 12. The intraoral scanning system according to any of the previous items, wherein the one or more processors is configured to determine a volumetric representation of the 3D graphical representation by a graph neural network, and wherein the 3D graphical representation is received by the graph neural network, and wherein the graph neural network is configured to determine the volumetric representation that includes segmented teeth and gingival of the dental object.
[0172] 13 The intraoral scanning system according to item 12, wherein the one or more processors is configured to train the graph neural network by volumetric representations determined based on CBCT scans and / or intraoral scans acquired by a handheld intraoral scanner.
[0173] 14. The intraoral scanning system according to any of the previous items, wherein the alignment of the point clouds of the non-visible 2D images and the visible sub-scans is provided by positions and directions of the handheld intraoral scanner during the non- visible sub scans and the visible sub-scans, and wherein the one or more processors is configured to:
[0174] • determine one or more volumetric optical coefficients for each of the plurality of vertices of the graphical representation based on the corresponding one or more optical coefficients and a neural radiance field algorithm, and
[0175] • determine a volumetric representation of a surface and an inner region of the dental object based on the one or more volumetric optical coefficients.
[0176] 15. The intraoral scanning system according to any of items 1 to 13, wherein the one or more processors is configured to:
[0177] • determine a reduced-resolution 3D finite element mesh of the 3D finite element mesh, and wherein the reduced-resolution 3D finite element mesh includes a plurality of reduced-resolution vertices, and wherein the reduced-resolution 3D finite element mesh includes a plurality of reduced-resolution finite elements,
[0178] • align the point clouds to the reduced-resolution 3D finite element mesh, wherein the alignment of the point clouds of the non-visible 2D images and the visible sub-scans is provided by positions and directions of the handheld intraoral scanner during the non-visible sub scans and the visible sub-scans,
[0179] • determine one or more reduced-resolution optical coefficients for each of the plurality of reduced-resolution vertices by an inverse photon scattering algorithm,
[0180] • determine intermediate one or more optical coefficients for each of the plurality of vertices of the graphical representation by interpolating between the one or more reduced-resolution optical coefficients of the plurality of reduced-resolution vertices of the reduced-resolution 3D finite element mesh, • determine one or more volumetric optical coefficients for each of the plurality of vertices of the graphical representation based on the corresponding intermediate one or more optical coefficients and a neural radiance field algorithm, and
[0181] • determine a volumetric representation of a surface and an inner region of the dental object based on the one or more volumetric optical coefficients.
[0182] 16. The intraoral scanning system according to item 14 or 15, wherein the neural radiance field algorithm is configured to:
[0183] • process the estimated positions and directions of the handheld intraoral scanner,
[0184] • determine ray or cone casting objects based on the estimated positions and directions for each pixels of an image sensor unit of the handheld intraoral scanner ,
[0185] • determine, based on a loss function and one or more optical coefficients, the one or more volumetric optical coefficients for each of the plurality of vertices which the ray or cone casting objects intersects, and wherein the neural radiance field algorithm is configured to be trained using the non- visible 2D images.
[0186] 17. The intraoral scanning system according to 16, wherein the one or more processors is configured to train the neural radiance field algorithm by:
[0187] • determining 2D non-visible images of the non-visible 2D images,
[0188] • receiving the estimated positions and directions of the handheld intraoral scanner that corresponds to ray or cone casting objects for each pixel of the 2D non- visible images,
[0189] • determining, based on the neural radiance field algorithm using the estimated positions and directions of the handheld intraoral scanner and one or more optical coefficients, the one or more volumetric optical coefficients for each of the plurality of vertices which the ray or cone casting objects intersects, and
[0190] • determining a synthetic pixel value for each of the ray or cone casting objects based on the corresponding determined one or more volumetric optical coefficients for each of the plurality of vertices which the ray or cone casting objects intersects; and minimizing a loss function between the synthetic pixel value and a corresponding true pixel value for each pixel of the 2D non-visible images.
[0191] 18. The intraoral scanning system according to any of the previous items, wherein the non-visible 2D images include infrared information or near-infrared information.
[0192] 19. The intraoral scanning system according to any of the previous items, comprising a displaying unit configured to display a 3D model of the dental object that includes the 3D graphical representation.
[0193] 20. The intraoral scanning system according to any of the previous items, comprising a displaying unit configured to display a 3D model of the dental object that includes a volumetric representation of a surface and an inner region of the dental object.
[0194] 21. The intraoral scanning system according to any of the previous items, wherein the one or more optical coefficients correspond to a dental feature, and wherein the one or more processors is configured to:
[0195] • determine a first dental feature boundary of a first dental feature based on the determined one or more optical coefficients or one or more volumertic optical coefficients, and wherein the first dental feature boundary corresponds to the non-visible 2D images, and
[0196] • determine an internal measurement of the first dental feature boundary.
[0197] 22. The intraoral scanning system according to item 21, wherein the internal measurement is determined between a first part of the first dental feature boundary and a second part of the first dental feature boundary.
[0198] 23. The intraoral scanning system according to any of items 21 and 22, wherein the one or more processors is configured to perform a fitting of a shape object to a geometry of the first dental feature boundary, and wherein the internal measurement is determined based on the fitted shape object. 24. The intraoral scanning system according to any of items 21 to 23, wherein the one or more processor is configured to determine multiple internal measurements of the first dental feature boundary or to a shape object fitted to the geometry of the first dental feature boundary, and wherein the one or more processor is configured to determine a minimum internal measurements of the multiple internal measurements.
[0199] 25. The intraoral scanning system according to any of items 21 to 24, wherein the one or more processor is configured to determine multiple internal measurements of the first dental feature boundary or to a shape object fitted to the geometry of the first dental feature boundary, and wherein the one or more processor is configured to determine a volumetric measurement based on the multiple internal measurements.
[0200] 26. The intraoral scanning system according to any of items 21 to 25, wherein the internal measurement is a sectional curvature measure of a part of the first dental feature boundary or to a shape object fitted to the geometry of the first dental feature boundary.
[0201] 27. The intraoral scanning system according to any of items 21 to 26, wherein the one or more processors is configured to determine a plurality of dental feature boundaries, and wherein the plurality of dental feature boundaries includes the first dental feature boundary and at least a second dental feature boundary, and wherein the at least second dental feature boundary corresponds to a second dental feature; and wherein the internal measurement is determined between the first dental feature boundary and the at least second dental feature boundary.
[0202] 28. The intraoral scanning system according to item 28, wherein the internal measurement is a distance between the first dental feature boundary and the at least second dental feature boundary or an overlap between the first dental feature boundary and the second dental feature boundary.
[0203] 29. The intraoral scanning system according to item 27 or 28, wherein the one or more processors is configured to determine multiple sub-internal measurements that include a first sub-internal measurement, a second sub-internal measurement, and a third subinternal measurement, and wherein:
[0204] • the first sub-internal measurement corresponds to a size measurement of the first dental feature that corresponds to the first dental feature boundary,
[0205] • the second sub-internal measurement corresponds to a size measurement of the second dental feature that corresponds to the second dental feature boundary,
[0206] • the third sub-internal measurement is a size measure of an overlap between the first dental feature boundary and the at least second dental feature boundary, and
[0207] • the internal measurement is a ratio between the first sub-internal measurement or the second sub-internal measurement and the third sub-internal measurement. 30. The intraoral scanning system according to any of items 27 to 29, wherein the plurality of dental feature boundaries is determined by a marching finite element algorithm.
[0208] 31. The intraoral scanning system according to any items 21 to 30, wherein the one or more processors is configured to:
[0209] • perform a covariance analysis of the one or more optical coefficients or the one or more volumetric optical coefficients for the multiple vertices,
[0210] • determine a plurality of descriptive parameters based on the covariance analysis, and wherein a number of the plurality of descriptive parameters is smaller than a number of the one or more optical coefficients or the one or more volumetric optical coefficients, and wherein the plurality of dental feature boundaries is determined by providing the plurality of descriptive parameters into a marching finite element method of the intraoral scanning system.
[0211] 32. The intraoral scanning system according to item 31, wherein the covariance analysis is a principal component analysis or an autoencoder deep learning algorithm, wherein the autoencoder deep learning algorithm includes a neural network, and wherein the neural network is configured to receive the one or more optical coefficients or the one or more volumetric optical coefficients of the multiple vertices that corresponds to the non- visible 2D images and determine the plurality of descriptive parameters.
[0212] 33. The intraoral scanning system according to item 32, wherein the one or more processors is configured to train the neural network by:
[0213] • receiving one or more CBCT scans of one or more dental objects,
[0214] • determining one or more training optical coefficients for each of the one or more CBCT scans,
[0215] • receiving one or more non-visible sub scans from a handheld intraoral scanner of the one or more dental objects, and training the neural network by mapping the one or more training optical coefficients onto the one or more non-visible sub scans.
[0216] 34. The intraoral scanning system according to any of items 32 and 33, wherein the neural network is trained to classify specific dental features based on the one or more training optical coefficients, and wherein the neural network is configured to receive the one or more optical coefficients or the one or more volumetric optical coefficients of the multiple vertices that corresponds to the non-visible 2D images, and determine a plurality of descriptive parameters for a specific dental feature.
[0217] 35. The intraoral scanning system according to any of items 21 to 34, wherein the first dental feature or a second dental feature corresponds to one of following:
[0218] • an anatomy feature being an enamel, a dentine, or a pulp,
[0219] • a disease feature being a crack or a caries, and
[0220] • a mechanical feature being a filling and / or a composite restoration.
[0221] 36. The intraoral scanning system according to any of items 21 to 34, wherein the one or more processors is configured to:
[0222] • determine a plurality of normal length from a second dental feature in a direction towards the first dental feature boundary and along a longitudinal axis of the point clouds,
[0223] • determine a maximum normal length of the plurality of normal lengths,
[0224] • determine a depth extension of the first dental feature boundary within the second dental feature boundary by the maximum normal length along, and wherein the internal measurement is the depth extension.
[0225] 37. The intraoral scanning system according to item 36, wherein the one or more processors is configured to a volumetric measurement by performing a volume measurement based on the plurality of normal lengths, and wherein the volume measurement includes a volume of the first dental feature boundary that is arranged within the second dental feature boundary, and wherein the internal measurement is the volume measurement. 38. The intraoral scanning system according to any of items 21 to 37, and wherein the one or more processors is configured to determine a notification signal when the internal measurement is above a measurement threshold, and wherein the notification signal is displayed on a user interface of the system.
[0226] 39. The intraoral scanning system according to any of the previous items, wherein the one or more processors is configured to align the point clouds to the 3D finite element mesh by determining shortest distance between a 3D point of the 3D point clouds and a vertex of the plurality of vertices,
Claims
CLAIMS1. An intraoral scanning system configured to determine a three-dimensional (3D) graphical representation of a dental object, the system comprising:• a handheld intraoral scanner configured to provide non-visible 2D images and visible sub-scans of a dental object; and• one or more processors configured to; o determine point clouds of the non-visible 2D images and the visible subscans, o determine a 3D finite element mesh and align the point clouds to the 3D finite element mesh, wherein the 3D finite element mesh includes a plurality of finite elements, and wherein each of the plurality of finite elements includes multiple vertices, o determine one or more optical coefficients for each of the plurality of vertices, and wherein the one or more optical coefficients correspond to a dental feature represented by the aligned point clouds; o determine a triangle mesh representation of the aligned point clouds that correspond to the visible sub-scans, and wherein the triangle mesh representation includes triangle arranged vertices of the plurality of vertices, o determine a tetrahedral mesh representation of the aligned point clouds that correspond to the non-visible 2D images, and wherein the tetrahedral mesh representation includes tetrahedral arranged vertices of the plurality of vertices, and wherein the 3D graphical representation is determined based on the triangle mesh representation and the tetrahedral mesh representation.
2. The intraoral scanning system according to claim 1, wherein the plurality of finite elements is skewed voxels.
3. The intraoral scanning system according to any of the previous claims, wherein the one or more optical coefficients is determined by an inverse photon scattering algorithmthat is configured to receive measured reflections, absorptions or refractions of the corresponding point clouds of the non-visible 2D images and the visible sub-scans.
4. The intraoral scanning system according to any of the previous claims, wherein the one or more processors is configured to subdivide the plurality of finite elements into a plurality of tetrahedrons and a plurality of triangles, and wherein the plurality of tetrahedrons corresponds to the tetrahedral mesh representation, and the plurality of triangles corresponds to the triangle mesh representation.
5. The intraoral scanning system according to any of the previous claims, wherein the one or more processors is configured to perform one or more subdivisions of the plurality of finite elements into a first group and a second group of a plurality of tetrahedrons and a plurality of triangles, and wherein the first group and the second group have different resolutions of the plurality of finite elements.
6. The intraoral scanning system according to claim 5, wherein the first group includes a higher resolution of the plurality of tetrahedrons and the plurality of triangles than in relation to the second group, and wherein the first group of the plurality of finite elements correspond to a first part of the dental object, and wherein the second group of the plurality of finite elements correspond to a second part of the dental object.
7. The intraoral scanning system according to any of the previous claims, wherein the one or more processors is configured to determine the 3D graphical representation by connecting the point clouds that are aligned closest to each of the plurality of vertices via a marching tetrahedral algorithm.
8. The intraoral scanning system according to any of the previous claims, wherein if the first group and the second group covers a same size of area of the dental object, the first group includes smaller and more tetrahedrons of the plurality of tetrahedrons and smaller and more triangles of the plurality of triangles than the second group.
9. The intraoral scanning system according to any of claims 5 to 8, wherein the first group of the plurality of finite elements corresponds to a disease feature and the second group of the plurality of finite elements corresponds to a dental feature other than the disease feature.
10. The intraoral scanning system according to any of the previous claims, wherein the one or more processors is configured to determine a surface of the dental object in the graphical representation based on the triangle mesh representation of the aligned point clouds.
11. The intraoral scanning according to any of the previous claims, wherein the one or more processors is configured to determine an inner region of the dental object in the graphical representation based on the tetrahedral mesh representation of the aligned point clouds.
12. The intraoral scanning system according to any of the previous claims, wherein the one or more processors is configured to determine a volumetric representation of the 3D graphical representation by a graph neural network, and wherein the 3D graphical representation is received by the graph neural network, and wherein the graph neural network is configured to determine the volumetric representation that includes segmented teeth and gingival of the dental object.13 The intraoral scanning system according to claim 12, wherein the one or more processors is configured to train the graph neural network by volumetric representations determined based on CBCT scans and / or intraoral scans acquired by a handheld intraoral scanner.
14. The intraoral scanning system according to any of the previous claims, wherein the alignment of the point clouds of the non-visible 2D images and the visible sub-scans is provided by positions and directions of the handheld intraoral scanner during the non- visible sub scans and the visible sub-scans, and wherein the one or more processors is configured to:• determine one or more volumetric optical coefficients for each of the plurality of vertices of the graphical representation based on the corresponding one or more optical coefficients and a neural radiance field algorithm, and• determine a volumetric representation of a surface and an inner region of the dental object based on the one or more volumetric optical coefficients.
15. The intraoral scanning system according to any of claims 1 to 13, wherein the one or more processors is configured to:• determine a reduced-resolution 3D finite element mesh of the 3D finite element mesh, and wherein the reduced-resolution 3D finite element mesh includes a plurality of reduced-resolution vertices, and wherein the reduced-resolution 3D finite element mesh includes a plurality of reduced-resolution finite elements,• align the point clouds to the reduced-resolution 3D finite element mesh, wherein the alignment of the point clouds of the non-visible 2D images and the visible sub-scans is provided by positions and directions of the handheld intraoral scanner during the non-visible sub scans and the visible sub-scans,• determine one or more reduced-resolution optical coefficients for each of the plurality of reduced-resolution vertices by an inverse photon scattering algorithm,• determine intermediate one or more optical coefficients for each of the plurality of vertices of the graphical representation by interpolating between the one or more reduced-resolution optical coefficients of the plurality of reduced-resolution vertices of the reduced-resolution 3D finite element mesh,• determine one or more volumetric optical coefficients for each of the plurality of vertices of the graphical representation based on the corresponding intermediate one or more optical coefficients and a neural radiance field algorithm, and• determine a volumetric representation of a surface and an inner region of the dental object based on the one or more volumetric optical coefficients.
16. The intraoral scanning system according to claim 14 or 15, wherein the neural radiance field algorithm is configured to:• process the estimated positions and directions of the handheld intraoral scanner,• determine ray or cone casting objects based on the estimated positions and directions for each pixels of an image sensor unit of the handheld intraoral scanner ,• determine, based on a loss function and one or more optical coefficients, the one or more volumetric optical coefficients for each of the plurality of vertices which the ray or cone casting objects intersects, and wherein the neural radiance field algorithm is configured to be trained using the non- visible 2D images.