System and method for intraoral scanning

Abstract

An intraoral scanning system (102) includes a handheld intraoral scanner (104), a processor (106), and a continuous volumetric ML model (108) for generating a 3D internal geometry (120) of a tooth (306). The intraoral scanner includes a sensor (214D) for detecting NIR and visible light. The processor receives the visible light and NIR information from the sensor, generates a 3D surface model (118) of the tooth using the visible light information, and determines input parameters of a continuous volume ML model by capturing a 2D NIR image (114) of an interior region of the tooth from the NIR information. Each NIR image contains pixels (722) having a projection object. The input parameters include spatial position and viewing angle, and the projected object contains point coordinates (726, 732). The continuous volume ML model processes the input parameters to determine intensity and density values for pixels in the 3D internal geometry of the 3D surface model.

Description

TECHNICAL FIELD

[0001] Exemplary embodiments of the present invention relate generally to intraoral scan registration, and more particularly to intraoral scanning systems and methods for intraoral scanning to generate three-dimensional internal geometry of a subject's teeth. BACKGROUND

[0002] Intraoral scanners are electronic devices that can be used, for example, to capture three-dimensional (3D) digital images. In one example, an intraoral scanner can include a light source that can project light rays onto an object to be scanned, such as teeth, gums, and other intraoral structures within a patient's mouth. In one example, images captured by the intraoral scanner can be processed to generate a digital impression, such as a 3D surface model of the patient's mouth. The 3D surface model can be displayed on a screen for examination of the mouth.

[0003] Generally, intraoral scanners are used for examination or treatment within a patient's mouth. Use of an intraoral scanner can eliminate the use of conventional impression materials, plaster models, and simplify the clinical treatment process for a dentist on a tooth while reducing discomfort for a patient. However, conventional intraoral scanners can not be suitable for detecting internal structures of a tooth, such as the structure of enamel and dentin within a patient's tooth, as they are typically designed only for detecting the surface of a tooth. In particular, an intraoral scanner can not be able to determine the general internal composition of a tooth within a patient's mouth. Thus, an intraoral scanner can not be suitable for detecting, for example, the development of caries and cracks within the enamel and underlying dentin of a tooth, bleeding or any other damage within the enamel and underlying dentin of a tooth, and / or deep cracks, margin lines, or other errors in a dental restoration (e.g., a crown, a bridge, an implant, an onlay, an inlay, etc.) prepared for a patient using a natural tooth or a prosthetic implant. In particular, internal structural information of a natural tooth or a prosthetic implant, in combination with surface information, is critical for effective treatment of a patient and ensuring long-span restorations using a natural tooth or a prosthetic implant.

[0004] Conventional methods for constructing a 3D internal geometry of an intraoral dental structure, such as a tooth, using an intraoral scanner have limitations. In one example, conventional methods can cause the intraoral scanner to capture a large number of images from various different viewpoints to reconstruct the internal geometry of the dental structure in 3D. In this regard, a plurality of points corresponding to the 3D internal geometry of the dental structure are projected based on a number of pixels and corresponding intensities from the large number of images captured from the different viewpoints. Then, only the minimum projected object intensity or the minimum scattering intensity corresponding to each artificial point is considered to reconstruct the 3D internal geometry of the dental structure in 3D.

[0005] Conventional methods of generating a 3D model of the 3D internal geometry of a dental structure have some drawbacks. In one example, conventional methods are unable to reliably disambiguate between different components of the 3D internal geometry of the dental structure. In addition, conventional methods require capturing a very large number of images from a large number of viewpoints because only covering multiple perspectives can determine reliable scattering coefficients. This increases the amount of computation and results in a delay in generating the 3D model. As a result, real-time operations can not be performed on such a 3D model of the 3D internal geometry of the dental structure. For example, all points along a given ray are determined to belong to the same material unless a point on the ray is also seen by another ray from another image with lower intensity, i.e., another ray that also sees the point in another image does not intersect any other part of the dental structure other than the point within the 3D internal geometry of the dental structure. For example, to determine that a point is enamel, at least one of the rays projected onto the dental structure and projected back forms an image at any point that does not intersect dentin and any other dental structure within the internal structure. Therefore, there is a need to improve systems and methods of intraoral scanning to overcome the drawbacks of conventional methods for generating a 3D model of the internal geometry of a dental structure. SUMMARY

[0006] The present invention provides an intraoral scanning system, method, and computer programmable product for generating a three-dimensional (3D) internal geometry of a subject’s teeth using a handheld intraoral scanner that includes a sensor for detecting near-infrared (NIR) and visible light, a tooth three-dimensional (3D) surface model rendering a subject’s teeth, and a continuum volumetric machine learning model.

[0007] Some embodiments are based on the understanding that the 3D internal geometry of a subject’s teeth is determined with visible light information and NIR information from one or more sensors, where the visible light information and the NIR information are either simultaneously from the one or more sensors or have a delay therebetween, such delay being caused by a switching of the visible light and the NIR light. Such switching of the visible light and the NIR light can be required because the intraoral scanning system is not configured to capture the visible light information and the NIR information simultaneously. The period of the delay can be within 200 ms or within 500 ms.

[0008] In one aspect, an intraoral scanning system for generating a three-dimensional (3D) internal geometry of a subject’s teeth is disclosed. The intraoral scanning system can include a handheld intraoral scanner configured to operate with one or more sensors to detect near-infrared (NIR) and visible light. The one or more sensors can include an image sensor. The intraoral scanning system can include one or more processors operably connected to the handheld intraoral scanner. The one or more processors can be configured to receive visible light information and near-infrared (NIR) information from the one or more sensors. The one or more processors can be configured to determine surface information in real-time from the visible light information to generate a three-dimensional (3D) surface model of the subject’s teeth with the surface information. The one or more processors can be configured to capture a plurality of two-dimensional (2D) NIR images of internal regions of the subject’s teeth in real-time from the NIR information with the image sensor. Each of the plurality of NIR images includes a plurality of corresponding pixels. The one or more processors can be configured to determine a set of input parameters of a projected object corresponding to each of the plurality of pixels, wherein the set of input parameters can include spatial position information and viewing angle information, and the projected object can include a plurality of point coordinates. The intraoral scanning system can include a continuous volumetric machine learning model configured to receive and process the set of input parameters to determine intensity values and density values of a three-dimensional internal geometry of the 3D surface model. The continuous volumetric machine learning model can be configured to be trained using the plurality of 2D NIR images.

[0009] In some embodiments, the continuous volumetric machine learning model can be trained by receiving, for each of the plurality of pixels corresponding to each of the plurality of 2D NIR images, the set of input parameters corresponding to the projected object. The projected object includes the plurality of point coordinates associated with the 3D surface model. The continuous volumetric machine learning model can be trained by determining, using the set of input parameters, intensity values and density values for each of the plurality of point coordinates based on the continuous volumetric machine learning model. The continuous volumetric machine learning model can be trained by determining, based on the corresponding determined intensity values and density values for each of the plurality of point coordinates, a synthetic pixel value of the projected object. Further, the continuous volumetric machine learning model can be trained by minimizing a loss function between the synthetic pixel value and corresponding true pixel values of the plurality of pixels in the plurality of 2D NIR images.

[0010] In some embodiments, the one or more processors can be further configured to determine a plurality of grid points within the 3D surface model and determine the three-dimensional internal geometry by arranging at least one of the intensity values or the density values at each of the plurality of grid points.

[0011] In some embodiments, the intraoral scanning system can further include a display unit configured to display the 3D internal geometry based on at least one of the intensity values or the density values determined by the continuous volume machine learning model.

[0012] In some embodiments, the display unit can be configured to display the 3D internal geometry within the 3D surface model.

[0013] In some embodiments, the one or more processors can be further configured to determine a boundary between the enamel and the dentin in the dentition of the subject’s teeth based on the intensity values and the density values of the projection object for each of the plurality of pixels.

[0014] In some embodiments, the projection object can be one of a ray or a cone.

[0015] In some embodiments, the handheld intraoral scanner can include a projector unit that can be configured to illuminate the subject’s teeth using one or more white wavelength pulses and one or more near-infrared (NIR) wavelength pulses. Further, the handheld intraoral scanner can include one or more sensors that can be configured to generate a set of white light images and a plurality of two-dimensional near-infrared images based on the illumination.

[0016] In some embodiments, the 3D surface model is determined based on the set of white light images.

[0017] In some embodiments, the one or more processors can be further configured to estimate a relative position between the handheld intraoral scanner and the subject’s teeth corresponding to each of the plurality of near-infrared images. The estimated relative position indicates the perspective information and the spatial position information of the corresponding projection object.

[0018] In some embodiments, the projection object can be a cone. In this regard, the one or more processors can be further configured to determine one or more frustums of each of the cones corresponding to each of the plurality of pixels. The one or more frustums are associated with a plurality of point coordinates. The one or more processors can be further configured to determine an integral position encoding for the one or more frustums that transforms the plurality of point coordinates of each cone. The integral position encoding includes at least a Gaussian encoding and a sinusoidal encoding. The one or more processors can be further configured to generate a synthesized pixel value for each cone based on the integral position encoding of the corresponding one or more cones and the continuous volume machine learning model.

[0019] In some embodiments, the one or more processors can be further configured to determine a radius of each cone indicating the projection object corresponding to each of the plurality of pixels based on a size of the corresponding pixel of the plurality of pixels.

[0020] In some embodiments, the one or more processors can be further configured to determine an average of content within the visible volume for each of the plurality of pixels, and render each frustum indicating the projected object for each of the plurality of pixels based on the corresponding average of content. The content indicates color intensity.

[0021] In some embodiments, the continuous volumetric machine learning model can include a machine learning based Neural Radiance Field (NeRF) network.

[0022] In another aspect, a method for generating a three-dimensional (3D) internal geometry of a subject's teeth with an intraoral scanning system is disclosed. The intraoral scanning system can include a handheld intraoral scanner configured to operate with one or more sensors to detect near-infrared (NIR) and visible light; one or more processors operably connected to the handheld intraoral scanner; and a continuous volumetric machine learning model. The one or more sensors can include an image sensor. The method can include receiving visible light information and near-infrared (NIR) information from the one or more sensors. The method can also include determining surface information from the visible light information to generate a three-dimensional (3D) surface model of the subject's teeth in real-time with the surface information. The method can further include capturing a plurality of two-dimensional (2D) near-infrared images of an internal region of the subject's teeth in real-time from the NIR information with the image sensor. Each of the plurality of NIR images includes a plurality of corresponding pixels. The method can include determining a set of input parameters for a projected object corresponding to each of the plurality of pixels. The set of input parameters can include spatial position information and viewing angle information, and the projected object can include a plurality of point coordinates. The method can also include processing the set of input parameters using the continuous volumetric machine learning model to determine intensity values and density values for the 3D internal geometry of the 3D surface model. The continuous volumetric machine learning model is configured to be trained using the plurality of NIR images.

[0023] In yet another aspect, a computer programmable product includes a non-transitory computer readable medium having stored thereon computer executable instructions that, when executed by a processing circuit, cause the processing circuit to perform operations. The operations can include receiving visible light information and near infrared (NIR) information from one or more sensors mounted within a handheld intraoral scanner. The one or more sensors can be configured to detect near infrared (NIR) and visible light, where the one or more sensors can include an image sensor. The operations can also include determining surface information from the visible light information to generate a three-dimensional (3D) surface model of a subject’s teeth in real-time using the surface information. The operations can also include capturing a plurality of two-dimensional (2D) near infrared images of internal regions of the subject’s teeth in real-time from the near infrared information using the image sensor. Each of the plurality of near infrared images can include a plurality of corresponding pixels. The operations can also include determining a set of input parameters of a projected object corresponding to each of the plurality of pixels. The set of input parameters can include spatial position information and viewing angle information, and the projected object can include a plurality of point coordinates. The operations can also include processing the set of input parameters using a continuous volumetric machine learning model to determine intensity values and density values of a three-dimensional internal geometry of the three-dimensional surface model. The continuous volumetric machine learning model can be configured to be trained using the plurality of near infrared images.

[0024] The foregoing overview is merely exemplary and is not intended to limit the application in any way. Additional aspects, embodiments, and features will become apparent from review of the drawings and the following detailed description.

[0025] According to the present application, intraoral scanning systems, methods, and computer programmable products are provided. One of the objectives of the present application is to provide enhanced processing capabilities in a handheld intraoral scanner to generate a three-dimensional (3D) internal geometry of a subject’s teeth.

[0026] Conventional systems can include intraoral scanners for capturing two-dimensional (2D) intraoral scans and / or three-dimensional (3D) information of a patient’s dental arch. However, conventional intraoral scanners have limited processing capabilities and can only be used to capture 2D intraoral scans and / or 3D models of a patient’s dental arch surface. However, a 3D surface model of a dental arch can not be able to detect internal structures of the teeth, such as the structure of the enamel and dentin within a patient’s dental arch. Thus, unless a disease or abnormality within the internal structure of a tooth develops to the surface of the tooth, or a dentist utilizes other devices such as an X-ray machine, such a disease or abnormality within the internal structure of a tooth can not be identified, however, X-rays are ionizing radiation that can cause mutations in human cells upon exposure and thus should be avoided. For example, a conventional intraoral scanner can not be able to identify early occurrences of abnormalities or diseases occurring within a tooth, such as dentin erosion, enamel erosion, internal cracks in a tooth, or caries, unless the bone of the tooth or the surface of the tooth is affected. Due to the delay in diagnosis of internal diseases or abnormalities of a tooth, irreversible damage can occur in the tooth, resulting in the use of surgical methods to remove such a tooth. This can cause great discomfort to the patient. Furthermore, a conventional intraoral scanner can not be able to identify abnormalities occurring within a dental prosthesis implant. Identifying any abnormalities occurring within a dental prosthesis implant can be crucial to ensure a long-term service life of such a dental prosthesis implant. To this end, information related to the internal geometry of a patient’s tooth or dental arch in combination with surface information can be crucial to early diagnosis, effective treatment, and ensuring long time-span restorations using natural teeth or prosthesis implants.

[0027] In certain conventional methods, intraoral scans can be used to reconstruct the internal geometry of a tooth. However, due to the large number of viewpoints and angles of intraoral 2D images required, conventional methods for generating the internal geometry of a tooth can not be practically feasible and have other limitations. To project a point indicative of a portion of the internal geometry of a tooth or teeth, the values of a plurality of pixels in near-infrared two-dimensional images from different perspectives corresponding to the portion of the three-dimensional internal geometry can be processed to identify an intensity of the portion in each of the plurality of pixels.

[0028] Then, based on the processing of the plurality of pixels, a three-dimensional point indicative of the portion can be projected using the minimum intensity value corresponding to the portion. For example, the minimum intensity value of a pixel of a two-dimensional image can be indicative of the reflection of the portion originating from the material of the portion. Thus, a three-dimensional point to be determined as belonging to a portion of enamel requires receiving at least one ray forming a pixel on a two-dimensional image such that the ray corresponding to the portion of enamel does not intersect with dentin at any point. In other words, when a ray reflects back from the portion of enamel to form a two-dimensional image, the pixel corresponding to the portion of enamel can have a minimum intensity. Furthermore, a three-dimensional point can then be projected as indicative of the portion of enamel based on the minimum intensity.

[0029] To this end, such processing of pixels from a large number of near-infrared two-dimensional images corresponding to the portion would be highly processing intensive. A large amount of computing power can be required, thereby increasing the size and cost of the intraoral scanning system. Moreover, since the projection of three-dimensional points relies on the minimum intensity value, the 3D model of the internal geometry generated using conventional methods can not be accurate. For example, all points along a given ray (e.g., a ray reflected from dentin) can be determined to be related to the same material (e.g., dentin), unless some point on the ray (corresponding to a portion of the internal geometry) is also seen by another ray (e.g., a ray reflected from enamel) from another image with a lower intensity. It can be appreciated that reflections in enamel are extremely rare, while reflections in dentin will be somewhat stronger, since the light is scattered much more. Thus, conventional methods can fail to delineate the boundary between enamel and dentin in the internal geometry of one or more teeth. Accordingly, generating a 3D model of the internal geometry using conventional systems for intraoral scanning can be inaccurate, time consuming, highly processing intensive, cumbersome to operate, and in some cases, practically infeasible.

[0030] In another aspect, the present disclosure provides an intraoral scanning system including a handheld intraoral scanner, a processor, and a continuous volumetric machine learning model for generating a three-dimensional internal geometry of a subject’s teeth. In one embodiment, the generation can be done in real-time, or can be done as a subsequent processing step after data acquisition. The intraoral scanning system can provide enhanced processing capabilities within the handheld intraoral scanner, thereby eliminating the need for additional equipment or devices to generate a three-dimensional internal geometry of the teeth. In particular, the internal geometry 3D model of the subject’s teeth and a three-dimensional surface model can be generated without the need for additional bulky equipment or devices to process the two-dimensional images. Moreover, a user (e.g., a dentist) can not have to carry and operate bulky equipment to generate the 3D model of the internal geometry and / or surface of the subject’s teeth.

[0031] The intraoral scanning system of the present disclosure includes a handheld intraoral scanner that emits light and generates two-dimensional images using a sensor. For example, the handheld intraoral scanner can continuously emit light in a spectral range corresponding to a visible white light wavelength range (e.g., 400-750 nm) as well as a near-infrared (NIR) wavelength range (e.g., 750-2500 nm). In this regard, there is no need to separately illuminate a subject’s teeth with white light and / or near-infrared light multiple times to generate white light and two-dimensional near-infrared image scans or images of the teeth. For example, based on a scan of a subject’s mouth or teeth using the handheld intraoral scanner, white light images and two-dimensional near-infrared images can be generated by sequentially switching on-off a visible light source and a near-infrared light source. The on-off switching can be required because the intraoral scanner can not be configured to capture visible light information and near-infrared information simultaneously. In one example, the switching time interval can be within 200 ms or 500 ms. Alternatively, white light illumination and near-infrared illumination can be performed simultaneously to avoid any relative motion between the intraoral scanner and the patient’s teeth between illuminations. In this case, the intraoral scanner can include suitable optical filters for selectively exposing individual pixels or groups of pixels (in the case of a single sensor) to a subset of the illuminating light. Alternatively, white light illumination and near-infrared illumination can be performed simultaneously using different image sensors within the intraoral scanner such that the different image sensors are configured to be sensitive only in the corresponding selective range related to white light or near-infrared light.

[0032] Further, the intraoral scanning system of the present disclosure alleviates issues related to inaccurate internal geometry of teeth by using a neural-radiance-field (NeRF) based continuous volumetric machine learning model. In particular, the processor of the intraoral scanning system does not estimate the material of a three-dimensional point based on the minimum intensity value of a pixel, i.e., whether the three-dimensional point corresponds to enamel or dentin. Rather, the point coordinates along a ray are integrated over a pixel in the two-dimensional near-infrared image. Further, the point coordinates of the integration for each pixel in multiple two-dimensional near-infrared images are optimized using a function defined by the continuous volumetric machine learning model to predict an intensity value and a density value at a three-dimensional point along a ray. The intensity value and the density value can describe material properties corresponding to the three-dimensional point, thereby enabling differentiation between enamel and dentin and identification of a boundary between enamel and dentin. Thus, the point coordinates on a ray of a pixel can not belong to the same material, and the optimization process is applied over all rays. As a result, the number of two-dimensional near-infrared images needed to reconstruct the three-dimensional internal geometry of a tooth in 3D is significantly reduced. This is advantageous because when the intraoral scanner moves relatively quickly over a patient’s dental arch, the coverage of two-dimensional near-infrared images from multiple angles and viewpoints obtained while performing an intraoral scanning session can be limited, and the intraoral scanner only observes or illuminates the same region of the patient’s teeth from a limited number of poses (viewpoints), such as occlusal, lingual, and labial orientations.

[0033] Further, the intraoral scanning system of the present disclosure includes or can be coupled to a display that can be communicatively coupled with the processor and the continuous volumetric machine learning model over a communication network. Based on the three-dimensional internal geometry of the teeth and the three-dimensional surface model of the teeth generated using the white light images, a comprehensive interactive rendering of the subject's teeth can be displayed on the display. For example, the display can be a display of a smart device, such as a smartphone, a monitor, a television, a tablet computer, a laptop computer, etc. Thus, the intraoral scanning system can provide a user-friendly and time-efficient process for generating a 3D model of the internal geometry and rendering a comprehensive interactive view or 3D model of the subject's teeth.

[0034] To this end, the intraoral scanning system of the present disclosure enables reconstruction of the internal geometry of the subject's teeth in 3D with a limited number of images of internal regions of the teeth obtained by acquiring two-dimensional near-infrared (NIR) images in combination with three-dimensional surface information. BRIEF DESCRIPTION OF DRAWINGS

[0035] The present disclosure is illustrated in the accompanying drawings, which are depicted by way of example and not limitation, in which like references indicate similar elements, and wherein: Figure 1 A network environment in which an intraoral scanning system for oral scanning and generation of 3D internal geometry is implemented is shown, in accordance with an example embodiment; Figure 2 A block diagram of a handheld intraoral scanner, in accordance with an example embodiment, is shown; Figure 3 A schematic diagram of a scanning session, in accordance with an example embodiment, is shown; Figure 4 A sequence diagram depicting generation of a 3D surface model, in accordance with an example embodiment, is shown; Figure 5 A method for generating a set of input parameters for a projection object, in accordance with an example embodiment, is shown; Figure 6 A method for training a continuous volumetric machine learning model, in accordance with an example embodiment, is shown; Figure 7A A sequence diagram depicting generation of a 3D internal geometry, in accordance with an example embodiment, is shown.

[0036] Figure 7B A schematic diagram of a first type of projection object, in accordance with an example embodiment, is shown; Figure 7C A schematic diagram of a second type of projection object, in accordance with an example embodiment, is shown; Figure 7D A schematic diagram of a generated 3D model of a subject's teeth, in accordance with an example embodiment, is shown; Figure 8 A method for generating synthetic pixel values utilizing a continuous volumetric machine learning model is shown in accordance with example embodiments; Figure 9 A method for generating a three-dimensional (3D) internal geometry and a 3D surface model of a subject's teeth is shown in accordance with example embodiments; and Figure 10 A schematic diagram depicting an example environment for generating a three-dimensional (3D) internal geometry and a 3D surface model of teeth and rendering an interactive 3D graphical representation in real-time is shown in accordance with example embodiments. DETAILED DESCRIPTION

[0037] For the purposes of explanation, numerous specific details are set forth in the following description in order to provide a thorough understanding of the present disclosure. It will be apparent, however, to one skilled in the art that the present disclosure can be practiced without these specific details. In other instances, well-known systems and methods have not been described in detail in order to avoid obscuring the present disclosure.

[0038] Reference throughout this specification to “an embodiment” or “embodiments” means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. The appearances of the phrase “in one embodiment” in various places in the specification are not necessarily all referring to the same embodiment, and the different embodiments or alternatives can be understood to be interchangeable in some instances. Furthermore, the terms “a” and “one” are not intended to be construed as limiting the quantity to one only, but rather are used to mean that “at least one” or “one or more.” Moreover, various features that are described in relation to some embodiments can be exhibited in other embodiments, and similar features can be exhibited in other embodiments.

[0039] Some embodiments of the present disclosure will now be described with reference to the attached drawings, which are meant to be exemplary only and not limiting in any way. In fact, the various embodiments of the present disclosure can be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. Like reference numerals refer to like elements throughout. As used herein, “data,” “content,” “information,” and similar terms can be used interchangeably to refer to data capable of being transmitted, received, and / or stored in accordance with embodiments of the present disclosure. Moreover, the terms “processor,” “controller,” and “processing circuitry,” and similar terms can be used interchangeably to refer to a processor capable of processing information in accordance with embodiments of the present disclosure. Furthermore, the terms “electronic device,” “electronic apparatus,” and “apparatus” can be used interchangeably to refer to an electronic device monitored by a system in accordance with embodiments of the present disclosure. Thus, none of these terms should be construed as limiting the broadest scope of the present disclosure.

[0040] The embodiments described herein are for illustrative purposes only and various modifications can be made. It should be understood that various omissions and substitutions and changes, in the form of the detail, can be made by those skilled in the art to the example embodiments without departing from the spirit of the disclosure. Furthermore, it is expressly intended that all permutations of these embodiments are to be encompassed within the spirit and scope of the disclosure. In addition, any use of a language known as being prior art as set forth in various portions of this specification is not to be construed as an admission that such terminology is prior art with respect to the present disclosure and all references contained herein are incorporated by reference herein individually as if each were individually directed to prior art with respect to the present disclosure.

[0041] In this specification and in the claims, the terms "for example," "e.g.," and "such as," and the like, where they are used to introduce an listed group of items, are to be construed as non-limiting, and are merely provided for the purposes of reciting a list of items, and are not to be construed as limiting the list of items to only those items specifically listed. Other terms are to be construed using their broadest reasonable meaning unless they are used in a context that requires a different interpretation.

[0042] The present disclosure provides an intraoral scanning system, method, and computer programmable product for generating a three-dimensional (3D) internal geometry of a subject's teeth and rendering the 3D internal geometry with 3D surface information of the teeth to provide a comprehensive view of the teeth.

[0043] For example, reference is made below to Figure 1 An exemplary network environment for an intraoral scanning system for oral scanning and generating 3D internal geometry is provided.

[0044] Figure 1 An exemplary network environment 100 in which an intraoral scanning system 102 for oral scanning and generating 3D internal geometry is implemented in accordance with an exemplary embodiment is shown. The intraoral scanning system 102 can be used to generate a 3D internal geometry of teeth or a dental arch within a subject's mouth. For example, the intraoral scanning system 102 can be used by a user, such as a person with dental knowledge, such as a dentist, dental technician, etc. Further, one or more components can be rearranged, altered, added and / or removed without departing from the scope of the disclosure.

[0045] The intraoral scanning system 102 can include a handheld intraoral scanner 104, one or more processors 106, and a continuous volumetric machine learning model 108. The network environment 100 can also include communication channels 110 that can be configured to establish communicative coupling between the components of the intraoral scanning system 102, namely the handheld intraoral scanner 104, the one or more processors 106 (hereinafter referred to as the processor 106), and the continuous volumetric machine learning model 108.

[0046] The network environment 100 can also include data generated by the intraoral scanning system 102. For example, the network environment 100 can include data captured by the handheld intraoral scanner 104 in the form of a plurality of 2D images 112. As shown, the plurality of 2D images 112 generated by the handheld intraoral scanner 104 includes a plurality of two-dimensional (2D) near-infrared (NIR) images 114 and white light images 116. Further, the data generated by the processor 106 can include a three-dimensional (3D) model. The 3D model includes a three-dimensional (3D) surface model 118 of the subject’s teeth and a 3D internal geometry 120 of the subject’s teeth. Further, the data generated by the continuous volume machine learning model 108 can include intensity values and density values 122 of the 3D internal geometry 120.

[0047] The intraoral scanning system 102 can be used to register intraoral scans and generate a three-dimensional internal geometry 120 and a three-dimensional surface model 118 of the teeth. The intraoral scanning system 102 can include a plurality of components, such as the handheld intraoral scanner 104, the processor 106, and the continuous volume machine learning model 108, which can be in communication with one another to register intraoral scans having a three-dimensional internal geometry 120 of the teeth.

[0048] In one example, the handheld intraoral scanner 104 can include the processor 106 and / or the continuous volume machine learning model 108. In another example, the handheld intraoral scanner 104 can be coupled with the processor 106 and / or the continuous volume machine learning model 108, where the processor 106 and / or the continuous volume machine learning model 108 can be remotely located and can perform operations associated with the intraoral scanning system 102. The intraoral scanning system 102 can have enhanced processing capabilities that can be required to process the plurality of two-dimensional near-infrared images 114 and visible light images 116 in real-time to generate the three-dimensional surface model and the three-dimensional internal geometry.

[0049] The handheld intraoral scanner 104 can be configured to capture the plurality of 2D images 112 during a scanning session of the subject’s teeth. The plurality of 2D images 112 can include a plurality of two-dimensional near-infrared images 114, which indicate images of the subject’s teeth captured using light emitted at wavelengths corresponding to a NIR wavelength range. The plurality of two-dimensional near-infrared images 114 can include images of the subject’s teeth from different viewing angles or viewpoints. Further, the plurality of 2D images 112 can include visible light images 116, which indicate images of the subject’s teeth captured using light emitted at wavelengths corresponding to a white light wavelength range or a selected sub-spectrum of visible light, such as blue light (400-495 nm). The white light images 116 can also include images of the subject’s teeth from different viewing angles. In certain instances, the plurality of 2D images 112 can also include images captured at other visible light wavelengths.

[0050] Based on the captured white light image 116 and visible light information, the processor 106 can be configured to determine three-dimensional surface information of the subject’s teeth. The three-dimensional surface information can be a digital representation of the subject’s dental arch depicted in 3D space. For example, the three-dimensional surface information can include three-dimensional point cloud data corresponding to the white light image 116. The three-dimensional point cloud data can correspond to three-dimensional real-world coordinates. Further, the three-dimensional surface information can also include color texture reflected from the surface of the teeth.

[0051] In one example, the handheld intraoral scanner 104 can include a web server that can be configured to communicate over the web and establish a connection to the communication channel 110. The handheld intraoral scanner 104 can include one or more sensors. The handheld intraoral scanner 104 can be configured to execute the web server to provide visible light information and near-infrared information obtained from the sensors to the processor 106. In one example, the handheld intraoral scanner 104 can be configured to provide three-dimensional surface information to the processor 106 based on the visible light information and / or the white light image 116. The handheld intraoral scanning device 104 can also include a processing unit, a memory unit, a communication interface, and additional components. The processing unit, the memory unit, the communication interface, and the additional components can be communicatively coupled to each other. Details of the components of the handheld intraoral scanner 104 are further provided, for example, in Figure 2

[0052] The processor 106 can include processing capabilities required to process the visible light information or the white light image 116 and the near-infrared information or the plurality of two-dimensional near-infrared images 114. The processor 106 can be configured to establish a connection with the communication channel 110. In one example, the processor 106 can be located within the handheld intraoral scanner 104. For example, the processor 106 can receive the visible light information or the white light image 116 from the handheld intraoral scanner 104 or the sensors of the handheld intraoral scanner 104 to generate three-dimensional surface information of the subject’s teeth. Further, based on the three-dimensional surface information derived from the white light image 116, a three-dimensional surface model 118 of the subject’s teeth is generated. The processor 106 can further render the 3D surface information or the 3D surface model 118 of the teeth into an interactive 3D graphical representation.

[0053] ​For example, the processor 106 can be implemented as one or more of various hardware processing means, such as a coprocessor, a microprocessor, a controller, a digital signal processor (DSP), a processing element with or without an accompanying DSP, or various other processing circuitry including integrated circuits such as, for example, an ASIC (application specific integrated circuit), an FPGA (field programmable gate array), a microcontroller unit (MCU), a hardware accelerator, a special-purpose computer chip, or the like. As such, in some embodiments, the processor 106 can include one or more processing cores configured to perform independently. A multi-core processing can enable parallel processing of data, thereby enabling the processor 106 to perform multiple functions simultaneously. Additionally or alternatively, the processor 106 can include one or more processors configured in tandem via the bus to enable independent execution of instructions, pipelining and / or multithreading. The processor 106 can also be accompanied with one or more cache memories to increase the efficiency of the processing operations. The processor 106 can be a part of or configured to run in a computing device, such as a desktop, a laptop, a handheld device, or any other device. The processor 106 can be implemented as a single- or multi-chip module with real-time and / or batch processing capabilities. The processor 106 can further be a part of one or more distributed systems, which together function as a distributed computing platform.

[0054] In one example, when the processor 106 is implemented as an executor of software instructions, the instructions can specifically configure the processor 106 to perform the algorithms and / or operations described herein when the instructions are executed. However, in some instances, the processor 106 can be a processor-specific device (e.g., a mobile terminal or a stationary computing device) configured to employ embodiments of the present disclosure by further configuring the processor 106 with instructions for performing the algorithms and / or operations described herein. The processor 106 can include a clock, an arithmetic logic unit (ALU), and logic gates configured to support the operation of the processor 106, among others. The network environment, such as the network environment 100, can be accessed using the communication network 110.

[0055] The continuous volumetric machine learning model 108 can include a fully connected neural network that can generate a view or model of a three-dimensional scene based on a set of two-dimensional images. The continuous volumetric machine learning model 108 can enhance the ability of the system 102 to generate a three-dimensional internal geometry of a subject’s tooth. The continuous volumetric machine learning model 108 can be configured to process near-infrared information or a plurality of two-dimensional near-infrared images 114 to generate an output indicating an intensity value and a density value for each point coordinate of each pixel of each of the plurality of two-dimensional near-infrared images 114. The continuous volumetric machine learning model can be used to generate a three-dimensional internal geometry 120 or internal volume of a tooth from a set of input parameters. The continuous volumetric machine learning model 108 can be based on the idea that the three-dimensional internal geometry 120 can be represented as a continuous field of optical density and intensity values, where each pixel in the two-dimensional near-infrared images 114 can correspond to a projected object in the continuous field of optical density and intensity values. By using a deep neural network, the continuous volumetric machine learning model 108 can be able to learn a mapping between the set of input parameters and the optical density and intensity values. This enables it to generate an internal volume 120 of a tooth from a given set of inputs. For example, the continuous volumetric machine learning model 108 can be trained using volume rendering to map values of view angles and spatial positions to density and intensity. The continuous volumetric machine learning model 108 can receive a plurality of two-dimensional near-infrared images 114 representing an internal region of a subject’s tooth and process the plurality of two-dimensional near-infrared images 114 to generate a composite pixel value for pixels of the plurality of two-dimensional near-infrared images 114. In one example, the continuous volumetric machine learning model 108 can include a neural radiance field (NeRF) based model, a multi-scale anti-aliased neural radiance field (mip-NeRF) model, or any other suitable architecture or variant associated with the same family of neural radiance field models.

[0056] In one example, the processor 106 can be configured to generate and render the three-dimensional internal geometry 120 of the tooth within the three-dimensional surface model 118 based on the trained continuous volumetric machine learning model. For example, the three-dimensional internal geometry 120 of the tooth can be rendered within the three-dimensional surface model 118 in the form of an interactive three-dimensional graphical representation. The interactive 3D graphical representation can be rendered on a display unit of a device. A user (e.g., a dentist) can view the interactive 3D graphical representation on the display to diagnose any disease or abnormality within the tooth and / or treat the subject. The user can modify the view of the interactive 3D graphical representation according to preference. For example, the perspective of the interactive 3D graphical representation can be altered or the interactive 3D graphical representation can be zoomed in or out according to user preference.

[0057] The display can be associated with any user-accessible device, such as a display unit, monitor, mobile phone, smart phone, tablet computer, computer, virtual reality (XR) device, etc. In some examples, the display can be part of the user-accessible device. For example, the display can be a touchscreen display. Additional, different, or fewer components can be provided. Furthermore, components can be rearranged, changed, added, and / or removed without departing from the scope of the disclosure.

[0058] The communication channel 110 can be any combination of wired, wireless, or wired and wireless communication networks such as a cellular, wireless fidelity (Wi-Fi), Internet, local area network (LAN), etc. According to embodiments, the communication channel 110 can be one or more wireless full-duplex communication channels. In one embodiment, the communication channel 110 can include one or more networks, such as a data network, a wireless network, a telephony network, or any combination thereof. It is contemplated that a data network can be any local area network (LAN), metropolitan area network (MAN), wide area network (WAN), a public data network (e.g., the Internet), short range wireless network, or any other suitable packet-switched network, such as a commercially owned, proprietary packet-switched network, and the like, or any combination thereof. In addition, the wireless network can be, for example, a cellular network and can employ various technologies including enhanced data rates for global evolution (EDGE), general packet radio service (GPRS), global system for mobile communications (GSM), Internet protocol multimedia subsystem (IMS), universal mobile telecommunications system (UMTS), etc., as well as any other suitable wireless medium, e.g., worldwide interoperability for microwave access (WiMAX), Long Term Evolution (LTE) networks, e.g., LTE-Advanced Pro, 5G, New Radio (NR) networks, ITU-IMT 2020 networks, code division multiple access (CDMA), wideband code division multiple access (WCDMA), wireless fidelity (Wi-Fi), wireless local area network (WLAN), Bluetooth, Internet Protocol (IP) data casting, satellite, mobile ad hoc network (MANET), and the like, or any combination thereof. The handheld intraoral scanner 104 can be configured to communicate with the processor 106 and the continuous volumetric machine learning model 108 through the communication channel 110.

[0059] For example, the subject can require dental treatment. In this case, the system 102 can be used by a user (e.g., a dentist) to provide dental treatment to the subject. In one embodiment, the subject can present at a dental clinic. In this case, the system 102 can be used in a treatment room of the dental clinic. In another embodiment, the subject can request a home visit for dental treatment. In this case, the system 102 can be used at the subject's home. To begin dental treatment, the user can use the handheld intraoral scanner 104 to capture a white light image 116 and a plurality of two-dimensional near-infrared images 114 of the subject's teeth using one or more sensors.

[0060] In operation, the one or more sensors of the handheld intraoral scanner 104 can be configured to detect near-infrared light and / or visible light. For example, near-infrared light and visible light can be emitted by the one or more sensors, and the near-infrared light and visible light can be reflected from interior regions and surfaces of the teeth. Characteristics (e.g., wavelength, frequency, etc.) of the emitted and received near-infrared light can be stored as near-infrared information, and can be used to generate the plurality of two-dimensional near-infrared images 114. In addition, characteristics (e.g., wavelength, frequency, etc.) of the emitted and received visible light can be stored as visible light information, and can be used to generate the white light image 116. The captured near-infrared information and visible light information can be provided to the processor 106 of the system 102.

[0061] Upon receiving the near-infrared information and visible light information, the processor 106 can determine three-dimensional surface information from the visible light information. Based on the three-dimensional surface information, the processor 106 can be configured to generate a three-dimensional surface model 118 of the subject's teeth. For example, in one embodiment, the three-dimensional surface model 118 can be generated by the processor 106 using a three-dimensional surface reconstruction algorithm. Figure 4 Details of capturing the visible light information of the white light image 116, determining the three-dimensional surface information, and generating the three-dimensional surface model 118 are further provided in

[0062] Once the three-dimensional surface model 118 of the tooth is created, it can be desirable to create a 3D model of the three-dimensional internal geometry of the tooth. In this regard, the processor 106 can use the received near-infrared information to capture (or generate) a plurality of two-dimensional near-infrared images 114 of the interior region of the tooth. For example, the processor 106 can filter the received NIR information to generate only the plurality of two-dimensional near-infrared images 114 of the interior region. Each of the plurality of two-dimensional near-infrared images 114 can include a plurality of pixels. That is, a two-dimensional near-infrared image from the plurality of two-dimensional near-infrared images 114 can be a collection of a plurality of pixels that indicate different portions of the interior region of the tooth that the two-dimensional near-infrared image captures. Further, the processor 106 is configured to determine a set of input parameters of a projection object corresponding to each of the plurality of pixels of each of the plurality of two-dimensional near-infrared images 114. The set of input parameters can include spatial location information and viewing angle information corresponding to each of the plurality of pixels. The projection object corresponding to the pixel includes a plurality of point coordinates along the projection object. The plurality of point coordinates can correspond to different materials within the interior region of the tooth, such as enamel or dentin. For example, the set of input parameters can include five-dimensional (5D) information that can be fed to the continuous volumetric machine learning model 108 for generating the intensity values and density values 122 of the generated mesh points in the 3D surface model, and the viewing directions of the plurality of 2D near-infrared images 114. Details of generating the set of input parameters for the continuous volumetric machine learning model 108 are further provided in Figure 5 , Figure 7A and Figure 8 .

[0063] The continuous volumetric machine learning model 108 of the system 102 can be configured to receive and process the set of input parameters. The continuous volumetric machine learning model 108 can be configured to be trained using the plurality of 2D near-infrared images 114. Based on the set of input parameters, the continuous volumetric machine learning model 108 can determine a synthesized pixel value 122 based on the intensity values and density values along the projection object of the corresponding pixel in the plurality of pixels. The intensity values and density values are associated with each of the plurality of point coordinates along the projection object. In one example, the continuous volumetric machine learning model 108 can generate synthesized data, i.e., the synthesized pixel value 122, that indicates a color intensity value or a density value of the material corresponding to each of the plurality of pixels of each of the plurality of two-dimensional near-infrared images 114. For example, the synthesized data for a pixel can indicate an integration and optimization of data related to the plurality of point coordinates associated with the pixel. The synthesized data for a pixel can be used to determine the material (e.g., enamel or dentin) related to the pixel and / or the corresponding 3D point in the three-dimensional internal geometry 120.

[0064] The processor 106 can be configured to generate and render the three-dimensional internal geometry 120 of the teeth within the three-dimensional surface model 118 based on the trained continuous volumetric machine learning model. For example, the one or more processors are configured to generate grid points within the 3D surface model at different viewing directions of the three-dimensional surface model and feed these grid points along with the viewing directions to the trained continuous volumetric machine learning model 108, which subsequently generates intensity values and density values for the corresponding grid points. The trained continuous volumetric machine learning model comprises a continuous definition of the three-dimensional internal geometry, which means that the trained continuous volumetric machine learning model is configured to evaluate and output intensity values and density values at any given grid point of the three-dimensional internal geometry. The manner in which the continuous volumetric machine learning model 108 generates the composite pixel values 122 will be described in detail in connection with, for example, Figure 7A and Figure 8 the figures that follow.

[0065] After generating the three-dimensional internal geometry 120 of the teeth of the subject, the system 102 can be connected with a display to render an interactive three-dimensional graphical representation comprising the three-dimensional internal geometry 120 and the three-dimensional surface model 118. For example, the display can be any display unit capable of rendering the interactive three-dimensional graphical representation. The dentist can use the interactive three-dimensional graphical representation of the teeth of the subject as a human-eye readable three-dimensional data of the white light image 116 and the plurality of two-dimensional near-infrared images 114. For example, the interactive three-dimensional graphical representation of the teeth of the subject can be used to make a comprehensive assessment of the condition of the teeth of the subject from the inside and the outside. The interactive three-dimensional graphical representation can be modified or edited on the display, for example, by using gestures provided by the dentist as input. Details of rendering the 3D internal geometry 120 and the 3D surface model 118 into an interactive 3D graphical representation are further provided in Figure 10 .

[0066] Figure 2 A block diagram 200 of the handheld intraoral scanning device 104 according to an exemplary embodiment is shown. Figure 2 is explained in connection with the elements in Figure 1 . The handheld intraoral scanning device 104 can comprise at least one processing unit (hereinafter referred to as “processing unit 202”), a memory unit 204, a web server 206, a monitoring unit 208, a temporary storage unit 210, a scanning feedback unit 212, an input / output (I / O) unit 214, and a communication interface 216.

[0067] The processing unit 202 can be implemented in a number of different fashions. For example, the processing unit 202 can be implemented as one or more of various hardware processing means such as a coprocessor, a microprocessor, a controller, a digital signal processor (DSP), a processing element with or without an accompanying DSP, or various other processing circuitry including integrated circuits such as, for example, an ASIC (application specific integrated circuit), an FPGA (field programmable gate array), a microcontroller unit (MCU), a hardware accelerator, a special-purpose computer chip, or the like. In an embodiment, the processing unit 202 can be implemented as a high- performance microprocessor, such as one having a series of system on a chip (SOC) that includes a relatively strong and energy-efficient graphics processing unit (GPU) and central processing unit (CPU) along with a small form factor. Thus, in some embodiments, the processing unit 202 can include one or more processing cores configured for independent execution. A multi-core processing unit can implement multiprocessing in a single physical package. Additionally or alternatively, the processing unit 202 can include one or more processors configured in a bus-based or point-to-point communication configuration. The processing unit 202 can be configured to execute instructions for performing various operations described herein.

[0068] In some embodiments, the processing unit 202 can be configured to detect near-infrared light and visible light using the I / O unit 214 during a scanning session of a subject’s (e.g., a patient in need of dental treatment) teeth. The detected near-infrared light and visible light can be used to generate a plurality of 2D images 112, such as a plurality of two-dimensional near-infrared images 114 and white light images 116. The plurality of 2D images 112 can include images of the subject’s teeth from different angles or viewpoints. For example, the processing unit 202 can be configured to generate 3D surface information of the teeth based on the white light images 116, and a continuous volumetric machine learning model can generate intensity values and density values 122 based on the plurality of 2D near-infrared images 114.

[0069] In one example embodiment, the processing unit 202 can communicate with the memory unit 204 via a bus to pass information between components of the handheld intraoral scanner 104.

[0070] The memory unit 204 can be non-transitory and can include, for example, one or more volatile memory and / or non-volatile memory. In other words, for example, the memory unit 204 can be an electronic storage device (e.g., computer readable storage medium) that includes gates configured to store data (e.g., bits) that can be retrievable by a machine (e.g., a computing device, like the processing unit 202). The memory unit 204 can be configured to store information, data, content, applications, instructions, etc. for enabling the device to perform various functions in accordance with exemplary embodiments of the present disclosure. For example, the memory unit 204 can be configured to store the detected near-infrared light and the detected visible light after a scanning session of a tooth has ended. The detected near-infrared light and visible light after the scanning session can be stored as near-infrared information and visible light information, respectively. In some cases, the memory unit 204 can be configured to store compressed near-infrared information and visible light information. In some embodiments, the memory unit 204 can be configured to store calibration data needed to measure the detected near-infrared and visible light to generate the near-infrared information, the visible light information, the white light image 116, and / or the plurality of two-dimensional near-infrared images 114. As Figure 2 The memory unit 204 can be configured to store instructions for execution by the processing unit 202, as shown by way of example. Thus, the processing unit 202, whether configured as hardware or software, or a combination of both, can be capable of performing operations according to embodiments of the present disclosure while configured accordingly. Thus, for example, when the processing unit 202 is implemented as a microprocessor, the processing unit 202 can be a specially configured hardware for performing the operations described herein. Alternatively, as another example, when the processing unit 202 is implemented as a software instruction executioner, the instructions can specially configure the processing unit 202 to perform the algorithms and / or operations described herein when the instructions are executed. The processing unit 202 can include a clock, an arithmetic logic unit (ALU), and logic gates, among other things, configured to support the operations of the processing unit 202.

[0071] The web server 206 can be software, hardware, or a combination thereof, configured to store data and provide the data to a web browser associated with the processor 106 and / or the continuous volumetric machine learning model 108. For example, visible light information and near-infrared (NIR) information can be provided to a web browser of the processor 106 through the web server 206. Since any web browser can access the web server 206, there is no need for the processor 106 to install additional software to connect to the web server 206. The web server 206 can communicate with one of the communication channels 110 over a web network. In one example, the web server 206 and the processor 106 and / or the continuous volumetric machine learning model 108 can communicate with a public wireless full-duplex communication channel over a web network to transmit and receive visible light information and NIR information. The web server 206 and the web browser can communicate through, for example, hypertext transfer protocol (HTTP), simple mail transfer protocol (SMTP), or file transfer protocol (FTP). After the web server 206 and the web browser are connected, the web server 206 can provide a web application on the web browser.

[0072] The monitoring unit 208 can be software, hardware, or a combination thereof, which can be configured to monitor a bandwidth of one of the communication channels 110, such as a wireless full-duplex communication channel, through which the handheld intraoral scanning device 104 and the processor 106 and / or the continuous volumetric machine learning model 108 can connect. In addition, the monitoring unit 208 can also be configured to monitor a connection of one of the communication channels 110, such as a wireless full-duplex communication channel, through which the handheld intraoral scanning device 104 and the processor 106 and / or the continuous volumetric machine learning model 108 can connect.

[0073] In one embodiment, if the monitoring unit 208 determines that the bandwidth of the communication channel 110 is below the minimum bandwidth, the monitoring unit 208 can provide such information to the processing unit 202. The processing unit 202 can downsample the visible light information and the NIR information based on the received information. In another embodiment, if the monitoring unit 208 can determine that the bandwidth of the communication channel 110 is below the minimum bandwidth for a duration longer than the maximum period, the monitoring unit 208 can provide such information to the processing unit 202. The processing unit 202 can compress and store the visible light information and the NIR information into the memory unit 204. In some embodiments, if the monitoring unit 208 can determine that the connection between the handheld intraoral scanning device 104 and the processor 106 and / or the continuous volumetric machine learning model 108 is lost, the monitoring unit 208 can provide such information to the processing unit 202. In this case, the processing unit 202 can compress and store the visible light information and the NIR information into the memory unit 204.

[0074] The temporary storage unit 210 can be software, hardware, or a combination thereof, configurable to store the visible light information and the NIR information when the bandwidth of the communication channel 110 (e.g., a wireless full-duplex communication channel) is determined to be lower than the minimum bandwidth. The temporary storage unit 210 can also be configurable to further transmit the stored visible light information and NIR information to the processor 106 and / or the continuum machine learning model 108 when the bandwidth is determined to be higher than or equal to the minimum bandwidth. Examples of the temporary storage unit 210 include, but are not limited to, a random access memory (RAM) or a cache memory.

[0075] The scan feedback unit 212 can be software, hardware, or a combination thereof, configurable to receive the status input from the monitoring unit 208. Based on the received status input, the scan feedback unit 212 can provide a scan feedback signal to a user (e.g., a dentist) of the handheld intraoral scanner 104. In one embodiment, the scan feedback signal is used to provide guidance to the user regarding a low scan quality of the scan session and insufficiently scanned regions of the teeth for visible light information and / or near-infrared information. For example, the scan feedback unit 212 can provide a scan feedback signal, such as an acoustic feedback signal, a haptic feedback, or a visual feedback.

[0076] The I / O unit 214 can include circuitry and / or software configurable to provide output to a user of the handheld intraoral scanning device 104 and to receive, measure, or sense input information. The I / O unit 214 can include a speaker 214A, a vibrator 214B, a projector unit 214C, and one or more sensors 214D. In one embodiment, the speaker 214A can be configured to output an acoustic feedback signal to guide the user. The vibrator 214B can be, for example, a transducer configured to convert a scan feedback signal (which can be an electrical signal) into a mechanical output, such as a haptic feedback in the form of a vibration, to guide the user.

[0077] It can be appreciated that the handheld intraoral scanner 104 can be configured to detect near-infrared light and visible light that can be reflected from the teeth of a subject. In this regard, the projector unit 214C can be configured to output one or more visible or white wavelength pulses and one or more near-infrared wavelength pulses. For example, the white or visible wavelength pulses and the near-infrared wavelength pulses can be projected onto the teeth to illuminate the teeth of a subject (e.g., a patient). Further, the white wavelength pulses and the wavelength pulses can be reflected or refracted from the tooth surface and / or internal regions. The one or more sensors 214D can be configured to detect the visible or white wavelength pulses and the near-infrared wavelength pulses that can be reflected and / or refracted from the tooth surface or internal regions. In one example, the one or more sensors 214D can include one or more image sensors, such as cameras. For example, the image sensors can be configured to generate the white light image 116 and the plurality of two-dimensional near-infrared images 114 based on the illumination of the teeth using the near-infrared light and the visible light.

[0078] For example, the I / O unit 214 can include a light emitting device, such as a light emitting diode (LED), to provide a scanning feedback signal in the form of light to guide the user or the dentist. For example, a plurality of LEDs can be divided into a left side LED group and a right side LED group to emit a flashing light according to which side of the teeth is not scanned properly. In one example, the I / O unit 214 can include additional input and output devices, such as buttons for receiving an operation signal from the user, a power sensor, a battery charging sensor, and the like.

[0079] The communication interface 216 can include input and output interfaces for supporting communications to and from the handheld intraoral scanner 104. The communication interface 216 can be a device or circuit implemented in hardware or a combination of hardware and software configured to receive and / or transmit data to / from the handheld intraoral scanner 104. In this regard, the communication interface 216 can include, for example, an antenna (or multiple antennas) and supporting hardware and / or software for enabling communications with a wireless communication network. Additionally or alternatively, the communication interface 216 can include a reception circuit that is used to interact with the antenna(s) to transmit signals through the antenna(s) or to process received signals received through the antenna(s). In certain environments, the communication interface 216 can additionally or alternatively support wired communications. Thus, for example, the communication interface 216 can include a communication modem and / or other hardware and / or software to support communications over a cable, a digital subscriber line (DSL), a universal serial bus (USB), or other mechanisms.

[0080] In operation, the handheld intraoral scanner 104 can be configured to capture two-dimensional images or two-dimensional scans of a subject's oral cavity. In one example, a user of the handheld intraoral scanner 104 can perform a scan session of a subject's oral cavity. The handheld intraoral scanner 104 can simultaneously illuminate the subject's oral cavity or teeth using one or more white wavelength pulses and one or more near-infrared wavelength pulses. Such one or more white wavelength pulses and one or more near-infrared wavelength pulses can be generated by the projector unit 214C. For example, in a single time frame, the pulses emitted by the projector unit 214C can include multiple wavelengths, such as one or more white wavelength pulses and one or more near-infrared wavelength pulses. The number of near-infrared wavelength pulses can be less than the number of white wavelength pulses. In one embodiment, the number of white wavelength pulses can be three times the number of near-infrared wavelength pulses in a given time frame. It is noted that this ratio of white wavelength pulses to near-infrared wavelength pulses is merely exemplary and should not be construed as limiting. In other examples, the projector unit 214C can be configured to illuminate the teeth using different numbers of white wavelength pulses and near-infrared wavelength pulses in a single time frame, to illuminate the teeth using only white wavelength pulses or near-infrared wavelength pulses in a single time frame, etc. In certain instances, the projector unit 214C can also be configured to illuminate the subject's teeth or oral cavity using visible light (e.g., corresponding to blue wavelength pulses, red wavelength pulses, and / or green wavelength pulses).

[0081] After illuminating the teeth or oral cavity, the reflected or refracted wavelength pulses can be detected by one or more sensors 214D. For example, the one or more sensors 214D can be image sensors. The one or more sensors 214D can be configured to generate a white light image 116 using the detected white light wavelength pulses and a plurality of two-dimensional near-infrared images 114 using the detected near-infrared wavelength pulses. The handheld intraoral scanner 104 can be configured to transmit the detected visible light information (or detected white light wavelength pulses) and near-infrared information (or detected near-infrared wavelength pulses) to the processor 106. Details of the operations performed by the processor 106 will be described in further detail in connection with FIGS. 2A-2B. Figure 3 Further description.

[0082] Figure 3is a schematic diagram 300 of a scanning session in accordance with an example embodiment. The schematic diagram 300 can include a user (e.g., a dentist 302 of the system 102) and a subject (e.g., a patient 304 whose teeth 306 can need treatment). The handheld intraoral scanner 104 can be available to the dentist 302 to capture a plurality of two-dimensional images 308 of the teeth 306 of the patient 304. During the scanning session, a plurality of two-dimensional images 308 of the teeth 306 of the patient 304 can be captured, e.g., including white light images (as shown by white light image 308A) and two-dimensional near-infrared images (as shown by two-dimensional near-infrared image 308B), to generate three-dimensional information 310, e.g., including a three-dimensional surface model 118 and a three-dimensional internal geometry 120 of the teeth 306 of the patient 304.

[0083] In one example scenario, the dentist 302 and the patient 304 can be present at a dental office. The dentist 302 can initiate a scanning session of the teeth 306 of the patient 304 to perform a diagnosis of the condition of the teeth 306 of the patient 304. The handheld intraoral scanner 104 can include a projector unit 214C and one or more sensors 214D. For example, the handheld intraoral scanner 104 is placed within the mouth or oral cavity of the patient 304 and moved around the teeth 306 and gums of the patient 304 to record the oral topography of the patient 304. In this regard, the projector unit 214C can illuminate the oral cavity of the patient 304 using visible (or white) light wavelength pulses and near-infrared wavelength pulses. Further, the one or more sensors (e.g., image sensors) can capture white light images and two-dimensional near-infrared images of the teeth 306.

[0084] In one example, the handheld intraoral scanner 104 can record the size and shape of each tooth, interproximal separations, palatal surface appearances, gums, implants, prosthetics, and other elements that make up the interior of the oral cavity of the patient 304. In one embodiment, the handheld intraoral scanner 104 can be moved multiple times over the teeth 306 and gums of the patient 304 to capture high-quality two-dimensional images. For example, the plurality of two-dimensional images 308 can include white light images 308A detected based on illuminating the teeth 306 with visible light wavelength pulses and two-dimensional near-infrared images 308B detected based on illuminating the teeth 306 with near-infrared wavelength pulses. It is noted that the plurality of two-dimensional images 308 can include a plurality of white light images captured using visible light wavelength pulses and a plurality of near-infrared images captured using near-infrared wavelength pulses. The plurality of two-dimensional images 308, e.g., the white light images 308A and the two-dimensional near-infrared images 308B, can be used to generate three-dimensional information 310, in particular, a three-dimensional surface model 118 and a three-dimensional internal geometry 120 of the teeth 306.

[0085] Once the scanning session begins, a plurality of two-dimensional images 308, such as a white light image 308A and a near-infrared image 308B, can be captured. The handheld intraoral scanner 104 can be configured to process the plurality of two-dimensional images 308 by applying calibration parameters and filtering noise from the plurality of two-dimensional images 308. For example, one or more calibration parameters of the sensor 214D can be applied to process the plurality of two-dimensional images 308. Further, noise can be filtered from the plurality of two-dimensional images 308. Based on the processing of the plurality of two-dimensional images 308, the handheld intraoral scanner 104 can be configured to provide a high-quality two-dimensional image to the processor 106 of the system 102. Details of the generation of the three-dimensional surface model 118 of the teeth 306 are further provided, for example, in Figure 4 .

[0086] Figure 4 A sequence diagram 400 depicting the generation of the three-dimensional surface model 118 is shown in accordance with example embodiments. Figure 4 in conjunction with the elements in Figure 1 , Figure 2 , and Figure 3 . The sequence diagram 400 can include the handheld intraoral scanner 104 and the one or more processors 106. The sequence diagram 400 can depict operations performed by at least one of the handheld intraoral scanner 104 and / or the processor 106.

[0087] In step 402, the projector unit 214C of the handheld intraoral scanner 104 can illuminate the oral cavity of the patient 304. The projector unit 214C illuminates the teeth 306 and other portions of the oral cavity using visible or white light wavelength pulses and near-infrared wavelength pulses. Details of the illumination of the oral cavity of the patient 304 and the teeth 306 are provided in detail, for example, in Figure 2 and Figure 3 .

[0088] In step 404, the one or more sensors 214D of the handheld intraoral scanner 104 can detect visible light and near-infrared light. Such detected visible light and near-infrared light can be reflected or refracted from the surface or internal regions of the teeth 306. In other words, the one or more sensors 214D can detect visible light information and near-infrared information reflected or refracted from the teeth 306.

[0089] In step 406, one or more sensors 214D of the handheld intraoral scanner 104 can capture the white light image 116 and a plurality of two-dimensional near-infrared images 114 (hereinafter NIR images 114). For example, the one or more sensors 214D can include an image sensor that can be configured to capture the white light image 116 based on detected visible light information and capture the NIR images 114 based on detected NIR information. In this way, the one or more sensors 214D can be configured to generate a plurality of 2D images 112 of the tooth 306.

[0090] In step 408, the processor 106 receives the plurality of 2D images 112 of the tooth 306. As described above, the plurality of 2D images 112 of the tooth 306 can include the white light image 116 and the NIR images 114.

[0091] In step 410, three-dimensional surface information is determined using the visible light information. In particular, the white light image 116 can be processed to determine the three-dimensional surface information. In one example, a sharp focus measurement in the white light image 116 can be determined and a projected feature of the tooth 306 can be tracked on the white light image 116. For example, a correspondence function can be performed or solved to triangulate depth information of the tooth 306 and determine 3D surface information of the tooth 306.

[0092] In step 412, a three-dimensional surface model 118 of the tooth 306 is generated based on the three-dimensional surface information. For example, a 3D patch corresponding to a portion of the surface of the tooth 306 can be generated by accessing calibration data of the one or more sensors 214D and transforming the three-dimensional surface information corresponding to the portion into real-world three-dimensional coordinates and texture information. For example, different 3D patches of different portions of the surface of the tooth 306 can be generated using the white light image 116. In some cases, there can be overlap in the portions covered by different 3D patches. For example, the 3D patch of the portion of the surface of the tooth 306 can be registered or associated with one or more previously generated 3D patches of other portions of the surface of the tooth 306 and / or overlapping portions by locating corresponding data points. Thereafter, the 3D patch of the portion can be fused with the one or more previously generated 3D patches related to other portions of the surface of the tooth 306 to generate the 3D surface model 118. For example, the 3D surface model 118 can include 3D points within a signed distance field and the signed distance field can be converted to a 3D mesh to render the 3D surface model 118. The 3D surface model 118 can also include texture data of the surface of the tooth 306.

[0093] After generating the 3D surface model 118 of the tooth 306, the processor 106 can be configured to generate a 3D internal geometry 120 of the tooth 306. The details of the generation of the 3D internal geometry can.

[0094] Figure 5 A method 500 for generating a set of input parameters for a projected object is shown in accordance with example embodiments. The set of input parameters is generated by the processor 106 using the near-infrared images 114. For example, the set of input parameters is generated for the continuum volumetric machine learning model 108. Figure 5 In conjunction with Figure 1 , Figure 2 , Figure 3 and Figure 4 elements are explained.

[0095] In step 502, a relative position between the handheld intraoral scanner 104 and the teeth 306 is estimated. Once the processor 106 receives the near-infrared images 114, the processor 106 can be configured to estimate the relative pose between the teeth 306 and the handheld intraoral scanner 104 for each near-infrared image 114.

[0096] Further, the estimated relative pose indicates, for example, perspective information and spatial position information of the handheld intraoral scanner, i.e. the position of the image sensor unit within the handheld intraoral scanner relative to the subject’s teeth at the time the corresponding near-infrared image was captured according to the near-infrared images 114. The perspective information of the near-infrared image indicates the angle formed between the handheld intraoral scanner 104 and the teeth 306 at the time the near-infrared image was captured. Further, the spatial position information indicates the position at which the position of the handheld intraoral scanner 104 relative to the teeth 306 was physically located.

[0097] For example, the projector unit 214C can illuminate the oral cavity and the teeth 306 with near-infrared wavelength pulses as well as white light wavelength pulses within a single time frame. Thus, the near-infrared images 114 can be captured together with or in between the white light images 116. In order to estimate the relative position between the handheld intraoral scanner 104 and the teeth 306 for a given near-infrared image, the pose of one or more white light images associated with the given near-infrared image can be interpolated. For example, the one or more white light images associated with the given near-infrared image can be captured during the same time frame as the given near-infrared image. Based on the pose of the one or more white light images associated with the given near-infrared image, the perspective information and the spatial position information of the given image can be determined.

[0098] In step 504, a set of input parameters is determined based on the estimated relative pose. For example, the set of input parameters includes spatial position information and viewing angle information of the casting object corresponding to each of the plurality of pixels of each near-infrared image 114. In this regard, the processor 106 can be configured to determine an object region of each near-infrared image 114. The object region in the near-infrared image can include a plurality of pixels that indicate the tooth 306 or a portion of the tooth 306 that can be captured in the corresponding near-infrared image. In one example, the object region or the plurality of pixels related to the tooth 306 in the near-infrared image can be determined by casting the 3D surface model 118 back to the near-infrared image and determining which pixels in the near-infrared image are hit by the casting. In this case, the plurality of pixels of the object region can be identified as the pixels within the pixels hit by the casting of the 3D surface model 118 back to the near-infrared image. In another example, the object region or the plurality of pixels related to the tooth 306 within the near-infrared image can be determined by performing edge detection on the near-infrared image to identify edges of the tooth 306 or the portion of the tooth 306 covered by the near-infrared image. In this case, the plurality of pixels of the object region can be identified as the pixels within the detected edges in the near-infrared image.

[0099] Once the plurality of pixels related to the tooth 306 is determined for each near-infrared image 114, a casting object can be determined for each of the plurality of pixels. For example, the casting object can be a camera ray or a cone that can cast through each of the plurality of pixels of each near-infrared image 114. Further, the casting object that casts through the pixel can include a plurality of point coordinates. Accordingly, the plurality of point coordinates of the casting object can correspond to different depths in the interior region of the tooth 306 and can be related to different materials of the interior region of the tooth 306. Based on the plurality of point coordinates of the casting object, the set of input parameters corresponding to the casting object can be determined.

[0100] For example, the set of input parameters indicating the casting object can include spatial position information (e.g., values of (x, y, z) coordinates) and viewing angle information (e.g., values of ). The set of input parameters indicating the casting object can be determined based on the estimated relative position at the time of capturing the corresponding near-infrared image. For example, the spatial position information of the casting object can indicate a spatial position of each of the plurality of point coordinates (x, y, z). Further, the viewing angle information of the casting object can indicate a viewing direction of the plurality of point coordinates or the casting object in the polar coordinates and at the time of capturing the corresponding near-infrared image. In this way, the set of input parameters can be determined for each casting object that casts through each of the plurality of pixels within the object region of each near-infrared image 114.

[0101] Figure 6 A method 600 for training a continuous volumetric machine learning model 108 is shown, in accordance with an example embodiment. The continuous volumetric machine learning model 108 can be trained based on a plurality of two-dimensional near-infrared images 114. Figure 6 In conjunction with Figure 1 , Figure 2 , Figure 3 , Figure 4 and Figure 5 elements are explained.

[0102] For example, the plurality of near-infrared images 114 can be captured by the handheld intraoral scanner 104 from different positions. In one example, the cast object can be generated from each of a plurality of pixels of the plurality of near-infrared images 114 captured from different positions of the handheld intraoral scanner. For example, an image sensor (e.g., a camera) of the handheld intraoral scanner 104 can capture a two-dimensional near-infrared image 114. Such two-dimensional pixels of the plurality of two-dimensional near-infrared images 114 can contain, for example, a portion of one or more teeth belonging to the patient 304. It is noted that the cast object can include a corresponding plurality of point coordinates. Further, a three-dimensional surface model 118 of the subject’s teeth can be generated based on the white light images.

[0103] In one example, the determined data, such as the plurality of near-infrared images 114, positions, cast objects, plurality of point coordinates, three-dimensional surface model, etc., can constitute training data for training the continuous volumetric machine learning model 108. The training data can be generated for each patient so as to train the continuous volumetric machine learning model 108 differently for each patient data.

[0104] In step 602, a set of input parameters corresponding to cast objects of the plurality of two-dimensional near-infrared images 114 is received. For example, the set of input parameters can be determined based on known or calibrated positions of a sensor (e.g., an image sensor or a camera of the handheld intraoral scanner 104 that can be used to capture the plurality of two-dimensional near-infrared images 114). It is noted that the cast objects can be cast or projected through two-dimensional pixels of the plurality of two-dimensional near-infrared images 114 containing a portion of a tooth. Further, each cast object includes a plurality of point coordinates.

[0105] For example, a three-dimensional surface model 118 of the subject can be previously generated, e.g., using a white light image 116 captured together with the two-dimensional near-infrared image 114. Further, based on the three-dimensional surface model 118, a grid point can be generated, within which an artificial pixel indicative of a corresponding point coordinate can be drawn or rendered. Further, the near-infrared camera ray or near-infrared wavelength ray is intersected with the external geometry, i.e., the three-dimensional surface model 118, to determine which portions of the camera ray or near-infrared ray (and corresponding two-dimensional pixel of the plurality of near-infrared images 114) are located within the outer boundary of one or more teeth.

[0106] The continuous volumetric machine learning model 108 can be trained to determine a synthesized pixel value of the artificial pixel for rendering the 3D internal geometry 120. Further, the set of input parameters, e.g., spatial position parameters (x, y, z) and viewing direction or perspective parameters , are determined based on a known calibration of the image sensor. For example, a plurality of 3D positions along the projection object located within the outer boundary of the corresponding tooth or within the 3D training surface model 118 may be sampled to determine a plurality of point coordinates. Further, based on the 3D positions indicative of the spatial position within the viewing direction for a plurality of point coordinate combinations, the set of input parameters for the plurality of point coordinates can be determined.

[0107] In step 604, an intensity value and a density value for each of the plurality of point coordinates are generated based on the continuous volumetric machine learning model 108. For example, the continuous volumetric machine learning model 108 can use the received set of input parameters. Based on the set of input parameters related to the point coordinates of the projection object, the continuous volumetric machine learning model 108 can be trained to determine or output an intensity value and a density value for each point coordinate of the projection object based on the corresponding input parameters . It is noted that the continuous volumetric machine learning model 108 can be trained to determine an intensity value and a density value for point coordinates of a plurality of different projection objects belonging to the two-dimensional near-infrared image 114 and related to the subject.

[0108] In step 606, a synthesized pixel value for each projection object is determined based on the corresponding determined intensity value and density value for each of the plurality of point coordinates. For example, the synthesized pixel value for the projection object can be determined as an expected intensity of the projection object along the camera ray capturing the corresponding two-dimensional pixel within the corresponding two-dimensional near-infrared image. The expected intensity may be computed based on an integration of the determined intensity value and density value of the point coordinate related to the projection object. Subsequently, the synthesized pixel value is the expected intensity value.

[0109] In step 608, a loss function between the synthetic pixel value and the corresponding true pixel values of the plurality of pixels of the plurality of two-dimensional near-infrared images 114 is minimized. For example, the synthetic pixel value generated for the cast object can indicate an intensity value for the corresponding artificial pixel.

[0110] For example, given the synthetic pixel values for the plurality of cast objects, the expected intensity values for a batch or group of computations are indicated A loss function for the synthetic pixel values can be computed based on the corresponding true pixel values as follows: wherein, is the set of cast objects, and is the ground truth intensity observed in the plurality of two-dimensional near-infrared images 114 used for training.

[0111] The generation of the loss function is based on the understanding that the cast objects in the set of cast objects do not necessarily come from the same viewpoint of the camera or image sensor. Furthermore, the loss function corresponding to the plurality of synthetic values is utilized to update the weights of the continuous volumetric machine learning model 108 and iterated through Figure 8 the training steps described until convergence or a fixed amount of iterations is performed.

[0112] To this end, the continuous volumetric machine learning model 108 can be trained every time a set of input parameters is generated for a different patient. As a result of the retraining of the continuous volumetric machine learning model 108, the synthetic pixel values can be generated for the patient such that the synthetic pixel values are user-specific. Furthermore, the training data required for training the continuous volumetric machine learning model 108 is also significantly less.

[0113] The set of input parameters is provided to the continuous volumetric machine learning model 108 for solving the continuous volumetric scene function. The operation of the continuous volumetric machine learning model 108 will be described in detail in Figure 7A connection with the following figures.

[0114] Figure 7A is a sequence diagram 700 according to an exemplary embodiment depicting the generation of the 3D internal geometry 120. The processor 106 generates the 3D internal geometry 120 using the continuous volumetric machine learning model 108. Figure 7A In connection with Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 and Figure 6The elements in the sequence diagram 700 are explained in more detail in the description of FIG. 7. The sequence diagram 700 can include the processor 106 and the continuous volume machine learning model 108. The sequence diagram 700 can depict operations performed by the processor 106 and the continuous volume machine learning model 108 of the system 102 for generating the 3D internal geometry 120 and the 3D model of the teeth 306 of the subject 304.

[0115] At step 702, a plurality of 2D near-infrared images 114 are captured. For example, the processor 106 can capture near-infrared images 114 indicative of internal regions of the teeth 306 of the subject. The near-infrared images 114 can be captured with one or more sensors 214D (in particular, image sensors) from near-infrared information received from the handheld intraoral scanner 104. For example, the near-infrared images 114 can be captured based on calibration information of the image sensors and near-infrared wavelength pulses detected by the image sensors.

[0116] At step 704, the set of input parameters of the continuous volume machine learning model 108 are determined. The set of input parameters can include values corresponding to input variables, such as spatial locations and viewing angles and viewing directions, i.e., and The processor 106 can determine the set of input parameters for each projected object corresponding to each pixel of the plurality of pixels of each near-infrared image 114. The set of input parameters can be generated to optimize a function of the continuous volume machine learning (ML) model 108. For example, the continuous volume scene function of the continuous volume ML model 108 can be optimized, Details of the generation of the set of input parameters are provided, for example, in Figure 5

[0117] At step 706, the continuous volume machine learning model 108 receives the plurality of near-infrared images 114 and the set of input parameters. For example, the processor 106 can forward the near-infrared images 114 and the set of input parameters to the continuous volume machine learning model 108. The continuous volume machine learning model 108 can be implemented as a continuous volume scene function, where the continuous volume scene function or the continuous volume machine learning model 108 is a fully connected deep neural network.

[0118] ​For example, the continuous volumetric machine learning model 108 can include a neural network model based on a neural radiance field (NeRF). In some cases, the continuous volumetric machine learning model 108 can include an enhanced version of NeRF, such as mip-NeRF. Mip-NeRF can be used to reduce aliasing, reveal fine details in 3D internal geometry, and reduce error rates. It can be appreciated that when the projection object is projected as a ray, the continuous volumetric machine learning model 108 can include a neural network model based on NeRF; while when the projection object is projected as a cone, the continuous volumetric machine learning model 108 can include a neural network model based on mip-NeRF. The advantage of using a cone-shaped projection object is that it incorporates model considerations that are embodied in the telecentric view of the intraoral scanner when capturing 2D near-infrared images. This can improve the final result.

[0119] The set of input parameters is related to a projection object that is projected through each of the plurality of pixels of the near-infrared image 114. The plurality of pixels of the near-infrared image 114 can correspond to the tooth 306, such as a portion of the tooth 306 or a point on the tooth 306. Further, the projection object (i.e., a ray or a cone) can be projected through each of the plurality of pixels of each near-infrared image 114. For example, in Figure 7B and Figure 7C Details of the type of projection object that is projected through the pixels are further provided in

[0120] In step 708, intensity values and density values are generated along each of the plurality of point coordinates of the projection object using the set of input parameters. For example, the projection object is projected from each of the plurality of pixels of each near-infrared image 114. Each projection object can include a plurality of point coordinates that can have corresponding values of, for example, spatial position and view angle or viewing direction. Further, each pixel that projects the projection object can also have corresponding values of, for example, color intensity and texture information.

[0121] The continuous volumetric machine learning model 108 can be configured to determine the intensity values and density values for each point coordinate based on the obtained near-infrared images 114 and the set of input parameters. For example, the intensity values and density values can be determined based on the input parameters (x, y, z) corresponding to the spatial position information of the point coordinate and the input parameters , the sampled point coordinates of the projection object optimize the continuous volumetric scene function . The optimized continuous volumetric scene function The generated output can be used. The one or more processors can be configured to generate grid points within the three-dimensional surface model at different viewing directions of the three-dimensional surface model. The three-dimensional surface model is determined based on the received visible light information. The model 108 receives the generated grid points and the viewing directions of the grid points, and determines intensity values and density values for each grid point and different viewing directions. Based on the intensity values and the density values, the one or more processors are configured to generate a three-dimensional internal geometry of the three-dimensional surface model by mapping the intensity values and the density values onto corresponding grid points of the three-dimensional surface model at corresponding viewing directions. The three-dimensional surface model containing the three-dimensional internal geometry can then be displayed. Based on user input, the one or more processors are configured to zoom in on the three-dimensional surface model so as to closely inspect the three-dimensional internal geometry of the three-dimensional surface model. Based on user input, the one or more processors are configured to rotate the three-dimensional surface model in a manner that the three-dimensional internal geometry is similarly rotated.

[0122] In step 710, a synthetic pixel value of the cast object is determined. For example, the intensity value and the density value of each of the plurality of point coordinates of the cast object can be integrated to determine a synthetic pixel value corresponding to the cast object. The determined synthetic pixel value 122 can be for a plurality of cast objects corresponding to a plurality of pixels of each near-infrared image 114. The synthetic pixel value of the cast object can be indicative of pixel information corresponding to an artificial pixel, such as color, depth, transparency, position, texture, and the like. Such artificial pixel can be cast as an artificial point in the three-dimensional internal geometry of the subject’s teeth 306. For example, for a two-dimensional pixel in the near-infrared image 114, a plurality of artificial points or artificial pixels can be determined based on processing the near-infrared image 114 using the continuum volumetric machine learning model 108.

[0123] In step 712, the processor 106 can be configured to generate grid points within the three-dimensional surface model 118 at different viewing directions of the three-dimensional surface model 118. It is noted that the processor 106 can generate the three-dimensional surface model 118 of the subject’s teeth 306 using the visible light information and / or the white light image 116 received from the handheld intraoral scanner 104. The generated grid points and the viewing directions are fed into the continuum volumetric machine learning model 108, which is then configured to determine intensity values and density values for the corresponding grid points and forward the values to the one or more processors 106. The one or more processors 106 are then configured to generate a three-dimensional internal geometry of the three-dimensional surface model 118 based on the intensity values and the density values of the corresponding grid points.

[0124] In one embodiment, the intensity values and the density values of the plurality of point coordinates can be indicative of properties of the material within the pixel of the near-infrared image at the plurality of coordinate points.

[0125] Further, the processor 106 can be configured to identify a distinction between enamel and dentin and a boundary between enamel and dentin within the three-dimensional internal geometry 120 of the tooth 306 based on the determined intensity values and density values that have been determined based on the grid points within the three-dimensional surface model and observations of the grid points. The processor 106 can identify a change in material in the three-dimensional internal geometry based on changes in intensity values and density values between the grid points. Changes in intensity values and density values that exceed a certain change threshold can correspond to a change in material. The one or more processors can be configured to determine whether the intensity values and density values correspond to a dental feature, such as an anatomical feature, a disease feature, or a mechanical feature. The anatomical feature can be enamel, dentin, or pulp. The disease feature can be a crack or a cavity. The mechanical feature can be a filling and / or a composite restoration.

[0126] To this end, the three-dimensional internal geometry 120 can be generated within the three-dimensional surface model 118 such that the three-dimensional internal geometry 120 identifies different materials in the dentition of the subject’s tooth 306. The different materials can correspond to enamel and dentin, but are not limited thereto. In some instances, the different materials can correspond to any prosthetic implant or any filling or material that can be inserted within a tooth, or artificial tooth.

[0127] Referring to Figure 7B , a schematic diagram 720 of a first type of cast object is shown in accordance with an example embodiment. Referring to Figure 7C , a schematic diagram 730 of a second type of cast object is shown in accordance with another example embodiment. For example, the first type of cast object is a ray, and the second type of cast object is a cone. For example, the near-infrared image 718 can be an image in the near-infrared image 114 captured by the processor 106 using the image sensor and the near-infrared information. Upon receiving the near-infrared image 718, the processor 106 can be configured to determine an object region, i.e., pixels that delineate portions of the subject’s tooth 306. For example, a plurality of pixels within the near-infrared image 718 that indicate the object region, i.e., the tooth 306, can be identified.

[0128] Further, the processor 106 can be configured to cast or render a cast object from each of the plurality of pixels from the near-infrared image 718 and other near-infrared images 114 captured by the processor 106. The cast object can be cast from the plurality of pixels to determine a set of input parameters for optimizing the continuous volumetric machine learning model 108. In accordance with the present example, the entire near-infrared image 718 can be rendered by rendering or casting the cast object through each of the plurality of pixels.

[0129] With particular reference to Figure 7BA ray-based projection object 724 (hereinafter referred to as ray 724) can be cast or projected through a pixel 722 of near-infrared image 718. Ray 724 can include a plurality of point coordinates, such as point coordinates 726A, 726B, 726C, 726D, and 726E (hereinafter collectively referred to as point coordinates 726) as shown in the figure. For example, each point coordinate 726 along ray 724 can need to be transformed using a positional encoding, such as using a gamma function. Point coordinates 726 can be sampled along ray 724. Spatial position information and viewing angle information corresponding to each transformed point coordinate is used to determine the set of input parameters for optimizing a continuous volumetric scene function of continuous volumetric machine learning model 108.

[0130] With particular reference to Figure 7C A cone-based projection object 728 (hereinafter referred to as cone 728) can be cast or projected through a pixel 722 of near-infrared image 718. For example, a radius of cone 728 indicating the projection object corresponding to pixel 722 can be determined based on the size of the corresponding pixel 722 on the image plane.

[0131] For example, processor 106 can be configured to determine an average of the content within visible volume 734 of pixel 722. For example, the content within visible volume 734 of pixel 722 can be indicative of the color intensity of pixel 722. Further, the average of the color intensity within visible volume 734 can enable identification of the presence of, for example, a portion or point of tooth 306, a cavity, a lesion, a crack, dentin, an enamel junction, a filling, or any other object present on or inside the point of tooth 306 captured by pixel 722. For example, processor 106 can be configured to render or cast cone 728 as a projection object corresponding to pixel 722 based on the corresponding average of the content. Further, cone 728 can be rendered such that cone 728 models the entire spatial volume that is visible through pixel 722 based on the average of the content or color intensity within visible volume 734.

[0132] Cone 728 can include a plurality of point coordinates, depicted as point coordinates 732A, 732B, 732C, 732D, and 732E (hereinafter collectively referred to as point coordinates 732). Further, cone 728 can be cut into conic frustums corresponding to point coordinates 732. For example, conic frustum 736 can correspond to point coordinate 732D. To this end, each point coordinate 732 along cone 728 can be transformed using a positional encoding of the volume of the corresponding conic frustum. Point coordinates 732 can be sampled along cone 728. Spatial position information and viewing angle information corresponding to each transformed point coordinate is used to determine the set of input parameters for optimizing a continuous volumetric scene function of continuous volumetric machine learning model 108, where continuous volumetric machine learning model 108 is a mip-NeRF based neural network. For example in Figure 8Details of this set of input parameters for generating a cone-based projected object are further described.

[0133] In conjunction Figure 7A and Figure 8 The detailed description describes the way the continuous volumetric machine learning model 108 uses this set of input parameters corresponding to the projected object.

[0134] Figure 7D A schematic diagram of a generated 3D model 740 of the subject’s teeth 306 is shown in accordance with an example embodiment. For example, the processor 106 can be configured to generate the 3D model 740. The 3D model 740 includes a 3D surface model 118 and a 3D internal geometry 120 of the teeth 306. It is noted that the 3D internal geometry 120 can be generated by rendering mesh points within the 3D surface model 118 at different viewing directions of the 3D surface model 118. Further, the intensity and density of the corresponding mesh points are determined based on the continuous volumetric machine learning model.

[0135] The 3D model 740 of the teeth 306 shows the geometry of the internal regions of the teeth 306. This enables a user or a dentist to identify any disease, abnormality, or irregularity within the surface of the teeth 306. In particular, the 3D model 740 provides an accurate representation of the condition of different layers within the surface of the teeth 306. For example, the 3D model can be used to identify caries, cracks, lesions, abnormal hyperplasia, fillings, boundaries between enamel and dentin, etc. within the surface of the teeth 306.

[0136] For example, the 3D model 740 can include a 3D internal geometry 120 rendered within the 3D surface model 118. Based on the 3D internal geometry 120, the 3D model 740 can represent a portion of the teeth 306 corresponding to the enamel 742, a portion of the teeth 306 corresponding to the dentin 744, and a boundary 746 or interface between the enamel 742 and the dentin 744 in the dentition of the teeth 306.

[0137] Figure 8 A method 800 for generating the synthetic pixel values 122 using the continuous volumetric machine learning model 108 is shown in accordance with an example embodiment. The method 800 describes the generation of the synthetic pixel values 122 when the projected projected object corresponds to a second type of projected object, such as the cone 728. In this case, the continuous volumetric machine learning model 108 can be implemented using, for example, a mip-NeRF based neural network model. FIG. 7 combines Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6 , Figure 7A , Figure 7B ,Figure 7C and Figure 7D The elements in

[0138] In step 802, for each cone corresponding to each pixel in the plurality of pixels, one or more frustums are determined. For example, a frustum 736 can be determined for the cone 728 corresponding to the pixel 722. The frustum 736 can be related to the point coordinate 732D in the plurality of point coordinates 732 of the cone 728. In one example, the processor 106 can determine the frustum 736 such that the frustum covers a visible volume around the point coordinate 732D. It is noted that a similar frustum can be determined for each point coordinate 732 of the cone 728.

[0139] In step 804, an integrated positional encoding of the one or more frustums is determined for transforming the point coordinates 732 of the cone 728. The integrated positional encoding of the frustum of the point coordinates 732 can include a Gaussian encoding and / or a sinusoidal encoding.

[0140] For example, fitting a multivariate Gaussian distribution in the positional encoding of the frustum 736 can analytically reduce the complexity of manipulating the positional encoding. Further, an expected integrated positional encoding can be determined based on the Gaussian fitted frustum 736 and each of the other Gaussian fitted frustums corresponding to, for example, the point coordinates 732 of the cone 728. Similarly, an expected integrated positional encoding of a cone projected from each of the plurality of pixels in the plurality of near-infrared images 114 can be determined. The integrated positional encoding of the frustum of the cone 728 can have characteristics of a sinusoidal function and a Gaussian function.

[0141] In step 806, a synthesized pixel value for each cone is generated based on the integrated positional encoding of the corresponding one or more frustums and the continuous volumetric machine learning model 108.

[0142] For example, the continuous volumetric machine learning model 108 can be configured to vary a width of the Gaussian fitting of the frustum in the integrated positional encoding at a higher frequency to determine the synthesized pixel value 122. For example, as the width widens, the integrated positional encoding can approach zero, indicating a small volume of the cone; and as the width narrows, the integrated positional encoding can increase towards a non-zero value, indicating a large volume of the cone. Thus, the MIP-Nerf based continuous volumetric machine learning model 108 is able to infer a scale of the input near-infrared image 114 by observing a scale of the integrated positional encoding of the projected cone. The continuous volumetric ML model 108 can distinguish between a small visible volume and a large visible volume corresponding to the cone.

[0143] Figure 9A method 900 for generating a three-dimensional internal geometry 120 and a three-dimensional surface model 118 of a subject's teeth 306 according to an exemplary embodiment is shown. Figure 9 In conjunction Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6 , Figure 7A , Figure 7B , Figure 7C , Figure 7D and Figure 8 are explained. The steps of the method 900 can be performed by the system 102. As previously mentioned, the system 102 includes the handheld intraoral scanner 104, the processor 106, and the continuous volumetric machine learning model 108. The handheld intraoral scanner 104 includes the projector unit 214C and the one or more sensors 214D. The projector unit 214C can illuminate the subject's teeth 306 using visible wavelength pulses and near-infrared wavelength pulses. The one or more sensors 214D can detect the near-infrared light and the visible light. For example, the one or more sensors 214D can include an image sensor that can be configured to capture a white light image 116 based on detecting the visible light and a near-infrared image 114 based on detecting the near-infrared light. Further, the processor 106 can be configured to generate the three-dimensional internal geometry 120 of the teeth 306 using the continuous volumetric machine learning model 108.

[0144] In step 902, visible light information and near-infrared information are received from the one or more sensors 214D. The visible light information and the near-infrared information can respectively indicate the visible wavelength pulses and the near-infrared wavelength pulses that can be detected by the one or more sensors 214D. For example, the detected visible wavelength pulses and the near-infrared wavelength pulses can be reflected from the subject's teeth 306. In some cases, the visible light information and the near-infrared information can also include calibration information related to the one or more sensors 214D based on which the visible wavelength pulses and the near-infrared wavelength pulses are detected. For example, the visible light information and the near-infrared information can also respectively include the white light image 116 and the near-infrared image 114.

[0145] In step 904, surface information is determined in real-time. For example, the processor 106 can be configured to generate surface information of the teeth 306 from the visible light information or the white light image 116. Based on the surface information, the processor 106 can be configured to generate a three-dimensional surface model 118 of the subject's teeth 306. In conjunction Figure 4 Details of generating the 3D surface model 118 are provided.

[0146] In step 906, a plurality of 2D near-infrared images 114 of the internal region of the subject’s tooth 306 can be captured according to the near-infrared information. For example, the processor 106 can capture the near-infrared images 114 using an image sensor of the one or more sensors 214D. It is noted that each near-infrared image 114 can contain a corresponding set of pixels. In one example, a plurality of pixels of each near-infrared image 114 corresponding to a portion or a point on the tooth 306 can be determined for further processing.

[0147] In step 908, a set of input parameters is determined. In this regard, a projection object can be projected from each of the plurality of pixels of each near-infrared image 114. The projection object can contain a corresponding plurality of point coordinates. The plurality of point coordinates correspond to different depths within the corresponding projection object and the corresponding pixel. For example, the plurality of point coordinates can correspond to different materials within the subject’s tooth 306. Thus, the plurality of point coordinates of the corresponding projection object can not belong to the same material. Further, the processor 106 can be configured to determine the set of input parameters of the plurality of point coordinates of the projection object projected from the plurality of pixels of the near-infrared image 114. For example, the set of input parameters can include spatial position information (e.g., values of (x, y, z)) and perspective information (e.g., values of Based on the set of input parameters, an optimization process is applied to all projection objects of the plurality of pixels, and a continuous volumetric scene function can be defined.

[0148] In step 910, the set of input parameters is processed using the continuous volumetric machine learning model 108. For example, the continuous volumetric machine learning model can process the set of input parameters to determine a synthesized pixel value of the projection object based on the intensity values and the density values of the point coordinates. The continuous volumetric machine learning model 108 is configured to be trained using the plurality of near-infrared images 114. Details of the training of the continuous volumetric machine learning model 108 are described in connection with, for example, Figure 8

[0149] ​It should be appreciated that each of the steps of the sequence diagram 900 can be implemented in various ways, such as in hardware, firmware, a processor, circuit, and / or other communication means related to the execution of software containing one or more computer program instructions. For example, one or more of the steps described above can be implemented by computer program instructions. In this regard, the computer program instructions implementing the steps described above can be stored in a memory of the system 102 employing embodiments of the present disclosure. It should be understood that any such computer program instructions can be loaded onto a computer or other programmable apparatus (e.g., hardware) to produce a machine, such that the resulting computer or other programmable apparatus implements the functions specified in the sequence diagram 1000. These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture that implements the functions specified in the sequence diagram 900. The computer program instructions can also be loaded onto a computer or other programmable apparatus to cause a series of operations to be performed on the computer or other programmable apparatus, thereby producing a computer-implemented process such that the instructions that execute on the computer or other programmable apparatus provide operations for implementing the functions specified in the sequence diagram 900.

[0150] Accordingly, the steps of the sequence diagram 900 support a combination of means for performing the specified functions and a combination of operations for performing the specified functions. It should also be understood that one or more of the steps in the sequence diagram 900, and combinations of steps in the sequence diagram 900, can be implemented by a special purpose hardware-based computer system which performs the specified functions, or combinations of special purpose hardware and computer instructions. Figure 9 The illustrated sequence diagram 900 is for generating and rendering a 3D model of a tooth that includes a three-dimensional internal geometry of the tooth. Fewer, more, or different steps can be provided.

[0151] Figure 10 A schematic diagram 1000 is shown that depicts an example environment for generating a three-dimensional internal geometry 120 and a three-dimensional surface model 118 of a tooth in real-time and rendering an interactive three-dimensional graphical representation 1010 in accordance with example embodiments. Figure 10 In conjunction with Figure 1 , Figure 2 , Figure 3 , Figure 4 , Figure 5 , Figure 6 , Figure 7A , Figure 7B , Figure 7C , Figure 7D , Figure 8 and Figure 9 The elements of the schematic diagram 1000 can be explained in conjunction with the elements of the sequence diagram 900. The schematic diagram 1000 can include a dentist 1002 and a patient 1004.

[0152] The dentist 1002 can use the handheld intraoral scanning device 104 to capture a plurality of 2D images 112 of the teeth 1006 of the patient 1004, including near-infrared images 114 and white light images 116. The handheld intraoral scanning device 104 can transmit the plurality of 2D images 112 to the processor 106. The processor 106 can process the white light images 116 and generate three-dimensional surface information of the teeth 1006. Based on the three-dimensional surface information, the processor 106 can generate a 3D surface model 118 of the teeth 1006 of the patient 1004.

[0153] Further, the processor 106 can process the near-infrared images 114 to generate a set of input parameters for optimizing a function of the continuous volume machine learning model 108. The set of input parameters is generated for each cast object cast through each of a plurality of pixels in each near-infrared image 114. The cast object can be a ray or a cone. For example, the set of input parameters can be used to optimize a continuous volume scene function of the continuous volume machine learning model 108. Given the optimized continuous volume scene function with the set of input parameters , the continuous volume machine learning model 108 can generate an intensity value and a density value for each of a plurality of point coordinates of the cast object. Further, the continuous volume machine learning model 108 can generate a composite pixel value of the cast object, for example, by integrating the intensity value and the density value of the plurality of point coordinates. In this way, a plurality of composite pixel values 122 of the plurality of cast objects corresponding to the plurality of pixels in each near-infrared image 114 can be determined.

[0154] The 3D model of the teeth 1006 of the patient 1004 can be displayed in real-time on the display unit 1008 as an interactive three-dimensional graphical representation 1010. Further, the interactive three-dimensional graphical representation 1010 can be generated in real-time and can be manipulated by the dentist 1002 as needed, for example, rotated or viewed at multiple viewing angles.

[0155] Therefore, the intraoral scanning system 102 may enable the processing of multiple 2D images 112 to generate a three-dimensional surface model 118 and a three-dimensional internal geometry 120 of the tooth 1006. The three-dimensional internal geometry 120 can display the condition, shape, size, and other characteristics of the internal regions of the tooth 1006. The generated three-dimensional internal geometry 120 accurately identifies different materials, such as enamel and dentin, within the internal regions of the tooth 1006 based on a comparison of the intensity and / or density values ​​of the projected objects with a variation threshold. For example, if the difference between a first intensity or density value of a first projected object and a second pixel value of a second projected object is greater than a variation threshold, the first and second projected objects can be identified as belonging to different materials. In this way, the boundary between the enamel and dentin within the tooth 1006 of the patient 1004 can be identified in the three-dimensional internal geometry 120.

[0156] Users (e.g., dentists) can access an interactive 3D graphical representation 1010 of a 3D model of a tooth 1006 via a display unit. Therefore, the intraoral scanning system 102 can process multiple 2D images 112 in real time and generate a 3D model for the user to access in real time via the display unit.

[0157] Based on the foregoing description and accompanying drawings, those skilled in the art will conceive of numerous modifications and other embodiments of the present invention. Therefore, it should be understood that the present invention is not limited to the specific embodiments disclosed, and modifications and other embodiments should be included within the scope of the appended claims. Furthermore, although the foregoing description and drawings depict exemplary embodiments of certain combinations of elements and / or functions, it should be understood that other embodiments may provide different combinations of elements and / or functions without departing from the scope of the appended claims. In this regard, for example, other combinations of elements and / or functions besides those explicitly described above, as set forth in some of the appended claims, may also be considered. Although specific terminology has been used herein, it is for general and descriptive purposes only and not for limiting purposes.

[0158] Parts list: Intraoral scanning system – 102 Handheld intraoral scanner – 104 Processor – 106 Continuous volume machine learning model – 108 Two-dimensional images – 112, 308 Two-dimensional near-infrared images – 114, 308B, 718 Two-dimensional white light image – 116, 308A 3D Surface Model – 118 Three-dimensional internal geometry – 120 Dentist – 302, 1002 Patient - 304, 1004 Subject's teeth - 306, 1006 Three-dimensional information - 310 Pixel - 722 Raycast object - 724 Point coordinates - 726A, 726B, 726C, 726D, 726E, 732A, 732B, 732C, 732D, and 732E Conecast object - 728 Visible volume - 734 Cone frustum - 736 3D model - 740 Enamel - 742 Dentin - 744 Boundary - 746 Display unit - 1008 Interactive three-dimensional graphical representation - 1010 List of items: 1. An intraoral scanning system (102) configured to generate a three-dimensional (3D) internal geometry (120) of a subject's teeth (306), the system comprising: a handheld intraoral scanner (104) configured to operate with one or more sensors (214D) to detect near-infrared (NIR) and visible light, wherein the one or more sensors include an image sensor; one or more processors (106) operably connected to the handheld intraoral scanner, the one or more processors configured to: receive visible light information and near-infrared (NIR) information from the one or more sensors; determine surface information in real-time from the visible light information to generate a three-dimensional (3D) surface model (118) of the subject's teeth using the surface information; capture a plurality of two-dimensional (2D) NIR images (114) of an internal region of the subject's teeth in real-time from the NIR information using the image sensor, wherein each of the plurality of NIR images contains a plurality of corresponding pixels (722); and determine a set of input parameters for a projection object corresponding to each of the plurality of pixels, wherein the set of input parameters includes spatial position information and viewing angle information, and the projection object includes a plurality of point coordinates (726, 732); and a continuous volumetric machine learning model (108) configured to receive and process the set of input parameters to determine intensity values and density values of a three-dimensional internal geometry of the 3D surface model, wherein the continuous volumetric machine learning model is configured to be trained using the plurality of 2D NIR images.

[0159] 2. The intraoral scanning system (102) of item 1, wherein the continuous volumetric machine learning model (108) is trained by: receiving (602) the set of input parameters corresponding to a projected object of each of the plurality of 2D NIR images (114), wherein the projected object comprises a plurality of point coordinates (726, 732) associated with the 3D surface model (118); generating (804), based on the continuous volumetric machine learning model, intensity values and density values for each of the plurality of point coordinates using the set of input parameters; determining (606) a synthesized pixel value of the projected object based on the corresponding determined intensity values and density values for each of the plurality of point coordinates; and minimizing (608) a loss function between the synthesized pixel value and corresponding true pixel values of the plurality of pixels (722) of the plurality of 2D NIR images by varying the intensity values and density values for each of the plurality of point coordinates.

[0160] 3. The intraoral scanning system (102) of any of the preceding items, wherein the one or more processors (106) are further configured to: determine a plurality of grid points within the 3D surface model (118); and determine the 3D internal geometry (120) by arranging at least one of: the intensity values or the density values at each of the plurality of grid points.

[0161] 4. The intraoral scanning system (102) of any of the preceding items, further comprising a display unit (1008) configured to display the 3D internal geometry (120) based on at least one of: the intensity values or the density values determined by the continuous volumetric machine learning model (108).

[0162] 5. The intraoral scanning system (102) of item 4, wherein the display unit (1008) is configured to display the 3D internal geometry (120) inside the 3D surface model (118).

[0163] 6. The intraoral scanning system (102) of any of the preceding items, wherein the one or more processors (106) are further configured to: Based on the determined intensity values and density values of the projection objects for each of the plurality of pixels (722), a boundary (746) between enamel (742) and dentin (744) in a dentition of the subject's teeth (306) is determined.

[0164] 7. The intraoral scanning system (102) of item 6, wherein the one or more processors (106) are further configured to: determine the boundary (746) based on a change in at least one of the intensity values or the density values of the at least two projection objects of the plurality of pixels (722) being above a change threshold.

[0165] 8. The intraoral scanning system (102) of any of the preceding items, wherein the projection objects are one of a ray (724) or a cone (728).

[0166] 9. The intraoral scanning system (102) of any of the preceding items, wherein: the handheld intraoral scanner (104) comprises a projector unit (214C) configured to illuminate the subject's teeth (306) using one or more white wavelength pulses and one or more near-infrared (NIR) wavelength pulses; and the one or more sensors (214D) are configured to generate a set of white light images (116) and a plurality of 2D NIR images (114) based on the illumination.

[0167] 10. The intraoral system (102) of item 9, wherein the three-dimensional surface model (118) is determined based on the set of white light images (116).

[0168] 11. The intraoral scanning system (102) of any of the preceding items, wherein the one or more processors (106) are further configured to: estimate a relative position between the handheld intraoral scanner (104) and the subject's teeth (306) corresponding to each of the plurality of NIR images (114), wherein the estimated relative position indicates perspective information and spatial position information of the corresponding projection object.

[0169] 12. The intraoral scanning system (102) of any of the preceding items, wherein the projection objects are cones (728), the one or more processors are further configured to: determine (802) one or more conical frustums (736) of each cone corresponding to each of the plurality of pixels (722), wherein the one or more conical frustums are related to the plurality of point coordinates (732); determine (804) an integral position encoding of the one or more conical frustums for transforming the plurality of point coordinates of each cone, the integral position encoding comprising at least a Gaussian encoding and a sinusoidal encoding; and Based on the integral position encoding of the corresponding one or more frustums and the continuous volumetric machine learning model (108), generate (806) a composite pixel value for each frustum.

[0170] 13. The intraoral scanning system (102) of item 12, wherein the one or more processors (106) are further configured to: determine, based on a size of the corresponding pixel of the plurality of pixels, a radius of each frustum (728) indicative of the projected object corresponding to each of the plurality of pixels (722).

[0171] 14. The intraoral scanning system (102) of item 12, wherein the one or more processors (106) are further configured to: determine an average value of content within the visible volume (734) for each of the plurality of pixels (722), wherein the content is indicative of color intensity; and render, based on the average value of the corresponding content, each frustum (728) indicative of the projected object for each of the plurality of pixels.

[0172] 15. The intraoral scanning system (102) of any of the preceding items, wherein the continuous volumetric machine learning model (108) is a machine learning based Neural Radiance Field (NeRF) network.

[0173] 16. A method (900) for generating a three-dimensional (3D) internal geometry (120) of a subject’s teeth (306) using an intraoral scanning system (102), the intraoral scanning system comprising: a handheld intraoral scanner (104) configured to operate with one or more sensors (214D) to detect near-infrared (NIR) and visible light, wherein the one or more sensors include an image sensor; one or more processors (106) operably connected to the handheld intraoral scanner; and a continuous volumetric machine learning model (108); wherein the method comprises: receiving (902) visible light information and near-infrared (NIR) information from the one or more sensors; determining (904) surface information in real-time from the visible light information to generate a three-dimensional (3D) surface model (118) of the subject’s teeth using the surface information; capturing (906) a plurality of two-dimensional (2D) NIR images (114) of an internal region of the subject’s teeth in real-time using the image sensor from the NIR information, wherein each of the plurality of NIR images contains a plurality of corresponding pixels (722); determining (908) a set of input parameters of a projection object corresponding to each of the plurality of pixels, wherein the set of input parameters comprises spatial position information and viewing angle information, and the projection object comprises a plurality of point coordinates (726, 732); and processing (910) the set of input parameters using a continuous volumetric machine learning model to determine intensity values and density values of a three-dimensional internal geometry of the three-dimensional surface model, wherein the continuous volumetric machine learning model is configured to be trained using the plurality of 2D NIR images.

[0174] 17. The method of item 17, further comprising: determining a plurality of grid points within the three-dimensional surface model (118); determining the three-dimensional internal geometry (120) by arranging at least one of the following based on the set of input parameters: intensity values or density values at each of the plurality of grid points; and displaying the three-dimensional internal geometry within the three-dimensional surface model.

[0175] 18. The method of any one of the preceding items, further comprising: determining a boundary (746) between enamel (742) and dentin (744) in a dentition of the subject’s teeth (306) based on at least one of the following: intensity values or density values of at least two projection objects of the plurality of pixels being above a change threshold.

[0176] 19. A computer-program product, comprising a transitory computer-readable medium having stored thereon computer-executable instructions that, when executed by a processing circuit, cause the processing circuit to perform operations comprising: receiving visible light information and near-infrared (NIR) information from one or more sensors (214D) mounted within a handheld intraoral scanner (104), the one or more sensors configured to detect near-infrared (NIR) and visible light, wherein the one or more sensors comprise an image sensor; determining surface information in real-time from the visible light information to generate a three-dimensional (3D) surface model (118) of the subject’s teeth (306) using the surface information; capturing a plurality of two-dimensional (2D) NIR images (114) of an internal region of the subject’s teeth in real-time from the NIR information using the image sensor, wherein each of the plurality of NIR images comprises a plurality of corresponding pixels (722); determining a set of input parameters of a projection object corresponding to each of the plurality of pixels, wherein the set of input parameters comprises spatial position information and viewing angle information, and the projection object comprises a plurality of point coordinates (726, 732); and processing the set of input parameters using a continuous volumetric machine learning model (118) to determine intensity values and density values of a three-dimensional internal geometry (120) of the three-dimensional surface model, wherein the continuous volumetric machine learning model is configured to be trained using a plurality of two-dimensional near-infrared images.

Claims

1. An intraoral scanning system (102) configured to generate a three-dimensional (3D) internal geometry (120) of a subject’s teeth (306), the intraoral scanning system comprising: a handheld intraoral scanner (104) configured to operate with one or more sensors (214D) to detect near-infrared (NIR) and visible light, wherein the one or more sensors include an image sensor; one or more processors (106) operably connected to the handheld intraoral scanner, the one or more processors configured to: receive visible light information and near-infrared (NIR) information from the one or more sensors; determine surface information from the visible light information in real-time to generate a three-dimensional (3D) surface model (118) of a subject’s teeth using the surface information; capture a plurality of two-dimensional (2D) NIR images (114) of an internal region of a subject’s teeth from the NIR information in real-time using the image sensor, wherein each of the plurality of NIR images contains a plurality of corresponding pixels (722); and determine a set of input parameters corresponding to each of the plurality of pixels for a projected object, wherein the set of input parameters includes spatial position information and viewing angle information, and the projected object includes a plurality of point coordinates (726, 732); and a continuous volumetric machine learning model (108) configured to receive and process the set of input parameters to determine intensity values and density values for a three-dimensional internal geometry of the 3D surface model, wherein the continuous volumetric machine learning model is configured to be trained using the plurality of 2D NIR images.

2. The intraoral scanning system (102) according to claim 1, wherein the continuous volumetric machine learning model (108) is trained by: receiving (602) the set of input parameters corresponding to a projected object for each of the plurality of 2D NIR images (114), wherein the projected object includes a plurality of point coordinates (726, 732) associated with the 3D surface model (118); generating (804) intensity values and density values for each of the plurality of point coordinates using the set of input parameters based on the continuous volumetric machine learning model; determining (606) a synthesized pixel value for the projected object based on the corresponding determined intensity values and density values for each of the plurality of point coordinates; and minimizing (608) a loss function between the synthesized pixel value and corresponding true pixel values for the plurality of pixels (722) of the plurality of 2D NIR images by varying the intensity values and density values for each of the plurality of point coordinates.

3. The intraoral scanning system (102) according to any one of the preceding claims, wherein, the one or more processors (106) are further configured to: determine a plurality of grid points within the 3D surface model (118); and determine the 3D internal geometry (120) by arranging at least one of: intensity values or density values at each of the plurality of grid points.

4. The intraoral scanning system (102) of any of the preceding claims, further comprising a display unit (1008) configured to display the 3D internal geometry (120) based on at least one of: intensity values or density values determined by the continuous volume machine learning model (108).

5. The intraoral scanning system (102) according to claim 4, wherein The display unit (1008) is configured to display the 3D internal geometry (120) inside the 3D surface model (118).

6. The intraoral scanning system (102) according to any one of the preceding claims, wherein The one or more processors (106) are further configured to: determine a boundary (746) between an enamel (742) and a dentin (744) in a dentition of the subject’s teeth (306) based on the determined intensity values and density values of the projection object for each of the plurality of pixels (722).

7. The intraoral scanning system (102) according to claim 6, wherein The one or more processors (106) are further configured to: determine the boundary (746) based on a change in at least one of: the intensity values or the density values of at least two projection objects of the plurality of pixels (722) being above a change threshold.

8. The intraoral scanning system (102) according to any one of the preceding claims, wherein, The projection object is one of a ray (724) or a cone (728).

9. The intraoral scanning system (102) of any of the preceding claims, wherein: the handheld intraoral scanner (104) comprises a projector unit (214C) configured to illuminate the subject’s teeth (306) using one or more white wavelength pulses and one or more near-infrared (NIR) wavelength pulses; and the one or more sensors (214D) are configured to generate a set of white light images (116) and the plurality of 2D NIR images (114) based on the illumination.

10. The intraoral system (102) of claim 9, wherein, The 3D surface model (118) is determined based on the set of white light images (116).

11. The intraoral scanning system (102) according to any one of the preceding claims, wherein, The one or more processors (106) are further configured to: estimate a relative position between the handheld intraoral scanner (104) and the subject’s teeth (306) corresponding to each of the plurality of NIR images (114), wherein the estimated relative position indicates perspective information and spatial position information of the corresponding projection object.

12. The intraoral scanning system (102) according to any one of the preceding claims, wherein, The projection object is a cone (728), the one or more processors are further configured to: determine (802) one or more frustums (736) of each cone corresponding to each of the plurality of pixels (722), wherein the one or more frustums are related to the plurality of point coordinates (732); determine (804) integral position encodings of the one or more frustums for transforming the plurality of point coordinates of each cone, the integral position encodings comprising at least a Gaussian encoding and a sinusoidal encoding; and generate (806) a synthesized pixel value for each cone based on the integral position encodings of the corresponding one or more frustums and the continuous volume machine learning model (108).

13. The intraoral scanning system (102) according to claim 12, wherein, The one or more processors (106) are further configured to: determine a radius of each cone (728) indicating the projection object corresponding to each of the plurality of pixels (722) based on a size of the corresponding pixel of the plurality of pixels.

14. The intraoral scanning system (102) according to claim 12, wherein, The one or more processors (106) are further configured to: determine a mean value of content within a visible volume (734) of each of the plurality of pixels (722), wherein the content is indicative of color intensity; and render each cone (728) indicative of the projected object of each of the plurality of pixels based on the corresponding mean value of content.

15. The intraoral scanning system (102) according to any one of the preceding claims, wherein, The continuous volumetric machine learning model (108) is a machine learning based Neural Radiance Field (NeRF) network.

Images