Device for estimating the three-dimensional structure of teeth

Abstract

Device for estimating the three-dimensional structure of teeth inside a mouth, comprising : - a central control unit, - a tray insertable inside the mouth, - a plurality of independent cameras arranged on the tray, each camera being equipped with a corresponding fixed focus lens, having a corresponding field of view and being designed to capture a corresponding image of the teeth within the field of view and to transmit the image to the central control unit, each image being composed by a plurality of pixels, and - illumination means arranged on the tray and designed to illuminate the fields of view of the cameras, wherein the central control unit is programmed to estimate the three-dimensional structure of the teeth by estimating independently for each image a depth value for each pixel thereof.

Description

[0001]DEVICE FOR ESTIMATING THE THREE-DIMENSIONAL STRUCTURE OF TEETH ------ The present invention relates to the technical field of dental technology, and refers to a device for estimating the three-dimensional structure of teeth inside a mouth. In the dental industry, the necessity of obtaining an estimate of the three-dimensional structure of one or more teeth of a patient is well-known, for example in order to manufacture dental appliances (such as dental braces, clear aligners, retainers, implants, dentures and other prosthetics) especially suited for the patient, or in order to help a dentist to diagnose dental diseases and devise corresponding treatment plans. Traditionally, the dental industry has relied heavily on physical dental impressions for obtaining an estimate of the three-dimensional structure of the teeth of a patient. These dental impressions are usually obtained by placing a semi-solid impression material (such as alginate) on a tray shaped so as to fit over the dental arch of the patient, and then placing the tray in the mouth over the arch: as soon as the impression material solidifies, it bears the physical negative imprint of the dental arch, from which a positive reproduction (such as a cast) can be formed obtaining the estimate of the three-dimensional structure of the dental arch. This solution, however, is relatively invasive, since the tray should be kept in the mouth for several minutes in order to allow the impression material to completely solidify, creating discomfort for the patient and the risk of gag reflex. In addition, the retrieval of the impression material without deformation is a complicated operation which is difficult to perform properly and requires specific training. Finally, each physical dental impression requires a fresh amount of impression material, the preparation of which is time consuming and relatively expensive. As an alternative, digital dental impressions are becoming more and more common in the dental industry. This technology involves the use of an intra-oral scanner which is inserted into the mouth of the patient and captures a set of images that are used to reconstruct the three-dimensional structure of the teeth. Actually, the digital dental impressions could be obtained by means of many different solutions. As a first example, an apparatus for active wavefront sampling could be used. This apparatus comprises a camera and a rotating module with an off-axis aperture: the aperture is moved in a circular path around the optical axis, and the depth of each point of the observed object is estimated on the basis of the pattern of the light rays coming from the object, passing through the rotating aperture and captured by the camera. This estimation can be very accurate, but it takes a long time to be obtained due to the need for the aperture to complete the movement along the circular path. In addition, an electric motor is needed to move the rotating module: together, the rotating module and the electric motor make this apparatus relatively bulky. As an alternative, an apparatus for “quasi” confocal microscopy could be used. This apparatus comprises a high-speed camera, a mobile lens and a light pattern projector: a light pattern with shallow depth of field is projected on the observed object, and the high-speed camera captures a multitude of images of the object at different depths. The points having the highest contrast are then related to their position in three dimensions. Again, the high-speed camera and the projector make the apparatus rather bulky. In addition, it has been observed that any relative motion between the camera and the observed object leads to non-negligible distortions of the images. As a further alternative, a stereo camera system could be used. This system comprises at least two cameras having overlapping fields of view. Each camera is designed to capture an image of the observed object within the overlapping field of view, but from a different point of view; the images are then compared by a central control unit in order to identify the features common to the different images, triangulate the scene in three dimensions and then reconstruct the three- dimensional structure of the teeth within the overlapping field of view. This solution is more compact than the other known solutions, but the cameras must have overlapping fields of view since each point of the observed object needs to be visible by at least two cameras, making it difficult to properly reconstruct inlays, holes and interdental spaces. Further, the at least two cameras must be precisely synchronized in time and space, since any synchronization error will negatively affect the accuracy of the estimation. In addition, the dental surfaces naturally lack any macroscopic texture, making the estimation of the structure of teeth by means of a stereo camera system particularly challenging. For this reason, the stereo camera system is usually associated with a structured light projector (forming an active stereo camera system), which projects a structured light pattern on the observed object in order to provide reference points for the triangulation. However, this makes the system bulky, does not solve the synchronization issues and does not change the fact that each point of the observed object needs to be visible by at least two cameras. Therefore, alongside with the other above-mentioned drawbacks, it appears that all the known apparatuses are rather bulky, and therefore allow only a limited number of cameras to be arranged on the intra-oral scanner, since the scanner needs to be easily insertable into the mouth of the patient. Therefore, at any given moment the known intra-oral scanners can capture images of only a limited part of the dental arch of the patient, thus requiring a manual movement of the scanner to be performed by a specialized operator if a complete reconstruction of the entire arch is desired: the need of easy and comfortable movement of the scanner inside the mouth further reduces the maximum dimensions of the scanner itself, further limiting the number of cameras that can be arranged on the scanner. However, the manual movement of the intra-oral scanner requires a specifically trained specialized operator and a considerable amount of time, with the risk that the patient moves during the scanning. In addition, each operator, even if specifically trained, will inevitably move the scanner in a different way, obtaining a three- dimensional reconstruction of the teeth that could be slightly different from the reconstructions obtained by other operators, or by the same operator in a different moment: this could lead for example to the manufacture of dental appliances which are not perfectly suited for the patient. An object of the present invention is to overcome the above-mentioned drawbacks and in particular to provide a device for estimating in a quick and accurate manner the three-dimensional structure of teeth, allowing to standardize the estimation and making it comfortable for the patient. Another object of the present invention is to provide a method for estimating the three-dimensional structure of teeth, by using the above-mentioned device. Another object of the present invention is to provide a method for producing a dental appliance suited for a patient, by first scanning the teeth of the patient by means of the above-mentioned device. These and other results are achieved according to the present invention providing a device for estimating the three-dimensional structure of teeth according to claim 1, a method for estimating the three-dimensional structure of teeth according to claim 12, and a method for producing a dental appliance suited for a patient according to claim 15. Further preferred characteristics are the object of the dependent claims. The present invention will be now described, in an illustrative and non-limiting way, referring to the attached schematic drawings, in which: - figure 1 is a top view sketch of a first embodiment of the device according to the present invention; - figure 2 is a top view sketch of a second embodiment of the device according to the present invention; - figure 3 is a see-through top view sketch of a device according to the present invention placed over a dental arch; - figure 4 is a cross section view of a device according to the present invention placed over a dental arch. With reference to figure 1, a device 1 for estimating the three-dimensional structure of a plurality of teeth is represented in a first embodiment. The device 1 comprises a tray 2, which is a support shaped so as to fit over at least a portion of the dental arch of the patient, that is insertable inside the mouth of the patient. The device 1 further comprises a plurality of independent cameras 3 arranged on the tray 2. Each camera 3 is equipped with a corresponding fixed focus lens 4 (visible in figure 4) and has a corresponding field of view. Each camera 3 is designed to capture a corresponding image of the teeth within its field of view, each image being composed by a plurality of pixels, and to transmit it to a central control unit 5. The device 1 further comprises illumination means 6 which are designed to illuminate the fields of view of the cameras 3. The central control unit 5 is programmed to estimate independently for each image a depth value (i.e. the distance from the corresponding camera) for each pixel thereof, therefore estimating the three-dimensional structure of the teeth. In practice, the three- dimensional structure of the teeth is estimated through monocular depth estimation, wherein each camera 3 operates independently and provides enough data to estimate the three-dimensional structure of the teeth (or of the portion of the teeth) within its field of view independently from the data provided by the other cameras 3. In this way, only the cameras 3, the fixed focus lenses 4 and the illumination means 6 need to be arranged on the tray 2 and inserted into the mouth: this makes it possible to arrange a larger number of cameras 3 on the tray 2 compared to the known systems and apparatuses, which instead need all the above-mentioned bulky equipment. In addition, since each image is independently processed by the central control unit 5, it is sufficient that each point of the teeth is in the field of view of at least one camera, not two: this makes it possible to arrange all the cameras 3 facing different directions, to make them capture images of different surfaces of the teeth. In turn, the large number of cameras 3 on the tray 2, all of them facing different directions, allows to capture with different cameras 3 in a substantially simultaneous way all the images necessary to completely estimate the three-dimensional structure of the teeth: therefore, the device 1 can estimate the three- dimensional structure of the teeth without being moved inside the mouth of the patient, since all the necessary images are captured by the different cameras 3 in a single “shot”. Therefore, the device 1 allows to standardize the estimation of the three-dimensional structure of the teeth, since it is sufficient to place the device 1 over the arch (as in figure 3) and to capture the images without moving anything: the result of the estimation is therefore always accurate and independent from the particular operator that performed it. In addition, since all the images can be captured substantially simultaneously, the tray 2 has to remain inside the mouth of the patient for only a limited time, making the procedure more comfortable for the patient and quicker in general. Finally, the components of the device 1 are relatively small and simple, making the device 1 itself compact. Preferably, the depth value for each pixel is estimated by the central control unit 5 at least on the basis of the blurriness of the image. In fact, a tooth appears sharp in an image if it is at a distance from the camera 3 that is equal to focal length of the fixed focus lens 4, while its blurriness increases with its distance from the focal plane: the central control unit 5 could estimate the distance of the point represented in each pixel from the camera 3 that captured the image on the basis of the blurriness of the image. Preferably, the depth value for each pixel is estimated by the central control unit 5 at least on the basis of the shadows created by the illumination means 6, increasing the accuracy of the estimation. The shadows created by the illumination means 6 could be used as an alternative to, or in combination with, the blurriness of the image to estimate the depth value for each pixel. Preferably, the depth value for each pixel is estimated by the central control unit 5 at least on the basis of the reflections created by the illumination means 6, increasing the accuracy of the estimation. The reflections created by the illumination means 6 could be used as an alternative to, or in combination with, the blurriness of the image and / or the shadows created by the illumination means 6 to estimate the depth value for each pixel. Preferably, the illumination means 6 comprise a plurality of illumination sources, for example LEDs in order to emphasize specific colours in the captured images. In particular, each camera 3 might be associated to a corresponding illumination unit comprising one or more corresponding LEDs, forming a plurality of independent units each comprising a camera 3 and a corresponding illumination unit synchronized with the camera 3 and designed to illuminate the field of view of the camera 3. Preferably, each independent unit is substantially identical to the other independent units: for example, each unit could be composed by a camera 3 and two adjacent LEDs (as in figure 2), or each unit could be composed by a camera 3 and one adjacent LED (as in figure 1), the cameras 3 and the LEDs being arranged alternating on the tray 2 so that the field of view of each camera 3 is illuminated by both the LED of the corresponding independent unit and the LED of the adjacent independent unit. The LEDs could be for example white LEDs. The illumination sources could be turned on and off, preferably independently from each other and / or randomly, by the central control unit 5, creating shadows and reflections that change in time and simplifying the estimation. The illumination means might be controlled directly by the central control unit 5, or through a specific card that depends on the central control unit 5. Preferably, the tray 2 is U-shaped, so as to fit over the entire dental arch of the patient. The tray 2 could have also a U-shaped cross section (visible in figure 4), so as to fit over the teeth from all sides (labial, occlusal and lingual), allowing a first group of independent cameras positioned on the labial portion of the U-shaped cross-surface to capture images of the labial surfaces of the teeth, a second group of independent cameras positioned on the occlusal portion of the U-shaped cross-surface to capture images of the occlusal surfaces of the teeth and a third group of independent cameras positioned on the lingual portion of the U-shaped cross-surface to capture images of the lingual surfaces of the teeth. In this way, the cameras 3 could be distributed over the tray 2 in such a way that the corresponding fields of view cover the entire surface of the dental arch, and all the images necessary to estimate the three- dimensional structure of the entire arch can be captured without repositioning the device 1. Each camera 3 may comprise a CCD sensor or a CMOS sensor, in particular a colour sensor, and may be arranged on the tray 2 with fixed and predetermined position and orientation, in order to simplify the estimation of the three-dimensional structure of the teeth. Preferably, for the same reason, also the illumination means 6 are arranged on the tray 2 with fixed and predetermined position and orientation. The fixed focus lens 4 of each camera 3 defines an optical system of fixed and pre-set focal length, that transmits the light rays arriving from the field of view to the corresponding sensor, without distortions, making each camera 3 a monocular camera. Preferably, the cameras 3 are arranged on the tray 2 so that the field of view of each camera 3 partially overlaps with the field of view of at least another (adjacent) camera 3. In particular, the field of view of each camera 3 might partially overlap with at least two adjacent cameras 3, in order make all the fields of view sequentially overlapping: this allows to join all the captured images using the overlapping regions, making it possible to simplify the estimation of the three- dimensional structure of large portions of the dental arch, or even of the entire dental arch. However, the overlap between the fields of view of the cameras 3 is only partial, and used only to join adjacent images: there is no need that each point of the surface of the teeth is comprised in the fields of view of at least two cameras 3, as it is instead necessary with stereo camera systems. Preferably, the device 1 comprises at least three cameras 3 in order to capture images of the labial, occlusal and lingual surfaces of the teeth. The device 1 could comprise up to 1000 cameras 3, or even more. The cameras 3 are connected to the central control unit 5, in order to transmit thereto the captured images: the central control unit 5 is programmed to collect, store and organize the captured images. The connection could be realized by means of a data transfer cable, or by means of a wireless data connection, for example a Wi- Fi connection. The central control unit 5 can be for example a computer, and the data transfer cable can be connected to the computer by means of a USB interface. As an alternative, at least a portion of the central control unit 5 might be located behind or proximate to the cameras 3. The central control unit 5 might also be held by the operator during use. The device 1 could also comprise one or more rechargeable batteries to provide power supply to the cameras 3 and the illumination means 6. In some embodiments, the tray 2 could be positioned on the face of a head end of an elongated body, wherein the elongated body further comprises a tail end that could be held by the operator while the images are captured, and that could comprise a connection to a power supply, which could be wireless. A shaft might connect the head end and the tail end, having a central axis extending from the head end to the tail end. The device 1 can be used in a method for estimating the three-dimensional structure of teeth inside the mouth of the patient. In particular, this method comprises the steps of inserting the tray 2 inside the mouth, independently capturing a plurality of images of the teeth by means of the cameras 3, transmitting the images to the central control unit 5, and estimating independently for each image a depth value for each pixel thereof. Preferably, all the images are captured by the cameras 3 in a substantially simultaneous way. In this way, the tray 2 has to remain inside the mouth of the patient for only a limited time. The images are considered captured in a substantially simultaneous way if the time needed to capture all the images is short enough to prevent substantial movements of the dental arch with respect to the tray 2. For example, a few seconds is considered a short enough time. Preferably, the cameras 3 are distributed over the tray 2 in such a way that the corresponding fields of view cover the entire surface of the dental arch, so that images of all the teeth inside the mouth are substantially simultaneously and independently captured without repositioning the tray 2. The device 1 can be used also in a method for producing dental appliances suited for a patient, for example dental braces, clear aligners, retainers, implants, dentures or other prosthetics. In particular, this method comprises the steps of estimating the three- dimensional structure of the teeth inside the mouth of the patient by means of the device 1, creating digital data associated to the estimated structure, transferring the digital data to a manufacturing system, and manufacturing the dental appliance by means of the manufacturing system on the basis of the digital data. In this way, the dental appliance can be manufactured to be perfectly suited for the patient. The manufacturing system could be a CAD / CAM system or a 3D printing system. Before use, the device 1 is calibrated outside the mouth, in order to define the relationship between the features of an image and the distance from the camera 3 of the teeth within its field of view. In this way, the central control unit 5 can run the appropriate image-processing algorithms where a priori knowledge is embedded. For example, the central control unit 5 may run a properly trained machine learning model. The machine learning model may be trained using a training set comprising a plurality of training images of reference teeth, each of the training image being associated to a training three-dimensional structure of the reference teeth. The machine learning model, in use, may compare each captured image with the training images, determine which training image is most similar to the captured image, and estimate the three-dimensional structure of the teeth within the field of view as the training three-dimensional structure of the reference teeth pictured in the selected training image. The present invention has been described in an illustrative and non-limiting way according to relative favourite embodiments, but it is clear that the person skilled in the art could perform many variations and modifications, all of which are within the scope of the invention.

Claims

CLAIMS 1) Device (1) for estimating the three-dimensional structure of teeth inside a mouth, comprising: - a central control unit (5), - a tray (2) insertable inside the mouth, - a plurality of independent cameras (3) arranged on the tray (2), each camera (3) being equipped with a corresponding fixed focus lens (4), having a corresponding field of view and being designed to capture a corresponding image of the teeth within the field of view and to transmit the image to the central control unit (5), each image being composed by a plurality of pixels, and - illumination means (6) arranged on the tray (2) and designed to illuminate the fields of view of the cameras (3), wherein the central control unit (5) is programmed to estimate the three-dimensional structure of the teeth by estimating independently for each image a depth value for each pixel thereof. 2) Device (1) according to claim 1, wherein the central control unit (5) is programmed to estimate independently for each image a depth value for each pixel thereof on the basis of the blurriness of the image and / or on the basis of the shadows created by the illumination means (6) and / or on the basis of the reflections created by the illumination means (6). 3) Device (1) according to any of the preceding claims, wherein the illumination means (6) comprise a plurality of illumination sources, in particular LEDs. 4) Device (1) according to claim 3, wherein the central control unit (5) is programmed to turn the illuminationsources on and off, preferably independently and / or randomly. 5) Device (1) according to any of claims 3 or 4, wherein each camera is associated to a corresponding illumination unit comprising at least one illumination source designed to illuminate the corresponding field of view, all the illumination units being substantially identical 6) Device (1) according to any of the preceding claims, wherein each camera (3) and the illumination means (6) are arranged on the tray (2) with fixed and predetermined position and orientation. 7) Device (1) according to any of the preceding claims, wherein the field of view of each camera (3) partially overlaps with the field of view of at least one other camera (3). 8) Device (1) according to any of the preceding claims, comprising 3 to 1000 independent cameras (3). 9) Device (1) according to any of the preceding claims, wherein each camera (3) comprises a CCD sensor or a CMOS sensor. 10) Device (1) according to any of the preceding claims, wherein the tray (2) is U-shaped. 11) Device (1) according to any of the preceding claims, wherein the tray (2) has a U-shaped cross section, and the device (1) comprises a first group of independent cameras (3) designed to capture images of the labial surfaces of the teeth, a second group of independent cameras (3) designed to capture images of the occlusal surfaces of the teeth and a third group of independent cameras (3) designed to capture images of the lingual surfaces of the teeth.12) Method for estimating the three-dimensional structure of teeth inside a mouth by means of a device (1) according to any of the preceding claims, comprising the steps of: - inserting the tray (2) inside the mouth; - independently capturing a plurality of images of the teeth by means of the cameras (3); - transmitting the images to the central control unit (5); - estimating independently for each image a depth value for each pixel thereof. 13) Method according to claim 12, wherein images of all the teeth inside the mouth are substantially simultaneously and independently captured. 14) Method according to any of claims 12 or 13, wherein for each image the depth value for each pixel thereof is estimated on the basis of the blurriness of the image and / or on the basis of the shadows created by the illumination means (6) and / or on the basis of the reflections created by the illumination means (6). 15) Method for producing a dental appliance suited for a patient, comprising the following steps: - estimating the three-dimensional structure of teeth inside the mouth of the patient, by means of a device (1) according to any of claims 1 to 11; - creating digital data associated to the estimated structure; - transferring the digital data to a manufacturing system; - manufacturing the dental appliance by means of the manufacturing system on the basis of the digital data. 16) Method according to claim 15, wherein themanufacturing system is a CAD / CAM system or a 3D printing system.

Images