Method for determining tooth characteristics from a tooth image

Abstract

The invention relates to a method for generating tooth characteristics, the method comprising providing a processing model, taking at least one tooth image of a tooth, and calculating the tooth characteristics from the tooth images by means of the processing model. The processing model was trained on the basis of a dataset. The dataset comprises at least one learning tooth image and a learning tooth characteristic which are linked to one another.

Description

[0001] Method for determining tooth characteristics from a tooth image

[0002] The present invention relates to a method for determining tooth characteristics from a tooth image.

[0003] Dental care is an important contribution to human hygiene. Removing biofilm from teeth is a particularly important part of dental care. This biofilm is a film that, if left untreated, can cause dental diseases such as caries. Regular removal, for example, with a toothbrush, is therefore strongly recommended.

[0004] To help a user know where to pay particular attention when brushing their teeth, the contrast agent fluorescein is known to fluoresce when illuminated by UV / blue light, for example.

[0005] If a color camera captures the fluorescent light from the biofilm in the spectrum of the human eye, the signal-to-noise ratio is very low and the biofilm is difficult to detect.

[0006] The not-yet-published German patent application DE 10 2022 102 045.2 proposes a dental dirt detection device that solves this problem by allowing the camera to capture only a specific color spectrum range. This significantly improves the signal-to-noise ratio. Biofilm is easily detectable.

[0007] A disadvantage of using a narrow band filter, however, is that other tooth characteristics, such as the edge of the tooth, i.e. the demarcation of the tooth from, for example, the gums or the back of the oral cavity, are very difficult to recognize in the recorded image. US 2020 / 0201266 A1 discloses a cleaning device for a household. This cleaning device can be designed for a wide variety of applications, such as cleaning the floor of a building, shaving a human body or cleaning teeth. This cleaning device can have a neural network with which different properties of an image of the object to be cleaned can be determined, such as the color of teeth, in order to influence the cleaning process.

[0008] US 2020 / 0146794 A1 describes a smart toothbrush equipped with a camera that captures images of the teeth being brushed. A neural network can evaluate the images to determine whether the teeth have plaque or tartar buildup, or whether there is inflammation in the oral cavity. The toothbrush's cleaning process is adjusted accordingly.

[0009] US 2020 / 0179089 A1 discloses an oral hygiene monitoring system that monitors the movement and orientation of oral hygiene devices, such as a toothbrush, during use. This is done by means of one or more cameras that monitor the movement of the toothbrush from outside the body of the person brushing their teeth.

[0010] US 2021 / 0393026 A1 describes an oral hygiene system that is a type of smart toothbrush with an optical sensor for optically scanning the oral cavity. The sensor data can be analyzed using a machine learning system, such as a neural network.

[0011] The invention is based on the object of creating a method for detecting teeth in a simple, reliable, and automated manner. This object is achieved by the subject matter of the independent claims. Advantageous further developments and preferred embodiments form the subject matter of the dependent claims.

[0012] A method for generating tooth characteristics comprises providing a processing model, acquiring at least one tooth image of a tooth, and calculating the tooth characteristics from the tooth images using the processing model. The processing model was trained using a dataset. The dataset includes at least one learning tooth image and a learning set of tooth characteristics, which are linked to one another. The tooth characteristics include at least boundaries of one or more teeth in the tooth image.

[0013] Tooth boundaries display a similar representation across multiple tooth images, making them particularly well-suited for machine learning.

[0014] During training, the trained processing model learned where the boundaries of the teeth are typically located in the image. The processing model can orient itself using specific markers in the image, such as individual teeth visible with greater contrast in the original tooth image. In the areas where a boundary should typically be present, the processing model evaluates subtle brightness differences in the grayscale as boundaries, provided they fit within the overall boundary contour of the teeth. However, it is also conceivable to evaluate brightness differences in color images, such as RGB images.

[0015] This allows the tooth contour to be determined very reliably and precisely, even if the lighting conditions are not optimal due to the system.

[0016] The boundaries of teeth, especially of the same tooth type, are similar across multiple dental images. For example, the boundaries of one incisor are similar to those of another user. The respective differences can then be easily detected using the machine learning processing model. By using a processing model, particularly one created using machine learning, tooth characteristics can also be recognized from dental images, even when no specialist is evaluating the images.

[0017] Conventional methods, such as thresholding, color spectrum separation, or various optical filters, do not reliably distinguish between tooth and gum or the background. In the worst case, this can lead to gum injury during cleaning.

[0018] The features in the dental image are so faint that they cannot be recognized using conventional methods.

[0019] However, the characteristics are certainly present and are similar from tooth to tooth. Depending on the tooth type, the characteristics can also be found in similar locations.

[0020] Due to these consistent features in the tooth image, the method is very well suited for machine learning.

[0021] The inventors have recognized that, unlike the prior art discussed above, in which machine learning systems are used to analyze images of the oral cavity for plaque, gingivitis, and the like, a machine learning system can very precisely and reliably detect the border between teeth and gums due to the similar shape of teeth in different individuals, even if the optical conditions are not optimal due to the system. Clear identification of the border between the teeth and gums significantly improves the quality of oral cleaning.

[0022] For an automated cleaning process, however, it is very advantageous if the border between the tooth and the gum is precisely known. As a result, a processing model trained on the tooth boundary significantly improves the quality of the treatment or cleaning process, which can be carried out automatically. Tooth characteristics are data that describe specific features of a tooth. In the simplest case, tooth characteristics are an image file with the same dimensions as the tooth image, whereby this image only contains binary data, where, for example, 0 indicates that there is no tooth at this point and 1 that there is a tooth. Depending on the representation, such an image would be displayed as a black contour drawing of the tooth.

[0023] The generated image of the tooth characteristics corresponds to the tooth image.

[0024] However, it is also conceivable that the tooth characteristics are position data or vector lines that run along the boundary of the tooth.

[0025] Biofilm, plaque, tartar, and dental dirt are synonymous within the scope of this application. They describe a substance that adheres to the teeth and typically contains saliva, bacteria, and food residue.

[0026] Preferably, the dental images are binary images.

[0027] Preferably, the dental images are reconstructed before being input into the processing model. The reconstruction may include at least one of the following features:

[0028] - Encoding the image,

[0029] - Color creation according to the Bayer pattern,

[0030] - Color signal amplification,

[0031] - Brightness adjustment and / or

[0032] - Gamma adjustment.

[0033] Preferably, the tooth characteristics represent boundaries of the tooth in the dental pattern, particularly in relation to the gums and / or the tongue.

[0034] The recognition of tooth characteristics can include a method for comparing and / or classifying or categorizing color auxiliaries. According to a preferred development, a segmentation model for tooth area detection is used in particular when recognizing the tooth characteristics using a machine learning algorithm.

[0035] Alternatively or additionally, an object detection model can be used. This includes a bounding box model and / or a tooth coordinate detection model.

[0036] Furthermore, it is conceivable that a thresholding algorithm, particularly an HSV, RGB, YCBCR, or LAB thresholding algorithm, could be used. Image sections can be highlighted using different colors.

[0037] The aforementioned segmentation model could also be used to assign features or characteristics to regions. This involves region detection. This is useful, for example, when the feature has clearly defined boundaries, as is the case with caries.

[0038] As a further development, it is conceivable that the boundary can be differentiated between a boundary from tooth to gum and from tooth to background.

[0039] However, it is also conceivable that the tooth characteristics include features of the teeth, such as tooth discoloration, implants, demineralized areas, caries, etc.

[0040] The tooth characteristics can be used, in combination with the tooth image depicting plaque, to create a map of the tooth that shows not only the plaque but also the boundaries of the teeth. This allows the plaque to be very precisely localized on the individual teeth, and a cleaning process can be controlled accordingly. Such a tooth map with the boundaries and the plaque can be used, for example, to control a tooth cleaning device, such as that described in German patent application DE 102022 102 045.2, for cleaning the teeth.

[0041] In a preferred further development method, the tooth image is generated using a dental dirt detection device.

[0042] By illuminating teeth with UV / blue (~405 nm) light, the teeth can be examined for early caries, cracks, and secondary caries. The contrast agent fluorescein exhibits fluorescence with its strongest intensity at 520 to 530 nm when excited with light with a wavelength of 465 to 500 nm. The very close wavelength spectrum of the exciting and emitted light results in a signal-to-noise ratio that is too low for reliable detection in standard intraoral camera units of a dental debris detection system. Reflections caused by focused lighting solutions and blurred images from the camera unit can currently only be compensated for by increasing the distance between the tooth and the sensor unit in intraoral cameras currently on the market.

[0043] Preferably, the dental dirt detection device has a light filter to allow light with wavelengths between 480 nm and 530 nm to pass through.

[0044] To ensure a good signal-to-noise ratio, an optical long-pass filter is preferably placed directly in front of the camera of the dental dirt detection device. Ideally, this filter has a cutoff wavelength of 480 nm to 530 nm, and especially around 510 nm, and cuts off signals below this wavelength. Additionally, a band-pass or short-pass filter can be placed in front of the LEDs illuminating the area to be detected to limit / focus the wavelength spectrum of the LEDs.

[0045] As an alternative to the long-pass filter, a circular polarizer can also be used. According to a further development, an additional parameter can be taken into account to determine tooth characteristics.

[0046] This can increase the accuracy in determining tooth characteristics.

[0047] For example, the processing model can generate better tooth characteristics if the tooth type (such as an incisor) is known. An incisor differs in shape from, for example, a molar.

[0048] Preferably, the additional parameters comprise at least one of the following parameters:

[0049] - dentures,

[0050] - Dental diseases,

[0051] - Position of demineralized areas,

[0052] - Position of caries,

[0053] - Position of fillings,

[0054] - Position of tartar,

[0055] - Position of discolorations,

[0056] - Position of rough spots,

[0057] - Whiteness of the tooth,

[0058] - misalignment of the tooth,

[0059] - Tooth positions, and / or

[0060] - Tooth type.

[0061] This not only allows for more accurate detection but also helps prevent errors. For example, an area with generalized calculus in the dental image may exhibit similar characteristics to the background. If such an area is known, the position of the tooth boundary can be more accurately calculated.

[0062] Preferably, the machine learning is supervised machine learning.

[0063] In supervised learning, the processing model is learned from given pairs of inputs and outputs. These inputs and outputs represent pre-formed tooth images and tooth characteristics. The correct functions are manually provided from the tooth images upon input. After several iterations with different inputs and outputs, the ability to make associations is trained.

[0064] According to a further development, the processing model is improved through independent machine learning during the application phase, with the user manually adjusting the tooth characteristics generated by the processing model. The processing model is further trained using a dataset consisting of the tooth image and the adjusted tooth characteristics.

[0065] This continuously improves the processing model. The more training data, including newly added data, is used to create the processing model, the more accurately the processing model can determine tooth characteristics from a tooth image.

[0066] In a method for generating a processing model, the processing model is trained using a machine learning algorithm based on a dataset. The dataset includes at least one learning tooth image and at least one learning tooth characteristic, which are linked to one another. The processing model is improved by generating a learning tooth image and target tooth characteristics, and then using a target algorithm to determine a measure of how much the target tooth characteristics and the learning tooth characteristics differ. The processing model is adapted based on the determined measure.

[0067] A computer program product comprises instructions which, when executed by a computer, cause the computer to carry out the method described above.

[0068] The computer is a processing unit. However, the processing unit, or computer, can also be a digital user device, such as a smartphone, a server, a microcontroller, a laptop, a tablet computer, a PDA, or another intelligent system, as well as a cloud-based system. Another aspect of the present invention is a method for automatically cleaning teeth, wherein at least one tooth characteristic is determined according to a method explained above, and the cleaning process is controlled according to the tooth characteristic thus determined.

[0069] The cleaning process is carried out, for example, using an automatically controlled toothbrush.

[0070] The invention will be explained in more detail below using examples shown in the drawings. The drawings show schematically: Figure 1 Schematic representation of tooth images and tooth characteristics,

[0071] Figure 2 Block diagram of a system for determining

[0072] Tooth characteristics from a dental image,

[0073] Figure 3a Block diagram of a system for forming a processing model,

[0074] Figure 3b Block diagram of a system for the use of a

[0075] processing model, and

[0076] Figure 4 is a flowchart of a method for determining tooth characteristics from a tooth image.

[0077] Examples of implementation

[0078] Device description

[0079] A system 1 for determining tooth characteristics comprises a learning unit 2, an execution unit 3 and a dental dirt detection device 4 for taking dental images 5.

[0080] The learning unit 2 comprises a machine learning module 6, which is designed to use assignments 7 of tooth images 5 to learning tooth characteristics 8 in order to generate a processing model 9 therefrom.

[0081] The application unit 3 comprises an application module 10, which uses the processing model 9 to automatically determine tooth characteristics 11 from tooth images 5. The learning unit 2 and the execution units 3 are typically computing units, such as computers. The machine learning module 6 and the application module 10 are software applications that can be executed on these computers.

[0082] In this embodiment, the learning unit 2 and the execution unit 3 are two different computers that are connected to each other via a computer network in order to exchange the processing model 9.

[0083] In other embodiments, it is conceivable that the learning unit 2 and the execution unit 3 are represented by the same computer.

[0084] The dental dirt detection device 4 preferably communicates wirelessly with the execution unit 3, for example via WLAN. However, wired communication is also conceivable.

[0085] Tooth dirt detection device 4 is described in detail in the unpublished German patent application DE 10 2022 102 045.2 and this patent application is incorporated by reference in its entirety.

[0086] Basically, the dental dirt detection device 4 is designed such that a U-shaped section of the dental dirt detection device 4 is inserted into the oral cavity of a user. The U-shaped section of the dental dirt detection device 4 has a sensor arrangement.

[0087] The U-shaped section is placed over the teeth so that all tooth surfaces can be detected by a sensor.

[0088] To make a biofilm more visible, a detection fluid is introduced into the oral cavity prior to the detection process. The detection fluid interacts with the biofilm and, under the influence of light of a predetermined wavelength, causes the biofilm to glow at a different predetermined wavelength. The detection fluid is housed in a detection capsule that can be inserted into the dental dirt detection device 4. The detection device then extracts the detection fluid and pumps it onto the teeth.

[0089] For this purpose, the dental dirt detection device 4 consists of a handpiece, which may have a display on the side facing away from the user. On the side facing the user, there is a mouthpiece that is inserted into the oral cavity and guides the sensor unit over the teeth.

[0090] The mouthpiece has at least one camera unit with a camera.

[0091] The camera unit alone, for example, measures 1 x 1 x 2.7 mm, while the entire sensor unit, including a protective glass, PCB (printed circuit board), filter, and camera mount, has a diameter of 8 mm and a height of 3.8 mm. These dimensions allow the unit to be easily inserted into the oral cavity.

[0092] To ensure a good signal-to-noise ratio, an optical long-pass filter is preferably placed directly in front of the camera, which ideally has a cutoff wavelength of about 510 nm and cuts off signals below this.

[0093] Additionally, a bandpass or shortpass filter can be placed in front of the LEDs illuminating the area to be detected to limit the wavelength spectrum of the LEDs.

[0094] Procedure description

[0095] A method for determining tooth characteristics 11 is explained below.

[0096] The process begins with step S1 (Figure 4).

[0097] In the next step (S2), tooth images 5 are manually assigned to tooth characteristics 11. Tooth images 5 are images of teeth captured by a dental debris detection device 4. They typically show strong signal-to-noise ratios of dental plaque, but the demarcation of teeth, in this case tooth characteristics 11, is very difficult to discern. However, qualified personnel are able to determine these demarcations manually.

[0098] If tooth images 5 have been assigned tooth characteristics 11, in the next step (S3) a processing model 9 is generated by machine learning of this manual assignment 7.

[0099] In the learning phase, an empty processing model 9 is initially used, which in this case consists of a neural network. "Empty" in this context means that the neural network has not yet been trained with any data, but contains the necessary fundamentals and is ready to learn.

[0100] To learn the processing model 9, learning tooth images 5 are read into the processing model 9, which then outputs target tooth characteristics 11 (Figure 3a).

[0101] To generate learning tooth characteristics 8, an image of the teeth is captured using a camera module of a dental dirt detection device 4. A contrast agent and, if necessary, a color filter are omitted. This creates an image of the tooth in which, although a biofilm is not easily recognizable, the tooth's boundary is clearly visible. This boundary can be calculated using an algorithm or manually from the tooth image 5, and then represents the learning tooth characteristics.

[0102] A target algorithm 12 determines how much the target characteristics differ from the learning tooth characteristics assigned to the learning tooth images 5.

[0103] Based on the measurement, the processing model 9 is automatically improved. This learning step is now repeated with the same learning tooth image and / or a different learning tooth image.

[0104] This repetition happens several times and based on this repetition of the learning step, processing model 9 is built.

[0105] This is followed by step S4, in which the processing model 9 is transferred from the learning unit 2 to the execution unit 3.

[0106] In step S5, new tooth images 5 are also recorded by the dental dirt detection device 4.

[0107] The new tooth images 5 are transmitted from the dental dirt detection device 4 to the execution unit 3.

[0108] Subsequently, in step S6, tooth characteristics 11 of the new tooth image 5 are generated based on the processing model 9 (Figure 3b).

[0109] The method ends with step S7.

[0110] According to one embodiment, the tooth surfaces visible to the camera are measured. For example, the heights, depths, and angles of inclination relative to the specific surface of the tooth can be determined in order to determine the surface of the tooth.

[0111] According to a further embodiment, teeth can be tracked, i.e., followed, across multiple recorded tooth images. In this case, teeth are identified on different tooth images. Furthermore, it is possible to determine tooth movements and calculate the camera movement from them. List of reference symbols

[0112] 1 System for determining tooth characteristics

[0113] 3 Execution unit 4 Tooth dirt detection device

[0114] 5 dental images

[0115] 2 learning units

[0116] 6 Learning Module

[0117] 7 Assignments 8 Learning tooth characteristics

[0118] 9 Processing model

[0119] 10 Application module

[0120] 11 Tooth characteristics

Claims

Patent claims Method for generating tooth characteristics (11) comprising: providing a processing model (9), Recording at least one tooth image (5) of a tooth, determining the tooth characteristics (11) from the tooth images (5) using the processing model (9), wherein the processing model (9) was trained on a data set, and the data set comprises at least one training tooth image (5) and training tooth characteristics (8) which are linked together, wherein the tooth characteristics (11) represent at least the boundaries of the tooth in the tooth image (5), in particular in relation to the gums and / or the tongue and / or other background in the oral cavity. The method according to claim 1, characterized in that the tooth image (5) was generated with a tooth stain detection device (4). The method according to claim 2, characterized in that the tooth stain detection device (4) has a light filter to allow light with wavelengths between 480 nm and 530 nm to pass through.Method according to one of claims 1 to 3, characterized in that additional parameters are taken into account to determine the tooth characteristics (11). The method according to claim 4, characterized in that the additional parameters comprise at least one of the following parameters: - dental prostheses, - Dental diseases, - Position of the demineralized areas, - Position of caries, - Position of fillings, - Position of tartar, - Position of discolorations, - Position of rough spots, - Wisdom level of the tooth, - Misalignment of the tooth, - Tooth positions, and / or - Tooth type. Method according to any one of claims 1 to 5, characterized in that the machine learning is supervised machine learning. Method according to any one of claims 1 to 6, characterized in that in the application phase the processing model (9) is improved by autonomous machine learning by a user manually adjusting the tooth characteristics (11) generated by the processing model (9), wherein the processing model (9) is additionally trained on the basis of a data set consisting of the tooth image (5) and the adjusted tooth characteristics (11). Method for generating a processing model (9) in which the processing model (9) is trained on the basis of a data set by a machine learning algorithm, and the data set comprises at least one training tooth image (5) and one training tooth characteristic (8). which are linked together, wherein the processing model (9) is improved by generating target tooth characteristics (11) from a training tooth image (5) and then determining, by means of a target algorithm, a measure of how much the target tooth characteristics (11) and the training tooth characteristics (8) differ, wherein the processing model (9) is adapted based on the determined measure and the tooth characteristics (11) represent at least the boundaries of the tooth in the tooth image (5), in particular in relation to the gums and / or the tongue and / or other background in the oral cavity. A computer program product comprising instructions which, when the program is executed by a computer, cause it to execute the method according to any one of the preceding claims 1 to 8.Method for automatically cleaning teeth, wherein at least one tooth characteristic is determined according to a method according to one of claims 1 to 7 and the cleaning process is controlled according to the tooth characteristic thus determined.

Images