Method for determining tooth characteristics from tooth images

Abstract

A method for generating tooth features includes providing a processing model, recording an image of at least one tooth of a tooth, and calculating tooth features from the image of the tooth using the processing model. The processing model is trained using a dataset. The dataset includes at least one training tooth image and one training tooth feature that are linked to each other.

Description

Technical Field

[0001] The present invention relates to a method for determining tooth characteristics from tooth images.

Background Art

[0002] Dental care makes an important contribution to human hygiene. In particular, removing biofilm from teeth is part of dental care. Biofilm is a film that, if not removed, can cause dental diseases such as tooth decay. Therefore, it is strongly recommended to remove it regularly with a toothbrush or the like.

[0003] It is known to use a contrast agent fluorescein that emits fluorescence when irradiated with, for example, ultraviolet / blue light so that the user can know the parts that should be particularly noted when brushing teeth.

[0004] When a color camera records fluorescence from biofilm in the spectrum of the human eye, the signal-to-noise ratio is very low, and it is difficult to detect biofilm.

[0005] German Patent No. 10 2022 102 045.2 (unpublished) describes a dental plaque detection device that solves this problem by allowing a camera to record only a specific color spectrum range. This significantly improves the signal-to-noise ratio. This makes it very easy to detect biofilm.

[0006] However, the drawback of using a narrowband filter is that it becomes difficult to recognize other tooth characteristics in the recorded image, such as the boundaries of teeth, i.e., the boundaries between teeth and, for example, the gums or the background of the oral cavity.

[0007] U.S. Patent No. 2020 / 0201266A1 discloses a household cleaning appliance. This cleaning appliance can be used for various purposes, such as cleaning the floor of a building, shaving the beard on a human body, and cleaning teeth. This cleaning appliance may have a neural network capable of discriminating various characteristics of an image of an object to be cleaned, for example, the color of teeth, so as to be able to affect the cleaning process.

[0008] U.S. Patent No. 2020 / 0146794 A1 describes an intelligent toothbrush equipped with a camera capable of recording an image of teeth to be polished. By using a neural network to evaluate the image, it is possible to determine whether there is dental plaque or tartar on the teeth or whether there is inflammation in the oral cavity. Accordingly, the cleaning process by the toothbrush is adjusted.

[0009] U.S. Patent No. 2020 / 0179089 A1 discloses an oral hygiene monitoring system that monitors the movement and orientation of an oral hygiene device such as a toothbrush in use. This is done by one or more cameras that monitor the movement of the toothbrush from outside the body of the person brushing their teeth.

[0010] U.S. Patent No. 2021 / 0393026 A1 describes an oral hygiene system that is a type of intelligent toothbrush, and this oral hygiene system has an optical sensor that optically scans the oral cavity. Sensor data can be analyzed by a machine learning system such as a neural network.

Prior Art Documents

Patent Documents

[0011]

Patent Document 1

Patent Document 2

Patent Document 3

Patent Document 4

[0012] An object of the present invention is to create a method for recognizing teeth in an automated manner simply and reliably. [Means for Solving the Problems]

[0013] This object is achieved by the subject matter of the independent claims. Advantageous and preferred embodiments are the subject matter of the dependent claims.

[0014] A method for generating tooth features includes providing a processing model, recording an image of at least one tooth, and calculating tooth features from the tooth image using the processing model. The processing model is trained using a dataset. The dataset includes at least one training tooth image and one training tooth feature linked to each other. The tooth features include at least the boundaries of one or more teeth within the tooth image.

[0015] Tooth boundaries are depicted similarly within the tooth image across multiple tooth images. This is particularly suitable for machine learning.

[0016] The processing model to be trained learns during training where tooth boundaries usually are within the image. The processing model can be based on specific markers within the image, such as individual teeth that appear to have stronger contrast in the original tooth image. In areas where there should generally be a boundary, extremely small differences in gray level are evaluated as boundaries by the processing model, provided that they adapt to the overall boundary contour of the tooth. It is also conceivable to evaluate differences in luminance in a color image such as an RGB image.

[0017] Thus, even when the lighting conditions are not optimal for system-related reasons, the tooth contour can be identified very reliably and accurately.

[0018] The boundaries of teeth, especially those of the same type, are similar across multiple tooth images. For example, the boundary of a certain incisor is similar to that of an incisor of another user. Each difference can be very easily recognized by a machine learning processing model.

[0019] Especially by using a processing model generated by machine learning, even when the image has not been evaluated by an expert, the features of the teeth can be recognized from the tooth image.

[0020] Conventional methods such as threshold processing (Schwellenwertbildung), color spectrum separation, or various optical filters cannot reliably distinguish teeth from gums or the background. In the worst case, it may even damage the gums during cleaning.

[0021] The features in tooth images are so weak that they cannot be recognized by conventional methods.

[0022] However, those features definitely exist and are similar for each tooth image. Also, depending on the type of tooth, features may be seen at similar positions.

[0023] Since there are consistent features in tooth images in this way, this method is very suitable for machine learning.

[0024] In contrast to the above-mentioned prior art, machine learning systems are used to analyze oral images for things like dental plaque and gingivitis. Since the tooth shapes are similar even among different people, the inventors have recognized that even when the optical conditions are not optimal for system-related reasons, the machine learning system can very accurately and reliably identify the boundary between teeth and gums. By clearly identifying the boundary between teeth and gums, the quality of oral cleaning is significantly improved.

[0025] To perform an automatic cleaning procedure, it is very beneficial to accurately grasp the boundary between the teeth and the gums. That is, the processing model that has learned the boundary of the teeth can significantly improve the quality of the procedures or cleaning processes that can be automatically executed.

[0026] Tooth features are data representing specific features of teeth. In the simplest case, the tooth feature is an image file having the same dimensions as the image of the teeth, where this image contains only binary data indicating, for example, 0 indicating no tooth at this position and 1 indicating a tooth at this position. Depending on the representation, such an image is displayed as a black contour line of the teeth.

[0027] The generated image of the tooth features corresponds to the image of the teeth.

[0028] It is also conceivable that the tooth features are position data or vector lines extending along the boundary of the teeth.

[0029] The terms biofilm and dental plaque are synonymous within the scope of this application. These represent substances that adhere to the teeth and usually contain saliva, bacteria, and food residues.

[0030] Preferably, the image of the teeth is a binary image.

[0031] Preferably, the image of the teeth is reconstructed before being input into the processing model. The reconstruction may include at least one of the following features: - Encoding of the image, - Color adjustment by the Bayer pattern, - Amplification of the color signal, - Brightness adjustment, and / or - Gamma adjustment. may include at least one of.

[0032] Preferably, the tooth features represent the boundary of the teeth in the image of the teeth, particularly in relation to the gums and / or the tongue.

[0033] The recognition of the characteristics of the teeth may include a method of matching and / or classifying or categorizing a shade guide.

[0034] According to one preferred variant, when using a machine learning algorithm to recognize the characteristics of the teeth, in particular, a segmentation model is used for recognizing the area of the teeth.

[0035] Alternatively or additionally, a model for object recognition may also be used. This model includes a bounding box model and / or a model for recognizing tooth coordinates (Zahnkoordinaten).

[0036] It is also conceivable to use an algorithm to determine a threshold value, in particular an HSV, RGB, YCBCR, or LAB threshold value. In this case, the image section can be highlighted in different colors.

[0037] The above-mentioned segmentation model can also be used to assign characteristics and features to an area. In this case, area recognition is performed. This is effective when there is a clear boundary to the feature, for example, in the case of dental caries.

[0038] As one variant, it is conceivable to distinguish between the boundary between the tooth and the tooth root and the boundary between the tooth and the background.

[0039] However, it is also conceivable that the characteristics of the teeth include characteristics of the teeth such as tooth discoloration, implants, demineralized areas, dental caries, etc.

[0040] By using the characteristics of the teeth in combination with an image of the teeth representing dental plaque, a map of the teeth can be generated that shows not only the dental plaque but also the boundaries of the teeth. Thereby, the dental plaque on individual teeth can be identified very accurately, and the cleaning process can be controlled accordingly.

[0041] Such a dental map representing the boundary and dental plaque can be used, for example, to control a tooth cleaning device for appropriately cleaning teeth, as described in German Patent Application No. 10 2022 102 045.2.

[0042] In one preferred embodiment, an image of the teeth is generated using a dental plaque detection device.

[0043] When teeth are irradiated with ultraviolet / blue light (about 405 nm), the presence or absence of initial dental caries, tooth cracks, and secondary dental caries in the teeth can be checked. The contrast agent fluorescein, when excited by light with a wavelength of 465 - 500 nm, has a maximum fluorescence intensity at 520 - 530 nm. Since the wavelength spectra of the excitation light and the emitted light are very close, the signal-to-noise ratio becomes too low, and reliable detection cannot be achieved in the standard intraoral camera unit of the dental plaque detection device. In currently commercially available intraoral cameras, reflections by the collected illumination solution and unclear images from the camera unit are currently corrected only by increasing the distance from the teeth to the sensor unit.

[0044] Preferably, the dental plaque detection device has an optical filter for passing light having a wavelength of 480 nm to 530 nm.

[0045] To achieve a good signal-to-noise ratio, preferably, an optical long-pass filter is directly placed in front of the camera of the dental plaque detection device. This filter ideally has a cut-off wavelength of 480 nm to 530 nm, especially around 510 nm, and cuts off signals with wavelengths below this. Further, a band-pass filter or a short-pass filter can be placed in front of the LED illuminating the area to be detected to limit / condense the wavelength spectrum of the LED.

[0046] A circular polarizer can also be used instead of the long-pass filter.

[0047] According to one variant, additional parameters can be considered to determine the characteristics of the teeth.

[0048] This can improve the accuracy in determining the characteristics of teeth.

[0049] Therefore, for example, if the type of tooth (such as incisors) is known, the processing model can generate better tooth characteristics. Incisors, for example, have a different shape from molars.

[0050] Preferably, the additional parameter is one of the following parameters: - Dental prosthesis, - Dental disease, - Position of the demineralized area, - Position of dental caries, - Position of fillings, - Position of dental calculus - Position of discoloration - Position of rough (rauen) areas, - Degree of tooth whiteness, - Tooth displacement, - Tooth position, and / or - Tooth type, and includes at least one of them.

[0051] This enables not only more accurate recognition but also avoidance of errors. For example, a general area with dental calculus in a tooth image may have features similar to the background. If such an area is known, the position of the tooth boundary can be calculated more accurately.

[0052] Preferably, the machine learning is supervised machine learning.

[0053] In supervised learning, the processing model is learned from a given pair of input and output. These inputs and outputs represent predefined tooth images and tooth characteristics. The correct function is manually provided by the tooth image in the input. In this case, if it is executed several times with different inputs and outputs, the ability to make associations is trained.

[0054] According to one variant, in the application phase, the processing model is improved by independent machine learning in which the user manually adjusts the tooth features generated by the processing model. The processing model is further trained using a data set composed of the tooth image and the adapted tooth features.

[0055] Thereby, the processing model is continuously improved. The more the training data including newly added data is used to generate the processing model, the more accurately the tooth features can be determined from the tooth image using the processing model.

[0056] In a method for generating a processing model, the processing model is trained by a machine learning algorithm using a data set. The data set includes at least one training tooth image and at least one training tooth feature linked to each other. The processing model generates a target tooth feature from the training tooth image, and then the processing model is improved by measuring a scale of how different the target tooth feature is from the training tooth feature using a target algorithm. The processing model is adapted based on the measured scale.

[0057] There is provided a computer program product including instructions that cause a computer to execute the above-described procedure when the program is executed by the computer.

[0058] The computer is a processing unit. However, the processing unit or computer may be a digital user device such as a smartphone, server, microcontroller, laptop, tablet computer, PDA, or other intelligent system, or a cloud-based system.

[0059] Another aspect of the present invention is a method for automatically cleaning teeth, wherein at least one tooth feature is determined according to the method described above, and the cleaning process is controlled according to the tooth feature thus determined.

[0060] The cleaning process is performed, for example, using an automatically controlled toothbrush.

[0061] Hereinafter, the present invention will be described in more detail with reference to the examples shown in the drawings.

Brief Description of the Drawings

[0062]

Figure 1

Figure 2

Figure 3a

Figure 3b

Figure 4

Modes for Carrying Out the Invention

[0063] (Exemplary Embodiment) Description of the Device A system 1 for determining tooth features includes a learning unit 2, an execution unit 3, and a dental plaque detection device 4 for recording a tooth image 5.

[0064] The learning unit 2 includes a machine learning module 6 embodied to learn tooth features 8 using an assignment 7 of tooth images 5 to generate a processing model 9.

[0065] The execution unit 3 includes an application module 10 that automatically determines tooth features 11 from a tooth image 5 using the processing model 9.

[0066] The learning unit 2 and the execution unit 3 are usually processing units such as a computer. The machine learning module 6 and the application module 10 are software applications executable on these computers.

[0067] In this exemplary embodiment, the learning unit 2 and the execution unit 3 are two different computers connected to each other via a computer network to exchange the processing model 9.

[0068] In other exemplary embodiments, it is conceivable that the learning unit 2 and the execution unit 3 are mapped by the same computer.

[0069] The dental plaque detection device 4 preferably communicates wirelessly with the execution unit 3 via, for example, WLAN. However, the use of wired communication is also conceivable.

[0070] The dental plaque detection device 4 is described in detail in the unpublished German patent application No. 10 2022 102 045.2, and the entire patent application is incorporated herein by reference.

[0071] In principle, the dental plaque detection device 4 is designed such that the U-shaped portion l of the dental plaque detection device 4 is inserted into the user's oral cavity. The U-shaped portion of the dental plaque detection device 4 has a sensor array.

[0072] The U-shaped portion is arranged on the teeth such that all tooth surfaces are recognized by the sensor.

[0073] To make the biofilm more visible, a detection fluid is placed in the oral cavity before the detection process. The detection fluid interacts with the biofilm and causes the biofilm to emit light at different predetermined wavelengths under the influence of light having a predetermined wavelength.

[0074] The detection fluid is placed in a detection capsule that can be inserted into the dental plaque detection device 4. Then, the detection device extracts the detection fluid and injects it onto the teeth.

[0075] For this purpose, the dental plaque detection device 4 is composed of a handpiece, which may have a display on the side facing away from the user. On the side facing the user, there is a mouthpiece that guides the sensor unit inserted into the oral cavity onto the teeth.

[0076] The mouthpiece has at least one camera unit equipped with a camera.

[0077] The dimensions of the camera unit alone are, for example, 1x1x2.7 mm, and the diameter of the entire sensor unit including the protective glass, PCB (printed circuit board), filter, and camera holder is 8 mm, and the height is 3.8 mm. With these dimensions, the unit can be easily inserted into the oral cavity.

[0078] To improve the signal-to-noise ratio, an optical long-pass filter is preferably placed directly in front of the camera. This filter ideally has a cut-off wavelength of approximately 510 nm and cuts off signals below this value. Furthermore, a band-pass filter or a short-pass filter can be placed in front of the LED that illuminates the area to be detected to limit the wavelength spectrum of the LED.

[0079] Explanation of the method A method for determining the dental feature 11 will be described below.

[0080] The method starts from step S1 (Figure 4).

[0081] In the next step (S2), an image 5 of the teeth is manually assigned to the dental feature 11.

[0082] The tooth image 5 is an image of a tooth recorded by the dental plaque detection device 4. These generally show a strong signal-to-noise ratio of dental plaque, but it is difficult to recognize the boundaries of the teeth (in this case, the tooth feature 11). However, an expert can manually determine these boundaries.

[0083] When the tooth feature 11 is assigned to the tooth image 5, in the next step (S3), a processing model 9 is generated by machine learning of this manual assignment 7.

[0084] In the learning phase, first an empty (leeres) processing model 9 is used, which in this case is composed of a neural network. In this context, "empty" means that the neural network has not yet been trained with any data, but contains the necessary basic information and is ready for learning.

[0085] To learn the processing model 9, a training tooth image 5 is loaded into the processing model 9, and the processing model 9 outputs the target tooth feature 11 (Figure 3a).

[0086] To generate the training tooth feature 8, a tooth image is taken with the camera module of the dental plaque detection device 4, where a contrast agent and, if necessary, a color filter are not used. As a result, although the biofilm cannot actually be easily detected, a tooth image in which the boundaries of the teeth are clearly visible is obtained. This boundary can be calculated algorithmically or manually from the tooth image 5, and these represent the training tooth features.

[0087] The target algorithm 12 is used to measure the scale of how different the target tooth feature is from the training tooth feature assigned to the training tooth image 5.

[0088] The processing model 9 is automatically improved based on the scale.

[0089] This learning step is then repeated with the same training tooth image and / or different training tooth images.

[0090] This repetition is performed multiple times, and based on the repetition of this learning step, the processing model 9 is established.

[0091] In the subsequent step S4, the processing model 9 is transferred from the learning unit 2 to the execution unit 3.

[0092] In step S5, a new tooth image 5 is also recorded by the dental plaque detection device 4.

[0093] The new tooth image 5 is transferred from the dental plaque detection device 4 to the execution unit 3.

[0094] Next, in step S6, the tooth features 11 of the new tooth image 5 are generated using the processing model 9 (Fig. 3b).

[0095] This method ends in step S7.

[0096] According to one embodiment, the tooth surface visible to the camera is measured. In this case, for example, the height, depth, and tilt angle can be determined in comparison with the teeth in order to determine the tooth surface based on them.

[0097] According to a further embodiment, the teeth can be tracked, i.e., monitored, across a plurality of recorded tooth images. In this case, the teeth are identified on different tooth images. It is also possible to determine the movement of the teeth and calculate the movement of the camera from that.

Explanation of Signs

[0098] 1 System for determining tooth features 3 Execution unit 4 Dental plaque detection device 5 Tooth image 2 Learning unit 6 Learning module 7 Assignment 8 Tooth features for learning 9 Processing model 10 Application Module 11 Tooth Characteristics

Claims

1. A method for generating tooth characteristics (11), To provide a processing model (9), Record an image of at least one tooth (5), The processing model (9) includes determining the characteristics (11) of the tooth from the tooth image (5), Here, the processing model (9) is trained using a dataset, the dataset includes at least one linked learning tooth image (5) and learning tooth features (8), A method wherein the tooth features (11) represent at least the tooth boundaries in the tooth image (5).

2. The method according to claim 1, characterized in that the tooth image (5) is generated using a plaque detection device (4).

3. The method according to claim 2, characterized in that the plaque detection device (4) has an optical filter for passing light having a wavelength of 480 nm to 530 nm.

4. The method according to any one of claims 1 to 3, characterized in that additional parameters are considered for determining the characteristics (11) of the tooth.

5. The method according to claim 4, characterized in that the additional parameters include at least one of the following parameters: - dental prosthesis, - dental disease, - location of demineralized area, - location of caries, - location of filling, - location of calculus, - location of discoloration, - location of rough areas, - degree of tooth whiteness, - tooth misalignment, - tooth position, and / or - tooth type.

6. The method according to claim 1 or 2, characterized in that the machine learning is supervised machine learning.

7. In the application phase, the processing model (9) is improved by independent machine learning, in which the user manually adapts the tooth features (11) generated by the processing model (9). The method according to claim 1 or 2, characterized in that the processing model (9) is further trained using a dataset consisting of the tooth image (5) and the adapted tooth features (11).

8. A method for generating a processing model (9), The processing model is trained using a machine learning algorithm with a dataset, wherein the dataset includes at least one image of a learning tooth (5) and a feature of one learning tooth (8) that are linked to each other. The processing model (9) is improved by the processing model (9) generating target tooth features (11) from a learning tooth image (5), and then a target algorithm is used to measure how much the target tooth features (11) differ from the learning tooth features (8), wherein the processing model (9) is adapted based on the measured scale so that the tooth features (11) represent at least the tooth boundaries in the tooth image (5).

9. A computer program product comprising, when the program is executed by a computer, an instruction causing the computer to perform the method according to claim 1 or 2.

10. A method for automatically cleaning teeth, wherein at least one tooth characteristic is determined according to the method of claim 1 or 2, and the cleaning process is controlled according to the tooth characteristic thus determined.

Images