Method for monitoring orthodontic treatment with an invisible tooth aligner

Abstract

One aspect of this application provides a computer-executed monitoring method for orthodontic treatment using invisible braces, comprising: acquiring N photographs of a patient's teeth wearing invisible braces, wherein N is a natural number greater than or equal to 1, and the N photographs are obtained from different angles; for each of the N photographs, segmenting each tooth using a trained first deep neural network and assigning a corresponding tooth number to the segmented tooth; for each of the N photographs, segmenting the non-fitting regions between each individual tooth and the invisible braces using a trained second deep neural network; and for each non-fitting region segmented in each of the N photographs, calculating its cross-union ratio (CUI) with each tooth segmented from that photograph, and assigning the tooth number of the tooth with the highest CUI to the non-fitting region.

Description

Technical Field

[0001] This application generally relates to a method for monitoring orthodontic treatment using invisible aligners. Background Technology

[0002] Due to their advantages in aesthetics, convenience, and ease of cleaning, invisible braces are becoming increasingly popular. Orthodontic treatment using invisible braces typically requires dozens of sessions of wearing these braces to gradually reposition the patient's teeth from their initial alignment to the target alignment.

[0003] During orthodontic treatment using invisible aligners, changes in the patient's tooth alignment may deviate from the treatment plan, potentially affecting the outcome and, in extreme cases, even causing the aligners to come off. Therefore, patients are required to regularly return to their dental clinic for checkups to ensure the aligners are worn correctly. This places a significant burden on both patients and dental professionals.

[0004] Currently, methods have emerged that use photographs taken by patients themselves to detect the gap between the clear aligners and their teeth. This reduces the number of times patients need to visit dental clinics, significantly alleviating the burden on both patients and dental professionals. However, existing detection methods cannot identify some abnormalities, such as excessively large gaps between the clear aligners and teeth caused by missing teeth or erupting permanent teeth.

[0005] In view of the above, it is necessary to provide a new method for monitoring orthodontic treatment using invisible orthodontic appliances. Summary of the Invention

[0006] One aspect of this application provides a computer-executed monitoring method for orthodontic treatment using invisible braces, comprising: acquiring N photographs of a patient's teeth wearing invisible braces, wherein N is a natural number greater than or equal to 1, and the N photographs are obtained from different angles; for each of the N photographs, segmenting each tooth using a trained first deep neural network and assigning a corresponding tooth number to the segmented tooth; for each of the N photographs, segmenting the non-fitting regions between each individual tooth and the invisible braces using a trained second deep neural network; and for each non-fitting region segmented in each of the N photographs, calculating its cross-union ratio (CUI) with each tooth segmented from that photograph, and assigning the tooth number of the tooth with the highest CUI to the non-fitting region.

[0007] In some embodiments, the method for monitoring orthodontic treatment using invisible aligners further includes: the second deep neural network classifying each of the segmented misalignment regions, the classification category being selected from a set of predetermined categories, the set of predetermined categories including at least one category defined according to a predetermined range of misalignment degree, a category of misalignment caused by erupting teeth, and a category of misalignment caused by missing teeth.

[0008] In some implementations, the first and second deep neural networks are instance segmentation networks.

[0009] In some embodiments, the monitoring method for orthodontic treatment using invisible aligners further includes: acquiring the patient's actual tooth number information, which includes which teeth of the patient; for each of the N dental photographs, continuously encoding the teeth segmented therefrom in a predetermined direction; continuously encoding the teeth in the patient's actual tooth number information in the predetermined direction; and updating the tooth number assigned by the second deep neural network based on the correspondence between the two sets of encodings and the tooth number in the patient's actual tooth number information.

[0010] In some implementations, the patient's teeth are divided into four quadrants, and the patient's teeth are coded in quadrant groups.

[0011] In some embodiments, the monitoring method for orthodontic treatment using invisible aligners further includes: acquiring M photographs of the patient's teeth without invisible aligners, where M is a natural number greater than or equal to 1, and the M photographs are taken from different angles; for each of the M photographs, segmenting each tooth using a first deep neural network and assigning a corresponding tooth number to the segmented tooth; for each of the M photographs, detecting attachments using a trained third deep neural network; and for each attachment detected in each of the M photographs, calculating its cross-union ratio (CUI) with each tooth segmented from that photograph, and assigning the tooth number of the tooth with the highest CUI to the attachment.

[0012] In some implementations, the third deep neural network is an object detection network.

[0013] In some embodiments, the monitoring method for orthodontic treatment using invisible orthodontic appliances further includes: obtaining the patient's attachment information, including which teeth the patient has attachments on; and determining whether any attachments have fallen off based on the patient's attachment information and the detected attachments.

[0014] In some embodiments, the monitoring method for orthodontic treatment using invisible aligners further includes: acquiring the patient's actual tooth number information, which includes which teeth of the patient; for each of the M dental photographs, continuously encoding the teeth segmented therefrom in a predetermined direction; continuously encoding the teeth in the patient's actual tooth number information in the predetermined direction; and updating the tooth number assigned by the second deep neural network based on the correspondence between the two sets of encodings and the tooth number in the patient's actual tooth number information.

[0015] In some implementations, the patient's teeth are divided into four quadrants, and the patient's teeth are coded in quadrant groups.

[0016] Another aspect of this application provides a computer system for monitoring orthodontic treatment using invisible braces, comprising a storage device and a processor, the storage device storing a computer program for monitoring orthodontic treatment using invisible braces, and when the program is run, the processor executing the monitoring method for orthodontic treatment using invisible braces. Attached Figure Description

[0017] The above and other features of this application will be further described below with reference to the accompanying drawings and their detailed description. It should be understood that these drawings only illustrate several exemplary embodiments according to this application and should not be considered as limiting the scope of protection of this application. Unless otherwise specified, the drawings are not necessarily to scale, and similar reference numerals denote similar parts.

[0018] Figure 1 This is a schematic flowchart illustrating a computer-executed monitoring method for orthodontic treatment using invisible aligners, as described in one embodiment of this application.

[0019] Figure 2A This is an example of a photograph of teeth wearing invisible orthodontic appliances taken from the right side by a patient using an oral radiography device.

[0020] Figure 2B Demonstrates the first deep neural network pair Figure 2A The result of segmenting teeth from the photograph shown;

[0021] Figure 2C Demonstrates the second deep neural network pair Figure 2A The results of non-fitting area detection are shown in the dental photographs.

[0022] Figure 3A This is an example of a photograph of a patient's teeth taken from the right side using an oral radiography device, showing the teeth without clear aligners.

[0023] Figure 3B Demonstrates the first deep neural network pair Figure 3A The results of tooth segmentation from the dental photographs shown; and

[0024] Figure 3C Demonstrates the third deep neural network based on Figure 3A The results of accessory examination of the teeth shown in the photograph. Detailed Implementation

[0025] The following detailed description incorporates the accompanying drawings, which form part of this specification. The illustrative embodiments mentioned in the specification and drawings are for illustrative purposes only and are not intended to limit the scope of this application. Those skilled in the art will understand, based on the teachings of this application, that many other embodiments can be employed and various changes can be made to the described embodiments without departing from the spirit and scope of this application. It should be understood that the various aspects of this application illustrated herein can be arranged, substituted, combined, separated, and designed in many different configurations, all of which are within the scope of this application.

[0026] One aspect of this application provides a computer-executed monitoring method for orthodontic treatment using invisible aligners. This method utilizes a trained deep neural network to segment out the misfit areas between each individual tooth and the invisible aligner from photographs of the teeth of a patient wearing invisible aligners, and then classifies the segmented misfit areas.

[0027] Another aspect of this application provides a computer system for monitoring orthodontic treatment using invisible braces, comprising a storage device and a processor, the storage device storing a computer program for monitoring orthodontic treatment using invisible braces, and when the program is run, the processor executing the method for monitoring orthodontic treatment using invisible braces.

[0028] Orthodontic treatment using invisible braces typically requires dozens of sessions of invisible braces. By wearing these invisible braces one after another, the patient's teeth are gradually repositioned from the initial tooth layout to the target tooth layout.

[0029] Invisible aligners are made of transparent polymer material and are a single, integrated shell that houses the patient's dentition. Each aligner corresponds to a treatment step, used to reposition the patient's dentition (maxillary or mandibular) from its current position to the target position for that step. Typically, the geometry of the aligner's housing closely matches the target position for that treatment step.

[0030] When a patient first starts wearing an invisible aligner, the aligner undergoes elastic deformation because the current tooth layout of the patient's dentition is inconsistent with the target tooth layout for the corresponding orthodontic step. The elastic force generated by this deformation forces the teeth to move.

[0031] Ideally, when a period of wearing an invisible aligner is completed (e.g., two weeks), the patient's dentition will be in a position that is essentially consistent with the target dentition for the corresponding treatment step, and the invisible aligner will fit the patient's dentition closely.

[0032] However, in reality, due to various reasons, when a period of wearing an invisible braces ends, the patient's dentition may deviate from the target dentition for that treatment step. Consequently, the invisible braces may not fit the patient's dentition properly, or there may be gaps or separation between the braces and the patient's dentition. If the gaps or separations are too large, it indicates that the orthodontic treatment has deviated from the design plan, and in severe cases, it may even lead to the invisible braces coming off (making them unwearable).

[0033] Therefore, during orthodontic treatment using invisible braces, it is necessary to monitor the fit of the invisible braces so that dental professionals can adjust the orthodontic treatment plan in a timely manner when abnormalities are found.

[0034] In some cases, it is difficult to apply the necessary force to the teeth for orthodontic treatment using only the clear aligner itself. In such cases, protruding attachments need to be bonded to the tooth surface. The required force is applied to the teeth through friction and compression between the attachments and the clear aligner. However, the attachments may fall off during orthodontic treatment. Therefore, in addition to monitoring the fit of the clear aligner, the condition of the attachments also needs to be monitored.

[0035] Currently, dental imaging devices that allow patients to take their own dental photographs are available on the market, such as the one developed by the applicant. Product. The monitoring method for orthodontic treatment using invisible aligners in this application is based on dental photographs taken by the patient themselves. Thus, patients only need to provide their dental photographs to a dental professional, who can then monitor the patient's orthodontic treatment based on these photographs, reducing the number of follow-up visits required during orthodontic treatment and alleviating the burden on both patients and dental professionals.

[0036] Please refer to Figure 1 This is a schematic flowchart of a computer-executed monitoring method 100 for orthodontic treatment using invisible orthodontic appliances, according to one embodiment of this application.

[0037] In 101, obtain N photos of a patient's teeth wearing invisible orthodontic appliances taken from different angles.

[0038] Where N is a natural number greater than or equal to 1.

[0039] In a preferred embodiment, in order to detect the fit of each tooth to the invisible aligner, three photographs of the patient's teeth wearing the invisible aligner can be taken from the left, center, and right angles respectively.

[0040] Please refer to Figure 2A Here is an example of a photograph of the teeth wearing invisible orthodontic appliances taken from the right side by the patient using the aforementioned oral imaging device.

[0041] In one embodiment, the N dental photographs can be taken at the end of a period of wearing an invisible orthodontic appliance.

[0042] In step 103, for each of the N dental photographs, each tooth is segmented using a trained first deep neural network and assigned a corresponding tooth number.

[0043] In one embodiment, the first deep neural network may be an instance segmentation network, for example, it may be a YOLO (You Only Look Once) network or a Fast R-CNN network.

[0044] In one embodiment, the tooth number can be a tooth number using the FDI tooth position representation method.

[0045] Please refer to Figure 2B This demonstrates the first deep neural network's... Figure 2A The results of tooth segmentation from the dental photograph shown.

[0046] The inventors of this application discovered through extensive testing that if one of the teeth 3, 4, or 5 in a quadrant is extracted, the first deep neural network may not be able to assign it the correct tooth number.

[0047] To avoid assigning incorrect tooth numbers, the actual tooth numbers of the patient can be obtained and encoded in a continuous sequence from one side to the other (e.g., from mesial to distal, or vice versa). Similarly, the teeth detected by the first deep neural network are encoded in the same way. Then, based on the correspondence between the two sets of codes, the tooth numbers assigned by the first deep neural network are updated with the actual tooth number information of the patient. The actual tooth number information includes the number of teeth the patient has and their corresponding numbers; this information can be provided by the dental professional responsible for the case.

[0048] In one embodiment, the patient's teeth can be coded by quadrant, with teeth in each quadrant forming a group.

[0049] For example, based on the actual tooth number information of a case, tooth number 15 of the case was extracted, and its tooth number in the first quadrant and its code are shown in Table 1 below.

[0050]

[0051]

[0052] Table 1

[0053] The tooth numbers and their codes of the teeth in the first quadrant detected by the first deep neural network are shown in Table 2 below. Among them, tooth number 14 was misdetected as tooth number 15.

[0054] Tooth 11 12 13 15 16 17 coding A1 A2 A3 A4 A5 A6

[0055] Table 2

[0056] In step 105, for each of the N dental photographs, a trained second deep neural network is used to segment the non-fitting areas between each individual tooth and the invisible orthodontic appliance, and these areas are then classified.

[0057] The misfit area refers to the area corresponding to the gap between a tooth and the invisible aligner.

[0058] In one embodiment, the second deep neural network can be an instance segmentation network, for example, it can be a YOLO (You Only Look Once) network or a Fast R-CNN network.

[0059] The inventors of this application have discovered that, in order to determine whether there is an abnormality in the wearing of invisible orthodontic appliances, it is not necessary to know the specific value of the gap between the invisible orthodontic appliances and the teeth, but only to know which range the gap falls within.

[0060] Therefore, a deep neural network can be used to directly classify the detected misfit regions according to one or more pre-defined intervals. Correspondingly, the training data used to train this deep neural network needs to be labeled according to the pre-defined intervals.

[0061] For example, three intervals can be set as shown in Table 3 below.

[0062]

[0063]

[0064] Table 3

[0065] Accordingly, the photographs of individuals wearing invisible braces used to train the second deep neural network are labeled according to the three intervals in Table 1. For example, if a misfit area in a dental photograph represents a gap of less than or equal to 0.75 mm, it is labeled as "slight misfit". In this way, the second deep neural network can classify the detected misfit areas according to the intervals in Table 1.

[0066] It is understood that the misfit ranges listed above are merely examples, and dental professionals can set different misfit ranges based on experience data.

[0067] Furthermore, gaps caused by newly erupted permanent teeth and missing teeth are not considered abnormal wear conditions. To differentiate between these two situations, the second deep neural network can also classify non-fitting areas according to these two scenarios. Accordingly, the training data used to train this deep neural network includes data labeled according to these two scenarios.

[0068] In other words, in the above example, the second deep neural network classifies the non-fitting areas into five categories: slight non-fitting, moderate non-fitting, severe non-fitting, budding, and missing teeth.

[0069] Please refer to Figure 2C This demonstrates the effect of the second deep neural network on... Figure 2A The results of the non-fitting area detection were obtained from the dental photograph shown.

[0070] In step 107, for each detected misfit region, the crossover ratio (CRR) between the misfit region and each detected tooth region is calculated, and the tooth number corresponding to the tooth region with the highest CRR is assigned to the misfit region.

[0071] The calculation of the intersection-union ratio is well known in this field, so it will not be elaborated here.

[0072] After obtaining the results of the fit test for the invisible aligners, dental professionals can determine whether there are any abnormalities in the wearing of the invisible aligners, and whether it is necessary to provide compensating aligners or redesign the orthodontic treatment plan.

[0073] In 109, M photographs of the patient's teeth taken from different angles without the patient wearing invisible orthodontic appliances were obtained.

[0074] Where M is a natural number greater than or equal to 1.

[0075] To more accurately detect attachments on the tooth surface, photographs of the teeth of patients not wearing clear aligners can be used.

[0076] Based on the teachings of this application, it is understood that, in addition to photographs of the teeth of patients not wearing clear aligners, attachment testing can also be performed based on photographs of the teeth of patients wearing clear aligners. In this case, attachment testing and clear aligner misfit testing can be based on the same set of dental photographs.

[0077] In a preferred embodiment, in order to detect whether there are attachments on the surface of each tooth, three photos of the patient's teeth without clear aligners can be taken from the left, center and right angles respectively.

[0078] Please refer to Figure 3A This is an example of a dental photograph taken from the right side of the patient's teeth without invisible braces, using the aforementioned oral imaging device.

[0079] In step 111, for each of the M dental photographs, the first deep neural network is used to segment each tooth and assign it a corresponding tooth number.

[0080] This operation is similar to 103, so it will not be described in detail here.

[0081] In one embodiment, in order for the first deep neural network to accurately segment tooth regions in both photographs of teeth with and without clear aligners, the dataset used to train the first deep neural network may include photographs of teeth with and without clear aligners.

[0082] Please refer to Figure 3B This demonstrates the first deep neural network's... Figure 3A The results of tooth segmentation from the dental photograph shown.

[0083] Similar to 103, to avoid assigning incorrect tooth numbers, the tooth numbers assigned by the first deep neural network can be updated based on the actual tooth number information of the case.

[0084] In step 113, for each of the M dental photographs, attachments are detected using a trained third deep neural network.

[0085] In one embodiment, the third deep neural network can be an object detection network, for example, it can be a YOLO network or a Fast R-CNN network. It is understood that, in addition to an object detection network, the third deep neural network can also be an instance segmentation network.

[0086] Please refer to Figure 3C This demonstrates that the third deep neural network is based on Figure 3A The results of accessory examination of the teeth shown in the photograph.

[0087] In step 115, for each detected attachment, the crossover ratio (CRR) between it and each detected tooth region is calculated, and the tooth number corresponding to the tooth region with the highest CRR is assigned to that attachment.

[0088] At this point, the detected attachments and their corresponding tooth numbers can be compared with the attachment information in the patient's medical record to confirm whether any attachments have fallen off. If attachments are found to be detached, the patient can be notified to return to a dental clinic to have the detached attachments reattached to the tooth surface. The attachment information in the medical record includes which tooth surfaces the patient has attachments on, and this information can be provided by the dental professional handling the case.

[0089] The detection of non-fitting invisible orthodontic appliances and the detection of attachments share the first deep neural network for tooth segmentation, making full use of the first deep neural network.

[0090] Based on the detection of non-fitting and attachments of invisible aligners using dental photographs, a more comprehensive monitoring of orthodontic treatment using invisible aligners can be achieved.

[0091] Although various aspects and embodiments of this application have been disclosed herein, other aspects and embodiments of this application will be apparent to those skilled in the art upon inspiration from this application. The various aspects and embodiments disclosed herein are for illustrative purposes only and not for limiting purposes. The scope and spirit of this application are determined solely by the appended claims.

[0092] Similarly, the diagrams may illustrate exemplary architectures or other configurations of the disclosed methods and systems, which aid in understanding the features and functions that may be included in the disclosed methods and systems. The claims are not limited to the exemplary architectures or configurations shown, and the desired features may be implemented with various alternative architectures and configurations. Furthermore, the order of the blocks given herein with respect to flowcharts, functional descriptions, and method claims should not be limited to various embodiments implemented in the same order to perform the said functions, unless explicitly indicated in the context.

[0093] Unless otherwise expressly stated, the terms and phrases used herein, and their variations thereof, should be interpreted as open-ended rather than restrictive. In some instances, the appearance of extended words and phrases such as “one or more,” “at least,” “but not limited to,” or other similar expressions should not be construed as an intention or necessity to indicate a narrower scope in examples where such extended expressions might not exist.

Claims

1. A computer-executed monitoring method for orthodontic treatment using invisible aligners, comprising: Obtain N photographs of the teeth of a patient wearing invisible orthodontic appliances, where N is a natural number greater than or equal to 1, and the N dental photographs are obtained from different angles; For each of the N dental photographs, each tooth is segmented using a trained first deep neural network and assigned a corresponding tooth number. For each of the N dental photographs, a trained second deep neural network is used to segment the non-fitting areas between each individual tooth and the invisible orthodontic appliance; and For each non-fitting region segmented from each of the N dental photographs, calculate its cross-union ratio (CUI) with each tooth segmented from that dental photograph, and assign the tooth number of the tooth with the highest CUI to the non-fitting region.

2. The monitoring method for orthodontic treatment using invisible orthodontic appliances as described in claim 1, characterized in that, It also includes: the second deep neural network classifying each of the segmented misfit regions, the classification category being selected from a set of predetermined categories, the set of predetermined categories including at least one category defined according to a predetermined range of misfit degree, a category of misfit caused by erupting teeth, and a category of misfit caused by missing teeth.

3. The monitoring method for orthodontic treatment using invisible orthodontic appliances as described in claim 1, characterized in that, The first and second deep neural networks are instance segmentation networks.

4. The monitoring method for orthodontic treatment using invisible orthodontic appliances as described in claim 1, characterized in that, It also includes: Obtain the patient's actual tooth number information, which includes which teeth of the patient; For each of the N dental photographs, the teeth segmented from them are sequentially encoded according to a predetermined direction; The teeth in the patient's actual dental identification information are sequentially encoded according to the predetermined direction; and The tooth number is updated by the second deep neural network based on the correspondence between the two sets of codes and the tooth number in the patient's real tooth number information.

5. The monitoring method for orthodontic treatment using invisible orthodontic appliances as described in claim 4, characterized in that, The patient's teeth were divided into four quadrants, and the teeth were coded in a quadrant-group manner.

6. The monitoring method for orthodontic treatment using invisible orthodontic appliances as described in claim 1, characterized in that, It also includes: Obtain M photographs of the patient's teeth without wearing invisible orthodontic appliances, where M is a natural number greater than or equal to 1, and the M photographs are obtained from different angles; For each of the M dental photographs, the first deep neural network is used to segment each tooth and assign a corresponding tooth number to the segmented tooth; For each of the M dental photographs, attachments are detected using a trained third deep neural network; and For each attachment detected in each of the M dental photographs, calculate its cross-union ratio (CUI) with each tooth segmented from that dental photograph, and assign the tooth number of the tooth with the highest CUI to that attachment.

7. The monitoring method for orthodontic treatment using invisible orthodontic appliances as described in claim 6, characterized in that, The third deep neural network is an object detection network.

8. The monitoring method for orthodontic treatment using invisible orthodontic appliances as described in claim 6, characterized in that, It also includes: Obtain the patient's attachment information, including which teeth the patient has attachments on; and Based on the patient's accessory information and the detected accessories, it is determined whether any accessories have fallen out.

9. The monitoring method for orthodontic treatment using invisible orthodontic appliances as described in claim 6, characterized in that, It also includes: Obtain the patient's actual tooth number information, which includes which teeth of the patient; For each of the M dental photographs, the teeth segmented from them are sequentially encoded in a predetermined direction; The teeth in the patient's actual dental identification information are sequentially encoded according to the predetermined direction; and The tooth number is updated by the second deep neural network based on the correspondence between the two sets of codes and the tooth number in the patient's real tooth number information.

10. The monitoring method for orthodontic treatment using invisible orthodontic appliances as described in claim 9, characterized in that, The patient's teeth were divided into four quadrants, and the teeth were coded in a quadrant-group manner.

11. A computer system for monitoring orthodontic treatment using invisible aligners, comprising a storage device and a processor, the storage device storing a computer program for monitoring orthodontic treatment using invisible aligners, wherein when the program is run, the processor executes the monitoring method for orthodontic treatment using invisible aligners as described in claim 1.

Images