Method of generating digital data set representing target tooth layout

Abstract

One aspect of the invention provides a computer-implemented method for generating a collision relief and / or gap closure scheme. The method comprises the following steps: acquiring a first three-dimensional digital model representing upper and lower jaw dentitions under an initial tooth layout of a patient; obtaining a set of predetermined constraints; aligning the first three-dimensional digital models based on the constraint; using a trained first deep neural network to generate a collision release and / or gap closing scheme based on the first three-dimensional digital model and the aligned first three-dimensional digital model, and updating the constraint with the collision release and / or gap closing scheme; iterating the aligned first three-dimensional digital model and the process of generating the collision relieving and / or gap closing scheme for at least two times to obtain a final collision relieving and / or gap closing scheme.

Description

Technical Field

[0001] The present application generally relates to methods of generating a digital dataset representing a target tooth configuration in conjunction with a deep learning neural network and predetermined constraints. Background Art

[0002] With the rapid development of computer technology, dental diagnosis and treatment are increasingly relying on computer technology. When formulating a dental orthodontic treatment plan, a three-dimensional digital model representing the dentition under the target tooth layout is usually generated based on the three-dimensional digital model representing the dentition under the initial tooth layout.

[0003] At present, there are mainly two methods for generating a three-dimensional digital model of the dentition under the target tooth layout based on the three-dimensional digital model representing the dentition under the initial tooth layout. One is to align the teeth based on pre-established rules, which are generally derived from the doctor's experience. The inventors of the present application found that due to the large differences in tooth shape and posture (position and posture) of different patients, it is difficult to formulate a set of universal rules applicable to various situations. Therefore, this method has great limitations. The other is a method based on deep learning, for example, a deep neural network is trained with a training data set (including multiple sets of three-dimensional digital models representing the dentition under the initial layout and a three-dimensional digital model representing the dentition under the target layout), and the teeth are aligned using the deep neural network trained by the path. The inventors of the present application found that due to the lack of constraints, the results produced by this method may not meet the requirements of clinical medicine.

[0004] Therefore, it is necessary to provide a new method for generating a digital data set representing a target tooth layout. Summary of the invention

[0005] One aspect of the present application provides a computer-implemented method for generating a collision resolution and / or gap closing solution, comprising: obtaining a first three-dimensional digital model representing the upper and lower jaw dentition under the patient's initial tooth layout; obtaining a set of predetermined constraints; aligning the first three-dimensional digital model based on the constraints; using a trained first deep neural network to generate a collision resolution and / or gap closing solution based on the first three-dimensional digital model and the aligned first three-dimensional digital model, and updating the constraints with the collision resolution and / or gap closing solution; iterating the process of aligning the first three-dimensional digital model and generating the collision resolution and / or gap closing solution at least twice to obtain a final collision resolution and / or gap closing solution.

[0006] In some embodiments, the training digital data set for training the first deep neural network includes multiple sets of training data, each set of training data includes a three-dimensional digital model of the upper and lower dentition under an initial dental layout of a patient, a three-dimensional digital model of the upper and lower dentition under a target dental layout of the patient, and a collision resolution and / or space closing plan in the orthodontic treatment plan for the patient.

[0007] In some embodiments, the process of aligning the first three-dimensional digital model and generating the collision resolution and / or gap closing solution is iterated N times, where N is a predetermined natural number greater than or equal to 2.

[0008] In some embodiments, the predetermined constraints include target positions of selected teeth of a first dentition in a selected direction, and slice cutting and space reservation schemes of the maxillary and mandibular dentitions, wherein the first dentition is one of the maxillary and mandibular dentitions.

[0009] Another aspect of the present application provides a method for generating a digital data set representing a target tooth layout based on the collision resolution and / or gap closing scheme, comprising: using a trained second deep neural network to preliminarily align a first dental arch model of the first three-dimensional digital model; and performing a first optimization on the preliminarily aligned first dental arch model, the first optimization comprising: updating the preliminarily aligned first dental arch model according to the target position of the selected tooth in the selected direction; and optimizing the dental arch curve of the first dental arch model, while performing collision resolution and / or gap closing on the first dental arch model according to the portion corresponding to the first dental arch according to the collision resolution and / or gap closing scheme, the collision resolution and / or gap closing requirement being determined according to the portion corresponding to the first dental arch according to the slice cutting and gap reservation scheme.

[0010] In some embodiments, the collision resolution and / or gap closing scheme for the first dental arch model includes one of the following: tooth extraction, tooth implantation, posterior tooth distalization, posterior tooth mesialization, arch expansion, arch retraction, lip inclination, anterior tooth retraction, slice cutting, gap reservation and any combination of the above.

[0011] In some embodiments, in the dental arch curve optimization, the position of the selected tooth of the first dentition in the selected direction is maintained unchanged at the target position.

[0012] In some embodiments, the updating of the preliminarily aligned first dental arch model and the optimization of the dental arch curve of the first dental arch model as well as the collision elimination and / or gap closing are performed simultaneously.

[0013] In some embodiments, the method for generating a digital data set representing a target tooth layout further includes: using a trained third deep neural network to preliminarily align a second dental arch model of the first three-dimensional digital model based on the first optimized first dental arch model, the second dental arch being the opposing dental arch of the first dental arch; performing a second optimization on the preliminarily aligned second dental arch model, the second optimization comprising: performing occlusal optimization on the preliminarily aligned second dental arch model based on the first optimized first dental arch model, so that the occlusal relationship between the second optimized second dental arch model and the first optimized first dental arch model meets the requirements, and at the same time, performing collision relief and / or gap closing on the preliminarily aligned second dental arch model according to the collision relief and / or gap closing scheme corresponding to the second dental arch, the collision relief and / or gap closing requirement being determined according to the slice cutting and gap reservation scheme corresponding to the second dental arch.

[0014] In some embodiments, the collision relief and / or gap closing scheme for the second dentition includes one of the following: tooth extraction, tooth implantation, posterior tooth distalization, posterior tooth mesialization, arch expansion, arch retraction, lip inclination, anterior tooth retraction, slice cutting, gap reservation and any combination of the above.

[0015] In some embodiments, the method of generating a digital data set representing a target tooth layout further includes: further aligning the first optimized first dental arch model and the second optimized second dental arch model using a trained fourth deep neural network.

[0016] In some embodiments, the first dentition is the maxillary dentition and the second dentition is the mandibular dentition.

[0017] In some embodiments, the target position of the selected tooth in the selected direction includes one of the following: a tooth midline target position, anterior tooth sagittal target position, anterior tooth vertical target position, posterior tooth vertical target position, posterior tooth sagittal target position, posterior tooth horizontal target position, canine horizontal target position, and any combination of the above. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The above and other features of the present application will be further described below in conjunction with the accompanying drawings and detailed descriptions thereof. It should be understood that these drawings only illustrate several exemplary embodiments according to the present application and should not be considered as limiting the scope of protection of the present application. Unless otherwise specified, the drawings are not necessarily to scale and similar reference numerals represent similar components.

[0019] Figure 1 A schematic flow chart of a method for generating a digital data set representing a target tooth layout in one embodiment of the present application;

[0020] Figure 2 The basic structure of the first deep neural network in one embodiment of the present application is schematically shown;

[0021] Figure 3 A schematic flow chart of a method for generating constraints for tooth arrangement in one embodiment of the present application;

[0022] Figure 4 An example is shown in which the target position of the maxillary anterior teeth (including sagittal and vertical directions) and the vertical target position of the maxillary posterior teeth are marked on a lateral radiograph by the fifth deep neural network; and

[0023] Figure 5 A schematic module diagram of a computer program for generating a collision resolution and gap closing solution in one embodiment of the present application. DETAILED DESCRIPTION

[0024] The following detailed description refers to the drawings that form part of this specification. The illustrative embodiments mentioned in the specification and drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present application. Under the guidance of the present application, those skilled in the art will understand that many other embodiments can be adopted and various changes can be made to the described embodiments without departing from the subject matter and scope of protection of the present application. It should be understood that the various aspects of the present application described and illustrated herein can be arranged, replaced, combined, separated and designed according to many different configurations, and these different configurations are within the scope of protection of the present application.

[0025] One aspect of the present application provides a computer-implemented method for generating a digital data set representing a target tooth layout, which combines tooth arrangement using a deep neural network and optimization based on pre-established constraints, has wide applicability, and the results generated meet clinical medical requirements and the special requirements of dental professionals for orthodontic treatment of specific cases (if any).

[0026] The target tooth arrangement refers to the tooth arrangement that is expected to be achieved by orthodontic treatment. It is understandable that the target tooth arrangement may be different for different patients.

[0027] Please refer to Figure 1 , which is a schematic flowchart of a method 100 for generating a digital data set representing a target tooth layout and executed by a computer in one embodiment of the present application.

[0028] In 101 , a first three-dimensional digital model of a first dentition and a second three-dimensional digital model of a second dentition representing an initial tooth layout are acquired.

[0029] The first dentition may be the maxillary dentition or the mandibular dentition of the patient, and the second dentition is the opposing dentition of the first dentition.

[0030] The initial tooth layout is the layout of the patient's teeth based on which the digital data set representing the target tooth layout is generated using the method of the present application. For example, the initial tooth layout can be the patient's tooth layout before orthodontic treatment, or it can be the patient's tooth layout at any time during orthodontic treatment (re-formulating the treatment plan during orthodontic treatment).

[0031] The first and second three-dimensional digital models are segmented three-dimensional digital models, that is, the teeth are independent of each other and each tooth can be moved individually.

[0032] In one embodiment, the three-dimensional digital model representing the dentition of the patient in the initial tooth layout can be obtained by directly scanning the patient's jaw. In another embodiment, the three-dimensional digital model representing the dentition of the patient in the initial tooth layout can be obtained by scanning a physical model of the patient's jaw, such as a plaster model. In another embodiment, the three-dimensional digital model representing the dentition of the patient in the initial tooth layout can be obtained by scanning an impression of the patient's jaw.

[0033] In 103, the first three-dimensional digital model is preliminarily aligned using a trained first deep neural network.

[0034] In one embodiment, the first three-dimensional digital model can be input into the trained first deep neural network, which preliminarily aligns the first three-dimensional digital model and outputs the rotation matrix and translation matrix of each tooth. These rotation matrices and translation matrices are applied to the first three-dimensional digital model to obtain the preliminarily aligned first three-dimensional digital model.

[0035] In one embodiment, the training data set for training the first deep neural network may include multiple sets of training data. When the first dentition is the maxillary dentition, each set of training data includes a three-dimensional digital model of the misaligned maxillary dentition and a three-dimensional digital model of the aligned maxillary dentition.

[0036] In one embodiment, the first deep neural network may be a Transformer-based deep neural network.

[0037] Please refer to Figure 2 , schematically illustrating the basic structure of the first deep neural network 200 in one embodiment of the present application.

[0038] The first deep neural network 200 includes a Point Encoder 201 , a Transformer Layer 203 , and a Transformer Decoder 205 .

[0039] Before inputting the first three-dimensional digital model into the first deep neural network 200, it is first converted into a point cloud.

[0040] Point Encoder 201 encodes a single tooth and the neighborhood of the single tooth selected by Point Encoder 201 to generate an Embedding (also referred to as a feature vector) of the single tooth. In one embodiment, Point Encoder 201 may encode based on the three-dimensional coordinates and normal vector of the point.

[0041] The Transformer Layer 203 merges the Embeddings of the teeth of the first dentition to generate the Embeddings of the teeth of the first dentition that are preliminarily aligned.

[0042] Transform Decoder 205 converts the embedding of each tooth of the first dentition that has been preliminarily aligned into a rotation matrix and a translation matrix. The rotation matrix and the translation matrix corresponding to each tooth represent the difference between the initial position (including position and posture) of the tooth and its position after the preliminarily alignment. These rotation matrices and translation matrices are applied to the corresponding teeth of the first three-dimensional digital model to obtain the first three-dimensional digital model that has been preliminarily aligned.

[0043] In light of the present application, it can be understood that, in addition to the above network structure, the first deep neural network can adopt any other applicable neural network.

[0044] In a preferred embodiment, the first dentition is the maxillary dentition. In the light of the present application, it can be understood that the first dentition can also be the mandibular dentition.

[0045] In 105 , a first optimization is performed on the preliminarily aligned first three-dimensional digital model based on predetermined constraints.

[0046] In the process of formulating an orthodontic treatment plan, some parameters of the orthodontic treatment goal can be determined based on the specific situation of the patient's teeth, such as the target position of the selected teeth in the selected direction, the collision relief and / or space closing plan, and the slice cutting and space reservation plan based on the analysis of the tooth width.

[0047] In one embodiment, the target position of the selected tooth in the selected direction may include one of the following: tooth midline target position, anterior tooth sagittal target position, anterior tooth vertical target position, posterior tooth sagittal target position, posterior tooth horizontal target position, canine horizontal target position, posterior tooth vertical target position and any combination of the above.

[0048] In one embodiment, the first three-dimensional digital model of the preliminary alignment can be operated accordingly according to the target position of the selected teeth of the first dental row in the constraint in the selected direction, that is, the position of the selected teeth of the first three-dimensional digital model of the preliminary alignment in the selected direction can be repositioned to the target position.

[0049] In one embodiment, the tooth centerline target position can be achieved by adjusting the position of the anterior teeth in the mesiodistal direction.

[0050] Moving the pre-selected teeth in the preliminarily aligned first three-dimensional digital model to the target position in the pre-selected direction may cause the dental arch curve of the first three-dimensional digital model to be not smooth enough. In one embodiment, while moving the pre-selected teeth to the target position in the pre-selected direction, the dental arch curve can be optimized.

[0051] In one embodiment, an initial dental arch curve may be fitted based on the preliminarily aligned first three-dimensional digital model, and the dental arch curve may be optimized based on the initial dental arch curve.

[0052] The method of fitting the dental arch curve and the characteristic points used are well known in the art. For example, a spline curve or a polynomial function can be used to fit the dental arch curve. For example, the FA point of the tooth, or the molar cusp, canine cusp and central incisor midpoint can be used as characteristic points to fit the dental arch curve.

[0053] In one embodiment, the pre-selected teeth may be moved one by one to the target position in the pre-selected direction, or a plurality of the pre-selected teeth may be moved simultaneously to the target position in the pre-selected direction.

[0054] In one embodiment, for the pre-selected teeth that have been moved to the target position in the pre-selected direction, their feature points are used as control points, and during the optimization process of the dental arch curve, the positions of these control points are kept fixed, so that the optimization of the dental arch curve will not change the posture of these teeth, thereby preventing the posture of these teeth from deviating from the target position in the pre-selected direction.

[0055] In one embodiment, while optimizing the dental arch curve of the first three-dimensional digital model, collision contact and / or gap closing can also be performed on it, so that the first optimized first three-dimensional digital model meets the collision and gap requirements. If the constraint includes a collision elimination and / or gap closing scheme for the first dentition, then the collision contact and / or gap closing is performed according to the scheme.

[0056] The collision resolution and / or space closing plan may include one of the following: tooth extraction, tooth implantation, arch expansion, arch reduction, posterior tooth distalization, posterior tooth mesialization, lip inclination, anterior tooth retraction, slice cutting, space reservation and any combination of the above.

[0057] In one embodiment, the collision resolution and / or gap closing scheme may only specify which means may be used to perform collision resolution and / or gap closing without specifying specific parameters for these means.

[0058] If the constraints include a slice cutting and / or space reservation scheme for the first dentition obtained based on an analysis of tooth width, the requirements for collision resolution and / or gap closing can be determined based on the slice cutting and / or space reservation scheme, and the first three-dimensional digital model can be subjected to collision resolution and / or gap closing according to the requirements.

[0059] In one embodiment, the allowed collision amount may represent the slice amount, and the allowed gap amount may represent the gap reserve amount, so that the first 3D digital model does not need to be modified before collision resolution and / or gap closing is performed on the first 3D digital model.

[0060] In another embodiment, before performing collision elimination and / or gap closing on the first three-dimensional digital model, the first three-dimensional digital model may also be modified according to the slice cutting and / or gap reservation scheme. For example, the corresponding teeth may be sliced ​​according to the slice cutting amount, and denture cavitation may be added according to the gap reservation amount, and then the modified first three-dimensional digital model may be subjected to collision elimination and / or gap closing. It should be noted that both schemes belong to the above-mentioned determination of the requirements for collision elimination and / or gap closing based on the slice cutting and / or gap reservation scheme, and collision elimination and / or gap closing are performed on the first three-dimensional digital model according to the requirements.

[0061] In yet another embodiment, the pre-selected teeth may be moved to the target position in the pre-selected direction first, and then the dental arch curve may be optimized and the collision contact and / or gap closing may be performed.

[0062] The pre-selected teeth of the first three-dimensional digital model that have been preliminarily aligned are moved to the target position in the pre-selected direction, and the dental arch curve is optimized and the collision contact and / or gap closing are performed on the first three-dimensional digital model, so as to obtain the first optimized three-dimensional digital model, which meets the aesthetic requirements as well as the clinical medical requirements and the special requirements (if any) of dental professionals for orthodontic treatment of the case.

[0063] Next, the second three-dimensional digital model may be aligned based on the first optimized first three-dimensional digital model.

[0064] In 107 , the second three-dimensional digital model is preliminarily aligned based on the first optimized first three-dimensional digital model using a trained second deep neural network.

[0065] The first optimized first three-dimensional digital model and the second three-dimensional digital model are input into the trained second deep neural network, which uses the first optimized first three-dimensional digital model as a reference to preliminarily align the second three-dimensional digital model and outputs a preliminarily aligned second three-dimensional digital model.

[0066] In one embodiment, the second deep neural network may adopt the same structure as the first deep neural network. Under the enlightenment of this application, it can be understood that the second deep neural network may also adopt any other applicable deep neural network.

[0067] Unlike the first deep neural network, the input of the second deep neural network includes a three-dimensional digital model of the aligned first dentition and a three-dimensional digital model of the unaligned second dentition. When preliminarily aligning the second three-dimensional digital model, not only the positional relationship between the corresponding teeth of the second dentition is considered, but also the relative positional relationship between each tooth of the second dentition and the relevant teeth of the first dentition is considered.

[0068] In one embodiment, the training data set used to train the second deep neural network may include multiple groups of training data. When the first dentition is the maxillary dentition, each group of training data includes a three-dimensional digital model of the aligned maxillary dentition, a three-dimensional digital model of the misaligned mandibular dentition, and a three-dimensional digital model of the aligned mandibular dentition.

[0069] In 109 , a second optimization is performed on the preliminarily aligned second three-dimensional digital model based on the predetermined constraints and the first optimized first three-dimensional digital model.

[0070] The occlusal relationship between the first optimized first three-dimensional digital model and the preliminarily aligned second three-dimensional digital model may not meet the requirements.

[0071] Therefore, the first optimized first three-dimensional digital model can be used as a reference to perform occlusal optimization on the preliminarily aligned second three-dimensional digital model, so that the occlusal relationship of the upper and lower teeth is normal after optimization.

[0072] In one embodiment, an energy function may be defined according to the occlusal relationship between the upper and lower teeth, and then an optimization algorithm may be used to optimize the overall position of the updated second three-dimensional digital model and the posture of the corresponding teeth.

[0073] In one embodiment, the occlusal relationship of the upper and lower teeth can be defined based on the positional relationship between key points on the corresponding teeth, such as the cusp point and the central groove.

[0074] It can be understood that the optimization based on the energy function can adopt any applicable optimization algorithm, such as second-order Newton optimization, numerical optimization, stochastic gradient descent optimization, etc.

[0075] In one embodiment, if the constraint includes a collision resolution and / or gap closing scheme for the second dental arch, during the occlusion optimization process, the collision resolution and / or gap closing scheme can be used to simultaneously resolve collisions and / or close gaps on the second three-dimensional digital model that is initially aligned, so that the second optimized second three-dimensional digital model meets the collision and gap requirements.

[0076] The collision resolution and / or space closing plan may include one of the following: tooth extraction, tooth implantation, arch expansion, arch reduction, posterior tooth distalization, posterior tooth mesialization, lip inclination, anterior tooth retraction, slice cutting, space reservation and any combination of the above.

[0077] For some special cases, the desired occlusal relationship is different from the standard occlusal relationship. For example, for overbite and overjet, it is desired to correct to a deep overbite instead of a normal overbite. In this case, the constraint may include this special occlusal relationship provided by the dental professional, so that the occlusal optimization is performed according to the special occlusal relationship.

[0078] If the constraint includes a slice cutting and / or space reservation scheme for the second dentition obtained based on the analysis of the tooth width, the requirement for collision elimination and / or space closing can be determined based on the slice cutting and / or space reservation scheme. Similarly, in one embodiment, the slice cutting amount can represent the allowable collision amount, and the space reservation amount can represent the allowable space amount, and when the collision elimination and / or space closing of the second three-dimensional digital model is performed, it is performed according to the requirements.

[0079] At this time, the first three-dimensional digital model after the first optimization and the second three-dimensional digital model after the second optimization may not completely conform to the statistical laws of aligning teeth. Therefore, the trained third deep neural network can be used to further align the two to make them further conform to the statistical laws of aligning teeth.

[0080] In 111, the first optimized first three-dimensional digital model and the second optimized second three-dimensional digital model are further aligned using a trained third deep neural network.

[0081] The first optimized first three-dimensional digital model and the second optimized second three-dimensional digital model are input into the trained third deep neural network, which further aligns the first and second three-dimensional digital models, and outputs the further aligned first and second three-dimensional digital models, which are the goals of the orthodontic treatment plan.

[0082] In light of this application, it can be understood that, based on different network designs, the third deep neural network can output a three-dimensional digital model, or a translation and rotation matrix of each tooth, or a target position of each tooth, which can all be referred to as a digital data set representing the target tooth layout. Similarly, the first and second deep neural networks can also output a three-dimensional digital model of the dentition, or a translation and rotation matrix or position of each tooth in the dentition based on different network designs.

[0083] In one embodiment, the third deep neural network may adopt the same structure as the first deep neural network. Under the inspiration of this application, it can be understood that the third deep neural network may also adopt any other applicable deep neural network.

[0084] In one embodiment, the training data set used to train the third deep neural network may include multiple sets of training data, each set of training data including a three-dimensional digital model of misaligned upper and lower dentition and a three-dimensional digital model of aligned upper and lower dentition.

[0085] Another aspect of the present application provides a computer-implemented method for generating the constraint conditions.

[0086] In one embodiment, the constraints may be generated based on a three-dimensional digital model representing the initial dental configuration of the patient, a frontal open-mouth radiograph, and a lateral radiograph.

[0087] Please refer to Figure 3 , which is a schematic flowchart of a method 300 for generating constraint conditions for tooth arrangement executed by a computer in one embodiment of the present application.

[0088] In 301 , a third three-dimensional digital model, a frontal open-mouth radiograph, and a lateral radiograph representing an initial dental arrangement of the patient are acquired.

[0089] The method of obtaining the third three-dimensional digital model, the front opening photograph, and the lateral view showing the initial dental layout of the patient is well known in the art and will not be described in detail here.

[0090] The front open-mouth photo is a photo of the patient taken from the front with the mouth open and teeth showing. A typical example is a front smiling photo of the patient.

[0091] In one embodiment, the third three-dimensional digital model includes the upper and lower jaw dentition in an occluded state, that is, it includes the first and second three-dimensional digital models in an occluded state.

[0092] In 303, the third three-dimensional digital model and the front opening photo are first registered.

[0093] In one embodiment, the third three-dimensional digital model and the front opening photograph may be registered using a simple size matching method.

[0094] In some cases, the direction of the front opening shot may not be strictly correct. Therefore, in one embodiment, a trained deep neural network can be used to adjust the direction of the front opening shot to correct it.

[0095] In one embodiment, the training data set for straightening the front open-face photo includes multiple sets of training data, each set of training data includes an original front open-face photo and the rotation angle required to straighten it.

[0096] In one embodiment, the third three-dimensional digital model can be projected in a front-to-back direction, and then the front open-mouth photo can be scaled so that the teeth in the front open-mouth photo overlap with the corresponding teeth in the projection. After registration, the physical size of the front open-mouth photo based on the third three-dimensional digital model is obtained.

[0097] In light of the present application, it can be understood that, in addition to the size matching method, the registration of the third three-dimensional digital model and the front opening photograph may also be performed by any other applicable method.

[0098] In 305, the third three-dimensional digital model and the lateral film are second-registered.

[0099] The registration of the third three-dimensional digital model and the lateral film can be regarded as a process of searching for camera intrinsic parameters and extrinsic parameters, that is, finding a set of camera intrinsic parameters and extrinsic parameters in the parameter space, and the projection of the key points pre-selected on the third three-dimensional digital model based on the set of camera intrinsic parameters and extrinsic parameters has the highest matching degree with the corresponding key points in the lateral film.

[0100] In one embodiment, the registration may adopt a PnP (Perspective-n-Point) algorithm or an iterative optimization method.

[0101] After registration, the physical size of the lateral view based on the third three-dimensional digital model is obtained.

[0102] In 307, a trained fourth deep neural network is used to generate a facial midline based on the frontal open-mouth radiograph, and a tooth midline target position is determined based on the position of the facial midline and the result of the first registration.

[0103] In one embodiment, it is only necessary to determine the position of the plane midline in the horizontal direction.

[0104] In one embodiment, the fourth deep neural network can be used to generate a heat map based on the front open-face photo, and then the heat map is post-processed to obtain the position of the face midline. In the heat map, the key point used to generate the face midline has the highest value. In one example, in the heat map, the nose tip point has the highest value, and a vertical straight line through the nose tip point is used to obtain the face midline.

[0105] In one embodiment, the training data set used to train the fourth deep neural network may include multiple groups of training data, each group of training data includes a frontal open-face photo and the position of the midline of the frontal open-face photo.

[0106] In one embodiment, the fourth deep neural network may adopt HRNet. In light of the present application, it can be understood that in addition to HRNet, the fourth neural network may also adopt any other applicable deep neural network.

[0107] In light of the present application, it can be understood that in addition to deep neural networks, a regression method using machine learning can also be used to generate a facial midline based on the frontal open-face photo.

[0108] In another embodiment, the fourth deep neural network can also be used to directly predict the position of the face midline based on the front open-face photo without converting the front open-face photo into a heat map. For example, the fourth deep neural network can be used to determine the position of the nose tip in the front open-face photo, and a straight line perpendicular to the horizontal direction can be drawn through the nose tip position to obtain the face midline.

[0109] After the facial midline is obtained, the tooth midline target position of the third three-dimensional digital model may be determined based on the result of the first registration.

[0110] In 309, the trained fifth deep neural network is used to predict the sagittal target position of the anterior teeth, the vertical target position of the anterior teeth, the sagittal target position of the posterior teeth, and the vertical target position of the posterior teeth in the lateral film based on the lateral film, and the sagittal target position of the anterior teeth, the vertical target position of the anterior teeth, the sagittal target position of the posterior teeth, and the vertical target position of the posterior teeth in the third three-dimensional digital model are calculated based on the result of the second registration and the predicted target positions.

[0111] In one embodiment, the fifth deep neural network may adopt HRNet. In light of the present application, it can be understood that in addition to HRNet, the fifth neural network may also adopt any other applicable deep neural network.

[0112] In one embodiment, the training data set used to train the fifth deep neural network may include multiple groups of training data, each group of training data includes a lateral view film without the target position marked and a lateral view film with the target position marked.

[0113] Please refer to Figure 4 , showing an example of the target position (including sagittal target position and vertical target position) of the maxillary anterior teeth (tooth numbers 11 and 21) and the lateral radiographs of the vertical target position of the maxillary posterior teeth (tooth numbers 16 and 26) annotated by the fifth deep neural network.

[0114] Since the lateral view includes not only teeth but also other tissues, during training, the fifth deep neural network not only learns the positional relationship between teeth, but also learns the positional relationship between teeth and other corresponding tissues, so that the predicted target position further meets the medical requirements.

[0115] In 311, a trained sixth deep neural network is used to predict a target position of the posterior teeth in the horizontal direction based on the WALA ridge and corresponding key points of the first or second three-dimensional digital model.

[0116] It should be noted that the horizontal direction in this application refers to the left and right direction of the patient's jaw.

[0117] In one embodiment, the trained seventh deep neural network may be used to annotate the WALA ridges and pre-selected key points of each tooth in the first or second three-dimensional digital model, such as the FA point and the cusp point.

[0118] In one embodiment, the seventh deep neural network may adopt a Point Transformer network. In the light of this application, it can be understood that in addition to Point Transformer, the seventh deep neural network may also adopt any other applicable network.

[0119] Next, the WALA ridge and key points are input into the sixth deep neural network to obtain the horizontal target position of the posterior teeth.

[0120] In one embodiment, the sixth deep neural network may adopt HRNet. In light of the present application, it can be understood that in addition to HRNet, the sixth neural network may also adopt any other applicable deep neural network.

[0121] In one embodiment, the training data set used to train the sixth deep neural network can be extracted from existing tooth arrangement cases.

[0122] In one embodiment, the sixth deep neural network may be used to predict the horizontal target position of the posterior teeth and / or canines of the first dentition based on the WALA ridge of the first three-dimensional digital model and selected key points of each tooth.

[0123] In another embodiment, the sixth deep neural network can also be used to predict the horizontal target position of the posterior teeth of the second dental arch based on the WALA ridge and selected key points of each tooth of the second three-dimensional digital model, and then calculate the horizontal target position of the posterior teeth of the first dental arch according to the normal occlusal relationship.

[0124] According to clinical experience, the mandible is usually more stable. Therefore, if the first dentition is the maxillary dentition, a preferred implementation is to use the sixth neural network to predict the horizontal target position of the posterior teeth of the second dentition based on the WALA ridge of the second three-dimensional digital model and the selected key points of each tooth, and then calculate the horizontal target position of the posterior teeth of the first dentition according to the normal occlusal relationship.

[0125] In one embodiment, the positions of the corresponding key points of the teeth may be used to represent the sagittal target position of the anterior teeth, the vertical target position of the anterior teeth, the vertical target position of the posterior teeth, and the horizontal target position of the posterior teeth.

[0126] In 313, the trained eighth deep neural network is used to predict whether slice cutting or reserved gaps are required on the left and right sides of each tooth based on the width of each tooth in the third three-dimensional digital model.

[0127] In orthodontic treatment, in addition to tooth position, the width ratio between the upper and lower teeth, namely the Bolton index, needs to be considered. For example, the ratio of the total crown width of the upper and lower anterior teeth, and the ratio of the total crown width of all teeth in the upper and lower jaws. Based on the Bolton index, it can be determined whether there is a problem of inconsistent crown width between the patient's upper and lower dentition.

[0128] The eighth deep neural network is essentially a Bolton exponential optimization network.

[0129] The width and tooth number of each tooth in the third three-dimensional digital model are input into the eighth deep neural network to obtain the required slice cutting amount and reserved gap amount on the left and right sides of each tooth.

[0130] In one embodiment, for teeth that are too small, it is not recommended to slice the opposing teeth in order to obtain the required tooth width ratio. In this case, gaps can be reserved on both sides of the teeth, which is equivalent to artificially increasing their width.

[0131] In one embodiment, the width of a tooth in the third three-dimensional digital model may be the maximum width of the tooth in the direction of the Y-axis (the coordinate axis having the smallest angle with the mesiodistal direction) of its local coordinate system.

[0132] In one embodiment, the sixth deep neural network may adopt Tranformer. In light of the present application, it can be understood that in addition to Transformer, the sixth neural network may also adopt any other applicable deep neural network, such as CNN (Convolutional Neural Network) or RNN (Recurrent Neural Network).

[0133] The training data set of the eighth deep neural network includes multiple sets of training data, each set of training data includes the tooth number and width of each tooth in an existing case and the slice cutting amount and / or reserved gap amount of each tooth given by optimizing the Bolton index in the orthodontic treatment plan of the case. It should be noted that the training data set of the eighth deep neural network may include slice cutting and space reservation plans manually designed by dental professionals based on the specific circumstances of actual cases, and some of these plans may not strictly comply with the standard Bolton index.

[0134] So far, the target position of the pre-selected teeth in the pre-selected direction and the slice cutting amount or gap reserve amount required for each tooth to meet the upper and lower teeth width ratio requirements have been obtained. However, there is still a lack of collision elimination and gap closing solutions.

[0135] In 315, the third three-dimensional digital model is arranged based on the constraints that have been obtained, and a collision resolution and gap closing solution is generated based on the aligned third three-dimensional digital model. The constraints are then updated based on the generated collision resolution and gap closing solution, and a final collision resolution and gap closing solution is obtained after multiple iterations.

[0136] Ways to resolve collisions and close gaps include, but are not limited to: tooth extraction, tooth implants, slice cutting, reserved gaps, molar distalization, molar mesialization, arch expansion, arch retraction, lip inclination, anterior tooth retraction, slice cutting, and space reservation.

[0137] In one embodiment, the tooth arrangement method used to generate the collision resolution and gap closing solution can be the above-mentioned method 100 for generating a digital data set representing the target tooth layout. In the light of the present application, it can be understood that in addition to the above-mentioned tooth arrangement method, any other suitable tooth arrangement method can also be used.

[0138] Please refer to Figure 5, schematically showing a module diagram of a computer program for generating collision resolution and gap closing solutions in one embodiment of the present application.

[0139] The obtained constraints 401 and the third three-dimensional digital model 403 are input into the tooth arrangement module 405, which generates a tooth arrangement plan (i.e., the aligned third three-dimensional digital model) and outputs it to the collision elimination and gap closing module 407. Since the constraints at this time do not include the collision elimination and gap closing plan, collision elimination and gap closing may not be performed during the first tooth arrangement.

[0140] In one embodiment, the tooth arrangement module 405 may be a computer program that implements the method 100 of generating a digital data set representing a target tooth arrangement.

[0141] The collision resolution and gap closing module 407 generates a collision resolution and gap closing scheme based on the initial third three-dimensional digital model and the aligned third three-dimensional digital model. The collision resolution and gap closing scheme generated for the first time may not be perfect, that is, the collision amount and gap amount of the tooth arrangement result obtained by performing tooth arrangement again based on the obtained constraints and the collision resolution and gap closing scheme may not meet the requirements, or be greater than the predetermined collision threshold and gap threshold. Therefore, the current constraints can be updated based on the current collision resolution and gap closing scheme, and the tooth arrangement can be performed again based on the updated constraints and a new collision resolution and gap closing scheme can be generated, and it can be iterated multiple times until the collision and gap amount of the tooth arrangement result obtained meet the requirements.

[0142] In one embodiment, the number of iterations N can be preset, and the iteration is stopped after N iterations, and the collision elimination and gap closing scheme outputted last is used as the final result. In another embodiment, the collision amount threshold and the gap amount threshold can also be preset, and the iteration is stopped when the collision amount and the gap amount of the tooth arrangement result obtained by the iteration converge to less than the collision amount threshold and the gap amount threshold, and the collision elimination and gap closing scheme outputted last is used as the final result.

[0143] In one embodiment, a trained ninth deep neural network may be used to generate collision resolution and gap closing solutions based on the initial third three-dimensional digital model and the third three-dimensional digital model aligned based on the current constraints.

[0144] In one embodiment, the ninth deep neural network may include an encoder and a decoder. The encoder is used to encode the initial third three-dimensional digital model and the aligned third three-dimensional digital model. The decoder generates a collision elimination and gap closing scheme based on the data output by the encoder. In one embodiment, the encoder may adopt Point Transformer, and the decoder may adopt RNN. Under the enlightenment of this application, it can be understood that in addition to the above network structure, the ninth deep neural network may also adopt any other applicable network.

[0145] In one embodiment, the training data set of the ninth deep neural network may include multiple groups of training data, each group of training data including a three-dimensional digital model of an unaligned jaw, a three-dimensional digital model of an aligned jaw, and corresponding collision resolution and gap closing solutions.

[0146] In one embodiment, in addition to the deep neural network, the computer may also generate collision resolution and gap closing solutions based on the initial third three-dimensional digital model and the aligned third three-dimensional digital model according to pre-defined rules.

[0147] In fact, there are many rules for relieving collisions and closing gaps in this field. Different doctors may use different rules, so these rules will not be listed in detail here.

[0148] Although various aspects and embodiments of the present application are disclosed herein, other aspects and embodiments of the present application will be apparent to those skilled in the art in light of the present application. The various aspects and embodiments disclosed herein are for illustrative purposes only and not for limiting purposes. The scope and subject matter of the present application are determined solely by the appended claims.

[0149] Likewise, various diagrams may illustrate exemplary architectures or other configurations of the disclosed methods and systems that aid in understanding the features and functions that may be included in the disclosed methods and systems. The claimed content is not limited to the exemplary architectures or configurations shown, and the desired features may be implemented with various alternative architectures and configurations. In addition, for flow charts, functional descriptions, and method claims, the order of blocks presented herein should not be limited to various embodiments that are implemented in the same order to perform the described functions, unless otherwise clearly indicated in the context.

[0150] Unless otherwise expressly noted, the terms and phrases used herein and their variations should be interpreted as open ended, not restrictive. In some instances, the appearance of broad words and phrases such as "one or more", "at least", "but not limited to", or other similar terms should not be understood as an intent or need to indicate a narrowing of the example where such broad terms may not be present.

Claims

1. A computer-implemented method for generating a collision resolution and / or gap closure solution, comprising: Acquire a first three-dimensional digital model representing the upper and lower dentition of the patient in an initial tooth arrangement; Get a set of predetermined constraints; aligning the first three-dimensional digital models based on the constraints; generating a collision resolution and / or gap closing solution based on the first three-dimensional digital model and the aligned first three-dimensional digital model using a trained first deep neural network, and updating the constraint with the collision resolution and / or gap closing solution; The process of aligning the first three-dimensional digital model and generating the collision resolution and / or gap closing solution is iterated at least twice to obtain a final collision resolution and / or gap closing solution.

2. The method for generating a collision resolution and / or gap closing scheme according to claim 1, characterized in that: The training digital data set for training the first deep neural network includes multiple groups of training data, each group of training data includes a three-dimensional digital model of the upper and lower dentition under a patient's initial tooth layout, a three-dimensional digital model of the upper and lower dentition under the patient's target tooth layout, and a collision resolution and / or space closing plan in the patient's orthodontic treatment plan.

3. The method for generating a collision resolution and / or gap closing scheme according to claim 1, characterized in that: The process of aligning the first three-dimensional digital model and generating the collision resolution and / or gap closing solution is iterated N times, wherein N is a predetermined natural number greater than or equal to 2.

4. The method for generating a collision resolution and / or gap closing scheme according to claim 1, characterized in that: The predetermined constraints include target positions of selected teeth of a first dental arch in a selected direction, and slice cutting and space reservation schemes of the maxillary and mandibular dental arches, wherein the first dental arch is one of the maxillary and mandibular dental arches.

5. A method for generating a digital data set representing a target tooth layout based on the collision resolution and / or gap closing solution generated according to claim 4, characterized in that: It includes: Preliminarily aligning the first dental arch model of the first three-dimensional digital model using the trained second deep neural network; and The first dentition model preliminarily aligned is first optimized, and the first optimization includes: updating the first dental model of preliminary alignment according to the target position of the selected tooth in the selected direction; and Optimize the dental arch curve of the first dental arch model, and perform collision relief and / or gap closing on the first dental arch model according to the collision relief and / or gap closing scheme corresponding to the first dental arch. The collision relief and / or gap closing requirement is determined based on the slice cutting and space reservation scheme corresponding to the first dental arch.

6. The method of generating a digital data set representing a target tooth configuration according to claim 5, wherein: The collision elimination and / or gap closing scheme for the first dental arch model includes one of the following: tooth extraction, tooth implantation, posterior tooth distalization, posterior tooth mesialization, arch expansion, arch contraction, lip inclination, anterior tooth retraction, slice cutting, gap reservation and any combination thereof.

7. The method of generating a digital data set representing a target tooth configuration according to claim 5, wherein: In the dental arch curve optimization, after the selected tooth moves to the target position in the selected direction, its position and posture remain unchanged.

8. The method of generating a digital data set representing a target tooth configuration according to claim 5, wherein: The updating of the first dental arch model of the preliminary alignment and the optimization of the dental arch curve of the first dental arch model as well as the collision elimination and / or gap closing are performed simultaneously.

9. The method for generating a digital data set representing a target tooth configuration according to claim 5, wherein: It also includes: Using a trained third deep neural network to preliminarily align a second dentition model of the first three-dimensional digital model based on the first optimized first dentition model, wherein the second dentition is the opposing dentition of the first dentition; The second dentition model preliminarily aligned is subjected to a second optimization, the second optimization comprising: Based on the first optimized first dental arch model, the occlusion of the second dental arch model that is preliminarily aligned is optimized, so that the occlusion relationship between the second optimized second dental arch model and the first optimized first dental arch model meets the requirements. At the same time, according to the collision relief and / or gap closing scheme, the part corresponding to the second dental arch is subjected to collision relief and / or gap closing. The requirement of collision relief and / or gap closing is determined according to the part corresponding to the second dental arch in the slice cutting and gap reservation scheme.

10. The method of generating a digital data set representing a target tooth configuration according to claim 9, wherein: The collision relief and / or gap closing scheme for the second dentition includes one of the following: tooth extraction, tooth implantation, posterior tooth distalization, posterior tooth mesialization, arch expansion, arch retraction, lip inclination, anterior tooth retraction, slice cutting, gap reservation and any combination thereof.

11. The method of generating a digital data set representing a target tooth configuration according to claim 10, wherein: It also includes: using a trained fourth deep neural network to further align the first optimized first dental arch model and the second optimized second dental arch model.

12. The method of generating a digital data set representing a target tooth configuration according to claim 9, wherein: The first dentition is the maxillary dentition, and the second dentition is the mandibular dentition.

13. The method of generating a digital data set representing a target tooth configuration according to claim 5, wherein: The target position of the selected tooth in the selected direction includes one of the following: tooth midline target position, anterior tooth sagittal target position, anterior tooth vertical target position, posterior tooth vertical target position, posterior tooth sagittal target position, posterior tooth horizontal target position, canine horizontal target position and any combination of the above.

Images