Neural network-based automatic tooth arrangement method and electronic device

Abstract

The present application relates to the technical field of orthodontic digital design and discloses a neural network-based automatic tooth arrangement method and an electronic device. The neural network-based automatic tooth arrangement method comprises: acquiring an initial tooth arrangement; extracting a tooth type feature parameter group of each tooth from the initial tooth arrangement; and using a trained sequential neural network model for automatic tooth arrangement to sequentially process the tooth type feature parameter groups of the teeth in a preset tooth position order to obtain a target position for each tooth. A sequential neural network model is selected, and the long-short term memory characteristic thereof is utilized. In predicting the features of a certain tooth, both the features of adjacent teeth and the features of teeth that are far apart are taken into consideration, so that tooth arrangement characteristics can be learned faster. In the whole processing process, compared to existing image network models, the sequential neural network model used enables better and faster learning of tooth arrangement characteristics and more accurate predictions about tooth arrangement.

Description

Neural Network-Based Automatic Tooth Alignment Method and Electronic Device Cross-reference to related applications

[0001] This application is based on and claims priority to Chinese Patent Application No. 202411197875.5, filed on August 28, 2024, the entire contents of which are hereby incorporated herein by reference. Technical Field

[0002] This application relates to the field of orthodontic digital design technology, and in particular to an automatic tooth alignment method and electronic device based on neural networks. Background Technology

[0003] Shell-shaped orthodontic appliances are a type of orthodontic device made of safe, elastic, transparent polymer material. They have the advantages of being completely invisible during the orthodontic process, aesthetically pleasing, easy to operate, and convenient for oral cleaning. Moreover, due to their transparent and aesthetically pleasing characteristics, the orthodontic process is completed almost imperceptibly to others, and they have gradually become the first choice for orthodontic patients.

[0004] With the rapid development of computer technology, dental treatment is increasingly relying on computer technology. For example, in orthodontic treatment using shell-shaped orthodontic appliances, a three-dimensional digital model representing the dentition in the target dentition layout is typically generated based on a three-dimensional digital model representing the dentition in the initial dentition layout. Then, based on both, three-dimensional digital models representing the dentition in several successive intermediate dentition layouts are generated. Finally, a series of successive shell-shaped orthodontic appliances are fabricated based on these three-dimensional digital models.

[0005] Current technology primarily generates the target tooth layout by manually moving each tooth in a 3D digital model representing the initial tooth arrangement. On one hand, this method is inefficient and costly; on the other hand, the results are highly dependent on the operator's skill, leading to inconsistent tooth placement.

[0006] Among current technologies, considering that the three-dimensional digital model of the dental arch is actually a geometric image, some methods are based on computer image algorithms to perform arrangement calculations on the model. However, tooth layout adjustment involves many medical parameters and is not simply an image processing problem. Therefore, when current image algorithms are applied to tooth arrangement, due to the lack of medical parameter limitations, the results differ greatly from the actual tooth arrangement and have low accuracy. Summary of the Invention

[0007] The purpose of this application is to provide an automatic tooth alignment method and electronic device based on neural networks, which can solve the problem of low accuracy when tooth layout adjustment relies on manual or automatic adjustment.

[0008] To address the aforementioned technical problems, embodiments of this application provide an automatic tooth alignment method based on a neural network, comprising: obtaining an initial tooth arrangement; extracting a set of tooth-like feature parameters for each tooth from the initial tooth arrangement; and using a trained Sequence Neural Network model for automatic tooth alignment to process the set of tooth-like feature parameters of each tooth sequentially according to a preset tooth position order to obtain the target position of each tooth.

[0009] Embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described neural network-based automatic tooth alignment method.

[0010] Embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described neural network-based automatic tooth alignment method.

[0011] Compared to existing technologies, this application's embodiments change the prior art's concept of processing tooth arrangement as a geometric image. Instead, it processes different teeth as sequential data, using a sequence neural network to process each tooth in a predetermined order to obtain the overall tooth arrangement result. Furthermore, the feature data of each tooth remains separate during this process, effectively improving the accuracy of feature learning. Based on these characteristics, the inventors of this application selected a sequence neural network model, utilizing its long short-term memory properties. When predicting the features of a tooth, it references not only the features of adjacent teeth but also those of other teeth located far apart, enabling faster learning of tooth arrangement characteristics. Throughout the entire processing, compared to existing image network models, the sequence neural network model learns tooth arrangement characteristics better and faster, resulting in more accurate tooth arrangement predictions. Attached Figure Description

[0012] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0013] Figure 1 is a flowchart of an automatic tooth alignment method based on a neural network provided in an embodiment of this application;

[0014] Figure 2 is a schematic diagram of the sequence neural network model training process in an automatic tooth alignment method based on neural networks provided in an embodiment of this application;

[0015] Figure 3 is a schematic diagram of a sequence neural network model with an LSTM structure used in an automatic tooth alignment method based on a neural network provided in an embodiment of this application;

[0016] Figure 4 is a schematic diagram of a single LSTM unit in a sequence neural network model with an LSTM structure used in an embodiment of the automatic tooth alignment method based on neural networks provided in this application;

[0017] Figure 5 is a schematic diagram of the process of calculating mesh vertices using model output results in an automatic tooth alignment method based on neural networks provided in an embodiment of this application;

[0018] Figure 6a is a schematic diagram of the initial tooth arrangement;

[0019] Figure 6b is a schematic diagram of the target dentition obtained by artificial tooth arrangement;

[0020] Figure 6c is a schematic diagram of the target dentition obtained after implementing the neural network-based automatic tooth alignment method according to an embodiment of this application;

[0021] Figure 6d is a schematic diagram showing the overlap and comparison of Figures 6b and 6c;

[0022] Figure 7 is a schematic diagram of a sequence neural network model with an LSTM structure used in an automatic tooth alignment method based on a neural network provided in another embodiment of this application;

[0023] Figure 8 is a schematic diagram of the steps in the automatic tooth alignment method based on neural networks provided in an embodiment of this application, which uses a sequence neural network model to process tooth-related feature parameters and medical plan-related parameters.

[0024] Figure 9 is a schematic diagram of an electronic device provided in another embodiment of this application;

[0025] Figure 10 is a schematic diagram of an automatic tooth alignment system based on a neural network provided in another embodiment of this application. Detailed Implementation

[0026] To make the objectives, technical solutions, and advantages of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been presented in the various embodiments of this application to facilitate a better understanding of the application. However, the technical solutions claimed in the claims of this application can be implemented even without these technical details and various variations and modifications based on the following embodiments.

[0027] In the embodiments of this application, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings. These terms are primarily for the purpose of better describing this application and its embodiments, and are not intended to limit the indicated devices, elements, or components to having a specific orientation, or to be constructed and operated in a specific orientation.

[0028] Furthermore, in addition to indicating location or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in some cases to indicate a certain dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0029] Furthermore, the terms "installation," "setup," "equipped with," "opening," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral structure; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of these terms in this application based on the specific circumstances.

[0030] Furthermore, the terms "first," "second," etc., are primarily used to distinguish different devices, elements, or components (which may be the same or different in specific type and construction), and are not intended to indicate or imply the relative importance or quantity of the indicated devices, elements, or components. Unless otherwise stated, "a plurality of" means two or more.

[0031] The terms "anterior region" and "posterior region" mentioned in the various embodiments of this application are defined according to the classification of teeth in the 2nd edition of *Introduction to Stomatology*, published by Peking University Medical Press, pages 36-38. These include premolars and molars, teeth designated as 4-8 by the FDI (Fédération Dentaire Internationale) notation, and teeth designated as 1-3 by the FDI notation for the anterior region. Teeth in the anterior region include the central incisors, lateral incisors, and canines.

[0032] The terms "horizontal plane," "coronal plane," and "sagittal plane" mentioned in the various embodiments of this application refer to terms in biomedical anatomy: the horizontal plane, also known as the "transverse plane," is a cross-section perpendicular to the vertical axis that divides the human body into upper and lower parts; the coronal plane is a cross-section that longitudinally cuts the human body into anterior and posterior parts along the left and right directions; the sagittal plane divides the human body into left and right parts, and the left and right cross-sections are called sagittal planes, while cross-sections that are equal on the left and right are called median sagittal planes.

[0033] The inventors of this application discovered in their research on digital design for orthodontic treatment that, in order to achieve automatic tooth alignment using computer technology, geometric calculation methods or machine learning models are often employed. However, in practice, the accuracy of these methods is often low. This is because, firstly, the dental arch layout is highly individualized, with each patient's layout being different; secondly, adjustments to the dental arch layout are limited by medical parameters, resulting in highly variable outcomes when actually adjusting the tooth layout, making it difficult to obtain an accurate and suitable tooth alignment plan for each patient. To address these technical problems, this application provides an automatic tooth alignment method based on a neural network, comprising: acquiring an initial tooth arrangement; extracting a set of tooth-like feature parameters for each tooth from the initial tooth arrangement; and using a trained sequence neural network model for automatic tooth alignment, sequentially processing the set of tooth-like feature parameters for each tooth according to a preset tooth position order to obtain the target position of each tooth. This method changes the existing concept of processing tooth alignment as a geometric image, instead processing different teeth as sequential data. Through a sequence neural network, each tooth is processed separately in a predetermined order to obtain the tooth alignment result for the entire row of teeth. Throughout the processing, compared with existing image network models, the sequence neural network model can learn the characteristics of tooth arrangement better and faster, and the prediction results of tooth arrangement are more accurate.

[0034] The following details the implementation of the neural network-based automatic tooth alignment method of this application. The following content is only for the convenience of understanding and is not necessary for implementing this solution.

[0035] First, it should be noted that the neural network-based automatic tooth alignment method in this application embodiment can be implemented through hardware or a combination of computer software and hardware. For hardware implementation, the neural network-based automatic tooth alignment method can be implemented through one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic devices for implementing the neural network-based automatic tooth alignment function, or a selection and combination of the above devices.

[0036] The specific flow of the automatic tooth alignment method based on neural networks provided in some embodiments of this application can be shown in Figure 1, and specifically includes:

[0037] Step 101: Obtain the initial tooth arrangement.

[0038] In some embodiments, the initial tooth arrangement can be represented by a digital model of the jaw, which can be a digital model of the jaw at the initial position of the patient or a digital model of the jaw at the initial position of the test case. The source of the model is determined as needed and will not be listed here. In some embodiments, the obtained initial tooth arrangement can be a jaw mesh model after incisor classification, in which each tooth is an independent model. In some embodiments, the initial tooth arrangement is the initial tooth arrangement of a single jaw, such as a single maxilla or a single mandible, which is not limited here.

[0039] Step 102: Extract the set of tooth class feature parameters for each tooth from the initial tooth arrangement.

[0040] In some embodiments, the tooth feature parameter group is a combination of multiple parameters representing tooth pose. The collected tooth feature data uses pose data. Since the shape of the teeth themselves remains almost unchanged during the tooth arrangement process, the neural network model only predicts the changes in tooth pose. Therefore, calculating only the changes in tooth pose can effectively reduce the number of feature parameters used, reduce the amount of data computation in the prediction, and speed up the model training and prediction.

[0041] In some embodiments, tooth-like feature parameters may include: the geometric center point of the tooth, and six intersection points formed by rays emanating from the geometric center point along the six directions of the three axes in the tooth coordinate system and the tooth mesh (i.e., the tooth model represented by the mesh model). The geometric center point of the tooth is relatively easy to calculate in the model; it can be directly averaged over all mesh vertices of a single tooth. Moreover, this feature effectively locates the tooth position, is well-defined, and has a clear relationship with other contour parameters. Calculations combined with the geometric center point of the tooth are highly accurate. Using the geometric center point and the six intersection points, a total of seven points are used as the feature points of the tooth. These seven feature points can be obtained for each tooth and then arranged in a uniform order. Each point has three spatial coordinates in 3D space, so the feature vector composed of the seven points of each tooth has 21 dimensions. The tooth coordinate system is generally constructed when the jaw model is established, and the data can be used directly. The three positive and three negative directions of the three axes are relatively clear. Rays emanating from the geometric center point along these directions will inevitably form six intersection points with the tooth mesh, effectively representing the current position and orientation of the tooth.

[0042] In some embodiments, tooth-like feature parameters may include: the eight vertices of the tooth's spatial bounding box and the tooth's geometric center. Each face of the bounding box is perpendicular to an axis in the tooth coordinate system, thus obtaining the 3D spatial coordinates of the eight vertices of each bounding box. Adding the tooth's geometric center, a total of nine points are used as the tooth's feature points. For different teeth, these nine points are arranged according to a uniform pattern. Because each point contains 3D spatial coordinates, each tooth's features have a total of 27 dimensions.

[0043] In some embodiments, the tooth-related feature parameters may include several feature points on the tooth facing the opposing jaw and the geometric center point of the tooth. The several feature points on the tooth facing the opposing jaw can be determined according to the tooth type. For example, when the tooth is a molar, the feature point is the buccal apex; or, when the tooth is a premolar, the feature point is the buccal apex and / or buccal groove; or, when the tooth is an incisor or canine, the feature point is the superior incisal edge. In one embodiment, four buccal cusps can be selected on the molars; for the premolars, one cusp can be selected on the lingual and one on the buccal side, and one distal point is selected mesially and distally in the buccal groove as a feature point; for the incisors, one point can be selected on the incisal edge in the direction of the nearest and farthest midpoints, and then two more points are selected at equal intervals on the incisal edge between these two points, thus also taking four feature points on the incisal surface; for the canines, two points are selected at the nearest and farthest midpoints of the incisal edge, the cusp of the canine is also used as a feature point, and then another point is selected at the midpoint of the incisal edge between the cusp and the farthest midpoint as a feature point. Furthermore, the geometric center of each tooth is also used as a feature point, thus each tooth has five feature points. Because each point contains 3-dimensional spatial coordinates, each tooth has a total of 15 dimensions in its features, demonstrating that the amount of feature data selected in this embodiment can be reduced.

[0044] In the above selection of feature parameters, each set of tooth feature parameters can be used individually or in combination. For example, in one embodiment, tooth feature parameters may include: the geometric center point of the tooth, the six intersection points formed by rays emanating from the geometric center point along the six directions of the three axes in the tooth coordinate system and the tooth mesh, and several feature points of the tooth facing the opposing jaw. More combinations of feature parameters can enrich the detailed features of the teeth, which is beneficial to improving the accuracy of the network model in predicting tooth alignment. It is understood that there are various types and combinations of feature parameters; to avoid repetition, they will not be listed here.

[0045] The number of teeth varies for different cases, and there may be missing teeth. In this case, 0 can be used to fill all dimensions of the tooth feature.

[0046] Since the adjustment of tooth layout is mainly achieved by translation and rotation, the pose of a single tooth can be determined by combining the geometric center with contour features, so as to minimize the amount of data representing tooth pose parameters, reduce the amount of parameter data involved in the calculation, and speed up the processing speed of the model.

[0047] Step 103: Using the trained sequence neural network model for automatic tooth alignment, process the tooth-type feature parameter groups of each tooth in the preset tooth position order to obtain the target position of each tooth.

[0048] In some embodiments, when the initial tooth arrangement is a single-jaw initial tooth arrangement, the preset tooth position sequence includes: an order from smallest to largest FDI tooth position number, such as 11, 12, 13, 14, ... 26, 27, 28 for the maxillary dentition. In other embodiments, the preset tooth position sequence may include: an order from the left terminal tooth to the right terminal tooth in the FDI tooth position; or an order from the right terminal tooth to the left terminal tooth in the FDI tooth position. For example, for the maxillary dentition, the sequence could be from the 8th molar in zone 1 to the 8th tooth in zone 2, that is, from 18, 17, 16... 26, 27, 28. In other embodiments, it may include an order from largest to smallest FDI tooth position number. It is understood that other tooth position sequences can be set besides the above-mentioned sequence settings, which will not be listed here. Because the arrangement of individual teeth in the current artificial tooth arrangement process has a certain order, the tooth arrangement process is closer to the current medical design method, and the prediction effect is better.

[0049] Regarding the sequence neural network model used for automatic tooth alignment, Figure 2 will be used as an example for explanation. The sequence neural network model used in this step for automatic tooth alignment can be trained and obtained in the following way:

[0050] Step 201: Combine the tooth class feature parameter groups of each tooth in the initial position and target position of an existing dentition, arrange the tooth class feature parameter groups of each tooth according to the preset tooth position order to obtain a dentition sample, and collect multiple dentition samples to construct a tooth arrangement training set.

[0051] Step 202: Construct a sequence neural network model.

[0052] In some embodiments, the constructed sequence neural network model can employ one or a combination of GRU (Gate Recurrent Unit) structure, LSTM (Long Short-Term Memory) structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model in some embodiments may further include adding an attention module to the constructed sequence neural network model. Adding an attention module to the constructed sequence neural network further improves the prediction accuracy of the network model. It is understood that different attention mechanisms can be added, such as the Luong attention mechanism and the Bahdanau attention mechanism. Furthermore, these models and attention mechanisms can be used to construct the network into an Encoder-Decoder structure, etc.

[0053] In some embodiments, the loss function used in the constructed sequence neural network model is one of the following: Smoothed L1 Loss, L2 Loss, Huber Loss, or Log-Cosh Loss. In some embodiments, the optimizer used in the constructed sequence neural network model is one of the following: Adam, AdamW, SGD (Stochastic Gradient Descent), AdaGrad, or RMSProp. The above loss function and optimizer can be defined simultaneously, and their combinations are diverse, and will not be listed here.

[0054] Step 203: Based on the tooth alignment training set, train the constructed sequence neural network model to obtain a sequence neural network model for automatic tooth alignment.

[0055] During training, the model sequentially learns the feature parameter sets of each tooth in the input dentition sample at its initial position through the recurrent module in the constructed sequence neural network model, following a preset tooth position order. It then outputs the feature parameter sets of each tooth at its predicted position according to the preset tooth position order. The model compares the feature parameter sets at the predicted and target positions to obtain the feature parameter difference value, which is the loss value between the model's predicted value and the labeled data. Using this loss value, the gradient of each parameter can be calculated. Then, using backpropagation and the chain rule, each parameter value of the model is updated along the direction from the model output to the model input. The model training is completed after processing a large amount of data (hundreds of thousands of cases) and multiple iterations. In some embodiments, mini-batches of data can be used to update model parameters during training, which can speed up the training process.

[0056] Taking the LSTM structure sequence neural network model in Figure 3 as an example, it specifically includes an encoding module and a decoding module, where the encoding module and the decoding module each include several recurrent units.

[0057] In the encoding stage, Xn represents the nth tooth-type feature parameter group. In one embodiment, tooth information can be directly used for encoding. In another embodiment, in addition to inputting each tooth-type feature parameter group, position information can also be encoded together. The position information can be the FDI number. Each tooth in the dental arch has a unique FDI number, so using the FDI number as the position information is simple and quick, so that the input of the model includes not only the spatial information of the teeth but also the position information of the teeth.

[0058] Continuing the explanation, each recurrent unit uses an LSTM structure, the internal structure of which can be shown in Figure 4. The input includes three parts: input information (X... tHidden state (H) t-1 ) and memory cell state (C t-1 (Simplified description in Figure 4: "memory cell"). The hidden state and memory cell state are derived from the previous LSTM unit; for the first LSTM unit, they are randomly initialized values. An LSTM unit typically includes three gating units: a forget gate (F...). t ), input gate / update gate and (I t Output gate (O) t The forget gate controls which parts of the historical information are unimportant and can be forgotten, and which need to be retained. It uses the Sigmoid activation function, with a value between 0 and 1, to control the degree of forgetting. The input / update gate also uses the Sigmoid function to determine how much new input information to use, and the tanh function to calculate the candidate cell state of the current input. Then, through the forget gate and the input gate, the cell state input from the previous unit and the current candidate cell state are linearly combined to obtain the cell state at the current time step. The output gate determines which parts of the memory unit will be used to calculate the current hidden state and output it to the next time step. It is controlled by a Sigmoid function to determine which parts of the memory unit will affect the final hidden state, and then scaled by the tanh function to the range [-1, 1]. In other words, the tanh function can be used to ensure numerical stability. The LSTM unit also includes candidate memory cells. It is used to calculate candidate update values ​​for the current unit. Based on the input information and the hidden state of the previous unit's input, it uses a gating mechanism to determine which information should be updated and passed to the next unit. Candidate memory cells work in conjunction with memory cells used to store and pass long-term memory information, enabling LSTM units to efficiently process sequential data and better capture and remember information in long-term dependencies.

[0059] In the decoding stage, Yn represents the predicted feature parameter set of the nth tooth. The input of the decoding module can be the output of the previous recurrent unit and the hidden state. For the recurrent unit of the LSTM structure, the input of the recurrent unit can also include the cell state of the previous recurrent unit.

[0060] As can be seen from steps 201 to 203, by pre-constructing a sample set, a sequence neural network model is built, and then trained to obtain a sequence neural network model that can be used for automatic tooth alignment, a sequence neural network model that better meets individual needs can be obtained.

[0061] In some embodiments, as shown in Figure 5, step 103 further includes the process of calculating mesh vertices using the model output results:

[0062] Step 501: Using the trained sequence neural network model for automatic tooth alignment, process the initial values ​​of the tooth class feature parameter groups of each tooth in the preset tooth position order to obtain the predicted target values ​​of the tooth class feature parameter groups.

[0063] In some embodiments, when the model makes predictions, the dimension of its output corresponds to the dimension of its input. For example, if the input is the three-dimensional spatial coordinates of the feature points of each tooth, then the output is the three-dimensional spatial coordinates of the corresponding feature points of each tooth.

[0064] Step 502: Based on the predicted target value and initial value of the tooth class feature parameter group of each tooth, the spatial coordinate transformation of each tooth is calculated by the point cloud registration algorithm.

[0065] Step 503: Based on the spatial coordinate transformation of each tooth, transform the mesh model of each tooth in the initial tooth arrangement to obtain the mesh model of the target position of each tooth.

[0066] In some embodiments, based on the transformation of the spatial coordinates of the corresponding points of each tooth, the translation matrix and rotation transformation matrix of each tooth can be calculated using a point cloud registration method (such as the Kabsch method). After applying these translation matrices and rotation transformations to the input grid vertex coordinates of each tooth, the grid vertex coordinates of each tooth at the predicted target position can be obtained.

[0067] Taking the Kabsch algorithm as an example, the specific calculation method is explained as follows:

[0068] (1) From the feature points of the model input and output, we can construct two matrices P and Q, each with a dimension of 3×N. Each column of these two matrices represents the coordinates of the feature points, N is the number of feature points, and the feature points represented by each column of these two matrices are in one-to-one correspondence.

[0069] (2) Decenter these two matrices (p0 and q0 are the centers of P and Q respectively), and the resulting new matrices are still denoted as P and Q;

[0070] (3) Construct the covariance matrix H = P T Q;

[0071] (4) Perform singular value decomposition (SVD) on the covariance matrix H, i.e., H = USV T U and V are two orthogonal matrices, and S is a diagonal matrix with singular values ​​on the diagonal.

[0072] (5) Next, calculate d = sign(det(VU) TTo determine whether a rotation correction matrix is ​​needed to ensure the use of a right-handed coordinate system, construct the following matrix based on d:

[0073] (6) The rotation matrix R is calculated by the following formula: R = VIU T R is a 3×3 matrix. The translation matrix r = p0 – R·q0. Applying these two matrices to the vertex coordinates of each tooth in its initial position yields the vertex coordinates of the tooth in the target position. That is:

[0074] X' = ​​RX + r;

[0075] Where X' is the coordinate of a vertex of a tooth in the target position, and X is the coordinate of a vertex of the corresponding initial position tooth.

[0076] As can be seen, steps 501 to 503 further define the process of calculating spatial coordinate transformation and then restoring the tooth mesh model through point cloud registration. Considering that the shape of the teeth themselves remains almost unchanged, restoring the dentition morphology through registration can increase the error tolerance of the predicted feature parameters. It is understood that in addition to using the Kabsch algorithm for registration and transformation as described above, other registration algorithms can also be used, which will not be listed here.

[0077] In some embodiments, after step 103, a post-processing procedure is further included for the predicted target positions output by the model. This may include: performing collision detection on the target positions of each tooth and updating the target positions of each tooth based on the detection results. Since the target positions predicted by the neural network model may contain collisions or gaps between teeth, it is necessary to adjust the relative positions of each pair of adjacent teeth in the same jaw if there are collisions or gaps.

[0078] In one embodiment, the method for fine-tuning between teeth in a single jaw is as follows: First, determine whether there is a collision or gap between the two central incisors. If there is a collision, calculate their collision depth d, and then move both teeth d / 2 along the distal direction. If there is a gap, calculate the gap width w, and then move each tooth w / 2 along the mesial direction. Then, starting with the incisors, perform collision detection on the incisors and lateral incisors. If there is a collision, move the lateral incisors distally; if there is a gap, move the lateral incisors mesially, and so on. For the canines, premolars, and molars behind them, fine-tune them in the same way.

[0079] Subsequently, the inventors of this application conducted a tooth arrangement test using the neural network-based automatic tooth arrangement method described in this application, and compared the target tooth positions obtained using the neural network-based automatic tooth arrangement method with those obtained manually, as shown in Figures 6a, 6b, 6c, and 6d. Figure 6a shows the initial tooth arrangement in the example, Figure 6b shows the target tooth row obtained manually, Figure 6c shows the target tooth row obtained using the automatic tooth arrangement method described in this application, and Figure 6d is an overlay comparison of Figures 6b and 6c. It can be seen that the difference between the target tooth positions obtained using the neural network-based automatic tooth arrangement method described in this application and those obtained manually is small, indicating that the neural network-based automatic tooth arrangement method described in this application has better practical effects and should be widely promoted.

[0080] As can be seen, the embodiments of this application change the prior art's concept of processing tooth arrangement as a geometric image, processing different teeth as sequential data, and using a sequence neural network to process each tooth separately in a predetermined order to obtain the tooth arrangement result of the entire row of teeth. Moreover, in this process, the feature data of each tooth are not mixed, which can effectively improve the accuracy of feature learning. Based on the above characteristics, the inventors of this application selected a sequence neural network model, utilizing its long short-term memory characteristics, to predict the features of a tooth, referring not only to the features of adjacent teeth, but also to the features of other teeth that are far away, thus learning the tooth arrangement characteristics faster. In the entire processing process, compared with existing image network models, the sequence neural network model can learn the tooth arrangement characteristics better and faster, and the prediction results of the tooth arrangement are also more accurate.

[0081] Other embodiments of this application provide an automatic tooth arrangement method based on a neural network. The main difference between this method and the foregoing embodiments is that the tooth arrangement processed in the foregoing embodiments is a single-jaw tooth arrangement, while the tooth arrangement processed in the current embodiment is a tooth arrangement that includes both the maxilla and mandible.

[0082] In some embodiments, the initial tooth arrangement of the maxilla and mandible of the same patient is obtained when obtaining the initial tooth arrangement. It is understood that the maxilla and mandible can be the maxilla and mandible tooth arrangement after occlusal registration.

[0083] Correspondingly, in addition to adopting the tooth arrangement order in the aforementioned embodiment in a single jaw, the preset tooth position order can be arranged in the order of arranging the teeth of the maxilla first and then the teeth of the mandibular dentition, or in the order of arranging the teeth of the mandibular dentition first and then the teeth of the maxilla.

[0084] The process of predicting the target tooth position using a sequence neural network model for automatic tooth alignment is similar to that in the aforementioned embodiments and will not be repeated here.

[0085] In some embodiments, after obtaining the target position, collision detection and adjustment can be performed on the teeth between the upper and lower jaws. During adjustment, fine-tuning can be performed first on the teeth between individual jaws, and then fine-tuning can be performed on the teeth between the upper and lower jaws. In one embodiment, collision is first checked on all teeth between the upper and lower jaws. If there is a collision depth d, the teeth colliding in the upper and lower jaws are moved a distance d / 2 in an upward or downward direction, respectively. If there is a gap w, the teeth with larger gaps in the upper and lower jaws are moved downward and upward, respectively, to reduce the gap between the teeth.

[0086] In summary, when the initial tooth arrangement obtained includes the arrangement of the maxillary and mandibular teeth, the neural network-based automatic tooth arrangement method in this application is also applicable, and the tooth arrangement effect is also good.

[0087] Other embodiments of this application provide an automatic tooth alignment method based on a neural network, which is a further improvement on the foregoing embodiments. The main improvement is that the data input to the neural network model in the foregoing embodiments only includes tooth-related feature parameters, while in the current embodiment, in addition to tooth-related feature parameters, medical plan-related parameters are also combined. By combining these medical plans to learn tooth alignment, the speed at which the sequence neural network model learns tooth alignment features can be further accelerated, and the accuracy of the predicted tooth alignment target position can be improved.

[0088] The current embodiment employs a sequence neural network model architecture as shown in Figure 7, specifically including: an encoding module, feature concatenation (simplified as "concatenation" in the figure), and a decoding module. In some embodiments, the features output by the last loop unit in the encoding module can also be combined with features associated with the medical protocol as input features for the decoder. In some embodiments, the medical protocol parameter group includes one or any combination of the following: arch expansion, gap closure, enamel reduction, tooth extraction, and ideal arch curve morphology.

[0089] As shown in Figure 8, when processing the dental feature parameters using the sequence neural network model in the current embodiment, the specific steps include:

[0090] Step 801: Using the encoding module in the trained sequence neural network model for automatic tooth alignment, the tooth-type feature parameter groups of each tooth are encoded according to the preset tooth position order. This step is similar to the previous embodiment and will not be repeated here.

[0091] Step 802: Using the decoding module in the trained sequence neural network model for automatic tooth alignment, the obtained encoding result and the pre-acquired medical plan parameter set are decoded to obtain the target position of each tooth.

[0092] In some embodiments, the encoding result obtained in step 801 is concatenated with the pre-acquired medical plan parameter set, and the concatenated result is decoded. In some embodiments, the pre-acquired medical plan parameter set can be encoded by a preset text encoder before concatenation. When applying medical plan feature parameters, the method of first encoding with a preset text encoder and then concatenating with the original parameters is more convenient to cover the rich expression methods in existing medical plans, making the medical plans more learnable. When combining medical plan parameters, parameter concatenation can be used to improve the combined application of multiple types of data. In some embodiments, after combining medical plan parameters, multiple forms of data are actually integrated in the overall dental alignment, realizing a multimodal solution. It is understood that in the feature concatenation part, the data output by the text encoder can be processed by a fully connected layer before concatenation. Since the numerical range of the encoded values ​​output by the text encoder and the numerical range of the encoded values ​​output by the encoding module may differ greatly, a fully connected layer can make the value ranges of the two closer, eliminating the interference of large values ​​on the results when the numerical differences are too large.

[0093] As can be seen, in the current embodiment, when using a sequence neural network model for tooth arrangement, in addition to using the characteristic parameters of the teeth themselves in the dentition, medical plan parameters can also be combined. Since the doctor or patient may specify some medical plans during the tooth arrangement process, combining these medical plans to learn tooth arrangement can further accelerate the speed at which the sequence neural network model learns the tooth arrangement features and improve the accuracy of the predicted tooth arrangement target position.

[0094] It is worth mentioning that the examples above in this application are merely illustrative for ease of understanding and do not constitute a limitation on the technical solutions of this application.

[0095] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0096] Other embodiments of this application provide an automatic tooth alignment system based on a neural network, as shown in Figure 10, specifically including:

[0097] The acquisition module is used to acquire the initial tooth arrangement;

[0098] The extraction module is used to extract the set of tooth class feature parameters for each tooth from the initial tooth arrangement;

[0099] The prediction module uses a trained sequence neural network model for automatic tooth alignment to process the tooth-type feature parameter sets of each tooth in a preset tooth position order to obtain the target position of each tooth.

[0100] Another embodiment of this application relates to an electronic device, as shown in FIG9, including: at least one processor 901; and a memory 902 communicatively connected to at least one processor 901; wherein the memory 902 stores instructions executable by at least one processor 901, the instructions being executed by at least one processor 901 to enable at least one processor 901 to perform the neural network-based automatic tooth alignment method in the above partial embodiments.

[0101] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0102] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0103] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the neural network-based automatic tooth alignment method in some of the above embodiments, or the digital modeling method for dental orthodontic jaw models in some of the above embodiments.

[0104] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0105] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. An automatic tooth alignment method based on neural networks, comprising: Obtain the initial tooth arrangement; Extract the set of tooth-type feature parameters for each tooth from the initial tooth arrangement; Using a trained sequence neural network model for automatic tooth alignment, the tooth-type feature parameter groups of each tooth are processed sequentially according to a preset tooth position order to obtain the target position of each tooth.

2. The automatic tooth alignment method based on neural networks according to claim 1, wherein, The initial tooth arrangement is a single-jaw initial tooth arrangement; The preset tooth position order includes: an order in ascending order of FDI tooth position number; or an order in descending order of FDI tooth position number; or an order from the left terminal tooth to the right terminal tooth of the FDI tooth position; or an order from the right terminal tooth to the left terminal tooth of the FDI tooth position.

3. The automatic tooth alignment method based on neural networks according to claim 1 or 2, wherein, The initial tooth arrangement includes the initial tooth arrangement of the maxilla and mandible of the same patient; The preset tooth position sequence includes: arranging the teeth of the maxillary dentition first and then arranging the teeth of the mandibular dentition, or arranging the teeth of the mandibular dentition first and then arranging the teeth of the maxillary dentition.

4. The automatic tooth alignment method based on neural networks according to any one of claims 1-3, wherein, The tooth-type feature parameter group is a combination of multiple parameters that characterize tooth pose.

5. The automatic tooth alignment method based on neural networks according to any one of claims 1 to 4, wherein, The set of dental feature parameters includes at least one of the following parameter sets: The geometric center of the tooth, and the six intersection points formed by rays emanating from the geometric center along the six directions of the three axes in the tooth coordinate system and the tooth mesh; The eight vertices of the spatial bounding box of the tooth and the geometric center of the tooth; Several characteristic points on the tooth facing the opposing jaw and the geometric center of the tooth.

6. The automatic tooth alignment method based on neural networks according to claim 5, wherein, The feature point on the tooth facing the opposing jaw is determined according to the tooth type, wherein when the tooth is a molar, the feature point is the buccal cusp; or when the tooth is a premolar, the feature point is the buccal cusp or buccal groove; or when the tooth is an incisor or canine, the feature point is the superior incisal edge.

7. The automatic tooth alignment method based on neural networks according to any one of claims 1-6, wherein, The process utilizes a trained sequence neural network model for automatic tooth alignment, sequentially processing the tooth-related feature parameter sets of each tooth according to a preset tooth position order, including: Using the encoding module in the trained sequence neural network model for automatic tooth alignment, the tooth-type feature parameter group of each tooth is encoded according to the preset tooth position order; The decoding module in the trained sequence neural network model for automatic tooth alignment is used to decode the obtained encoding results and the pre-acquired medical plan parameter set to obtain the target position of each tooth.

8. The automatic tooth alignment method based on neural networks according to claim 7, wherein, The decoding process of the obtained encoding result and the pre-acquired medical protocol parameter group includes: The obtained encoding result is concatenated with the pre-acquired medical plan parameter group, and the concatenated result is then decoded.

9. The automatic tooth alignment method based on neural networks according to claim 8, wherein, The process of concatenating the obtained encoding result with the pre-acquired medical plan parameter group includes: The pre-acquired medical plan parameter group is encoded by a preset text encoder and then concatenated.

10. The automatic tooth alignment method based on neural networks according to any one of claims 7-9, wherein, The pre-acquired medical protocol parameter set includes one or any combination of the following: arch expansion, gap closure, enamel removal, tooth extraction, and ideal arch curve morphology.

11. The automatic tooth alignment method based on neural networks according to any one of claims 1-10, wherein, The process utilizes a trained sequence neural network model for automatic tooth alignment, sequentially processing the tooth-type feature parameter sets of each tooth according to a preset tooth position order to obtain the target position of each tooth, including: Using a trained sequence neural network model for automatic tooth alignment, the initial values ​​of the tooth-type feature parameter group of each tooth are processed sequentially according to the preset tooth position order to obtain the predicted target value of the tooth-type feature parameter group. Based on the predicted target value and the initial value of the tooth class feature parameter group for each tooth, the spatial coordinate transformation of each tooth is calculated using a point cloud registration algorithm. Based on the spatial coordinate transformation of each tooth, the mesh model of each tooth in the initial tooth arrangement is transformed to obtain the mesh model of the target position of each tooth.

12. The automatic tooth alignment method based on neural networks according to any one of claims 1-11, wherein, Before processing the tooth-type feature parameter set of each tooth sequentially according to a preset tooth position order using a trained sequence neural network model for automatic tooth alignment, the process includes: Combine the tooth-type feature parameter groups of each tooth in the initial and target positions of an existing dentition, arrange the tooth-type feature parameter groups of each tooth according to the preset tooth position order to obtain a dentition sample, and collect multiple dentition samples to construct a tooth arrangement training set; Construct a sequence neural network model; Based on the tooth alignment training set, the constructed sequence neural network model is trained to obtain a sequence neural network model for automatic tooth alignment.

13. The automatic tooth alignment method based on neural networks according to claim 12, wherein, The sequence neural network model includes one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure.

14. The automatic tooth alignment method based on a neural network according to claim 12 or 13, wherein, The construction of the sequence neural network model includes adding an attention module to the constructed sequence neural network model.

15. The neural network-based automatic tooth alignment method according to any one of claims 12-14, wherein, The loss function used in the sequence neural network model is one of the following: SmoothedL1 Loss, L2 Loss, Huber Loss, Log-Cosh Loss; and / or, The optimizer used in the sequence neural network model is one of the following: Adam, AdamW, SGD, AdaGrad, or RMSProp.

16. The automatic tooth alignment method based on neural networks according to any one of claims 1-15, wherein, After obtaining the target position of each tooth, the process includes: performing collision detection on the target position of each tooth, and updating the target position of each tooth based on the detection results.

17. An electronic device comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the neural network-based automatic tooth alignment method as described in any one of claims 1-16.

Images