Automatic step-by-step method based on neural network and electronic equipment

Abstract

The invention relates to the technical field of orthodontic digital design, and discloses an automatic step-by-step method and electronic equipment based on a neural network, and the automatic step-by-step method comprises the following steps: a first acquisition step and a second acquisition step: respectively acquiring a first digital data set and a second digital data set representing a first dentition; a data fusion step: respectively fusing digital data of corresponding teeth in the first digital data set and the second digital data set according to a single tooth to obtain a fused digital data set; a step-by-step prediction step: processing digital data of each tooth in the fusion digital data set by using a trained neural network model for automatic step-by-step, and generating a stage digital data set representing the target bit of the first dentition in the next stage; a data updating step of updating the first digital data set based on the stage digital data set; repeating the data fusion step, the step-by-step prediction step and the data updating step until a preset prediction stop condition is met; and a step of step-by-step scheme output.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of orthodontic digital design, and particularly relates to an automatic step-by-step method based on a neural network and an electronic device. BACKGROUND

[0002] The shell-shaped tooth appliance is a kind of orthodontic appliance, which is made of safe, elastic, transparent and high-molecular material. It has the advantages of full invisibility in the correction process, good appearance, simple operation, easy oral cleaning, etc. Moreover, due to its transparent and beautiful appearance, the correction process is almost completed without being noticed by others, and it has gradually become the first choice for orthodontic patients.

[0003] With the rapid development of computer technology, computer technology is increasingly used in dental diagnosis and treatment. For example, in orthodontic treatment using a shell-shaped tooth appliance, a three-dimensional digital model representing a dental arch in an initial tooth arrangement is usually generated, and a three-dimensional digital model representing the dental arch in a target tooth arrangement is generated. Then, based on the initial tooth arrangement and the target tooth arrangement, three-dimensional digital models representing intermediate tooth arrangements that gradually change are generated between the two. Finally, based on these three-dimensional digital models, a series of successive shell-shaped tooth appliances are manufactured, and the patient wears these shell-shaped tooth appliances in sequence during treatment, so that the teeth move along the path formed by the intermediate tooth arrangements to achieve the final alignment.

[0004] In the current technology, a series of intermediate tooth arrangements are generated mainly by interpolating between the initial position and the target position. This method usually adopts uniform arrangement, but the movement of teeth is not uniform, so the step-by-step method according to uniform interpolation is sometimes unreasonable. In order to achieve more reasonable step-by-step, the existing technology specifies some key arrangements between the initial arrangement and the target arrangement, and then interpolates between the two adjacent key positions to improve the accuracy. However, the confirmation of the key arrangement and the arrival time needs to be specified by experienced designers in advance, and the result is more dependent on the professional skills of the operator, and the quality of the final result is unstable. SUMMARY

[0005] The present application aims to provide an automatic step-by-step method based on a neural network and an electronic device, which can solve the problem of low accuracy in the step-by-step process of orthodontic tooth arrangement.

[0006] To solve the above technical problems, the embodiment of the present application provides an automatic step-by-step method based on a neural network, comprising: a first acquisition step of acquiring a first digital data set representing the current pose of each tooth in a first dentition; a second acquisition step of acquiring a second digital data set representing the ideal pose of each tooth in the first dentition; a data fusion step of fusing the digital data of the corresponding teeth in the first digital data set and the second digital data set respectively according to individual teeth to obtain a fused digital data set; a step-by-step prediction step of processing the digital data of each tooth in the fused digital data set using a trained neural network model for automatic step-by-step prediction to generate a stage digital data set representing the target position of the first dentition in the next stage; a data updating step of updating the first digital data set based on the stage digital data set; repeating the data fusion step, the step-by-step prediction step and the data updating step until a preset prediction stop condition is reached; and a step-by-step scheme output step of collecting all the stage digital data sets obtained by prediction and outputting them as a treatment scheme for the first dentition.

[0007] The embodiment of the present application also provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the above-mentioned automatic step-by-step method based on a neural network.

[0008] Compared with the prior art, the embodiment of the present application uses a neural network model to predict the orthodontic path of the dentition, and fuses the current pose information and the target pose information of the teeth during prediction, so that the predicted step-by-step result gradually approaches the target pose, avoiding the situation that the moving direction of the teeth in the prediction result is unclear, causing the moving path to be repeated in the entire orthodontic process, which is inefficient and ineffective. Among them, the next stage step-by-step result is obtained through cyclic prediction, and the sequence obtained is the step-by-step sequence, avoiding confusion. At the same time, the prediction stop condition is set to avoid the dead loop caused by the inability to reach the ideal pose in the cyclic prediction. In addition, during the automatic prediction of the target position, the required number of correction steps does not need to be specified, and the actual situation can be predicted to obtain a more reasonable correction path and a more reasonable treatment scheme. It can be seen that the series of dentition layouts obtained by using the automatic step-by-step method based on the neural network in the present application not only realizes automatic and rapid, reduces the influence of artificial experience, and improves the accuracy of the output treatment scheme.

[0009] Optionally, the neural network model for automatic step-by-step belongs to a sequence neural network model, and the processing of the digital data of each tooth in the fused digital data set is in a preset tooth position sequence. In this embodiment, the concept of arranging teeth as geometric images in the prior art is changed, and different teeth are processed as sequence data. Through the sequence neural network, each tooth is processed in a predetermined order to obtain the tooth arrangement result of the full row of teeth. Moreover, the feature data of each tooth is not mixed in this process, which can effectively improve the accuracy of feature learning. Based on the above characteristics, the inventors of the present application select a sequence neural network model, which has a long short-term memory characteristic. When predicting the characteristics of a tooth, not only the characteristics of adjacent teeth are referred to, but also the characteristics of other teeth that are far apart are referred to, so that the step-by-step characteristics can be learned faster. Compared with the existing image network model, the sequence neural network model can better and faster learn the step-by-step characteristics, and the prediction result of the step-by-step is more accurate.

[0010] Optionally, in the first acquisition step, the acquired first dentition is a single jaw dentition; the preset tooth position sequence includes: a sequence from small to large FDI tooth position number; or a sequence from large to small FDI tooth position number; or a sequence from the left end tooth of the FDI tooth position to the right end tooth; or a sequence from the right end tooth of the FDI tooth position to the left end tooth. In this embodiment, since the arrangement of a single tooth has a certain order when the single step arrangement of artificial step-by-step is arranged, the step-by-step prediction process is closer to the current medical design method, and the prediction effect is better.

[0011] Optionally, the acquired first dentition includes the dentition of the upper jaw and the lower jaw of the same patient; and the preset tooth position sequence includes: a sequence of arranging the teeth of the upper jaw first and then arranging the teeth of the lower jaw, or a sequence of arranging the teeth of the lower jaw first and then arranging the teeth of the upper jaw.

[0012] Optionally, the first digital data set and / or the second digital data set includes tooth parameters of each tooth, which can be data representing the pose. In this embodiment, the digital data uses data representing the pose. Since the shape of the tooth itself hardly changes during the tooth arrangement process, the neural network model only predicts the pose change of the tooth, so calculating only the pose change of the tooth can effectively reduce the amount of feature parameters used, reduce the data operation amount in prediction, and speed up the model training and prediction speed.

[0013] Optionally, the tooth type parameter comprises one of the following: a geometric center point of the tooth, six intersection points formed by six rays respectively extending from the geometric center point along six directions of a three-axis tooth coordinate system and the tooth mesh; or eight vertices of a space bounding box of the tooth and the geometric center point of the tooth; or a plurality of feature points of the tooth facing the opposite tooth and the geometric center point of the tooth. In this embodiment, since the adjustment of the tooth layout is mainly achieved by translation and rotation, the pose of a single tooth can be confirmed from the geometric center in combination with the contour-related features, so as to reduce the amount of data representing the tooth pose parameters as much as possible, reduce the amount of parameter data participating in the operation, and accelerate the processing speed of the model.

[0014] Optionally, the feature point of the tooth facing the opposite tooth is determined according to the tooth type, wherein when the tooth is a molar, the feature point is a buccal cusp point; or when the tooth is a premolar, the feature point is a buccal cusp point or a buccal groove point; or when the tooth is an incisor or a canine, the feature point is an upper incisal edge point.

[0015] Optionally, the data fusion step comprises: performing the same weight operation on the digital data of the same tooth in the first digital data set and the second digital data set. In this embodiment, the same weight operation is used to better fuse the current pose and the ideal pose, so as to achieve equal weight and avoid excessive influence of a certain parameter.

[0016] Optionally, the data fusion step comprises: performing the same weight operation on the digital data of the same tooth in the first digital data set and the second digital data set. In this embodiment, the same weight operation is used to better fuse the current pose and the ideal pose, so as to achieve equal weight and avoid excessive influence of a certain parameter.

[0017] Optionally, there are M substep stages, and the neural network model for automatic substep has K, at least two substep stages correspond to different neural network models; wherein, M and K are natural numbers greater than 1. In this embodiment, the overall substep stage of a dentition is further divided into multiple segments, and different neural network models are allocated for prediction. Since the substep methods in different stages may be different, respectively constructing neural network models for prediction is beneficial to improving the accuracy of the substep results in each stage.

[0018] Optionally, the M substep stages are divided according to the number of steps, and the neural network models corresponding to different step numbers are trained using sample data corresponding to the steps in the historical cases. In this embodiment, the substep stages are divided according to the number of steps. Since the principles of substep may be different according to the time sequence in the arrangement of teeth, such as the early stage, the middle stage and the subsequent stage, it is convenient to select different historical data to form different sample data in model training according to the number of steps, so as to accurately train different neural network models.

[0019] Optionally, the K neural network models for automatic step-by-step have the same model architecture.

[0020] Optionally, the first dentition corresponds to a case type, the neural network model for automatic step-by-step has a plurality of models, and is trained by a training set corresponding to each case type; the step-by-step prediction step includes: selecting the neural network model corresponding to the first dentition according to the case type information to perform step-by-step prediction. In this embodiment, different neural network models are trained for different case types, and since the principles of different case types are quite different, training a special neural network model for different case types can be more accurate in step-by-step prediction.

[0021] Optionally, in the step-by-step prediction step, the next stage is the next treatment step or the next N treatment steps, wherein N is a natural number greater than 1. In this embodiment, one treatment step can be predicted at a time, or multiple treatment steps can be predicted at a time, and different prediction methods can be selected according to actual needs to facilitate flexible adjustment of the automatic step-by-step method.

[0022] Optionally, the data fusion step, the step-by-step prediction step, and the data updating step are repeated until a predetermined prediction stopping condition is reached, including a data correction step, wherein the stage digital data set generated by the step-by-step prediction step is verified according to a predetermined medical rule, and the stage digital data set is corrected when the verification fails; wherein the data correction step is performed once every several times the step-by-step prediction step is performed. In this embodiment, a data correction step is added in the process of cyclic prediction, and since there are some limited conditions based on medical rules in the step-by-step process, stage correction can avoid the inconsistency with the actual step-by-step effect caused by error accumulation, and improve the accuracy of the final output treatment plan.

[0023] Optionally, the predetermined medical rule includes: the movement of the teeth, the movement of the teeth, and the occlusal relationship with the occlusal teeth.

[0024] Optionally, the data correction step further includes a collision detection step, wherein the stage pose of each tooth is subjected to collision detection, and the stage pose of each tooth is corrected according to the detection result. In this embodiment, collision detection is further added in the data correction step, which can also adjust the collided teeth in time, reduce the accumulation of collision errors, and improve the accuracy of the final output treatment plan.

[0025] Optionally, the preset prediction stop condition comprises one of the following: the number of correction steps reaches a preset number; and a difference between the stage digital data set and the second digital data set is less than a predetermined range.

[0026] Optionally, before the step-by-step prediction step, the method comprises a model training step, wherein a plurality of pairs of fused digital data of teeth at a current step and a predetermined subsequent step are constructed; all the constructed pairs of digital data are collected to form a step-by-step training set; a neural network model is constructed; and the constructed neural network model is trained based on the step-by-step training set to obtain the neural network model for automatic step-by-step prediction. In this embodiment, a sample set is constructed in advance, a neural network model is constructed, and then a neural network model for automatic step-by-step prediction is obtained through training, so that a sequence neural network model that meets individual needs can be obtained.

[0027] Optionally, the current step and the predetermined subsequent step are two adjacent steps, and the next stage target position in the step-by-step prediction step is a target position of a next correction step. In this embodiment, the target positions of two adjacent correction steps are used to construct the pairs of digital data in the sample set, so that the learned step-by-step features are more accurate.

[0028] Optionally, the constructed neural network model is a sequence neural network model.

[0029] Optionally, the sequence neural network model comprises one of the following: a GRU structure, an LSTM structure, a bidirectional GRU structure, a bidirectional LSTM structure, a multi-layer GRU structure, a multi-layer LSTM structure, and a Transformer structure.

[0030] Optionally, the construction of the neural network model comprises adding an attention module to the constructed neural network model. In this embodiment, the attention module is added to the constructed sequence neural network model, so that the prediction accuracy of the network model is improved.

[0031] Optionally, the construction of the neural network model comprises using one of the following loss functions: SmoothedL1Loss, L2 Loss, Huber Loss, and Log-Cosh Loss; and / or using one of the following optimizers: Adam, AdamW, SGD, AdaGrad, and RMSProp.

[0032] Optionally, the first digital data set and the second digital data set comprise tooth type parameters and medical scheme type parameters of each tooth.

[0033] Optionally, the medical scheme type parameter group comprises one or any combination of the following: expansion of the arch, closure of the gap, enamel reduction, pushing of the molar, and ideal dental arch curve shape.

[0034] Optionally, the step-by-step scheme outputting step comprises: outputting the stage digital data set in the form of a grid model, the form of coordinate values, or the form of movement amount. The embodiment limits the output of the step-by-step scheme in multiple forms, so that the output form is flexible and diverse.

[0035] Optionally, if the output stage digital data set is in the form of a grid model, the step-by-step scheme outputting step comprises: a grid model generating step, wherein the spatial coordinate transformation of each tooth is calculated by using a point cloud registration algorithm based on the digital data of each tooth in the stage digital data set and the first digital data set; and the grid model of each tooth in the first dentition is subjected to spatial coordinate transformation to obtain the stage grid model of each tooth. The embodiment further limits the calculation of the spatial coordinate transformation by the point cloud registration and the restoration of the grid model of the tooth, and the morphology of the tooth itself is almost unchanged, so that the restoration of the dentition morphology by the registration method can increase the fault tolerance of the predicted value of the feature parameter. BRIEF DESCRIPTION OF DRAWINGS

[0036] One or more embodiments are illustrated by way of example in the figures that form a part of this patent document, these example are not intended to limit the embodiments, elements having the same reference numbers in the figures indicate like elements, unless otherwise expressly stated, the figures in the drawings do not constitute a proportional limit.

[0037] Figure 1 is a flowchart of an automatic step-by-step method based on a neural network provided by an embodiment of the present application;

[0038] Figure 2 is a schematic diagram of a sequence neural network model training process in an automatic step-by-step method based on a neural network provided by an embodiment of the present application;

[0039] Figure 3 is a schematic diagram of a sequence neural network model using an LSTM structure in an automatic step-by-step method based on a neural network provided by an embodiment of the present application;

[0040] Figure 4 is a schematic diagram of a single LSTM unit in a sequence neural network model using an LSTM structure in an automatic step-by-step method based on a neural network provided by an embodiment of the present application;

[0041] Figure 5 is a schematic diagram of a process for calculating grid vertices using model output results in an automatic step-by-step method based on a neural network provided by an embodiment of the present application;

[0042] Figure 6is a schematic diagram of a test example in a step of an automatic step-by-step method based on a neural network provided by an embodiment of the present application;

[0043] Figure 7 is a schematic diagram of a sequence neural network model using an LSTM structure in an automatic step-by-step method based on a neural network provided by another embodiment of the present application;

[0044] Figure 8 is a schematic diagram of a step of processing tooth feature parameters and medical plan parameters using a sequence neural network model in an automatic step-by-step method based on a neural network provided by an embodiment of the present application;

[0045] Figure 9 is a schematic diagram of an electronic device provided by another embodiment of the present application. DETAILED DESCRIPTION

[0046] To make the objectives, technical solutions and advantages of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. However, those skilled in the art can understand that, in the embodiments of the present application, many technical details are presented in order to make the readers better understand the present application. However, the technical solutions claimed by the claims of the present application can be implemented even without these technical details and various changes and modifications based on the following embodiments.

[0047] In addition, the terms "first", "second", and the like are mainly used to distinguish different devices, elements or components (the specific types and structures can be the same or different), and are not used to indicate or imply the relative importance and quantity of the indicated devices, elements or components. Unless otherwise specified, the meaning of "a plurality of" is two or more.

[0048] The "anterior tooth region" and "posterior tooth region" mentioned in various embodiments of the present application are defined according to the classification of teeth in "Introduction to Oral Medicine", 2nd Edition, pp. 36-38, published by Peking University Medical Press, including premolars and molars, which are shown as teeth 4-8 in FDI marking method, and the teeth in the anterior tooth region are shown as teeth 1-3 in FDI marking method. The teeth in the anterior tooth region include central incisors, lateral incisors and canines.

[0049] The inventors of this application discovered in their research on digital design for orthodontic treatment that, in order to automate the step-by-step process of tooth alignment using computer technology, a method of superimposing intermediate interpolation with specified keyframes is often employed. However, in practice, while this method can complete the step-by-step process, the quality of the step-by-step process is unstable due to the limitation of the keyframes specified by the designer based on experience. The reasons for this are twofold: firstly, the interaction of forces during tooth movement makes it not a simple problem of independent rigid body movement; moving tooth A may trigger unwanted movement of adjacent tooth B. Secondly, automatic step-by-step alignment essentially requires predicting the tooth movement path in the next stage. Therefore, achieving a more optimal and reasonable movement path is a very complex problem, and it is difficult to design simple rules to determine a more reasonable movement path. To address the aforementioned technical problems, this invention provides an automatic step-by-step method based on a neural network, comprising: acquiring a first dentition; a first acquisition step, acquiring a first digital dataset representing the current pose of each tooth in the first dentition; a second acquisition step, acquiring a second digital dataset representing the ideal pose of each tooth in the first dentition; a data fusion step, fusing the digital data of the corresponding tooth in the first digital dataset and the second digital dataset for each individual tooth to obtain a fused digital dataset; a step-by-step prediction step, using a trained neural network model for automatic step-by-step processing to process the digital data of each tooth in the fused digital dataset to generate a stage digital dataset representing the target position of the first dentition in the next stage; a data update step, updating the first digital dataset based on the stage digital dataset; repeating the data fusion step, the step-by-step prediction step, and the data update step until a preset prediction stopping condition is reached; and a step-by-step scheme output step, aggregating and outputting all predicted stage digital datasets as the treatment scheme for the first dentition. Throughout the process, this method can solve the problem of low accuracy in step-by-step layout of orthodontic tooth alignment that relies on manual or predetermined methods, obtaining a more reasonable movement path and improving orthodontic efficiency.

[0050] The following details the implementation of the neural network-based automatic step-by-step method of this application. The following content is only for the convenience of understanding and is not necessary for implementing this solution.

[0051] First, it should be noted that the automatic step-by-step method based on neural networks in this embodiment can be implemented through hardware or a combination of computer software and hardware. For hardware implementation, the automatic step-by-step method based on neural networks can be implemented through one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic devices for implementing the automatic step-by-step function based on neural networks, or a selection and combination of the above devices.

[0052] The specific flow of the automatic step-by-step method based on neural networks provided in one embodiment of the present invention can be as follows: Figure 1 As shown, it specifically includes:

[0053] Step 101, First Acquisition Step: Acquire the first digital dataset representing the current pose of each tooth in the first dental row.

[0054] Specifically, the digital dataset can be a digital dataset representing the geometric shape of teeth, which may include a digital model of the jaw and a combination of parameters characterizing tooth pose. The digital model of the jaw can be a digital model of the jaw in the patient's initial position or a digital model of the jaw in the initial position of a test case; the source of the model is determined as needed and will not be listed here. The current pose can be the layout presented by the patient before orthodontic treatment, or it can be the current tooth layout based on the automatic stepping process in this application.

[0055] In some embodiments, the obtained first dentition can be a dentition mesh model after incisor classification, in which each tooth is an independent model. In some embodiments, the first dentition is the first dentition of a single jaw, such as a single maxilla or a single mandible, which is not limited here.

[0056] Step 102, the second acquisition step, acquires a second digital dataset representing the ideal pose of each tooth in the first dentition.

[0057] Specifically, the ideal position of each tooth can be the target position after orthodontic design for the first dentition. The target position can be the position determined by the doctor and medical designers based on the patient's needs and intraoral condition to achieve the final orthodontic effect. Of course, it can also be recommended based on similar cases using intraoral digital design software, and even more targeted adjustments can be made to the patient's treatment based on the recommendations. This invention does not limit this. The ideal position includes aligning the teeth in their current state to a spatial position and posture that meets the doctor's / patient's expectations.

[0058] Step 103, data fusion step: fuse the digital data of the corresponding teeth in the first digital dataset and the second digital dataset for each individual tooth to obtain a fused digital dataset.

[0059] In some embodiments, parameters representing tooth pose are fused from the digital dataset. The digital data uses multiple tooth-type parameters for each tooth. These tooth-type parameters can be data representing pose. Since the shape of the teeth themselves remains almost unchanged during the tooth arrangement process, the neural network model only predicts the changes in tooth pose. Therefore, calculating only the changes in tooth pose can effectively reduce the amount of feature parameters used, reduce the amount of data computation in the prediction, and speed up the model training and prediction.

[0060] In some embodiments, the combination of tooth-like parameters may include: the geometric center point of the tooth, and the six intersection points formed by rays emanating from the geometric center point along the six directions of the three axes in the tooth coordinate system and the tooth mesh. The geometric center point of the tooth is relatively easy to calculate in the model; it can be directly averaged over all mesh vertices of a single tooth. Moreover, this feature effectively locates the tooth position, is well-defined, and has a clear relationship with other contour parameters. Calculations combined with the geometric center point of the tooth are highly accurate. Using the center point and the six intersection points, a total of seven points are used as the feature points of the tooth. These seven feature points can be obtained for each tooth and then arranged in a uniform order. Each point has three spatial coordinates in 3D space, so the feature vector composed of the seven points of each tooth has 21 dimensions. The tooth coordinate system is generally constructed when the dental model is established, and the data can be used directly. The three positive and three negative directions of the three axes are relatively clear. Rays emanating from the geometric center point along these directions will inevitably form six intersection points with the tooth mesh, effectively representing the current position and orientation of the tooth.

[0061] In some embodiments, the combination of tooth-like parameters may include: the eight vertices of the tooth's spatial bounding box and the tooth's geometric center. Each face of the bounding box is perpendicular to an axis in the tooth coordinate system, thus obtaining the 3D spatial coordinates of the eight vertices of each bounding box. Adding the tooth's geometric center, a total of nine points are used as the tooth's feature points. For different teeth, these nine points are arranged according to a uniform pattern. Because each point contains 3D spatial coordinates, each tooth's features have a total of 27 dimensions.

[0062] In other embodiments, the combination of tooth parameters may include: several feature points on the tooth facing the opposing jaw and the geometric center point of the tooth. These feature points can be determined based on the tooth type. Specifically, when the tooth is a molar, the feature point is the buccal apex; or, when the tooth is a premolar, the feature point is the buccal apex and / or buccal groove; or, when the tooth is an incisor or canine, the feature point is the superior incisal edge. In one embodiment, four buccal cusps can be selected on the molars; for the premolars, one cusp can be selected on the lingual and one on the buccal side, and one distal point is selected mesially and distally in the buccal groove as a feature point; for the incisors, one point can be selected on the incisal edge in the direction of the nearest and farthest midpoints, and then two more points are selected at equal intervals on the incisal edge between these two points, thus also taking four feature points on the incisal surface; for the canines, two points are selected at the nearest and farthest midpoints of the incisal edge, the cusp of the canine is also used as a feature point, and then another point is selected at the midpoint of the incisal edge between the cusp and the farthest midpoint as a feature point. Furthermore, the geometric center of each tooth is also used as a feature point, thus each tooth has five feature points. Because each point contains 3-dimensional spatial coordinates, each tooth has a total of 15 dimensions in its features, demonstrating that the amount of feature data selected in this embodiment can be reduced.

[0063] In the selection of the above parameter combinations, each set of tooth feature parameters can be used individually or in combination. For example, in one embodiment, the parameter combination may include: the geometric center point of the tooth, the six intersection points formed by rays emanating from the geometric center point along the six directions of the three axes in the tooth coordinate system and the tooth mesh, and several feature points of the tooth facing the opposing jaw. More types of feature parameter combinations can make the expressed tooth feature details richer, which is beneficial to improving the accuracy of the network model's prediction step-by-step results. It is understood that there are many ways to select and combine parameter types, and to avoid repetition, they will not be listed one by one here.

[0064] Furthermore, the number of teeth varies for different cases, and there may be missing teeth. In such cases, 0 can be used to fill all parameter dimensions of that tooth.

[0065] Since the adjustment of tooth layout is mainly achieved by translation and rotation, the pose of a single tooth can be determined by combining the geometric center with contour features, so as to minimize the amount of data representing tooth pose parameters, reduce the amount of parameter data involved in the calculation, and speed up the processing speed of the model.

[0066] In some embodiments, the data fusion step includes performing weighted operations on the digital data of the same tooth in the first digital dataset and the second digital dataset. This embodiment utilizes weighted operations to better fuse the current pose and the ideal pose, achieving equal weights and avoiding excessive influence from any one parameter. Furthermore, operations such as addition, subtraction, multiplication, averaging, and / or concatenation can be used, and during the calculation, the weight of each item can be 1.

[0067] Taking parameter subtraction as an example, let's calculate the position difference of a single feature point between the ideal pose and the initial pose:

[0068] The characteristic sequence of each tooth in the ideal pose is as follows:

[0069] S t1 ,S t2 ,S t3 ,...,S t16

[0070] Where ti represents the ideal target position of the i-th tooth (i = 1 to 16, indicating that there are 16 teeth in a row of teeth). Taking 9 feature points per tooth as an example:

[0071] S ti =[x ti1 ,y ti1 ,z ti1 ,x ti2 ,y ti2 ,z ti2 ,...,x ti9 ,y ti9 ,z ti9 ]

[0072] Where ti1 represents the first feature point of the i-th tooth in the ideal target position, and x, y, z represent the three-axis coordinates of the feature point, and so on.

[0073] The sequence characteristics of each tooth in a single jaw in the initial pose are as follows:

[0074] S n1 ,S n2 ,S n3 ,...,S n16

[0075] Where ni represents the position of the i-th tooth in the n-th step (i = 1 to 16, also indicating that there are 16 teeth in a row of teeth), and the characteristics of each tooth are as follows:

[0076] S ni =[x ni1 ,y ni1 ,z ni1 ,x ni2 ,y ni2,z ni2 ,...,x ni9 ,y ni9 ,z ni9 ]

[0077] Similarly, ni1 represents the first feature point of the i-th tooth in the n-th step, and x, y, z represent the three-axis coordinates of the feature point, and so on.

[0078] The fused data after subtracting the parameters is as follows:

[0079] S t1 -S n1, S t2 -S n2, S t3 -S n3,..., S t16 -S n16,

[0080] in:

[0081] S ti -S ni =[x ti1 -x ni1, y ti1 -y ni1, z ti1 -z ni1, x ti2 -x ni2, y ti2 -y ni2, z ti2 -z ni2,..., x ti9 -x ni9, y ti9 -y ni9, z ti9 -z ni9 ]

[0082] As can be seen, after feature fusion, the feature of each tooth is the three-axis coordinates of the nine feature points of the target location minus the three-axis coordinates of the corresponding nine feature points in the nth step.

[0083] It can be understood that when the current pose and ideal pose of each tooth are fused by subtracting parameters, the remaining pose difference of the predicted orthodontic steps to reach the final goal is utilized. In the prediction, the neural network model can learn the characteristics of difference convergence relatively easily, so that the prediction of subsequent orthodontic steps can better approach the goal.

[0084] Step 104, step-by-step prediction step: using a trained neural network model for automatic step-by-step processing, the digital data of each tooth in the fused digital dataset is processed to generate a stage digital dataset representing the target position of the first dentition in the next stage.

[0085] Specifically, the next stage can be the next corrective step or the next N corrective steps, where N is a natural number greater than 1. In this embodiment, the step-by-step prediction can predict one corrective step at a time, or multiple corrective steps at a time. That is, it can be set to input step n and predict and output step n+1, or input step n and predict and output steps n+1 and n+2, or input steps n-1 and n and output step n+1, or steps n+1 and n+2. It is understood that in practical applications, even more steps can be set as needed. Therefore, this embodiment selects different prediction methods according to actual needs, facilitating flexible adjustment of the automatic step-by-step method in this application.

[0086] In some embodiments, the neural network model used for automatic step-by-step processing is a sequence neural network model. Furthermore, in processing the digital data of each tooth in the fused digital dataset, the digital data of each tooth in the fused digital dataset is processed according to a preset tooth position order. This embodiment changes the prior art concept of processing tooth arrangement as a geometric image, instead processing different teeth as sequence data. Through a sequence neural network, each tooth is processed separately in a predetermined order to obtain the single-step tooth arrangement result for the entire row of teeth. Moreover, the feature data of each tooth is not mixed during this process, which can effectively improve the accuracy of feature learning. Based on the above characteristics, the inventors of this application selected a sequence neural network model, utilizing its long short-term memory characteristics. When predicting the features of a tooth, it not only refers to the features of adjacent teeth but also the features of other teeth that are far apart, allowing for faster learning of step-by-step characteristics. Throughout the entire processing, compared with existing image network models, the sequence neural network model can learn step-by-step characteristics better and faster, and the prediction results for step-by-step are more accurate.

[0087] In one embodiment, when the first dentition is a single-jaw dentition, the preset tooth position sequence includes: an ascending order of FDI tooth position numbers, such as 11, 12, 13, 14, ... 26, 27, 28 for the maxillary dentition. In other embodiments, the preset tooth position sequence may include: the order from the left terminal tooth to the right terminal tooth in the FDI tooth position sequence; or, the order from the right terminal tooth to the left terminal tooth in the FDI tooth position sequence. For example, for the maxillary dentition, the sequence could be from the 8th molar in zone 1 to the 8th tooth in zone 2, that is, from 18, 17, 16... 26, 27, 28. In other embodiments, the sequence may include descending order of FDI tooth position numbers. It is understood that other tooth position sequences can be set besides the above-mentioned methods, which will not be listed here. Because the arrangement of individual teeth in the current manual step-by-step tooth arrangement process has a certain order, the step-by-step tooth arrangement process is closer to current medical design methods, resulting in better predictive effects.

[0088] Taking a sequence neural network model used for automatic step-by-step processing as an example, combined with... Figure 2 To clarify, the sequence neural network model used in this step for automatic step-by-step processing can be pre-trained in the following way:

[0089] Step 201: Construct fused digital data pairs of teeth in the current step and in the planned subsequent steps of an existing dental arch.

[0090] Step 202: Collect all the constructed data pairs to form a step-by-step training set.

[0091] Specifically, the limitations of the current step and subsequent steps can be set as needed, such as limiting them to two adjacent steps. In this case, all step-by-step target positions of a historical case can be broken down into pairs of adjacent steps. Correspondingly, the trained model can predict the next adjacent corrective step based on the previous corrective step. Since adjacent steps are used as data pairs, more data pairs can be combined, and the pose difference between two adjacent steps is smaller, making it easier for the pose difference principle learned during model training to be more accurate.

[0092] Step 203: Construct a neural network model.

[0093] In some embodiments, a sequence neural network model can be constructed as needed, specifically employing one or a combination of GRU, LSTM, bidirectional GRU, bidirectional LSTM, multi-layer GRU, multi-layer LSTM, and Transformer structures. The sequence neural network model in some embodiments may further include adding an attention module to the constructed sequence neural network model. Further defining this as adding an attention module to the constructed sequence neural network improves the prediction accuracy of the network model. It is understood that different attention mechanisms can be added, such as the Luong attention mechanism and the Bahdanau attention mechanism. Furthermore, these models and attention mechanisms can be used to construct the network into an Encoder-Decoder structure, etc.

[0094] In some embodiments, the loss function used in the constructed sequence neural network model is one of the following: SmoothedL1 Loss, L2 Loss, Huber Loss, or Log-Cosh Loss. In some embodiments, the optimizer used in the constructed sequence neural network model is one of the following: Adam, AdamW, SGD, AdaGrad, or RMSProp. The above loss function and optimizer can be defined simultaneously, and their combinations are diverse, and will not be listed here.

[0095] by Figure 3Taking the LSTM structure sequence neural network model as an example, it specifically includes an encoding module and a decoding module, where the encoding module and the decoding module each include several recurrent units.

[0096] During the encoding phase, X n The nth tooth-type feature parameter group can be represented in one embodiment, where tooth information can be directly used for encoding. In another embodiment, in addition to inputting each tooth-type feature parameter group, location information can also be encoded together. The location information can be the FDI number. Each tooth in the dental arch has a unique FDI number, so using the FDI number as the location information is simple and quick, so that the input of the model includes not only the spatial information of the teeth but also the location information of the teeth.

[0097] To further explain, each loop unit uses an LSTM structure, and its internal structure can be as follows: Figure 4 As shown, the input consists of three parts: input information, hidden state, and memory cell state. The hidden state and memory cell state come from the previous LSTM unit; for the first LSTM unit, they use randomly initialized values. An LSTM unit typically includes three gating units: a forget gate, an input / update gate, and an output gate. The forget gate controls which parts of the historical information are unimportant and can be forgotten, and which need to be retained. It uses the Sigmoid activation function, with a value between 0 and 1, to control the degree of forgetting. The input / update gate also uses the Sigmoid function to determine how much new input information to use, and the tanh function to calculate the current candidate cell state. Then, through the forget gate and input gate, the cell state from the previous unit and the current candidate cell state are linearly combined to obtain the current cell state. The output gate determines which parts of the memory unit will be used to calculate the current hidden state and output to the next time step. It is controlled by a sigmoid function, which determines which parts of the memory cell will affect the final hidden state. Then, it is scaled to the range of [-1, 1] by a tanh function. In other words, the numerical stability can be guaranteed by the tanh function.

[0098] During the decoding phase, Y n This represents the predicted feature parameter set for the nth tooth. The input to the decoding module can be the output of the previous recurrent unit and the hidden state. For recurrent units in an LSTM structure, the input to the recurrent unit can also include the cell state of the previous recurrent unit.

[0099] Step 204: Based on the step-by-step training set, train the constructed neural network model to obtain the neural network model used for automatic step-by-step processing.

[0100] During training, the model sequentially learns the feature parameter sets of each tooth in the input dental arch sample at its initial position through the recurrent module in the constructed sequence neural network model, according to the preset tooth position order. The model then outputs the feature parameter sets of each tooth at its predicted position according to the preset tooth position order. The feature parameter sets of each tooth at its predicted position are compared with those at the target position in the current step to obtain the feature parameter difference value, which is the loss value between the model's predicted value and the labeled data. Using this loss value, the gradient of each parameter can be calculated. Then, using backpropagation and the chain rule, each parameter value of the model is updated along the direction from the model output to the model input. The model training is completed after processing a large amount of data (hundreds of thousands of cases) and multiple iterations. In some embodiments, mini-batch data can be used to update the model parameters during training, which can speed up the training process.

[0101] As can be seen from steps 201 to 204, by pre-constructing a sample set, a sequence neural network model is built, and then trained to obtain a sequence neural network model that can be used for automatic step-by-step processing, resulting in a sequence neural network model that better meets individual needs. The construction and training steps can be completed before implementing the automatic step-by-step processing method in this application, or they can be completed immediately when the automatic step-by-step processing method in this application is required; the execution order is not limited here.

[0102] In some embodiments, such as Figure 5 As shown, step 104 further includes a mesh model generation step, specifically the process of calculating the vertices of the mesh model based on the stage digital dataset of the predicted next-stage target position:

[0103] Step 501: Based on the digital data of each tooth in the stage digital dataset and the first digital dataset, the spatial coordinate transformation of each tooth is calculated using a point cloud registration algorithm.

[0104] Step 502: Based on the spatial coordinate transformation of each tooth, the mesh model of each tooth in the first dentition is transformed to obtain the stage mesh model of each tooth.

[0105] Specifically, based on the transformation of the spatial coordinates of the corresponding points of each tooth, the translation matrix and rotation transformation matrix of each tooth can be calculated using point cloud registration methods (such as the Kabsch method). After applying these translation matrices and rotation transformations to the input grid vertex coordinates of each tooth, the grid vertex coordinates of each tooth at the predicted target position can be obtained.

[0106] Taking the Kabsch algorithm as an example, the specific calculation method is explained as follows:

[0107] (1) From the feature points of the model input and output, we can construct two matrices P and Q, each with a dimension of 3xN. Each column of these two matrices represents the coordinates of the feature points, N is the number of feature points, and the feature points represented by each column of these two matrices are in one-to-one correspondence.

[0108] (2) Decenter these two matrices (p0 and q0 are the centers of P and Q respectively), and the resulting new matrices are still denoted as P and Q;

[0109] (3) Construct the covariance matrix H = P T Q;

[0110] (4) Perform singular value decomposition (SVD) on the covariance matrix H, i.e., H = US V T U and V are two orthogonal matrices, and S is a diagonal matrix with singular values ​​on the diagonal.

[0111] (5) Next, calculate d = sign(det(VU) T To determine whether a rotation correction matrix is ​​needed to ensure the use of a right-handed coordinate system, construct the following matrix based on d:

[0112] (6) The rotation matrix R is calculated by the following formula: R = VIU T R is a 3x3 matrix. The translation matrix r = p0 - R * q0. Applying these two matrices to the initial vertex coordinates of each tooth yields the vertex coordinates of the target tooth. That is:

[0113] X' = ​​RX + r;

[0114] Where X' is the coordinate of a vertex of a tooth in the target position, and X is the coordinate of a vertex of the corresponding initial position tooth.

[0115] As can be seen, steps 501 to 502 further define the calculation of spatial coordinate transformation through point cloud registration and then the reconstruction of the tooth mesh model. Considering that the shape of the teeth themselves is almost unchanged, reconstructing the dentition morphology through registration can increase the error tolerance of the predicted feature parameters. It is understood that in addition to using the Kabsch algorithm for registration and transformation as described above, other registration algorithms can also be used, which will not be listed here.

[0116] Step 105, data update step, update the first digital dataset based on the stage digital dataset.

[0117] Specifically, in this step, the first digital dataset obtained in the previous step is updated based on the predicted stage digital dataset, and the current pose is iteratively updated.

[0118] Step 106: Determine whether the preset prediction stop condition has been met; if not, repeat steps 103-105; if it has been met, proceed to step 107.

[0119] Specifically, the preset prediction stopping conditions may include one of the following: the number of orthodontic steps reaches a preset number, or the difference between the stage digital dataset and the second digital dataset is less than a predetermined range. This predetermined range can be set by the technician as needed, or a set of standard ranges can be preset, such as the movement of each tooth being less than a fixed value (e.g., 0.25mm, 0.1mm, 0.05mm, 0.04mm, 0.03mm, etc.), and the axial tilt or torsion of each tooth being less than a fixed value (e.g., 2.5 degrees, 1 degree, 0.5 degrees, 0.4 degrees, 0.3 degrees, etc.). It is worth noting that this difference can be the difference of a single feature point or the average difference of all feature points of a single tooth, and can be specifically set according to actual needs. Since the automatic prediction process may predict a pose very close to the ideal pose, but not exactly the same value, the prediction process can be paused when the difference between the predicted stage target pose and the ideal pose is sufficiently small. The current predicted target pose can then be used as the final design target pose, or the current model prediction can be ignored and directly used as the final target pose as the current prediction result.

[0120] In some embodiments, due to unknown reasons related to the input data, the automatic step-by-step process of cyclic prediction may fail to reach the allowable range of the ideal pose for an extended period, potentially leading to an infinite loop. Therefore, by limiting the prediction to a preset number of steps (e.g., 80, 100, 120, 200 steps, etc.) and stopping the prediction after reaching this preset number, manual verification can be used to confirm whether the target position is sufficiently close, or whether the excessive number of steps is due to errors in the input data. It is evident that including a preset number of steps in the prediction stopping condition allows the automatic step-by-step method of this application to be implemented more accurately.

[0121] Based on the combination of steps 101 to 106 above, the data fusion step, the step-by-step prediction step, and the data update step can be repeated until a preset prediction stopping condition is reached, including a data correction step, wherein the stage digital dataset generated by the step-by-step prediction step is verified according to preset medical rules, and the stage digital dataset is corrected when the verification fails; wherein the data correction step is executed once every certain number of times the step-by-step prediction step is executed. In this embodiment, a data correction step is added during the cyclic prediction process. Since some of the step-by-step process is based on the constraints of medical rules, the stage correction can avoid discrepancies between the actual step-by-step effect and the actual step-by-step effect caused by the accumulation of errors, thereby improving the accuracy and feasibility of the final output treatment plan.

[0122] In some embodiments, the preset medical rules include: the mode of tooth movement, the amount of tooth movement, and the occlusal relationship with opposing teeth. For example, anchor teeth may not move or may move only a very small range; no more than six teeth in a single jaw may move in the same direction (all towards the mesial or distal direction); and some teeth may even be required to remain immobile at certain stages, etc. These medical rules can be preset by professional designers based on experience, and will not be listed one by one here.

[0123] Step 107, Step-by-step solution output step, gather all the predicted stage digital datasets and output them as the treatment plan for the first dentition.

[0124] Specifically, during output, the staged digital dataset can be displayed in the form of a grid model, coordinate values, or movement quantities. From another perspective, it can be displayed in text form, image form, two-dimensional image form, or three-dimensional image form, and so on. This embodiment limits the output of the step-by-step scheme to multiple formats, making the output format flexible and varied.

[0125] In some embodiments, before the step-by-step output step in step 107, a post-processing procedure is further included for the stage target positions output by the model. This may include: performing collision detection on the stage target positions of each tooth and updating the target positions of each tooth based on the detection results. Since the target positions predicted by the neural network model may result in collisions between teeth, it is necessary to adjust the relative positions of each pair of adjacent teeth in the same jaw if a collision occurs.

[0126] In one embodiment, the method for fine-tuning between teeth in a single jaw is as follows: First, determine whether there is a collision between the two central incisors. If there is a collision, calculate their collision depth d. Then, move both teeth d / 2 distally. Next, starting with the incisors, perform collision detection on the incisors and lateral incisors. If a collision is found, move the lateral incisors distally, and so on. For the canines, premolars, and molars, fine-tune them in the same way. In addition to collision detection on teeth in a single jaw, collision detection can also be performed on corresponding teeth in occlusion by combining data from opposing teeth, thus avoiding excessive collisions on individual teeth during occlusion.

[0127] It is understood that although collision detection is performed before the step-by-step solution output in this embodiment, in practical applications, collision detection can also be performed during the intermediate prediction process. For example, collision detection can be performed on the stage target position obtained in each prediction, or on the stage target position obtained after several predictions. In the process of cyclic prediction, adding collision detection in the intermediate process is beneficial to reduce the intermediate error of the automatic step-by-step method in this application, adjust the teeth that have collided in time, reduce the accumulation of collision error, and improve the accuracy of the final output treatment plan.

[0128] Subsequently, the inventors of this application used the neural network-based automatic step-by-step method described in this application to conduct an accuracy test, and compared the stage target tooth position obtained using the neural network-based automatic step-by-step method with the stage target tooth position obtained manually. Figure 6 A test example shown in the paper demonstrates manual step-by-step segmentation of the dentition at its initial position (displayed in light color) and step-by-step segmentation using the automatic step-by-step method described in this embodiment (displayed in dark color). The images of the 10th, 20th, 30th, 40th, 50th, 60th, and final positions are extracted and superimposed. It can be seen that the target tooth position obtained by the automatic step-by-step method based on the neural network in this application is less different from the target tooth position obtained by manual step-by-step segmentation. This indicates that the automatic step-by-step method based on the neural network in this application has better practical effect and should be widely promoted.

[0129] It is understood that although the neural network model in this embodiment is limited to a sequence neural network model, other neural network models can be used in practical applications, and neural network models with different architectures can be constructed and trained accordingly. These will not be listed one by one here.

[0130] As can be seen, this embodiment utilizes a neural network model to predict the orthodontic path of the dental arch. During prediction, it integrates the current and target pose information of the teeth, ensuring that the predicted step-by-step results gradually approach the target pose. This avoids unclear tooth movement directions in the predicted results, preventing repetitive movement paths, low efficiency, and poor results throughout the orthodontic process. Specifically, it continuously obtains the step-by-step results of the next stage through iterative prediction, with the obtained order being the step-by-step sequence, avoiding confusion. Simultaneously, a prediction stop condition is set to avoid infinite loops caused by the inability to reach the ideal pose during iterative prediction. Furthermore, during the automatic prediction of the target position, there is no need to specify the required number of treatment steps; prediction can be based on the actual situation to obtain a more reasonable treatment path and a more appropriate treatment plan. Therefore, the series of dental arch layouts obtained using the neural network-based automatic step-by-step method in this application not only achieves automatic and rapid results but also reduces the influence of human experience and improves the accuracy of the output treatment plan. In addition, this embodiment uses a sequential neural network model, processing different teeth as sequential data. Through the sequential neural network, each tooth is processed separately in a predetermined order to obtain the tooth arrangement result of the entire dental arch in a certain treatment step. Furthermore, the feature data of each tooth remains separate during this process, effectively improving the accuracy of feature learning. Based on these characteristics, the inventors of this application selected a sequence neural network model, utilizing its long short-term memory feature. When predicting the features of a tooth, it references not only the features of adjacent teeth but also those of other teeth located far apart, enabling faster learning of step-by-step characteristics. Throughout the entire processing, compared to existing image network models, the sequence neural network model learns the automatic step-by-step characteristics better and faster, and the prediction results for each step are more accurate.

[0131] It is understood that the above implementation uses the same neural network model for automatic step-by-step segmentation of all cases. In practical applications, different neural network models can be trained according to different case types. Taking three case types as an example, the three case types include missing teeth, arch expansion, and distal molars. There are three corresponding neural network models for automatic step-by-step segmentation, trained from training sets corresponding to the above three case types. When the case type corresponding to the first dentition is distal molars, the step-by-step prediction step includes: selecting the corresponding neural network model for distal molars based on the case type information for step-by-step prediction. In this embodiment, different neural network models are trained for different case types. Since the tooth arrangement principles for different case types vary greatly, training dedicated neural network models for different case types can be more accurate in step-by-step prediction. It is understood that in addition to setting three case types in this embodiment, two case types, five case types, etc., can also be set in practical applications. The different numbers do not limit the core improvement of this application.

[0132] Another embodiment of this application provides an automatic step-by-step method based on neural networks. The main difference between this embodiment and the previous embodiment is that the tooth arrangement processed in the previous embodiment is a single jaw tooth arrangement, while the tooth arrangement processed in this embodiment includes the maxilla and mandible.

[0133] Specifically, when obtaining the first dentition, the first dentition of the maxilla and mandible of the same patient is obtained. It can be understood that the maxilla and mandible can be the maxillary and mandibular teeth arrangement after occlusal registration.

[0134] Correspondingly, in addition to adopting the tooth arrangement order in the previous embodiment in a single jaw, the preset tooth position order can be arranged in the order of arranging the teeth of the maxillary dentition first and then arranging the teeth of the mandibular dentition, or in the order of arranging the teeth of the mandibular dentition first and then arranging the teeth of the maxillary dentition.

[0135] The process of predicting the target tooth location using a sequence neural network model for automatic step-by-step analysis is similar to the previous implementation method and will not be described again here.

[0136] In some embodiments, after obtaining the target dataset for the stage, collision detection and adjustment can be performed on the teeth between the upper and lower jaws. During adjustment, fine-tuning can be performed first on the teeth between individual jaws, and then on the teeth between the upper and lower jaws. In one embodiment, collision is first checked on all teeth between the upper and lower jaws. If there is a collision depth d, the teeth colliding in the upper and lower jaws are moved a distance d / 2 in an upward or downward direction, respectively. If there is a gap, the teeth with larger gaps in the upper and lower jaws are moved downward and upward, respectively, to reduce the gaps between the teeth.

[0137] In summary, when the first dentition obtained includes the layout of the maxillary teeth and the layout of the mandibular teeth, the automatic step-by-step method based on neural networks in this application is also applicable, and the automatic step-by-step effect is also good.

[0138] Another embodiment of this application provides an automatic step-by-step method based on neural networks. This embodiment is an improvement on the previous embodiment. The main improvement is that the digital dataset input to the neural network model in the previous embodiment only includes dental feature parameters, while in this embodiment, in addition to dental feature parameters, medical plan parameters are also combined. By combining these medical plans for learning step-by-step, the speed at which the neural network model learns the step-by-step features can be further accelerated, and the accuracy of the predicted stage target position can be improved.

[0139] One architecture of the sequence neural network model used in this embodiment is as follows: Figure 7As shown, it specifically includes an encoding module, a feature concatenation module, and a decoding module. Specifically, the features output by the last loop unit in the encoding module can be combined with features associated with the medical protocol as input features for the decoder. In some embodiments, the medical protocol parameter group includes one or any combination of the following: arch expansion, gap closure, enamel reduction, tooth extraction, and ideal arch curve morphology.

[0140] like Figure 8 As shown, when processing tooth-like feature parameters using the sequence neural network model in this embodiment, the specific steps include:

[0141] Step 801: Using the encoding module in the trained sequence neural network model for automatic step-by-step processing, the tooth-type feature parameter groups of each tooth are encoded according to the preset tooth position order. Specifically, this step is similar to the first implementation method and will not be described again here.

[0142] Step 802: Using the decoding module in the trained sequence neural network model for automatic step-by-step processing, the obtained encoding result and the pre-acquired medical plan parameter group are decoded to obtain the target position of each tooth.

[0143] Specifically, the encoding result obtained in step 801 is concatenated with the pre-acquired medical plan parameter group, and the concatenated result is decoded. In some embodiments, the pre-acquired medical plan parameter group can be encoded by a preset text encoder before concatenation. When applying medical plan feature parameters, the method of first encoding with a preset text encoder and then concatenating with the original parameters is more convenient to cover the rich expression methods in existing medical plans, making the medical plans more learnable. When combining medical plan parameters, parameter concatenation can be used to improve the combined application of multiple types of data. In addition, since combining medical plan parameters actually integrates multiple forms of data in the overall dental alignment, a multimodal solution is realized. It can be understood that in the feature concatenation part, the data output by the text encoder can be processed by a fully connected layer before concatenation. Since the numerical range of the encoded data output by the text encoder and the numerical range of the encoded data output by the encoding module may differ greatly, a fully connected layer can make the value ranges of the two closer, eliminating the interference of large values ​​on the results when the numerical differences are too large.

[0144] As can be seen, in this embodiment, when using a neural network model for automatic step-by-step segmentation, in addition to using the characteristic parameters of the teeth themselves in the dentition, medical plan parameters can also be combined. Since doctors or patients may specify some medical plans during the automatic step-by-step segmentation process, combining these medical plans for learning automatic step-by-step segmentation can further accelerate the speed at which the neural network model learns the automatic step-by-step features and improve the accuracy of the predicted step-by-step target location.

[0145] It is worth mentioning that the examples above in this application are merely illustrative for ease of understanding and do not constitute a limitation on the technical solutions of this invention.

[0146] Another embodiment of this application provides an automatic step-by-step method based on neural networks. This embodiment is largely the same as the aforementioned embodiment, with the main difference being that the same neural network model was used in the step-by-step prediction process for a case in the aforementioned embodiment, while more than one neural network model is used in the step-by-step prediction process for a case in this embodiment.

[0147] Specifically, there can be M pre-defined step-by-step stages, and K neural network models for automatic step-by-step division, with at least two step-by-step stages corresponding to different neural network models; wherein, M and K are natural numbers greater than 1. In this embodiment, the overall step-by-step stages of a dental arch are further divided into multiple segments, and different neural network models are assigned for prediction. Since the step-by-step methods of different stages may be different, constructing neural network models for prediction separately is beneficial to improving the accuracy of the step-by-step results of each stage.

[0148] In some embodiments, the M step-by-step stages are divided by the number of steps, and the neural network models corresponding to different numbers of steps are trained using sample data from historical cases for the corresponding steps. In a specific embodiment, the stage is divided into two steps: the first stage is from the initial position to step 10, and the second stage is from step 11 to the final target. Two neural network models are trained, namely a first neural network model and a second neural network model. When predicting steps 1-10, the first neural network model is used for prediction. When predicting step 11 until the cutoff condition is reached, the second neural network model is used for prediction. In the step-by-step output step, the prediction results of the two stages are combined and output.

[0149] In some embodiments, the K neural network models used for automatic step-by-step processing have the same model architecture. Specifically, they all use sequence neural network models. In this embodiment, the different neural network models are limited to using the same model architecture, and different sample data are used only during training to simplify model selection.

[0150] As can be seen, in this embodiment, the steps are divided into stages based on the number of steps. Since the principles of tooth arrangement differ depending on the timing of the tooth arrangement process, such as the early stage, the middle stage, and the late stage, dividing the process based on the number of steps makes it easier to select different historical data to form different sample data during model training, thereby accurately training different neural network models.

[0151] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0152] Another embodiment of this application relates to an electronic device, such as... Figure 9 As shown, it includes: at least one processor 901; and a memory 902 communicatively connected to the at least one processor 901; wherein the memory 902 stores instructions executable by the at least one processor 901, the instructions being executed by the at least one processor 901 to enable the at least one processor 901 to execute the neural network-based automatic step-by-step method in the above partial embodiments.

[0153] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0154] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0155] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the neural network-based automatic step-by-step method in some of the above embodiments, or the digital modeling method for dental orthodontic jaw models in some of the above embodiments.

[0156] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A neural network-based automatic stepwise method, characterized by, The method comprises: a first acquisition step of acquiring a first digital dataset representing current poses of teeth in a first dentition; a second acquisition step of acquiring a second digital dataset representing ideal poses of teeth in the first dentition; a data fusion step of fusing digital data of corresponding teeth in the first digital dataset and the second digital dataset respectively for each tooth to obtain a fused digital dataset; a step-by-step prediction step of processing digital data of each tooth in the fused digital dataset by using a trained neural network model for automatic step-by-step prediction to generate a stage digital dataset representing a target position of the first dentition in a next stage; a data updating step of updating the first digital dataset based on the stage digital dataset; repeating the data fusion step, the step-by-step prediction step and the data updating step until a preset prediction stopping condition is reached; and a step-by-step scheme output step of collecting all predicted stage digital datasets and outputting them as a treatment scheme of the first dentition.

2. The neural network-based automatic step dividing method according to claim 1, wherein, The neural network model for automatic step-by-step prediction belongs to a sequence neural network model, and in the processing of digital data of each tooth in the fused digital dataset, the digital data of each tooth in the fused digital dataset is processed in a preset tooth position order.

3. The neural network-based automatic step dividing method according to claim 2, wherein, In the first acquisition step, the first dentition is a single-jaw dentition. The preset tooth position order includes: an order from small to large FDI tooth position number; or an order from large to small FDI tooth position number; or an order from the last tooth in the left region to the last tooth in the right region; or an order from the last tooth in the right region to the last tooth in the left region.

4. The neural network-based automatic step dividing method according to claim 2, wherein, The acquired first dentition includes the dentition of the upper jaw and the lower jaw of the same patient; and the preset tooth position order includes: an order of arranging the teeth of the upper jaw first and then arranging the teeth of the lower jaw, or an order of arranging the teeth of the lower jaw first and then arranging the teeth of the upper jaw.

5. The neural network-based automatic step dividing method according to any one of claims 1-4, wherein, The data fusion step includes: performing the same weight operation on the digital data of the same tooth in the first digital dataset and the second digital dataset.

6. The neural network-based automatic step dividing method according to claim 5, wherein, The data fusion step includes: performing addition, subtraction, multiplication, averaging and / or splicing operations on the digital data of the same tooth in the first digital dataset and the second digital dataset.

7. The neural network-based automatic step dividing method according to any one of claims 1-4, wherein, There are M step-by-step stages, and there are K neural network models for automatic step-by-step prediction, at least two step-by-step stages correspond to different neural network models; wherein M and K are natural numbers greater than 1.

8. The neural network-based automatic step dividing method according to claim 7, wherein, The M step-by-step stages are divided by the number of steps, and the neural network models corresponding to different step numbers are trained by sample data corresponding to the steps in the historical cases.

9. The neural network-based automatic step dividing method according to claim 7, wherein, The model architectures of the K neural network models for automatic step-by-step prediction are the same.

10. The neural network-based automatic step dividing method according to any one of claims 1-4, wherein, The first dentition corresponds to a case type, and there are multiple neural network models for automatic step-by-step prediction, which are trained by training sets corresponding to respective case types; The step-by-step prediction step includes: selecting a neural network model corresponding to the first dentition for step-by-step prediction according to case type information.

11. The neural network-based automatic step dividing method according to any one of claims 1-4, wherein, The next stage in the step-by-step prediction step is the next orthodontic step or the next N orthodontic steps, where N is a natural number greater than 1.

12. The neural network-based automatic step dividing method according to any one of claims 1-4, wherein, The repeated data fusion step, step-by-step prediction step, and data updating step until a predetermined prediction stopping condition is reached include a data correction step, in which the stage digital data set generated by the step-by-step prediction step is verified according to a predetermined medical rule, and the stage digital data set is corrected if the verification fails. The data correction step is performed once for every several times the step-by-step prediction step is performed.

13. The neural network-based automatic step dividing method according to claim 12, wherein, The predetermined medical rule includes the movement of the teeth, the movement of the teeth, and the occlusal relationship with the opposing teeth.

14. The neural network-based automatic step dividing method according to claim 12, wherein, The data correction step further includes a collision detection step, in which the stage pose of each tooth is detected for collision, and the stage pose of each tooth is corrected according to the detection result.

15. The neural network-based automatic step dividing method according to any one of claims 1-4, wherein, The predetermined prediction stopping condition includes one of the following: the number of orthodontic steps reaches a predetermined number, and the difference between the stage digital data set and the second digital data set is less than a predetermined range.

16. The neural network-based automatic step dividing method according to any one of claims 1-4, wherein, Before the step-by-step prediction step, a model training step is included, in which A fusion digital data pair is constructed for each tooth in the current step and a predetermined subsequent step with the existing dentition; All constructed data pairs are collected to form a step-by-step training set; A neural network model is constructed; Based on the step-by-step training set, the constructed neural network model is trained to obtain the neural network model for automatic step-by-step prediction.

17. The neural network-based automatic step dividing method according to claim 16, wherein, The current step and the predetermined subsequent step are two adjacent steps, and the next stage target position in the step-by-step prediction step is the target position of the next orthodontic step.

18. The neural network-based automatic step dividing method of claim 16, wherein, The constructed neural network model is a sequence neural network model.

19. The neural network-based automatic step dividing method according to claim 18, wherein, The sequence neural network model includes one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure.

20. The neural network-based automatic step dividing method of claim 16, wherein, The construction of the neural network model includes adding an attention module to the constructed neural network model.

21. The neural network-based automatic step dividing method of claim 16, wherein, The loss function used in the construction of the neural network model is one of the following: SmoothedL1Loss, L2 Loss, Huber Loss, and Log-Cosh Loss; and / or The optimizer used is one of the following: Adam, AdamW, SGD, AdaGrad, and RMSProp.

22. The neural network-based automatic step dividing method according to any one of claims 1-4, wherein, The first digital data set and the second digital data set include tooth class parameters of each tooth; or, include tooth class parameters and medical scheme class parameters of each tooth.

23. The neural network-based automatic step dividing method of claim 22, wherein, The tooth class parameters include one of the following combinations: The geometric center of the tooth, and the six intersection points formed by the six rays along the three axes of the tooth coordinate system from the geometric center of the tooth to the tooth mesh; Or, The eight vertices of the space bounding box of the tooth and the geometric center of the tooth; Or, The geometric center of the tooth and the several feature points of the tooth facing the opposing tooth.

24. The neural network-based automatic step dividing method according to claim 23, wherein, The feature point of the tooth-facing counter-jaw is determined according to the tooth type, wherein when the tooth is a molar, the feature point is a buccal cusp point; or when the tooth is a premolar, the feature point is a buccal cusp or a buccal groove point; or when the tooth is an incisor or a canine, the feature point is an upper incisal edge point.

25. The neural network-based automatic step dividing method of claim 22, wherein, The medical scheme type parameter group comprises one or any combination of the following: expansion of arch, closure of gap, reduction of enamel, pushing of molar, ideal dental arch curve shape.

26. The neural network-based automatic step dividing method according to any one of claims 1-4, wherein, The step-by-step scheme output step comprises: outputting the stage digital data set in the form of a grid model, coordinate value or movement amount.

27. The neural network-based automatic step dividing method of claim 24, wherein, If the output stage digital data set is in the form of a grid model, the step-by-step scheme output step comprises: a grid model generation step, wherein, Based on the digital data of each tooth in the stage digital data set and the first digital data set, the spatial coordinate transformation of each tooth is calculated by using a point cloud registration algorithm; According to the spatial coordinate transformation of each tooth, the grid model of each tooth in the first dentition is subjected to spatial coordinate transformation to obtain the stage grid model of each tooth.

28. An electronic device, comprising: Comprise: At least one processor; And The memory is in communication connection with the at least one processor; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the automatic step-by-step method based on the neural network as claimed in any one of claims 1 to 27.

Images