Neural network-based automatic staging method and electronic device

Abstract

The present application relates to the technical field of orthodontic digital design. Disclosed are a neural network-based automatic staging method and an electronic device. The automatic staging method comprises: a first acquisition step and a second acquisition step of respectively acquiring a first digital data set and a second digital data set representing a first dentition; a data fusion step of respectively fusing, on a per-tooth basis, digital data of corresponding teeth in the first digital data set and the second digital data set to obtain a fused digital data set; a staging prediction step of processing digital data of each tooth in the fused digital data set by using a trained neural network model for automatic staging to generate a stage digital data set representing a target position of the first dentition in the next stage; a data update step of updating the first digital data set on the basis of the stage digital data set; repeating the data fusion step, the staging prediction step, and the data update step until a preset prediction stopping condition is met; and a staging scheme output step.

Description

Neural Network-Based Automatic Step-by-Step Methods and Electronic Devices Cross-reference to related applications

[0001] This application is based on and claims priority to Chinese Patent Application No. 202411199198.0, filed on August 28, 2024, the entire contents of which are hereby incorporated herein by reference. Technical Field

[0002] This application relates to the field of orthodontic digital design technology, and in particular to an automatic step-by-step method and electronic device based on neural networks. Background Technology

[0003] Shell-shaped orthodontic appliances are a type of orthodontic device made of safe, elastic, transparent polymer material. They have the advantages of being completely invisible during the orthodontic process, aesthetically pleasing, easy to operate, and convenient for oral cleaning. Moreover, due to their transparent and aesthetically pleasing characteristics, the orthodontic process is completed almost imperceptibly to others, and they have gradually become the first choice for orthodontic patients.

[0004] With the rapid development of computer technology, dental treatment is increasingly relying on computer technology. For example, in orthodontic treatment using shell-shaped orthodontic appliances, a three-dimensional digital model representing the target tooth layout is typically generated based on a three-dimensional digital model of the dentition representing the initial tooth layout. Then, based on the initial and target tooth layouts, a three-dimensional digital model representing the progressively changing intermediate tooth layout is generated. Finally, a series of shell-shaped orthodontic appliances are fabricated based on these three-dimensional digital models. During treatment, the patient wears these shell-shaped orthodontic appliances sequentially, allowing the teeth to move along the path formed by the intermediate tooth layout, achieving the final neat alignment.

[0005] Current techniques primarily generate a series of intermediate tooth layouts by interpolating between the initial and target positions. This method typically employs a uniform layout, but tooth movement is not uniform, making uniform interpolation sometimes unreasonable. To achieve a more reasonable step-by-step approach, existing techniques specify key layouts between the initial and target layouts, then interpolate between adjacent key positions to improve accuracy. However, confirming the key layouts and their arrival times requires prior specification by experienced designers, making the results highly dependent on the operator's skill, resulting in inconsistent final quality. Summary of the Invention

[0006] The purpose of this application is to provide an automatic step-by-step method and electronic device based on neural networks, which can solve the problem of low accuracy in the step-by-step process of orthodontic tooth alignment that relies on manual or predetermined methods.

[0007] To address the aforementioned technical problems, embodiments of this application provide an automatic step-by-step method based on a neural network, comprising: a first acquisition step, acquiring a first digital dataset representing the current pose of each tooth in a first dentition; a second acquisition step, acquiring a second digital dataset representing the ideal pose of each tooth in the first dentition; a data fusion step, fusing the digital data of the corresponding tooth in the first digital dataset and the second digital dataset for each individual tooth to obtain a fused digital dataset; a step-by-step prediction step, using a trained neural network model for automatic step-by-step processing to process the digital data of each tooth in the fused digital dataset to generate a stage digital dataset representing the target position of the first dentition in the next stage; a data update step, updating the first digital dataset based on the stage digital dataset; repeating the data fusion step, the step-by-step prediction step, and the data update step until a preset prediction stopping condition is reached; and a step-by-step scheme output step, aggregating and outputting all predicted stage digital datasets as a treatment scheme for the first dentition.

[0008] Embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described neural network-based automatic step-by-step method.

[0009] Compared to existing technologies, this application utilizes a neural network model to predict the orthodontic path of the dentition. During prediction, it integrates the current and target tooth pose information, ensuring that the predicted step-by-step results gradually approach the target pose. This avoids unclear tooth movement directions in the predicted results, preventing repetitive movement paths, inefficiency, and poor results throughout the orthodontic process. Specifically, it uses a cyclic prediction method to continuously obtain the next stage's step-by-step results, with the obtained results forming the step-by-step sequence, thus avoiding confusion. Simultaneously, a prediction stopping condition is set to prevent infinite loops caused by the inability to reach the ideal pose. Furthermore, during the automatic prediction of the target position, there is no need to specify the required number of treatment steps; predictions can be made based on the actual situation, resulting in a more reasonable treatment path and a more appropriate treatment plan. Therefore, the series of dentition layouts obtained using the neural network-based automatic step-by-step method in this application not only achieves automatic and rapid results, but also reduces the influence of human experience and improves the accuracy of the output treatment plan. Attached Figure Description

[0010] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0011] Figure 1 is a flowchart of an automatic step-by-step method based on a neural network provided in one embodiment of this application;

[0012] Figure 2 is a schematic diagram of the sequence neural network model training process in the automatic step-by-step method based on neural networks provided in one embodiment of this application;

[0013] Figure 3 is a schematic diagram of a sequence neural network model with an LSTM structure used in the automatic step-by-step method based on neural networks provided in one embodiment of this application;

[0014] Figure 4 is a schematic diagram of a single LSTM unit in a sequence neural network model with an LSTM structure used in an automatic step-by-step method based on neural networks provided in an embodiment of this application.

[0015] Figure 5 is a schematic diagram of the process of calculating mesh vertices using model output results in the automatic step-by-step method based on neural networks provided in one embodiment of this application;

[0016] Figure 6 is a schematic diagram of a step-by-step test example in the automatic step-by-step method based on neural networks provided in one embodiment of this application;

[0017] Figure 7 is a schematic diagram of a sequence neural network model with an LSTM structure used in an automatic step-by-step method based on a neural network provided in another embodiment of this application;

[0018] Figure 8 is a schematic diagram of the steps in the automatic step-by-step method based on neural networks provided in an embodiment of this application, which uses a sequence neural network model to process dental feature parameters and medical plan parameters.

[0019] Figure 9 is a schematic diagram of an electronic device provided in another embodiment of this application. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been presented in the various embodiments of this application to facilitate a better understanding of the application. However, the technical solutions claimed in the claims of this application can be implemented even without these technical details and with various variations and modifications based on the following embodiments.

[0021] Furthermore, the terms "first," "second," etc., are primarily used to distinguish different devices, components, or parts (which may be the same or different in specific type and construction), and are not intended to indicate or imply the relative importance or quantity of the indicated devices, components, or parts. Unless otherwise stated, "a plurality of" means two or more.

[0022] The terms "anterior region" and "posterior region" mentioned in the various embodiments of this application are defined according to the classification of teeth in the 2nd edition of *Introduction to Stomatology*, published by Peking University Medical Press, pages 36-38. These include premolars and molars, teeth marked as 4-8 according to the FDI (Fédération Dentaire Internationale) designation, and teeth marked as 1-3 according to the FDI designation for the anterior region. Teeth in the anterior region include the central incisors, lateral incisors, and canines.

[0023] The inventors of this application discovered in their research on digital design for orthodontic treatment that, in order to automate the step-by-step process of tooth alignment using computer technology, a method of superimposing intermediate interpolation with specified keyframes is often employed. However, in practice, while this method can complete the step-by-step process, the quality of the step-by-step process is unstable due to the limitation of the keyframes specified by the designer based on experience. The reasons for this are twofold: firstly, the interaction of forces during tooth movement makes it not a simple problem of independent rigid body movement; moving tooth A may trigger unwanted movement of adjacent tooth B. Secondly, automatic step-by-step alignment essentially requires predicting the tooth movement path in the next stage. Therefore, achieving a more optimal and reasonable movement path is a very complex problem, and it is difficult to design simple rules to determine a more reasonable movement path. To address the aforementioned technical problems, this application provides an automatic step-by-step method based on a neural network, comprising: acquiring a first dentition; a first acquisition step, acquiring a first digital dataset representing the current pose of each tooth in the first dentition; a second acquisition step, acquiring a second digital dataset representing the ideal pose of each tooth in the first dentition; a data fusion step, fusing the digital data of the corresponding tooth in the first digital dataset and the second digital dataset for each individual tooth to obtain a fused digital dataset; a step-by-step prediction step, using a trained neural network model for automatic step-by-step processing to process the digital data of each tooth in the fused digital dataset to generate a stage digital dataset representing the target position of the first dentition in the next stage; a data update step, updating the first digital dataset based on the stage digital dataset; repeating the data fusion step, the step-by-step prediction step, and the data update step until a preset prediction stopping condition is reached; and a step-by-step scheme output step, aggregating and outputting all predicted stage digital datasets as the treatment scheme for the first dentition. Throughout the process, this method can solve the problem of low accuracy in step-by-step layout of orthodontic tooth alignment that relies on manual or predetermined methods, obtaining a more reasonable movement path and improving orthodontic efficiency.

[0024] The following details the implementation of the neural network-based automatic step-by-step method of this application. The following content is only for the convenience of understanding and is not necessary for implementing this solution.

[0025] First, it should be noted that the automatic step-by-step method based on neural networks in this embodiment can be implemented through hardware or a combination of computer software and hardware. For hardware implementation, the automatic step-by-step method based on neural networks can be implemented through one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic devices for implementing the automatic step-by-step function based on neural networks, or a selection and combination of the above devices.

[0026] The specific flow of the automatic step-by-step method based on neural networks provided in some embodiments of this application can be shown in Figure 1, and specifically includes:

[0027] Step 101, First Acquisition Step: Acquire the first digital dataset representing the current pose of each tooth in the first dental row.

[0028] In some embodiments, the digital dataset can be a digital dataset representing the geometry of teeth, which may include a digital model of the jaw and a combination of parameters characterizing tooth pose. The digital model of the jaw can be a digital model of the jaw in the patient's initial position or a digital model of the jaw in the initial position of a test case. The source of the model is determined as needed and will not be listed here. The current pose can be the layout presented to the patient before orthodontic treatment, or it can be the current tooth layout based on the automatic stepping process in this application.

[0029] In some embodiments, the obtained first dentition can be a dentition mesh model after incisor classification, in which each tooth is an independent model. In some embodiments, the first dentition is the first dentition of a single jaw, such as a single maxilla or a single mandible, which is not limited here.

[0030] Step 102, the second acquisition step, acquires a second digital dataset representing the ideal pose of each tooth in the first dentition.

[0031] In some embodiments, the ideal position of each tooth can be the target position after orthodontic design for the first dentition. The target position can be the position determined by the doctor and medical designers based on the patient's needs and intraoral condition to achieve the final orthodontic effect. Of course, it can also be recommended based on similar cases using intraoral digital design software. Furthermore, it can be used to make more targeted adjustments to the patient's treatment based on the recommendations. This application does not limit this. The ideal position includes aligning the teeth in their current state to a spatial position and posture that meets the doctor's / patient's expectations.

[0032] Step 103, data fusion step: fuse the digital data of the corresponding teeth in the first digital dataset and the second digital dataset separately for each individual tooth to obtain a fused digital dataset.

[0033] In some embodiments, parameters representing tooth pose are fused from the digital dataset. The digital data uses one or more tooth-type parameters for each tooth. These tooth-type parameters can be data representing pose. Since the shape of the teeth themselves remains almost unchanged during the tooth arrangement process, the neural network model only predicts the changes in tooth pose. Therefore, calculating only the changes in tooth pose can effectively reduce the amount of feature parameters used, reduce the amount of data computation in the prediction, and speed up the model training and prediction.

[0034] In some embodiments, the combination of tooth-like parameters may include: the geometric center point of the tooth, and the six intersection points formed by rays emanating from the geometric center point along the six directions of the three axes in the tooth coordinate system and the tooth mesh. The geometric center point of the tooth is relatively easy to calculate in the model; it can be directly averaged over all mesh vertices of a single tooth. Moreover, this feature effectively locates the tooth position, is well-defined, and has a clear relationship with other contour parameters. Calculations combined with the geometric center point of the tooth are highly accurate. Using the center point and the six intersection points, a total of seven points are used as the feature points of the tooth. These seven feature points can be obtained for each tooth and then arranged in a uniform order. Each point has three spatial coordinates in 3D space, so the feature vector composed of the seven points of each tooth has 21 dimensions. The tooth coordinate system is generally constructed when the dental model is established, and the data can be used directly. The three positive and three negative directions of the three axes are relatively clear. Rays emanating from the geometric center point along these directions will inevitably form six intersection points with the tooth mesh, effectively representing the current position and orientation of the tooth.

[0035] In some embodiments, the combination of tooth-like parameters may include: the eight vertices of the tooth's spatial bounding box and the tooth's geometric center point. Each face of the bounding box is perpendicular to an axis in the tooth coordinate system, thus obtaining the 3D spatial coordinates of the eight vertices of each bounding box. Adding the tooth's geometric center point, a total of nine points are used as the tooth's feature points. For different teeth, these nine points are arranged according to a uniform pattern. Because each point contains 3D spatial coordinates, each tooth's features have a total of 27 dimensions.

[0036] In some embodiments, the combination of tooth-related parameters may include: several feature points on the tooth facing the opposing jaw and the geometric center point of the tooth. These feature points can be determined based on the tooth type; for example, when the tooth is a molar, the feature point is the buccal apex; or, when the tooth is a premolar, the feature point is the buccal apex and / or buccal groove; or, when the tooth is an incisor or canine, the feature point is the superior incisal edge. In one embodiment, four buccal cusps can be selected on the molars; for the premolars, one cusp can be selected on the lingual and one on the buccal side, and one distal point is selected mesially and distally in the buccal groove as a feature point; for the incisors, one point can be selected on the incisal edge in the direction of the nearest and farthest midpoints, and then two more points are selected at equal intervals on the incisal edge between these two points, thus also taking four feature points on the incisal surface; for the canines, two points are selected at the nearest and farthest midpoints of the incisal edge, the cusp of the canine is also used as a feature point, and then another point is selected at the midpoint of the incisal edge between the cusp and the farthest midpoint as a feature point. Furthermore, the geometric center of each tooth is also used as a feature point, thus each tooth has five feature points. Because each point contains 3-dimensional spatial coordinates, each tooth has a total of 15 dimensions in its features, demonstrating that the amount of feature data selected in this embodiment can be reduced.

[0037] In the selection of the above parameter combinations, each set of tooth feature parameters can be used individually or in combination. For example, in one embodiment, the parameter combination may include: the geometric center point of the tooth, the six intersection points formed by rays emanating from the geometric center point along the six directions of the three axes in the tooth coordinate system and the tooth mesh, and several feature points of the tooth facing the opposing jaw. More types of feature parameter combinations can make the expressed tooth feature details richer, which is beneficial to improving the accuracy of the network model's prediction step-by-step results. It is understood that there are many ways to select and combine parameter types, and to avoid repetition, they will not be listed one by one here.

[0038] Furthermore, the number of teeth varies for different cases, and there may be missing teeth. In such cases, 0 can be used to fill all parameter dimensions of that tooth.

[0039] Since the adjustment of tooth layout is mainly achieved by translation and rotation, the pose of a single tooth can be determined by combining the geometric center with contour features, so as to minimize the amount of data representing tooth pose parameters, reduce the amount of parameter data involved in the calculation, and speed up the processing speed of the model.

[0040] In some embodiments, the data fusion step includes performing weighted operations on the digital data of the same tooth in the first and second digital datasets. This application embodiment utilizes weighted operations to better fuse the current pose and the ideal pose, achieving equal weights and avoiding excessive influence from any single parameter. In some embodiments, addition, subtraction, multiplication, averaging, and / or concatenation operations can be used. That is, the data fusion step includes performing addition, subtraction, multiplication, averaging, and / or concatenation operations on the digital data of the same tooth in the first and second digital datasets. During the calculation, the weight of each item can be 1.

[0041] Taking parameter subtraction as an example, let's calculate the position difference of a single feature point between the ideal pose and the initial pose:

[0042] The characteristic sequence of each tooth in the ideal pose is as follows:

[0043] S t1 ,S t2 ,S t3 ,...,S t16 ;

[0044] Where ti represents the ideal target position of the i-th tooth (i = 1 to 16, indicating that there are 16 teeth in a row of teeth). Taking 9 feature points per tooth as an example:

[0045] S ti =[x ti1 ,y ti1 ,z ti1 ,x ti2 ,y ti2 ,z ti2 ,...,x ti9 ,y ti9 ,z ti9 ];

[0046] Where ti1 represents the first feature point of the i-th tooth in the ideal target position, and x, y, z represent the three-axis coordinates of the feature point, and so on.

[0047] The sequence characteristics of each tooth in a single jaw in the initial pose are as follows:

[0048] S n1 ,S n2 ,S n3 ,...,S n16 ;

[0049] Where ni represents the position of the i-th tooth in the n-th step (i = 1 to 16, also indicating that there are 16 teeth in a row of teeth), and the characteristics of each tooth are as follows:

[0050] S ni =[x ni1 y ni1 , z ni1 x ni2 y ni2 , z ni2 , ..., x ni9 y ni9 , z ni9 ];

[0051] Similarly, ni1 represents the first feature point of the i-th tooth in the n-th step, and x, y, z represent the three-axis coordinates of the feature point, and so on.

[0052] The fused data after subtracting the parameters is as follows:

[0053] S t1 -S n1 ,S t2 -S n2 ,S t3 -S n3 ,...,S t16 -S n16 ,

[0054] in:

[0055] S ti -S ni =[x ti1 -x ni1 ,y ti1 -y ni1 ,z ti1 -z ni1 ,x ti2 -x ni2 ,y ti2 -y ni2 ,z ti2 -z ni2 ,...,x ti9 -x ni9 ,y ti9 -y ni9 ,z ti9 -z ni9 ];

[0056] As can be seen, after feature fusion, the feature of each tooth is the three-axis coordinates of the nine feature points of the target location minus the three-axis coordinates of the corresponding nine feature points in the nth step.

[0057] It can be understood that when the current pose and ideal pose of each tooth are fused by subtracting parameters, the remaining pose difference of the predicted orthodontic steps to reach the final goal is utilized. In the prediction, the neural network model can learn the characteristics of difference convergence relatively easily, so that the prediction of subsequent orthodontic steps can better approach the goal.

[0058] Step 104, the step-by-step prediction step, uses a trained neural network model for automatic step-by-step processing to process the digital data of each tooth in the fused digital dataset, and generates a stage digital dataset representing the target position of the first dentition in the next stage.

[0059] In some embodiments, the next stage can be the next corrective step or the next N corrective steps, where N is a natural number greater than 1. In this application embodiment, the step-by-step prediction can be limited to predicting one corrective step at a time, or multiple corrective steps at a time. That is, it can be set to input the nth step and predict the output of the (n+1)th step, or it can be set to input the nth step and predict the outputs of the (n+1)th and (n+2)th steps, or it can input the (n-1)th and nth steps and output the (n+1)th step, or the (n+1)th and (n+2)th steps. It is understood that in practical applications, even more steps can be set as needed. Therefore, this application embodiment selects different prediction methods according to actual needs, facilitating flexible adjustment of the automatic step-by-step method in this application.

[0060] In some embodiments, the neural network model used for automatic step-by-step processing is a sequence neural network model. Furthermore, in processing the digital data of each tooth in the fused digital dataset, the digital data of each tooth in the fused digital dataset is processed according to a preset tooth position order. This application embodiment changes the prior art concept of processing tooth arrangement as a geometric image, processing different teeth as sequence data. Through a sequence neural network, each tooth is processed separately in a predetermined order to obtain the single-step tooth arrangement result of the entire row of teeth. Moreover, the feature data of each tooth is not mixed during this process, which can effectively improve the accuracy of feature learning. Based on the above characteristics, the inventors of this application selected a sequence neural network model, utilizing its long short-term memory characteristics. When predicting the features of a tooth, it not only refers to the features of adjacent teeth but also the features of other teeth that are far apart, allowing for faster learning of step-by-step characteristics. Throughout the entire processing, compared with existing image network models, the sequence neural network model can learn step-by-step characteristics better and faster, and the prediction results for step-by-step are more accurate.

[0061] In one embodiment, when the first dentition is a single-jaw dentition, the preset tooth position sequence includes: an ascending order of FDI tooth position numbers, such as 11, 12, 13, 14, ..., 26, 27, 28 for the maxillary dentition. In other embodiments, the preset tooth position sequence may include: the order from the left terminal tooth to the right terminal tooth in the FDI tooth position sequence; or, the order from the right terminal tooth to the left terminal tooth in the FDI tooth position sequence. For example, for the maxillary dentition, the sequence could be from the 8th molar in zone 1 to the 8th tooth in zone 2, that is, from 18, 17, 16, ..., 26, 27, 28. In other embodiments, the sequence may include descending order of FDI tooth position numbers. It is understood that other tooth position sequences can be set besides the above-mentioned methods, which will not be listed here. Because the arrangement of individual teeth in the current manual step-by-step tooth arrangement process has a certain order, the step-by-step tooth arrangement process is closer to current medical design methods, resulting in better predictive effects.

[0062] Taking the sequence neural network model used for automatic step-by-step segmentation as an example, and referring to Figure 2, the sequence neural network model used in this step for automatic step-by-step segmentation can be pre-trained in the following way (i.e., the model training steps include the following process):

[0063] Step 201: Construct fused digital data pairs of teeth in the current step and in the planned subsequent steps of an existing dental arch.

[0064] Step 202: Collect all the constructed data pairs to form a step-by-step training set.

[0065] In some embodiments, the limitations of the current step and subsequent steps can be set as needed, such as limiting them to two adjacent steps. In this case, all step-by-step target positions of a historical case can be split into pairs of adjacent steps. Correspondingly, the trained model can also predict the next adjacent correction step based on the previous correction step. Since adjacent steps are used as data pairs, more data pairs can be combined, and the pose difference between two adjacent steps is smaller, making it easier to accurately learn the pose difference principle during model training.

[0066] In some embodiments, the current step and the predetermined subsequent step are two adjacent steps, and the target position of the next stage in the step-by-step prediction step is the target position of the next correction step. This embodiment limits the construction of data pairs in the sample set to using the target positions of two adjacent correction steps, making the learned step-by-step features more accurate.

[0067] Step 203: Construct a neural network model.

[0068] In some embodiments, a sequence neural network model can be constructed as needed, specifically employing one or a combination of GRU (Gated Recurrent Unit) structures, LSTM (Long Short-Term Memory) structures, bidirectional GRU structures, bidirectional LSTM structures, multi-layer GRU structures, multi-layer LSTM structures, and Transformer structures. The sequence neural network model in some embodiments may further include adding an attention module to the constructed sequence neural network model. Adding an attention module to the constructed sequence neural network further improves the prediction accuracy of the network model. It is understood that different attention mechanisms can be added, such as the Luong attention mechanism and the Bahdanau attention mechanism. Furthermore, these models and attention mechanisms can be used to construct the network into an Encoder-Decoder structure, etc.

[0069] In some embodiments, the loss function used in the constructed sequence neural network model is one of the following: SmoothedL1 Loss, L2 Loss, Huber Loss, or Log-Cosh Loss. In some embodiments, the optimizer used in the constructed sequence neural network model is one of the following: Adam, AdamW, SGD (Stochastic Gradient Descent), AdaGrad, or RMSProp. The above loss function and optimizer can be defined simultaneously, and their combinations are diverse, and will not be listed here.

[0070] Taking the LSTM structure sequence neural network model in Figure 3 as an example, it specifically includes an encoding module and a decoding module, where the encoding module and the decoding module each include several recurrent units.

[0071] In the encoding stage, Xn represents the nth tooth-type feature parameter group. In one embodiment, tooth information can be directly used for encoding. In another embodiment, in addition to inputting each tooth-type feature parameter group, position information can also be encoded together. The position information can be the FDI number. Each tooth in the dental arch has a unique FDI number, so using the FDI number as the position information is simple and quick, so that the input of the model includes not only the spatial information of the teeth but also the position information of the teeth.

[0072] Continuing the explanation, each recurrent unit uses an LSTM structure, the internal structure of which can be shown in Figure 4. The input includes three parts: input information (simplified as "input" in the figure, denoted by X). t (represented by H in the diagram) Hidden state (represented by H in the diagram)t-1 (represented by) and memory cell states (simplified as "memory cells" in the diagram, denoted by C) t-1 (Represented). The hidden state and memory cell state are derived from the previous LSTM unit; for the first LSTM unit, they use randomly initialized values. An LSTM unit typically includes three gating units: the forget gate (denoted as F in the diagram). t (represented by I in the diagram), input gate / update gate (represented by I in the diagram) t (represented by O) and output gate (represented by O in the diagram) t (Represented by...) The forget gate controls which parts of the historical information are unimportant and can be forgotten, and which need to be retained. It uses the Sigmoid activation function, with a value between 0 and 1, to control the degree of forgetting. The input / update gate also uses the Sigmoid function to determine how much new input information to use, and the tanh function to calculate the candidate cell state of the current input. Then, through the forget gate and the input gate, the cell state input from the previous unit and the current candidate cell state are linearly combined to obtain the cell state at the current time step. The output gate determines which parts of the memory unit will be used to calculate the current hidden state and output it to the next time step. It is controlled by a Sigmoid function, which determines which parts of the memory unit will affect the final hidden state, and then scaled to the range [-1, 1] by the tanh function. That is, the tanh function can be used to ensure numerical stability. The LSTM unit also includes candidate memory cells. It is used to calculate candidate update values ​​for the current unit. Based on the input information and the hidden state of the previous unit's input, it uses a gating mechanism to determine which information should be updated and passed to the next unit. Candidate memory cells work in conjunction with memory cells used to store and pass long-term memory information, enabling LSTM units to efficiently process sequential data and better capture and remember information in long-term dependencies.

[0073] In the decoding stage, Yn represents the predicted feature parameter set of the nth tooth. The input of the decoding module can be the output of the previous recurrent unit and the hidden state. For the recurrent unit of the LSTM structure, the input of the recurrent unit can also include the cell state of the previous recurrent unit.

[0074] Step 204: Based on the step-by-step training set, train the constructed neural network model to obtain a neural network model for automatic step-by-step processing.

[0075] During training, the model sequentially learns the feature parameter sets of each tooth in the input dentition sample at its initial position through the recurrent module in the constructed sequence neural network model, following a preset tooth position order. It then outputs the feature parameter sets of each tooth at its predicted position according to the preset tooth position order. The model compares the feature parameter sets of each tooth at its predicted position with those at the target position in the current step to obtain the feature parameter difference value, which is the loss value between the model's predicted value and the labeled data. Using this loss value, the gradient of each parameter can be calculated. Then, using backpropagation and the chain rule, each parameter value of the model is updated along the direction from the model output to the model input. The model training is completed after processing a large amount of data (hundreds of thousands of cases) and multiple iterations. In some embodiments, mini-batch data can be used to update model parameters during training, which can speed up the model training process.

[0076] As can be seen from steps 201 to 204, by pre-constructing a sample set, a sequence neural network model is built, and then trained to obtain a sequence neural network model that can be used for automatic step-by-step processing, resulting in a sequence neural network model that better meets individual needs. The construction and training steps can be completed before implementing the automatic step-by-step processing method in this application, or they can be completed immediately when the automatic step-by-step processing method in this application is required; the execution order is not limited here.

[0077] In some embodiments, as shown in Figure 5, step 104 further includes a mesh model generation step, specifically the process of calculating the vertices of the mesh model based on the stage digital dataset of the predicted next-stage target position:

[0078] Step 501: Based on the digital data of each tooth in the stage digital dataset and the first digital dataset, the spatial coordinate transformation of each tooth is calculated using a point cloud registration algorithm.

[0079] Step 502: Based on the spatial coordinate transformation of each tooth, the mesh model of each tooth in the first dentition is transformed to obtain the stage mesh model of each tooth.

[0080] In some embodiments, based on the transformation of the spatial coordinates of the corresponding points of each tooth, the translation matrix and rotation transformation matrix of each tooth can be calculated using a point cloud registration method (such as the Kabsch method). After applying these translation and rotation transformation matrices to the input grid vertex coordinates of each tooth, the grid vertex coordinates of each tooth at the predicted target position can be obtained.

[0081] Taking the Kabsch algorithm as an example, the specific calculation method is explained as follows:

[0082] (1) From the feature points of the model input and output, we can construct two matrices P and Q, each with a dimension of 3×N. Each column of these two matrices represents the coordinates of the feature points, N is the number of feature points, and the feature points represented by each column of these two matrices are in one-to-one correspondence.

[0083] (2) Decenter these two matrices (p0 and q0 are the centers of P and Q respectively), and the resulting new matrices are still denoted as P and Q;

[0084] (3) Construct the covariance matrix H = P T Q;

[0085] (4) Perform singular value decomposition (SVD) on the covariance matrix H, i.e., H = USV T U and V are two orthogonal matrices, and S is a diagonal matrix with singular values ​​on the diagonal.

[0086] (5) Next, calculate d = sign(det(VUT)) to determine whether a rotation correction matrix is ​​needed to ensure the use of a right-handed coordinate system. Construct the following matrix based on d:

[0087] (6) The rotation matrix R is calculated by the following formula: R = VIU T R is a 3×3 matrix. The translation matrix r = p0 - R·q0. Applying these two matrices to the vertex coordinates of each tooth in its initial position yields the vertex coordinates of the tooth in the target position. That is:

[0088] X' = ​​RX + r;

[0089] Where X' is the coordinate of a vertex of a tooth in the target position, and X is the coordinate of a vertex of the corresponding initial position tooth.

[0090] As can be seen, steps 501 to 502 further define the calculation of spatial coordinate transformation through point cloud registration and then the reconstruction of the tooth mesh model. Considering that the shape of the teeth themselves is almost unchanged, reconstructing the dentition morphology through registration can increase the error tolerance of the predicted feature parameters. It is understood that in addition to using the Kabsch algorithm for registration and transformation as described above, other registration algorithms can also be used, which will not be listed here.

[0091] Step 105, data update step, update the first digital dataset based on the stage digital dataset.

[0092] In some embodiments, this step updates the first digital dataset obtained in the preceding steps based on the predicted stage digital dataset, and performs iterative updates to the current pose.

[0093] Step 106: Determine whether the preset prediction stop condition has been met; if not, repeat steps 103-105; if it has been met, proceed to step 107.

[0094] In some embodiments, the preset prediction stopping condition may include one of the following: the number of orthodontic steps reaches a preset number, or the difference between the stage digital dataset and the second digital dataset is less than a predetermined range. This predetermined range can be set by the technician as needed, or a set of standard ranges can be preset, such as the movement of each tooth being less than a fixed value (e.g., 0.25mm, 0.1mm, 0.05mm, 0.04mm, 0.03mm, etc.), and the axial tilt or torsion of each tooth being less than a fixed value (e.g., 2.5 degrees, 1 degree, 0.5 degrees, 0.4 degrees, 0.3 degrees, etc.). It is worth noting that this difference can be the difference of a single feature point or the average difference of all feature points of a single tooth, and can be specifically set according to actual needs. Since the automatic prediction process may predict a pose very close to the ideal pose, but not exactly the same value, setting the difference between the predicted stage target pose and the ideal pose to be sufficiently small allows the cyclic prediction process to be paused, and the current predicted target pose can be used as the final design target pose, or the current model prediction can be ignored and directly used as the final target pose as the current prediction result.

[0095] In some embodiments, due to unknown reasons related to the input data, the automatic step-by-step process of cyclic prediction may fail to reach the allowable range of the ideal pose for an extended period, potentially leading to an infinite loop. Therefore, by limiting the prediction to a preset number of steps (e.g., 80, 100, 120, 200 steps, etc.) and stopping the prediction after reaching this preset number, manual verification can be used to confirm whether the target position is sufficiently close, or whether the excessive number of steps is due to errors in the input data. It is evident that including a preset number of steps in the prediction stopping condition allows the automatic step-by-step method of this application to be implemented more accurately.

[0096] Based on the combination of steps 101 to 106 above, the data fusion step, step-by-step prediction step, and data update step can be repeated until a preset prediction stopping condition is reached, including a data correction step, wherein the stage digital dataset generated by the step-by-step prediction step is verified according to preset medical rules, and the stage digital dataset is corrected when the verification fails; wherein, the data correction step is executed once every certain number of step-by-step prediction steps. In this embodiment of the application, a data correction step is added to the cyclic prediction process. Since some of the step-by-step process is based on the constraints of medical rules, the stage correction can avoid discrepancies between the actual step-by-step effect and the actual step-by-step effect caused by the accumulation of errors, thereby improving the accuracy and feasibility of the final output treatment plan.

[0097] In some embodiments, the preset medical rules include one or any combination of the following: the mode of tooth movement, the amount of tooth movement, and the occlusal relationship with opposing teeth. For example, anchor teeth may not move or may move only a very small range; no more than six teeth in a single jaw may move in the same direction (all towards the mesial or distal direction); and some teeth may even be required to remain immobile at certain stages, etc. These medical rules can be preset by professional designers based on experience, and will not be listed one by one here.

[0098] In some embodiments, the data correction step further includes a collision detection step, wherein collision detection is performed on the stage pose of each tooth, and the stage pose of each tooth is corrected based on the detection results. This embodiment further specifies the addition of collision detection in the data correction step, which can also promptly adjust teeth that have collided, reduce the accumulation of collision errors, and improve the accuracy of the final output treatment plan.

[0099] Step 107: Output the step-by-step plan, gather all the predicted stage data sets and output them as the treatment plan for the first dentition.

[0100] In some embodiments, the staged digital dataset can be displayed in the form of a grid model, coordinate values, or movement measurements during output. From another perspective, it can be displayed as text, images, two-dimensional images, or three-dimensional images, and these will not be listed here. This application's embodiments limit the output of the step-by-step scheme to multiple formats, making the output format flexible and varied.

[0101] In some embodiments, before the step-by-step output step in step 107, a post-processing procedure is further included for the stage target positions output by the model. This may include: performing collision detection on the stage target positions of each tooth and updating the target positions of each tooth based on the detection results. Since the target positions predicted by the neural network model may result in collisions between teeth, it is necessary to adjust the relative positions of each pair of adjacent teeth in the same jaw if a collision occurs.

[0102] In one embodiment, the method for fine-tuning between teeth in a single jaw is as follows: First, determine whether there is a collision between the two central incisors. If there is a collision, calculate their collision depth d. Then, move both teeth d / 2 distally. Next, starting with the incisors, perform collision detection on the incisors and lateral incisors. If a collision is found, move the lateral incisors distally, and so on. For the canines, premolars, and molars, fine-tune them in the same way. In addition to collision detection on teeth in a single jaw, collision detection can also be performed on corresponding teeth in occlusion by combining data from opposing teeth, thus avoiding excessive collisions on individual teeth during occlusion.

[0103] It is understood that although collision detection is performed before the step-by-step solution output in the embodiments of this application, in practical applications, collision detection can also be performed during the intermediate prediction process. For example, collision detection can be performed on the stage target position obtained in each prediction, or on the stage target position obtained after several predictions. In the process of cyclic prediction, adding collision detection in the intermediate process is beneficial to reduce the intermediate error of the automatic step-by-step method in this application, adjust the teeth that have collided in time, reduce the accumulation of collision error, and improve the accuracy of the final output treatment plan.

[0104] Subsequently, the inventors of this application conducted an accuracy test using the automatic step-by-step method based on neural networks described in this application. The target tooth positions obtained using the automatic step-by-step method based on neural networks described in this application were compared with those obtained through manual step-by-step analysis. As shown in Figure 6, a test example is presented, where manual step-by-step analysis is performed using the initial position of the dentition (light-colored display), and step-by-step analysis is performed using the automatic step-by-step method described in this embodiment (dark-colored display). Steps 10, 20, 30, 40, 50, and 60, along with the final position, are extracted and the dentition images are superimposed. It can be seen that the difference between the target tooth positions obtained using the automatic step-by-step method based on neural networks described in this application and those obtained through manual step-by-step analysis is small, indicating that the automatic step-by-step method based on neural networks described in this application has better practical effects and should be widely promoted.

[0105] It is understood that although the neural network model in the embodiments of this application is limited to a sequence neural network model, other neural network models can also be used in practical applications, and neural network models with different architectures can be constructed and trained accordingly. These will not be listed one by one here.

[0106] As can be seen, this application embodiment utilizes a neural network model to predict the orthodontic path of the dental arch. During prediction, it integrates the current and target pose information of the teeth, ensuring that the predicted step-by-step results gradually approach the target pose. This avoids unclear tooth movement directions in the predicted results, preventing repeated movement paths, low efficiency, and poor results throughout the orthodontic process. Specifically, it continuously obtains the step-by-step results of the next stage through iterative prediction, with the obtained order being the step-by-step sequence, avoiding confusion. Simultaneously, a prediction stop condition is set to avoid infinite loops caused by the inability to reach the ideal pose during iterative prediction. Furthermore, during the automatic prediction of the target position, there is no need to specify the required number of treatment steps; prediction can be based on the actual situation to obtain a more reasonable treatment path and a more reasonable treatment plan. Therefore, the series of dental arch layouts obtained using the neural network-based automatic step-by-step method in this application not only achieves automatic and rapid results, but also reduces the influence of human experience and improves the accuracy of the output treatment plan. In addition, the neural network model in this application embodiment uses a sequential neural network model, processing different teeth as sequential data. Through the sequential neural network, each tooth is processed separately in a predetermined order to obtain the tooth arrangement result of the entire row of teeth in a certain treatment step. Furthermore, the feature data of each tooth remains separate during this process, effectively improving the accuracy of feature learning. Based on these characteristics, the inventors of this application selected a sequence neural network model, utilizing its long short-term memory feature. When predicting the features of a tooth, it references not only the features of adjacent teeth but also those of other teeth located far apart, enabling faster learning of step-by-step characteristics. Throughout the entire processing, compared to existing image network models, the sequence neural network model learns the automatic step-by-step characteristics better and faster, and the prediction results for each step are more accurate.

[0107] It is understood that the above embodiments use the same neural network model for automatic step-by-step segmentation of all cases. In practical applications, different neural network models can be trained according to different case types. Taking three case types as an example, these three case types include cases of missing teeth, cases of arch expansion, and cases of distalized molars. There are three corresponding neural network models for automatic step-by-step segmentation, trained from training sets corresponding to the above three case types. When the case type corresponding to the first dentition is a case of distalized molars, the step-by-step prediction step includes: selecting the neural network model corresponding to the case of distalized molars for step-by-step prediction based on the case type information. That is, there are case types corresponding to the first dentition, and multiple neural network models for automatic step-by-step segmentation are used, trained from training sets corresponding to their respective case types. The step-by-step prediction step includes: selecting the neural network model corresponding to the first dentition for step-by-step prediction based on the case type information. In this embodiment, different neural network models are trained for different case types. Since the tooth arrangement principles for different case types vary greatly, training dedicated neural network models for different case types can be more accurate in step-by-step prediction. It is understood that, in addition to setting three case types in the embodiments of this application, two case types, five case types, etc. can also be set in actual applications. The different numbers do not limit the core improvement of this application.

[0108] Other embodiments of this application provide an automatic step-by-step method based on a neural network. The main difference between the current embodiment and the above embodiment is that the tooth arrangement processed in the above embodiment is a single jaw tooth arrangement, while the tooth arrangement processed in the current embodiment includes the maxilla and mandible.

[0109] In some embodiments, when obtaining the first dentition, the dentition including the maxilla and mandible of the same patient is obtained. It is understood that the maxilla and mandible can be the maxillary and mandibular tooth arrangement after occlusal registration.

[0110] Correspondingly, in addition to adopting the tooth arrangement order in a single jaw as described in the above embodiment, the preset tooth position order can be arranged in the order of arranging the teeth of the maxilla first and then the teeth of the mandibular dentition, or in the order of arranging the teeth of the mandibular dentition first and then the teeth of the maxilla.

[0111] The process of predicting the target tooth location using a sequence neural network model for automatic step-by-step analysis is similar to the above embodiment and will not be described again here.

[0112] In some embodiments, after obtaining the target dataset for the stage, collision detection and adjustment can be performed on the teeth between the upper and lower jaws. During adjustment, fine-tuning can be performed first on the teeth between individual jaws, and then on the teeth between the upper and lower jaws. In one embodiment, collision is first checked on all teeth between the upper and lower jaws. If there is a collision depth d, the teeth colliding in the upper and lower jaws are moved a distance d / 2 in an upward or downward direction, respectively. If there is a gap, the teeth with larger gaps in the upper and lower jaws are moved downward and upward, respectively, to reduce the gaps between the teeth.

[0113] In summary, when the first dentition obtained includes the layout of the maxillary teeth and the layout of the mandibular teeth, the automatic step-by-step method based on neural networks in this application is also applicable, and the automatic step-by-step effect is also good.

[0114] Other embodiments of this application provide an automatic step-by-step method based on neural networks. The current embodiment is a further improvement on the above embodiments. The main improvement is that the digital dataset input to the neural network model in the above embodiments only includes dental feature parameters, while in the current embodiment, in addition to dental feature parameters, medical plan parameters are also combined. By combining these medical plans for learning step-by-step, the speed at which the neural network model learns the step-by-step features can be further accelerated, and the accuracy of the predicted stage target position can be improved.

[0115] The current embodiment employs a sequence neural network model architecture as shown in Figure 7, which specifically includes an encoding module, a feature concatenation module, and a decoding module. In some embodiments, the features output by the last loop unit in the encoding module can also be combined with features associated with the medical protocol as input features for the decoder. In some embodiments, the medical protocol parameter group includes one or any combination of the following: arch expansion, gap closure, enamel reduction, tooth extraction, and ideal arch curve morphology.

[0116] As shown in Figure 8, when processing the dental feature parameters using the sequence neural network model in the current embodiment, the specific steps include:

[0117] Step 801: Using the encoding module in the trained sequence neural network model for automatic step-by-step processing, the tooth-type feature parameter groups of each tooth are encoded according to the preset tooth position order. This step is similar to the previous embodiment and will not be repeated here.

[0118] Step 802: Using the decoding module in the trained sequence neural network model for automatic step-by-step processing, the obtained encoding result and the pre-acquired medical plan parameter group are decoded to obtain the target position of each tooth.

[0119] In some embodiments, the encoding result obtained in step 801 is concatenated with the pre-acquired medical plan parameter set, and the concatenated result is decoded. In some embodiments, the pre-acquired medical plan parameter set can be encoded by a preset text encoder before concatenation. When applying medical plan feature parameters, the method of first encoding with a preset text encoder and then concatenating with the original parameters is more convenient to cover the rich expression methods in existing medical plans, making the medical plans more learnable. When combining medical plan parameters, parameter concatenation can be used to improve the combined application of multiple types of data. Since combining medical plan parameters actually integrates multiple forms of data in the overall dental alignment, a multimodal solution is realized. It can be understood that in the feature concatenation part, the data output by the text encoder can be processed by a fully connected layer before concatenation. Since the numerical range of the encoded values ​​output by the text encoder and the numerical range of the encoded values ​​output by the encoding module may differ greatly, a fully connected layer can make the value ranges of the two closer, eliminating the interference of large values ​​on the results when the numerical differences are too large.

[0120] As can be seen, in the embodiments of this application, when using a neural network model for automatic step-by-step segmentation, in addition to using the characteristic parameters of the teeth themselves in the dentition, medical plan parameters can also be combined. Since doctors or patients may specify some medical plans during the automatic step-by-step segmentation process, combining these medical plans for learning automatic step-by-step segmentation can further accelerate the speed at which the neural network model learns the automatic step-by-step features and improve the accuracy of the predicted step-by-step target location.

[0121] It is worth mentioning that the examples above in this application are merely illustrative for ease of understanding and do not constitute a limitation on the technical solutions of this application.

[0122] Other embodiments of this application provide an automatic step-by-step method based on neural networks. The current embodiment is largely the same as the aforementioned implementation method, with the main difference being that the same neural network model was used in the step-by-step prediction process for a case in the aforementioned embodiments, while more than one neural network model is used in the step-by-step prediction process for a case in the current embodiment.

[0123] In some embodiments, M step-by-step stages can be preset, and K neural network models can be used for automatic step-by-step division, with at least two step-by-step stages corresponding to different neural network models; where M and K are natural numbers greater than 1. In this embodiment, the overall step-by-step stages of a dental arch are further divided into multiple segments, and different neural network models are assigned for prediction. Since the step-by-step methods of different stages may be different, constructing neural network models for prediction separately is beneficial to improving the accuracy of the step-by-step results of each stage.

[0124] In some embodiments, the M step-by-step stages are divided by the number of steps, and the neural network models corresponding to different numbers of steps are trained using sample data from historical cases for the corresponding steps. In a specific embodiment, the stage is divided into two steps: the first stage is from the initial position to step 10, and the second stage is from step 11 to the final target. Two neural network models are trained, namely a first neural network model and a second neural network model. When predicting steps 1-10, the first neural network model is used for prediction. When predicting step 11 until the cutoff condition is reached, the second neural network model is used for prediction. In the step-by-step output step, the prediction results of the two stages are combined and output.

[0125] In some embodiments, the K neural network models used for automatic step-by-step processing have the same model architecture. Specifically, if all models use a sequence neural network model, this application embodiment limits the different neural network models to using the same model architecture, only using different sample data for training, thus simplifying model selection.

[0126] As can be seen, in this embodiment, the steps are divided into stages based on the number of steps. Since the principles of tooth arrangement differ depending on the timing of the tooth arrangement process, such as the early stage, the middle stage, and the late stage, dividing the process based on the number of steps makes it easier to select different historical data to form different sample data during model training, thereby accurately training different neural network models.

[0127] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0128] Other embodiments of this application relate to an electronic device, as shown in FIG9, including: at least one processor 901; and a memory 902 communicatively connected to at least one processor 901; wherein the memory 902 stores instructions executable by at least one processor 901, the instructions being executed by at least one processor 901 to enable at least one processor 901 to perform the neural network-based automatic step-by-step method in some of the above embodiments.

[0129] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0130] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0131] Other embodiments of this application relate to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the neural network-based automatic step-by-step method described in some of the above embodiments.

[0132] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0133] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. An automatic step-by-step method based on neural networks, comprising: The first acquisition step is to acquire a first digital dataset representing the current pose of each tooth in the first dental row. The second acquisition step is to acquire a second digital dataset representing the ideal pose of each tooth in the first dental arch. The data fusion step involves fusing the digital data of the corresponding teeth in the first digital dataset and the second digital dataset separately for each individual tooth to obtain a fused digital dataset. The step-by-step prediction step utilizes a trained neural network model for automatic step-by-step processing to process the digital data of each tooth in the fused digital dataset, generating a stage digital dataset representing the target position of the first dentition in the next stage. The data update step involves updating the first digital dataset based on the stage digital dataset. Repeat the data fusion step, the step-by-step prediction step, and the data update step until the preset prediction stopping condition is met; and, The step-by-step solution outputs steps, gathers all the predicted stage digital datasets and outputs them as the treatment plan for the first dentition.

2. The automatic step-by-step method based on neural networks according to claim 1, wherein, The neural network model used for automatic step-by-step division belongs to the sequence neural network model, and, The processing of digital data for each tooth in the fused digital dataset includes: The digital data of each tooth in the fused digital dataset is processed according to a preset tooth position order.

3. The automatic step-by-step method based on neural networks according to claim 2, wherein, In the first acquisition step, the first dentition acquired is a single-jaw dentition: The preset tooth position order includes: an order in ascending order of FDI tooth position number; or an order in descending order of FDI tooth position number; or an order from the left terminal tooth to the right terminal tooth of the FDI tooth position; or an order from the right terminal tooth to the left terminal tooth of the FDI tooth position.

4. The automatic step-by-step method based on neural networks according to claim 2, wherein, In the first acquisition step, the first dentition acquired includes the maxillary and mandibular dentitions of the same patient; The preset tooth position sequence includes: arranging the teeth of the maxillary dentition first and then arranging the teeth of the mandibular dentition, or arranging the teeth of the mandibular dentition first and then arranging the teeth of the maxillary dentition.

5. The automatic step-by-step method based on neural networks according to any one of claims 1-4, wherein, The data fusion step includes: performing equal-weighted calculations on the digital data of the same tooth in the first digital dataset and the second digital dataset.

6. The automatic step-by-step method based on neural networks according to claim 5, wherein, The data fusion step includes: performing addition, subtraction, multiplication, averaging, and / or splicing operations on the digital data of the same tooth in the first digital dataset and the second digital dataset.

7. The automatic step-by-step method based on neural networks according to any one of claims 1-6, wherein, There are M preset step-by-step stages, and K neural network models for automatic step-by-step operation, with at least two step-by-step stages corresponding to different neural network models for automatic step-by-step operation; where M and K are natural numbers greater than 1.

8. The automatic step-by-step method based on neural networks according to claim 7, wherein, The M step-by-step stages are divided by the number of steps, and the neural network models for automatic step-by-step division corresponding to different numbers of steps are trained using sample data of the corresponding steps in historical cases.

9. The automatic step-by-step method based on a neural network according to claim 7 or 8, wherein, The K neural network models described for automatic step-by-step processing have the same model architecture.

10. The automatic step-by-step method based on neural networks according to any one of claims 1-9, wherein, The first dental row corresponds to a case type, and there are multiple neural network models for automatic step-by-step division, which are trained by training sets corresponding to their respective case types; The step-by-step prediction step includes: selecting the neural network model for automatic step-by-step prediction corresponding to the first dental arch based on the case type information.

11. The automatic step-by-step method based on neural networks according to any one of claims 1-10, wherein, In the step-by-step prediction step, the next stage is the next correction step or the next N correction steps, where N is a natural number greater than 1.

12. The automatic step-by-step method based on neural networks according to any one of claims 1-11, wherein, The process of repeating the data fusion step, the step-by-step prediction step, and the data update step until a preset prediction stop condition is met includes a data correction step, wherein the stage digital dataset generated by the step-by-step prediction step is verified according to preset medical rules, and the stage digital dataset is corrected when the verification fails. Specifically, after each set of step-by-step prediction steps is executed several times, a data correction step is executed once.

13. The automatic step-by-step method based on neural networks according to claim 12, wherein, The preset medical rules include one or any combination of the following: the mode of tooth movement, the amount of tooth movement, and the occlusal relationship with the opposing teeth.

14. The automatic step-by-step method based on a neural network according to claim 12 or 13, wherein, The data correction step also includes a collision detection step, wherein collision detection is performed on the stage pose of each tooth, and the stage pose of each tooth is corrected based on the detection results.

15. The automatic step-by-step method based on neural networks according to any one of claims 1-14, wherein, The preset prediction stopping conditions include one of the following: the number of correction steps reaches a preset number of steps, or the difference between the stage digital dataset and the second digital dataset is less than a predetermined range.

16. The automatic step-by-step method based on neural networks according to any one of claims 1-15, wherein, Prior to the step-by-step prediction step, a model training step is included, wherein, Construct fused digital data pairs of each tooth in an existing dentition for the current step and the planned subsequent steps; All constructed data pairs are combined to form a step-by-step training set; Build a neural network model; Based on the step-by-step training set, the constructed neural network model is trained to obtain the neural network model used for automatic step-by-step processing.

17. The automatic step-by-step method based on neural networks according to claim 16, wherein, The current step and the predetermined subsequent step are two adjacent steps, and the next stage target position in the step-by-step prediction step is the target position of the next correction step.

18. The automatic step-by-step method based on a neural network according to claim 16 or 17, wherein, The constructed neural network model is a sequence neural network model.

19. The automatic step-by-step method based on neural networks according to claim 18, wherein, The sequence neural network model includes one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure.

20. The automatic step-by-step method based on neural networks according to any one of claims 16-19, wherein, The construction of the neural network model includes adding an attention module to the constructed neural network model.

21. The automatic step-by-step method based on neural networks according to any one of claims 16-20, wherein, The construction of the neural network model includes: using one of the following as the loss function: SmoothedL1 Loss, L2 Loss, Huber Loss, Log-Cosh Loss; and / or, Use one of the following as the optimizer: Adam, AdamW, SGD, AdaGrad, RMSProp.

22. The automatic step-by-step method based on neural networks according to any one of claims 1-21, wherein, The first digital dataset and the second digital dataset include tooth-type parameters for each tooth; or, they include tooth-type parameters for each tooth and medical treatment parameters.

23. The automatic step-by-step method based on neural networks according to claim 22, wherein, The dental parameters include one or any combination of the following: The geometric center of the tooth, and the six intersection points formed by rays emanating from the geometric center along the six directions of the three axes in the tooth coordinate system and the tooth mesh; The eight vertices of the spatial bounding box of the tooth and the geometric center of the tooth; Several characteristic points on the tooth facing the opposing jaw and the geometric center of the tooth.

24. The automatic step-by-step method based on neural networks according to claim 23, wherein, The feature point is determined according to the tooth type. When the tooth is a molar, the feature point is the buccal apex; or, when the tooth is a premolar, the feature point is the buccal apex or buccal groove; or, when the tooth is an incisor or canine, the feature point is the superior incisal edge.

25. The automatic step-by-step method based on neural networks according to any one of claims 22-24, wherein, The medical protocol parameter group includes one or any combination of the following: arch expansion, gap closure, enamel reduction, molar reduction, and ideal arch curve morphology.

26. The automatic step-by-step method based on neural networks according to any one of claims 1-25, wherein, The step-by-step scheme outputs a stage-specific digital dataset in the form of a grid model, coordinate values, and movement amounts.

27. The automatic step-by-step method based on neural networks according to claim 24, wherein, If the output stage digital dataset is in the form of a grid model, then the step-by-step output steps of the scheme include: a grid model generation step, wherein, Based on the stage digital dataset and the digital data of each tooth in the first digital dataset, the spatial coordinate transformation of each tooth is calculated using a point cloud registration algorithm. Based on the spatial coordinate transformation of each tooth, the mesh model of each tooth in the first dentition is transformed to obtain the stage mesh model of each tooth.

28. An electronic device comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the neural network-based automatic step-by-step method as described in any one of claims 1-27.

Images