Automatic tooth arrangement method based on neural network and electronic equipment

Abstract

The invention relates to the technical field of orthodontic digital design, and discloses an automatic tooth arrangement method based on a neural network and electronic equipment, the automatic tooth arrangement method based on the neural network comprises the following steps: obtaining initial tooth arrangement; extracting a tooth type characteristic parameter group of each tooth from the initial tooth arrangement; and sequentially processing the tooth type characteristic parameter group of each tooth according to a preset tooth position sequence by using the trained sequence neural network model for automatic tooth arrangement to obtain a target position of each tooth. A sequence neural network model is selected, the characteristics of long and short-term memory of the sequence neural network model are utilized, and when the characteristics of a certain tooth are predicted, the characteristics of adjacent teeth and the characteristics of other teeth which are far away from each other are referred to, so that the tooth arrangement characteristics can be learned more quickly. In the whole processing process, compared with an existing image network model, the tooth arrangement characteristics can be better and faster learned by adopting the sequence neural network model, and the tooth arrangement prediction result is more accurate.

Description

Technical Field

[0001] This invention relates to the field of orthodontic digital design technology, and in particular to an automatic tooth alignment method and electronic device based on neural networks. Background Technology

[0002] Shell-shaped orthodontic appliances are a type of orthodontic device made of safe, elastic, transparent polymer material. They have the advantages of being completely invisible during the orthodontic process, aesthetically pleasing, easy to operate, and convenient for oral cleaning. Moreover, due to their transparent and aesthetically pleasing characteristics, the orthodontic process is completed almost imperceptibly to others, and they have gradually become the first choice for orthodontic patients.

[0003] With the rapid development of computer technology, dental treatment is increasingly relying on computer technology. For example, in orthodontic treatment using shell-shaped orthodontic appliances, a three-dimensional digital model representing the dentition in the target dentition layout is typically generated based on a three-dimensional digital model representing the dentition in the initial dentition layout. Then, based on both, three-dimensional digital models representing the dentition in several successive intermediate dentition layouts are generated. Finally, a series of successive shell-shaped orthodontic appliances are fabricated based on these three-dimensional digital models.

[0004] Current technology primarily generates the target tooth layout by manually moving each tooth in a 3D digital model representing the initial tooth arrangement. On one hand, this method is inefficient and costly; on the other hand, the results are highly dependent on the operator's skill, leading to inconsistent tooth placement.

[0005] Among current technologies, considering that the three-dimensional digital model of the dental arch is actually a geometric image, some methods are based on computer image algorithms to perform arrangement calculations on the model. However, tooth layout adjustment involves many medical parameters and is not simply an image processing problem. Therefore, when current image algorithms are applied to tooth arrangement, due to the lack of medical parameter limitations, the results differ greatly from the actual tooth arrangement and have low accuracy. Summary of the Invention

[0006] The purpose of this invention is to provide an automatic tooth alignment method and electronic device based on neural networks, which can solve the problem of low accuracy when tooth layout adjustment relies on manual or automatic adjustment.

[0007] To address the aforementioned technical problems, embodiments of the present invention provide an automatic tooth alignment method based on a neural network, comprising: obtaining an initial tooth arrangement; extracting a set of tooth-like feature parameters for each tooth from the initial tooth arrangement; and using a trained Sequence Neural Network model for automatic tooth alignment to process the set of tooth-like feature parameters of each tooth sequentially according to a preset tooth position order to obtain the target position of each tooth.

[0008] Embodiments of the present invention also provide an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described neural network-based automatic tooth alignment method.

[0009] Embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described neural network-based automatic tooth alignment method.

[0010] Compared to existing technologies, this invention changes the conventional approach of treating tooth arrangement as a geometric image. Instead, it processes different teeth as sequential data, using a sequence neural network to process each tooth in a predetermined order to obtain the complete tooth arrangement. Furthermore, the feature data of each tooth remains separate during this process, effectively improving the accuracy of feature learning. Based on these characteristics, the inventors of this application selected a sequence neural network model, utilizing its long short-term memory (LSTM) properties. When predicting the features of a tooth, it references not only the features of adjacent teeth but also those of other teeth located further apart, enabling faster learning of tooth arrangement characteristics. Throughout the entire processing, compared to existing image network models, the sequence neural network model learns tooth arrangement characteristics better and faster, resulting in more accurate predictions.

[0011] Optionally, in obtaining the initial tooth arrangement, the obtained initial tooth arrangement is a single-jaw initial tooth arrangement: the preset tooth position sequence includes: an order of FDI tooth position number from smallest to largest; or an order of FDI tooth position number from largest to smallest; or an order of FDI tooth position number from left terminal tooth to right terminal tooth; or an order of FDI tooth position from right terminal tooth to left terminal tooth. Because the arrangement of individual teeth in current artificial tooth arrangement processes has a certain sequence, the tooth arrangement process is closer to current medical design methods, resulting in better predictive effects.

[0012] Optionally, in obtaining the initial tooth arrangement, the initial tooth arrangement includes the initial tooth arrangement of the maxilla and mandible of the same patient; the preset tooth position order includes: arranging the teeth of the maxilla first and then arranging the teeth of the mandible, or arranging the teeth of the mandible first and then arranging the teeth of the maxilla.

[0013] Optionally, the tooth feature parameter set is a combination of multiple parameters representing tooth pose. The collected tooth feature data uses pose-representing data. Since the shape of the teeth themselves remains almost unchanged during the tooth arrangement process, the neural network model only predicts the pose changes of the teeth. Therefore, calculating only the pose changes of the teeth can effectively reduce the number of feature parameters used, reduce the amount of data computation in the prediction, and speed up the model training and prediction.

[0014] Optionally, the tooth-type feature parameters may include at least one of the following parameter groups: the geometric center point of the tooth, and the six intersection points formed by rays emanating from the geometric center point along the six directions of the three axes in the tooth coordinate system and the tooth mesh; or, the eight vertices of the spatial bounding box of the tooth and the geometric center point of the tooth; or, several feature points of the tooth facing the opposing jaw and the geometric center point of the tooth. Since the adjustment of the tooth layout is mainly achieved by translation and rotation, the pose of a single tooth can be determined by combining the geometric center with contour-related features, minimizing the amount of data representing the tooth pose parameters, reducing the amount of parameter data involved in the calculation, and speeding up the model processing speed.

[0015] Optionally, the feature point of the tooth facing the opposing jaw is determined according to the tooth type, wherein when the tooth is a molar, the feature point is the buccal cusp; or, when the tooth is a premolar, the feature point is the buccal cusp or buccal groove; or, when the tooth is an incisor or canine, the feature point is the superior incisal edge.

[0016] Optionally, the step of using a trained sequence neural network model for automatic tooth alignment to process the tooth-like feature parameter sets of each tooth sequentially according to a preset tooth position order includes: using the encoding module in the trained sequence neural network model for automatic tooth alignment to encode the tooth-like feature parameter sets of each tooth according to the preset tooth position order; and using the decoding module in the trained sequence neural network model for automatic tooth alignment to decode the obtained encoding results and the pre-acquired medical plan parameter sets to obtain the target position of each tooth. Further, when using the sequence neural network model for tooth alignment, in addition to using the feature parameters of the teeth themselves in the dentition, medical plan parameters can also be combined. Since doctors or patients may specify some medical plans during the tooth alignment process, combining these medical plans for learning tooth alignment can further accelerate the speed at which the sequence neural network model learns tooth alignment features and improve the accuracy of the predicted tooth alignment target position.

[0017] Optionally, the decoding process of the obtained encoded result and the pre-acquired medical plan parameter group includes: concatenating the obtained encoded result and the pre-acquired medical plan parameter group, and decoding the concatenated result. When combining medical plan parameters, parameter concatenation can be used to improve the combined application of multiple data types. Furthermore, since combining medical plan parameters actually integrates multiple forms of data in the overall dental alignment, a multimodal solution is achieved.

[0018] Optionally, the step of concatenating the obtained encoding result with the pre-acquired medical plan parameter group includes: encoding the pre-acquired medical plan parameter group using a preset text encoder before concatenation. When applying medical plan feature parameters, the method of first encoding with a preset text encoder and then concatenating with the original parameters is more convenient for covering the rich expressions in existing medical plans, making the medical plans more learnable.

[0019] Optionally, the medical protocol parameter group includes one or any combination of the following: arch expansion, gap closure, enamel removal, tooth extraction, and ideal arch curve morphology.

[0020] Optionally, the step of using a trained sequence neural network model for automatic tooth alignment to process the tooth-like feature parameter groups of each tooth sequentially according to a preset tooth position order to obtain the target position of each tooth includes: using the trained sequence neural network model for automatic tooth alignment to process the initial values ​​of the tooth-like feature parameter groups of each tooth sequentially according to a preset tooth position order to obtain the predicted target values ​​of the tooth-like feature parameter groups; based on the predicted target values ​​and the initial values ​​of the tooth-like feature parameter groups of each tooth, using a point cloud registration algorithm to calculate the spatial coordinate transformation of each tooth; and according to the spatial coordinate transformation of each tooth, performing a spatial coordinate transformation on the mesh model of each tooth in the initial tooth arrangement to obtain the mesh model of the target position of each tooth. Further limiting the calculation of spatial coordinate transformation and then restoring the tooth mesh model through point cloud registration, considering that the shape of the teeth themselves is almost unchanged, restoring the dentition shape through registration can increase the error tolerance of the predicted feature parameter values.

[0021] Optionally, before calculating the sorted tooth feature parameter sets using the trained sequence neural network model for automatic tooth alignment, the process includes: combining the tooth feature parameter sets of each tooth in the initial and target positions of an existing tooth row; arranging the tooth feature parameter sets of each tooth according to the preset tooth position order to obtain a tooth row sample; collecting multiple tooth row samples to construct a tooth alignment training set; constructing a sequence neural network model; and training the constructed sequence neural network model based on the tooth alignment training set to obtain the sequence neural network model for automatic tooth alignment. Further, by pre-constructing a sample set, constructing a sequence neural network model, and then training it to obtain a sequence neural network model suitable for automatic tooth alignment, a sequence neural network model that better meets individual needs can be obtained.

[0022] Optionally, the sequence neural network model includes one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure.

[0023] Optionally, constructing the sequence neural network model includes adding an attention module to the constructed sequence neural network model. Further, adding an attention module to the constructed sequence neural network improves the prediction accuracy of the network model.

[0024] Optionally, the construction of the sequence neural network model includes: using one of the following loss functions: SmoothedL1Loss, L2 Loss, Huber Loss, Log-Cosh Loss; and / or using one of the following optimizers: Adam, AdamW, SGD, AdaGrad, RMSProp.

[0025] Optionally, after obtaining the target position of each tooth, the process includes: performing collision detection on the target position of each tooth, and updating the target position of each tooth based on the detection results. Attached Figure Description

[0026] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0027] Figure 1 This is a flowchart of an automatic tooth alignment method based on a neural network provided in one embodiment of this application;

[0028] Figure 2 This is a schematic diagram of the sequence neural network model training process in the automatic tooth alignment method based on neural networks provided in one embodiment of this application;

[0029] Figure 3 This is a schematic diagram of a sequence neural network model with an LSTM structure used in the automatic tooth alignment method based on neural networks provided in one embodiment of this application;

[0030] Figure 4 This is a schematic diagram of a single LSTM unit in a sequence neural network model with an LSTM structure used in an automatic tooth alignment method based on a neural network provided in one embodiment of this application;

[0031] Figure 5 This is a schematic diagram illustrating the process of calculating mesh vertices using model output results in an automatic tooth alignment method based on neural networks provided in one embodiment of this application;

[0032] Figure 6 This is a schematic diagram of a tooth alignment test example in an automatic tooth alignment method based on neural networks provided in one embodiment of this application;

[0033] Figure 7 This is a schematic diagram of a sequence neural network model with an LSTM structure used in an automatic tooth alignment method based on a neural network provided in another embodiment of this application;

[0034] Figure 8 This is a schematic diagram illustrating the steps of using a sequence neural network model to process tooth-related feature parameters and medical plan-related parameters in the automatic tooth alignment method based on neural networks provided in one embodiment of this application.

[0035] Figure 9 This is a schematic diagram of an electronic device provided in another embodiment of this application;

[0036] Figure 10 This is a schematic diagram of an automatic tooth alignment system based on a neural network, provided in another embodiment of this application. Detailed Implementation

[0037] To make the objectives, technical solutions, and advantages of this invention clearer, the various embodiments of this invention will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been provided in the various embodiments of this invention to facilitate a better understanding of this application. However, the technical solutions claimed in the claims of this application can be implemented even without these technical details and with various variations and modifications based on the following embodiments.

[0038] In embodiments of the present invention, the terms "upper," "lower," "left," "right," "front," "rear," "top," "bottom," "inner," "outer," "middle," "vertical," "horizontal," "lateral," and "longitudinal" indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings. These terms are primarily for the purpose of better describing the present invention and its embodiments, and are not intended to limit the indicated devices, elements, or components to having a specific orientation, or to be constructed and operated in a specific orientation.

[0039] Furthermore, in addition to indicating direction or positional relationship, some of the aforementioned terms may also have other meanings. For example, the term "above" may also be used in certain situations to indicate a dependency or connection. Those skilled in the art can understand the specific meaning of these terms in this invention based on the specific circumstances.

[0040] Furthermore, the terms "installation," "setting," "equipped with," "opening," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, a detachable connection, or an integral structure; they can refer to a mechanical connection or an electrical connection; they can refer to a direct connection or an indirect connection through an intermediate medium, or an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of these terms in this invention according to the specific circumstances.

[0041] Furthermore, the terms "first," "second," etc., are primarily used to distinguish different devices, elements, or components (which may be the same or different in specific type and construction), and are not intended to indicate or imply the relative importance or quantity of the indicated devices, elements, or components. Unless otherwise stated, "a plurality of" means two or more.

[0042] The terms "anterior region" and "posterior region" mentioned in the various embodiments of this application are defined according to the classification of teeth in the 2nd edition of *Introduction to Stomatology*, published by Peking University Medical Press, pages 36-38. These include premolars and molars, teeth marked as 4-8 using the FDI notation, and teeth marked as 1-3 using the FDI notation for the anterior region. Teeth in the anterior region include the central incisors, lateral incisors, and canines.

[0043] The terms "horizontal plane," "coronal plane," and "sagittal plane" mentioned in the various embodiments of this application refer to terms in biomedical anatomy: a horizontal plane (also called a "transverse plane") is a cross-section perpendicular to the vertical axis that divides the human body into upper and lower parts; a coronal plane is a cross-section that longitudinally cuts the human body into anterior and posterior parts along the left and right directions; a sagittal plane is a cross-section that divides the human body into left and right parts, and the left and right cross-sections are called sagittal planes, while cross-sections that are equal on the left and right are called median sagittal planes.

[0044] The inventors of this application discovered in their research on digital design for orthodontic treatment that, in order to achieve automatic tooth alignment using computer technology, geometric calculation methods or machine learning models are often employed. However, in practice, the accuracy of these methods is often low. This is because, on the one hand, the dental arch layout is highly individualized, with each patient's layout being different; on the other hand, adjustments to the dental arch layout are limited by medical parameters, resulting in highly varied actual adjustments and making it difficult to obtain an accurate and suitable tooth alignment plan for each patient. To address these technical problems, this invention provides an automatic tooth alignment method based on a neural network, comprising: acquiring an initial tooth arrangement; extracting a set of tooth-like feature parameters for each tooth from the initial tooth arrangement; and using a trained sequence neural network model for automatic tooth alignment, sequentially processing the set of tooth-like feature parameters for each tooth according to a preset tooth position order to obtain the target position of each tooth. This method changes the existing concept of processing tooth alignment as a geometric image, instead processing different teeth as sequential data. Through a sequence neural network, each tooth is processed separately in a predetermined order to obtain the tooth alignment result for the entire row of teeth. Throughout the processing, compared with existing image network models, the sequence neural network model can learn the characteristics of tooth arrangement better and faster, and the prediction results of tooth arrangement are more accurate.

[0045] The following details the implementation of the neural network-based automatic tooth alignment method of this application. The following content is only for the convenience of understanding and is not necessary for implementing this solution.

[0046] First, it should be noted that the neural network-based automatic tooth alignment method in this embodiment can be implemented through hardware or a combination of computer software and hardware. For hardware implementation, the neural network-based automatic tooth alignment method can be implemented through one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DAPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic devices for implementing the neural network-based automatic tooth alignment function, or a selection and combination of the above devices.

[0047] The specific process of the automatic tooth alignment method based on neural networks provided in one embodiment of the present invention can be as follows: Figure 1 As shown, it specifically includes:

[0048] Step 101: Obtain the initial tooth arrangement.

[0049] Specifically, the initial tooth arrangement can be represented using a digital model of the jaw, which can be a digital model of the jaw in the initial position of the patient or a digital model of the jaw in the initial position of the test case. The source of the model is determined as needed and will not be listed here. In some embodiments, the obtained initial tooth arrangement can be a jaw mesh model after incisor classification, in which each tooth is an independent model. In some embodiments, the initial tooth arrangement is the initial tooth arrangement of a single jaw, such as a single maxilla or a single mandible, which is not limited here.

[0050] Step 102: Extract the set of tooth-type feature parameters for each tooth from the initial tooth arrangement.

[0051] In some embodiments, the tooth feature parameter group is a combination of multiple parameters representing tooth pose. The collected tooth feature data uses pose data. Since the shape of the teeth themselves remains almost unchanged during the tooth arrangement process, the neural network model only predicts the changes in tooth pose. Therefore, calculating only the changes in tooth pose can effectively reduce the number of feature parameters used, reduce the amount of data computation in the prediction, and speed up the model training and prediction.

[0052] In some embodiments, tooth-like feature parameters may include: the geometric center point of the tooth, and six intersection points formed by rays emanating from the geometric center point along the six directions of the three axes in the tooth coordinate system and the tooth mesh. The geometric center point of the tooth is relatively easy to calculate in the model; it can be directly averaged over all mesh vertices of a single tooth. Moreover, this feature effectively locates the tooth position, is well-defined, and has a clear relationship with other contour parameters. Calculations combined with the geometric center point of the tooth are highly accurate. Using the center point and the six intersection points, a total of seven points are used as the feature points of the tooth. These seven feature points can be obtained for each tooth and then arranged in a uniform order. Each point has three spatial coordinates in 3D space, so the feature vector composed of the seven points of each tooth has 21 dimensions. The tooth coordinate system is generally constructed when the dental model is established, and the data can be used directly. The three positive and three negative directions of the three axes are relatively clear. Rays emanating from the geometric center point along these directions will inevitably form six intersection points with the tooth mesh, effectively representing the current position and orientation of the tooth.

[0053] In some embodiments, tooth-like feature parameters may include: the eight vertices of the tooth's spatial bounding box and the tooth's geometric center. Each face of the bounding box is perpendicular to an axis in the tooth coordinate system, thus obtaining the 3D spatial coordinates of the eight vertices of each bounding box. Adding the tooth's geometric center, a total of nine points are used as the tooth's feature points. For different teeth, these nine points are arranged according to a uniform pattern. Because each point contains 3D spatial coordinates, each tooth's features have a total of 27 dimensions.

[0054] In other embodiments, the tooth-related feature parameters may include several feature points on the tooth facing the opposing jaw and the geometric center point of the tooth. These feature points can be determined based on the tooth type. Specifically, when the tooth is a molar, the feature point is the buccal apex; or, when the tooth is a premolar, the feature point is the buccal apex and / or buccal groove; or, when the tooth is an incisor or canine, the feature point is the superior incisal edge. In one embodiment, four buccal cusps can be selected on the molars; for the premolars, one cusp can be selected on the lingual and one on the buccal side, and one distal point is selected mesially and distally in the buccal groove as a feature point; for the incisors, one point can be selected on the incisal edge in the direction of the nearest and farthest midpoints, and then two more points are selected at equal intervals on the incisal edge between these two points, thus also taking four feature points on the incisal surface; for the canines, two points are selected at the nearest and farthest midpoints of the incisal edge, the cusp of the canine is also used as a feature point, and then another point is selected at the midpoint of the incisal edge between the cusp and the farthest midpoint as a feature point. Furthermore, the geometric center of each tooth is also used as a feature point, thus each tooth has five feature points. Because each point contains 3-dimensional spatial coordinates, each tooth has a total of 15 dimensions in its features, demonstrating that the amount of feature data selected in this embodiment can be reduced.

[0055] In the above selection of feature parameters, each set of tooth feature parameters can be used individually or in combination. For example, in one embodiment, tooth feature parameters may include: the geometric center point of the tooth, the six intersection points formed by rays emanating from the geometric center point along the six directions of the three axes in the tooth coordinate system and the tooth mesh, and several feature points of the tooth facing the opposing jaw. More combinations of feature parameters can enrich the detailed features of the teeth, which is beneficial to improving the accuracy of the network model in predicting tooth alignment. It is understood that there are various types and combinations of feature parameters; to avoid repetition, they will not be listed here.

[0056] Furthermore, the number of teeth varies for different cases, and there may be missing teeth. In such cases, 0 can be used to fill all dimensions of the tooth feature.

[0057] Since the adjustment of tooth layout is mainly achieved by translation and rotation, the pose of a single tooth can be determined by combining the geometric center with contour features, so as to minimize the amount of data representing tooth pose parameters, reduce the amount of parameter data involved in the calculation, and speed up the processing speed of the model.

[0058] Step 103: Using the trained sequence neural network model for automatic tooth alignment, process the tooth-type feature parameter groups of each tooth in the preset tooth position order to obtain the target position of each tooth.

[0059] Specifically, when the initial tooth arrangement is obtained as a single-jaw initial tooth arrangement, the preset tooth position sequence includes: an order from smallest to largest FDI tooth position number, such as 11, 12, 13, 14, ... 26, 27, 28 for the maxillary dentition. In other embodiments, the preset tooth position sequence may include: the order from the left terminal tooth to the right terminal tooth in the FDI tooth position; or the order from the right terminal tooth to the left terminal tooth in the FDI tooth position. For example, for the maxillary dentition, the sequence could be from the 8th molar in zone 1 to the 8th tooth in zone 2, that is, from 18, 17, 16... 26, 27, 28. In other embodiments, it may include an order from largest to smallest FDI tooth position number. It is understood that in addition to the above-mentioned sequence setting methods, other tooth position sequences can be set, which will not be listed here. Because the arrangement of individual teeth in the current artificial tooth arrangement process has a certain order, the tooth arrangement process is closer to the current medical design method, and the prediction effect is better.

[0060] Regarding the sequence neural network model used for automatic tooth alignment, Figure 2 Taking this as an example, the sequence neural network model used in this step for automatic tooth alignment can be trained and obtained in the following way:

[0061] Step 201: Combine the tooth-type feature parameter groups of each tooth in the initial position and target position of an existing dentition, arrange the tooth-type feature parameter groups of each tooth according to the preset tooth position order to obtain a dentition sample, and collect multiple dentition samples to construct a tooth arrangement training set.

[0062] Step 202: Construct a sequence neural network model.

[0063] Specifically, the constructed sequence neural network model can employ one or a combination of GRU, LSTM, bidirectional GRU, bidirectional LSTM, multi-layer GRU, multi-layer LSTM, and Transformer structures. In some embodiments, the sequence neural network model may further include adding an attention module. Adding an attention module to the constructed sequence neural network improves the prediction accuracy of the network model. It is understood that different attention mechanisms can be added, such as the Luong attention mechanism and the Bahdanau attention mechanism. Furthermore, these models and attention mechanisms can be used to construct the network into an Encoder-Decoder structure, etc.

[0064] In some embodiments, the loss function used in the constructed sequence neural network model is one of the following: SmoothedL1 Loss, L2 Loss, Huber Loss, or Log-Cosh Loss. In some embodiments, the optimizer used in the constructed sequence neural network model is one of the following: Adam, AdamW, SGD, AdaGrad, or RMSProp. The above loss function and optimizer can be defined simultaneously, and their combinations are diverse, and will not be listed here.

[0065] Step 203: Based on the tooth alignment training set, train the constructed sequence neural network model to obtain the sequence neural network model for automatic tooth alignment.

[0066] During training, the model sequentially learns the feature parameter sets of each tooth in the input dentition sample at its initial position through the recurrent module in the constructed sequence neural network model, according to the preset tooth position order. The model then outputs the feature parameter sets of each tooth at its predicted position according to the preset tooth position order. The feature parameter sets of each tooth at the predicted position are compared with those at the target position to obtain the feature parameter difference value, which is the loss value between the model's predicted value and the labeled data. Using this loss value, the gradient of each parameter can be calculated. Then, using backpropagation and the chain rule, each parameter value of the model is updated along the direction from the model output to the model input. The model training is completed after processing a large amount of data (hundreds of thousands of cases) and multiple iterations. In some embodiments, mini-batch data can be used to update model parameters during model training, which can speed up the training process.

[0067] by Figure 3 Taking the LSTM structure sequence neural network model as an example, it specifically includes an encoding module and a decoding module, where the encoding module and the decoding module each include several recurrent units.

[0068] During the encoding phase, X n The nth tooth-type feature parameter group can be represented in one embodiment, where tooth information can be directly used for encoding. In another embodiment, in addition to inputting each tooth-type feature parameter group, location information can also be encoded together. The location information can be the FDI number. Each tooth in the dental arch has a unique FDI number, so using the FDI number as the location information is simple and quick, so that the input of the model includes not only the spatial information of the teeth but also the location information of the teeth.

[0069] To further explain, each loop unit uses an LSTM structure, and its internal structure can be as follows: Figure 4As shown, the input consists of three parts: input information, hidden state, and memory cell state. The hidden state and memory cell state are derived from the previous LSTM unit; for the first LSTM unit, they use randomly initialized values. An LSTM unit typically includes three gating units: a forget gate, an input / update gate, and an output gate. The forget gate controls which parts of the historical information are unimportant and can be forgotten, and which need to be retained. It uses the Sigmoid activation function, with a value between 0 and 1, to control the degree of forgetting. The input / update gate also uses the Sigmoid function to determine how much new input information to use, and the tanh function to calculate the current candidate cell state. Then, through the forget gate and input gate, the cell state from the previous unit and the current candidate cell state are linearly combined to obtain the current cell state. The output gate determines which parts of the memory unit will be used to calculate the current hidden state and output to the next time step. It is controlled by a sigmoid function, which determines which parts of the memory cell will affect the final hidden state. Then, it is scaled to the range of [-1, 1] by a tanh function. In other words, the numerical stability can be guaranteed by the tanh function.

[0070] During the decoding phase, Y n This represents the predicted feature parameter set for the nth tooth. The input to the decoding module can be the output of the previous recurrent unit and the hidden state. For recurrent units in an LSTM structure, the input to the recurrent unit can also include the cell state of the previous recurrent unit.

[0071] As can be seen from steps 201 to 203, by pre-constructing a sample set, a sequence neural network model is built, and then trained to obtain a sequence neural network model that can be used for automatic tooth alignment, a sequence neural network model that better meets individual needs can be obtained.

[0072] In some embodiments, such as Figure 5 As shown, step 103 further includes the process of calculating mesh vertices using the model output results:

[0073] Step 501: Using the trained sequence neural network model for automatic tooth alignment, the initial values ​​of the tooth-type feature parameter groups of each tooth are processed sequentially according to the preset tooth position order to obtain the predicted target values ​​of the tooth-type feature parameter groups.

[0074] Specifically, when the model makes predictions, the dimension of its output corresponds to the dimension of its input. For example, if the input is the three-dimensional spatial coordinates of the feature points of each tooth, then the output is the three-dimensional spatial coordinates of the corresponding feature points of each tooth.

[0075] Step 502: Based on the predicted target value and the initial value of the tooth class feature parameter group for each tooth, the spatial coordinate transformation of each tooth is calculated using a point cloud registration algorithm.

[0076] Step 503: Based on the spatial coordinate transformation of each tooth, the mesh model of each tooth in the initial tooth arrangement is transformed to obtain the mesh model of the target position of each tooth.

[0077] Specifically, based on the transformation of the spatial coordinates of the corresponding points of each tooth, the translation matrix and rotation transformation matrix of each tooth can be calculated using point cloud registration methods (such as the Kabsch method). After applying these translation matrices and rotation transformations to the input grid vertex coordinates of each tooth, the grid vertex coordinates of each tooth at the predicted target position can be obtained.

[0078] Taking the Kabsch algorithm as an example, the specific calculation method is explained as follows:

[0079] (1) From the feature points of the model input and output, we can construct two matrices P and Q, each with a dimension of 3xN. Each column of these two matrices represents the coordinates of the feature points, N is the number of feature points, and the feature points represented by each column of these two matrices are in one-to-one correspondence.

[0080] (2) Decenter these two matrices (p0 and q0 are the centers of P and Q respectively), and the resulting new matrices are still denoted as P and Q;

[0081] (3) Construct the covariance matrix H = P T Q;

[0082] (4) Perform singular value decomposition (SVD) on the covariance matrix H, i.e., H = US V T U and V are two orthogonal matrices, and S is a diagonal matrix with singular values ​​on the diagonal.

[0083] (5) Next, calculate d = sign(det(VU) T To determine whether a rotation correction matrix is ​​needed to ensure the use of a right-handed coordinate system, construct the following matrix based on d:

[0084] (6) The rotation matrix R is calculated by the following formula: R = VIU T R is a 3x3 matrix. The translation matrix r = p0 - R * q0. Applying these two matrices to the initial vertex coordinates of each tooth yields the vertex coordinates of the target tooth. That is:

[0085] X' = ​​RX + r;

[0086] Where X' is the coordinate of a vertex of a tooth in the target position, and X is the coordinate of a vertex of the corresponding initial position tooth.

[0087] As can be seen, steps 501 to 503 further define the process of calculating spatial coordinate transformation and then restoring the tooth mesh model through point cloud registration. Considering that the shape of the teeth themselves remains almost unchanged, restoring the dentition morphology through registration can increase the error tolerance of the predicted feature parameters. It is understood that in addition to using the Kabsch algorithm for registration and transformation as described above, other registration algorithms can also be used, which will not be listed here.

[0088] In some embodiments, after step 103, a post-processing procedure is further included for the predicted target positions output by the model. This may include: performing collision detection on the target positions of each tooth and updating the target positions of each tooth based on the detection results. Since the target positions predicted by the neural network model may contain collisions or gaps between teeth, it is necessary to adjust the relative positions of each pair of adjacent teeth in the same jaw if there are collisions or gaps.

[0089] In one embodiment, the method for fine-tuning between teeth in a single jaw is as follows: First, determine whether there is a collision or gap between the two central incisors. If there is a collision, calculate their collision depth d, and then move both teeth d / 2 along the distal direction. If there is a gap, calculate the gap width w, and then move each tooth w / 2 along the mesial direction. Then, starting with the incisors, perform collision detection on the incisors and lateral incisors. If there is a collision, move the lateral incisors distally; if there is a gap, move the lateral incisors mesially, and so on. For the canines, premolars, and molars behind them, fine-tune them in the same way.

[0090] Subsequently, the inventors of this application used the neural network-based automatic tooth alignment method described in this application to conduct tooth alignment tests, and compared the target tooth positions obtained using the neural network-based automatic tooth alignment method with the target tooth positions obtained through manual tooth alignment. Figure 6 The test example shown in the paper is as follows: a is the initial tooth arrangement in the example, b is the target tooth row obtained by manual tooth arrangement, c is the target tooth row obtained by the automatic tooth arrangement method in this application, and d is an overlapping comparison diagram of b and c. It can be seen that the difference between the target tooth position obtained by the automatic tooth arrangement method based on neural network in this application and the target tooth position obtained by manual tooth arrangement is small, indicating that the automatic tooth arrangement method based on neural network in this application has better practical effect and should be widely promoted.

[0091] As can be seen, this embodiment changes the existing concept of processing tooth arrangement as a geometric image, treating different teeth as sequential data. Through a sequence neural network, each tooth is processed separately in a predetermined order to obtain the tooth arrangement result for the entire row. Moreover, the feature data of each tooth remains separate during this process, effectively improving the accuracy of feature learning. Based on these characteristics, the inventors of this application selected a sequence neural network model, utilizing its long short-term memory characteristics. When predicting the features of a tooth, it references not only the features of adjacent teeth but also the features of other teeth that are far apart, allowing for faster learning of tooth arrangement characteristics. Throughout the entire processing, compared with existing image network models, the sequence neural network model can learn tooth arrangement characteristics better and faster, and the prediction results for tooth arrangement are more accurate.

[0092] Another embodiment of this application provides an automatic tooth arrangement method based on neural networks. The main difference between this embodiment and the previous embodiment is that the tooth arrangement processed in the previous embodiment is a single jaw tooth arrangement, while the tooth arrangement processed in this embodiment includes the upper and lower jaw teeth.

[0093] Specifically, when obtaining the initial tooth alignment, the initial tooth alignment of the maxilla and mandible of the same patient is obtained. It can be understood that the maxilla and mandible can be the maxilla and mandible tooth alignment after occlusal registration.

[0094] Correspondingly, in addition to adopting the tooth arrangement order in the previous embodiment in a single jaw, the preset tooth position order can be arranged in the order of arranging the teeth of the maxillary dentition first and then arranging the teeth of the mandibular dentition, or in the order of arranging the teeth of the mandibular dentition first and then arranging the teeth of the maxillary dentition.

[0095] The process of predicting the target tooth position using a sequence neural network model for automatic tooth alignment is similar to the previous implementation method and will not be described again here.

[0096] In some embodiments, after obtaining the target position, collision detection and adjustment can be performed on the teeth between the upper and lower jaws. During adjustment, fine-tuning can be performed first on the teeth between individual jaws, and then fine-tuning can be performed on the teeth between the upper and lower jaws. In one embodiment, collision is first checked on all teeth between the upper and lower jaws. If there is a collision depth d, the teeth colliding in the upper and lower jaws are moved a distance d / 2 in an upward or downward direction, respectively. If there is a gap w, the teeth with larger gaps in the upper and lower jaws are moved downward and upward, respectively, to reduce the gap between the teeth.

[0097] In summary, when the initial tooth arrangement obtained includes the arrangement of the maxillary and mandibular teeth, the neural network-based automatic tooth arrangement method in this application is also applicable, and the tooth arrangement effect is also good.

[0098] Another embodiment of this application provides an automatic tooth alignment method based on neural networks. This embodiment is an improvement on the previous embodiment. The main improvement is that the data input to the neural network model in the previous embodiment only includes tooth-related feature parameters, while in this embodiment, in addition to tooth-related feature parameters, medical plan-related parameters are also combined. By combining these medical plans to learn tooth alignment, the speed at which the sequence neural network model learns tooth alignment features can be further accelerated, and the accuracy of the predicted tooth alignment target position can be improved.

[0099] One architecture of the sequence neural network model used in this embodiment is as follows: Figure 7 As shown, it specifically includes an encoding module, a feature concatenation module, and a decoding module. Specifically, the features output by the last loop unit in the encoding module can be combined with features associated with the medical protocol as input features for the decoder. In some embodiments, the medical protocol parameter group includes one or any combination of the following: arch expansion, gap closure, enamel reduction, tooth extraction, and ideal arch curve morphology.

[0100] like Figure 8 As shown, when processing tooth-like feature parameters using the sequence neural network model in this embodiment, the specific steps include:

[0101] Step 801: Using the encoding module in the trained sequence neural network model for automatic tooth alignment, the tooth-type feature parameter groups of each tooth are encoded according to the preset tooth position order. Specifically, this step is similar to the first implementation method and will not be described again here.

[0102] Step 802: Using the decoding module in the trained sequence neural network model for automatic tooth alignment, the obtained encoding result and the pre-acquired medical plan parameter set are decoded to obtain the target position of each tooth.

[0103] Specifically, the encoding result obtained in step 801 is concatenated with the pre-acquired medical plan parameter group, and the concatenated result is decoded. In some embodiments, the pre-acquired medical plan parameter group can be encoded by a preset text encoder before concatenation. When applying medical plan feature parameters, the method of first encoding with a preset text encoder and then concatenating with the original parameters is more convenient to cover the rich expression methods in existing medical plans, making the medical plans more learnable. When combining medical plan parameters, parameter concatenation can be used to improve the combined application of multiple types of data. In addition, since combining medical plan parameters actually integrates multiple forms of data in the overall dental alignment, a multimodal solution is realized. It can be understood that in the feature concatenation part, the data output by the text encoder can be processed by a fully connected layer before concatenation. Since the numerical range of the encoded data output by the text encoder and the numerical range of the encoded data output by the encoding module may differ greatly, a fully connected layer can make the value ranges of the two closer, eliminating the interference of large values ​​on the results when the numerical differences are too large.

[0104] As can be seen, in this embodiment, when using a sequence neural network model for tooth arrangement, in addition to using the characteristic parameters of the teeth themselves in the dentition, medical plan parameters can also be combined. Since the doctor or patient may specify some medical plans during the tooth arrangement process, combining these medical plans to learn tooth arrangement can further accelerate the speed at which the sequence neural network model learns the tooth arrangement features and improve the accuracy of the predicted tooth arrangement target position.

[0105] It is worth mentioning that the examples above in this application are merely illustrative for ease of understanding and do not constitute a limitation on the technical solutions of this invention.

[0106] The steps of the various methods described above are only for clarity. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this patent. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this patent.

[0107] Another embodiment of this application provides an automatic tooth alignment system based on a neural network, such as... Figure 10 As shown, it specifically includes:

[0108] The acquisition module is used to obtain the initial tooth arrangement;

[0109] An extraction module is used to extract a set of tooth-type feature parameters for each tooth from the initial tooth arrangement;

[0110] The prediction module uses a trained sequence neural network model for automatic tooth alignment to process the tooth-type feature parameter sets of each tooth in a preset tooth position order to obtain the target position of each tooth.

[0111] Another embodiment of this application relates to an electronic device, such as... Figure 9 As shown, it includes: at least one processor 901; and a memory 902 communicatively connected to the at least one processor 901; wherein the memory 902 stores instructions executable by the at least one processor 901, the instructions being executed by the at least one processor 901 to enable the at least one processor 901 to perform the neural network-based automatic tooth alignment method in the above partial embodiments.

[0112] The memory and processor are connected via a bus, which can include any number of interconnecting buses and bridges, connecting various circuits of one or more processors and memories. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and will not be described further herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by the processor is transmitted over the wireless medium via an antenna, which further receives data and transmits it to the processor.

[0113] The processor manages the bus and general processing, and also provides various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory is used to store data used by the processor during operation.

[0114] Another embodiment of this application relates to a computer-readable storage medium storing a computer program. When executed by a processor, the computer program implements the neural network-based automatic tooth alignment method in some of the above embodiments, or the digital modeling method for dental orthodontic jaw models in some of the above embodiments.

[0115] That is, those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0116] Those skilled in the art will understand that the above embodiments are specific embodiments for implementing this application, and in practical applications, various changes can be made to them in form and detail without departing from the spirit and scope of this application.

Claims

1. A neural network-based automatic tooth alignment method, characterized by, The method comprises the following steps: obtaining an initial tooth arrangement; extracting a tooth feature parameter set of each tooth from the initial tooth arrangement; using a trained sequence neural network model for automatic tooth arrangement to sequentially process the tooth feature parameter set of each tooth in a preset tooth position order to obtain a target position of each tooth.

2. The neural network-based automatic tooth alignment method of claim 1, wherein, In the step of obtaining the initial tooth arrangement, the obtained initial tooth arrangement is a single-jaw initial tooth arrangement. The preset tooth position order comprises: an order from small to large FDI tooth position number; or an order from large to small FDI tooth position number; or an order from the last tooth in the left region to the last tooth in the right region; or an order from the last tooth in the right region to the last tooth in the left region.

3. The neural network-based automatic tooth alignment method of claim 1, wherein, In the step of obtaining the initial tooth arrangement, the obtained initial tooth arrangement comprises initial tooth arrangements of the upper jaw and the lower jaw of the same patient. The preset tooth position order comprises: an order of arranging the teeth of the upper jaw first and then arranging the teeth of the lower jaw, or an order of arranging the teeth of the lower jaw first and then arranging the teeth of the upper jaw.

4. The neural network-based automatic tooth alignment method of claim 1, wherein, The tooth feature parameter set is a combination of a plurality of parameters representing the pose of the tooth.

5. The neural network-based automatic tooth alignment method of claim 4, wherein, The tooth feature parameter set comprises at least one of the following parameter sets: a geometric center point of the tooth, and six intersection points formed by six rays emitted from the geometric center point along six directions of a three-axis tooth coordinate system and a tooth mesh; or eight vertices of a space bounding box of the tooth and the geometric center point of the tooth; or a plurality of feature points of the tooth facing the opposite jaw and the geometric center point of the tooth. The feature points of the tooth facing the opposite jaw are determined according to the type of the tooth, wherein when the tooth is a molar, the feature point is a buccal cusp point; or when the tooth is a premolar, the feature point is a buccal cusp point or a buccal groove point; or when the tooth is an incisor or a canine, the feature point is an upper incisal edge point.

6. The neural network-based automatic tooth alignment method of claim 5, wherein, The calculation of using the trained sequence neural network model for automatic tooth arrangement to sequentially process the tooth feature parameter set of each tooth in the preset tooth position order comprises:

7. The neural network-based automatic tooth alignment method according to any one of claims 1-6, wherein, using an encoding module in the trained sequence neural network model for automatic tooth arrangement to encode the tooth feature parameter set of each tooth in the preset tooth position order; using a decoding module in the trained sequence neural network model for automatic tooth arrangement to decode the obtained encoding result and a pre-obtained medical scheme parameter set to obtain the target position of each tooth. The decoding of the obtained encoding result and the pre-obtained medical scheme parameter set comprises:

8. The neural network-based automatic tooth alignment method of claim 7, wherein, splicing the obtained encoding result and the pre-obtained medical scheme parameter set, and decoding the spliced result. The splicing of the obtained encoding result and the pre-obtained medical scheme parameter set comprises: encoding the pre-obtained medical scheme parameter set through a preset text encoder and then splicing.

9. The neural network-based automatic tooth alignment method of claim 8, wherein, The medical scheme parameter set comprises one or any combination of the following: expansion of the arch, closure of the gap, enamel reduction, tooth extraction, and ideal dental arch curve shape.

10. The neural network-based automatic tooth alignment method of claim 7, wherein, The calculation of using the trained sequence neural network model for automatic tooth arrangement to sequentially process the tooth feature parameter set of each tooth in the preset tooth position order to obtain the target position of each tooth comprises:

11. The neural network-based automatic tooth alignment method according to any one of claims 1-6, wherein, ​ The trained sequence neural network model for automatic tooth arrangement is used to process initial values of the tooth feature parameter group of each tooth in a preset tooth position sequence to obtain predicted target values of the tooth feature parameter group. Based on the predicted target values and the initial values of the tooth feature parameter group of each tooth, a point cloud registration algorithm is used to calculate the spatial coordinate transformation of each tooth. According to the spatial coordinate transformation of each tooth, the mesh model of each tooth in the initial tooth arrangement is subjected to spatial coordinate transformation to obtain the mesh model of the target position of each tooth.

12. The neural network-based automatic tooth arrangement method according to any one of claims 1-6, characterized in that, Before the trained sequence neural network model for automatic tooth arrangement is used to calculate the sorted tooth feature parameter group, the method comprises: Combining the tooth feature parameter groups of each tooth in the initial position and the target position of an existing dentition, arranging the tooth feature parameter groups of each tooth in the preset tooth position sequence to obtain a dentition sample, collecting multiple dentition samples, and constructing a tooth arrangement training set; Constructing a sequence neural network model; Based on the tooth arrangement training set, the constructed sequence neural network model is trained to obtain the sequence neural network model for automatic tooth arrangement.

13. The neural network-based automatic tooth alignment method of claim 12, wherein, The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure.

14. The neural network-based automatic tooth alignment method of claim 12, wherein, The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure.

15. The neural network-based automatic tooth alignment method of claim 12, wherein, The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure.

16. The neural network-based automatic tooth alignment method according to any one of claims 1-6, wherein, The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure.

17. An electronic device, comprising: The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GRU structure, bidirectional LSTM structure, multi-layer GRU structure, multi-layer LSTM structure, and Transformer structure. The sequence neural network model comprises one of the following: GRU structure, LSTM structure, bidirectional GR

Images