Tooth arrangement method, tooth orthodontic simulation method, storage medium and oral CT device

Abstract

The present invention discloses a tooth arrangement method, a tooth orthodontic simulation method, a storage medium and an oral CT device. The tooth arrangement method comprises the following steps: obtaining an oral CT image; segmenting the oral CT image to obtain a first positioning frame of each tooth; determining multiple tooth arrangement reference points according to the first positioning frame, and sorting all the first positioning frames according to the multiple tooth arrangement reference points to obtain a sorting result; using a pre-trained classification model to identify the tooth type of each tooth; and determining the tooth arrangement result according to the sorting result and the tooth type. Thus, the tooth arrangement method can realize fully automatic tooth arrangement based on oral CT images.

Description

Technical Field

[0001] The present invention relates to the technical field of oral medicine, and in particular to a teeth arrangement method, a tooth orthodontic simulation method, a storage medium and an oral CT device. Background Art

[0002] In the field of stomatology, oral disease is a common and multiple disease. Dentomaxillary deformity has a great impact on oral health, oral function, and appearance. Oral deformity correction is considered an indispensable part of oral health care. Before treatment, tooth arrangement experiments can help doctors predict the treatment process involved in orthodontic treatment. However, the current method of tooth arrangement is mainly manual tooth arrangement, which has the problems of low efficiency, large material consumption, and low precision. Summary of the invention

[0003] The present invention aims to solve one of the technical problems in the related art at least to a certain extent. To this end, the first object of the present invention is to provide a tooth arrangement method to achieve full-automatic tooth arrangement based on oral CT images.

[0004] The second objective of the present invention is to provide a tooth orthodontic simulation method.

[0005] A third object of the present invention is to provide a computer-readable storage medium.

[0006] The fourth objective of the present invention is to provide an oral CT device.

[0007] To achieve the above-mentioned purpose, the first aspect of the present invention proposes a tooth arrangement method, comprising the following steps: acquiring an oral CT image; segmenting the oral CT image to obtain a first positioning frame for each tooth; determining multiple tooth arrangement reference points based on the first positioning frame; sorting all the first positioning frames based on the multiple tooth arrangement reference points to obtain a sorting result; identifying the tooth type of each tooth using a pre-trained classification model; and determining the tooth arrangement result based on the sorting result and the tooth type.

[0008] To achieve the above-mentioned purpose, the second aspect of the present invention proposes a method for orthodontic simulation, which includes the following steps: determining the dental arch line of the model to be corrected, and generating a target model according to the dental arch line and a pre-stored standard tooth model; performing a first adjustment on the corresponding tooth in the model to be corrected according to each tooth of the target model to achieve orthodontic simulation of the dentition in the model to be corrected, wherein the target model and the corresponding teeth in the model to be corrected are identified based on the tooth arrangement result of the model to be corrected, and the model to be corrected is obtained by tooth arrangement according to the above-mentioned tooth arrangement method.

[0009] To achieve the above-mentioned purpose, the third aspect of the present invention proposes a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the above-mentioned tooth arrangement method or the above-mentioned tooth orthodontic simulation method is implemented.

[0010] To achieve the above-mentioned purpose, the fourth aspect of the present invention proposes an oral CT device, including a memory, a processor and a computer program stored in the memory. When the computer program is executed by the processor, the above-mentioned tooth arrangement method or the above-mentioned tooth orthodontic simulation method is implemented.

[0011] The teeth arrangement method, tooth orthodontic simulation method, storage medium and oral CT equipment of the embodiments of the present invention can realize fully automatic teeth arrangement based on oral CT images.

[0012] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] Figure 1 is a flow chart of a tooth segmentation method according to an embodiment of the present invention;

[0014] Figure 2 is a schematic diagram of an oral CT image to be segmented according to an example of the present invention;

[0015] Figure 3 is a schematic diagram of a framework of a first neural network of an example of the present invention;

[0016] Figure 4 is a schematic diagram of a tooth overall area positioning frame according to an example of the present invention;

[0017] Figure 5 is a schematic diagram of a framework of a second neural network of an example of the present invention;

[0018] Figure 6 is a schematic diagram of a framework of a third neural network of an example of the present invention;

[0019] Figure 7 is a schematic diagram of a software perspective of an example of the present invention;

[0020] Figure 8 is a schematic diagram of annotated viewing angles of an example of the present invention;

[0021] Fig. 9 This is an effect diagram of Otsu threshold segmentation according to an example of the present invention;

[0022] Fig.10 This is an effect diagram of adaptive three-dimensional threshold segmentation of an example of the present invention;

[0023] Fig.11 is a flow chart of misdetection or missed detection of a positioning frame according to an example of the present invention;

[0024] Fig.12 , Fig.13 is a schematic diagram of mis-positioning of the first example of the present invention;

[0025] Fig.14 , Fig.15 is a schematic diagram of mis-positioning of a second example of the present invention;

[0026] FIG. 16( a ) and FIG. 16( b ) are schematic diagrams of leak location according to an example of the present invention;

[0027] Fig.17 is a flow chart of a method for acquiring tooth surface data according to an embodiment of the present invention;

[0028] Fig.18 is a flow chart of a tooth arrangement method according to an embodiment of the present invention;

[0029] Fig.19 is a schematic diagram of a tooth overall area positioning frame according to an example of the present invention;

[0030] Figure 20-22 is a schematic diagram of a process of determining multiple tooth arrangement reference points according to an example of the present invention;

[0031] Fig.23 is a schematic diagram of a tooth arrangement result of an example of the present invention;

[0032] Fig.24 is a flow chart of a tooth orthodontic simulation method according to an embodiment of the present invention;

[0033] Fig.25 It is a schematic diagram of the framework of a neural network model of an example of the present invention. DETAILED DESCRIPTION

[0034] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0035] In the embodiment of the present invention, before arranging teeth, the oral CT image needs to be segmented to obtain the positioning frame of each tooth. Figure 1-17 A tooth segmentation method according to an embodiment of the present invention is described.

[0036] Figure 1 4 is a flow chart of a tooth segmentation method according to an embodiment of the present invention.

[0037] like Figure 1 As shown, the tooth segmentation method includes the following steps:

[0038] S101, obtaining an oral CT image to be segmented.

[0039] Specifically, the oral CT images to be segmented can be obtained from the database of the constructed medical image management system, such as Figure 2 shown.

[0040] S102, using a pre-trained first neural network to locate the entire tooth region in the oral CT image to be segmented.

[0041] The overall tooth area is an area smaller than the overall area of ​​the oral CT image and covers all teeth in the oral CT image. For example, it can be an area selected by the minimum circumscribed three-dimensional frame covering all teeth, or an area corresponding to a three-dimensional frame slightly larger than the minimum three-dimensional frame. Of course, an area corresponding to a three-dimensional frame larger than the minimum three-dimensional frame within a preset range is also possible, such as the preset range being the range below the nose. The located overall tooth area can be represented by the first 3D positioning data, and the first 3D positioning data can be the image data corresponding to the above-mentioned three-dimensional frame in the oral CT image.

[0042] Specifically, the first neural network may include a 3D convolution layer, a 3D pooling layer, and a normalization layer, which can locate the three-dimensional tooth area in the oral CT image, and then further locate the single tooth area through the second neural network. The first neural network framework can be as follows: Figure 3 The first neural network can quickly locate the entire tooth area in the oral CT image, that is, the first 3D positioning data mentioned above, as shown in FIG. Figure 4 The reason for doing so is that it can quickly determine whether there are teeth in the oral CT image. If not, no subsequent operation will be performed; at the same time, it can avoid misidentification in the nasal cavity and other parts, reduce the amount of input data of the second neural network, and thus improve the overall speed and accuracy of tooth segmentation.

[0043] S103, using the pre-trained second neural network to locate a single tooth region within a preset range in the entire tooth region.

[0044] The preset range is a custom tooth type, which can be all teeth or partial teeth. That is to say, the second neural network is not limited to identifying the area of ​​each tooth among all teeth, but can also be used to identify a single tooth area of ​​a specific type (such as all central incisors) or several types (such as all lateral incisors and canines) among all teeth, or can also be used to identify a single tooth area of ​​a certain area (such as the left side of the upper central incisor) or multiple areas (such as the left side of the upper central incisor and the right side of the lower central incisor) among all teeth.

[0045] Of course, tooth types may include central incisors, lateral incisors, canines, first premolars, second premolars, first permanent molars, second permanent molars, and third permanent molars. A single tooth region is a region corresponding to a single tooth selected by using a minimum or slightly larger three-dimensional frame; a single tooth region may be a single tooth image selected by the above three-dimensional frame in an oral CT image or the first 3D positioning data.

[0046] Specifically, if step S102 determines that there are teeth in the oral CT image, the identified whole tooth area is input into the second neural network to locate the single tooth area and obtain the second 3D positioning data, thereby avoiding mis-positioning in the nasal cavity and other parts, while reducing the input data volume of the second neural network and improving the overall speed and recognition accuracy. Figure 5 As shown, the second neural network may include multiple 3D detection layers (e.g., 3), and the 3D detection layer includes a 3D convolution layer and a 3D pooling layer.

[0047] As an example, using a pre-trained second neural network to locate a single tooth region within a preset range in the overall tooth region may include: performing three-dimensional Otsu threshold segmentation, adaptive threshold segmentation, and morphological processing on the overall tooth region in sequence to obtain first prior knowledge data; using a pre-trained second neural network to locate a single tooth region within a preset range in the overall tooth region based on the first prior knowledge data and the overall tooth region. The first prior knowledge data eliminates the influence of soft tissue, etc., and referring to the first prior knowledge and combining the overall tooth region for identification can reduce the identification accuracy of the single tooth region.

[0048] Specifically, the second neural network mainly completes the positioning of a single tooth area based on the first neural network. In order to reduce the missed detection area, while inputting the overall tooth area data positioned by the first neural network at the input end, the prior knowledge data (i.e., the first prior knowledge data mentioned above) can also be input to reduce the learning difficulty of the second neural network and accelerate convergence. The steps for obtaining the prior knowledge data can be as follows:

[0049] Step 1: Since oral CT images include air, soft tissue and other parts, and in order to achieve tooth segmentation, only the tooth part needs to be marked, so the oral CT images are first segmented by 3D Otsu threshold method to remove the interference of the air part;

[0050] Step 2: Perform a second adaptive 3D threshold segmentation on the data in Step 1 to separate the bone part including the tooth area in the oral CT image;

[0051] Step 3: Perform morphological processing on the separated bone data to clearly separate and extract the overall tooth part.

[0052] Due to the prior knowledge of the bone area and the labeled real data as supervision, the second neural network can quickly locate and identify the area of ​​​​each tooth. In order to better cope with teeth of different sizes, three detection layers can be set in the second neural network to detect teeth of different shapes and sizes.

[0053] As an example, when using a pre-trained second neural network to locate a single tooth area within a preset range in the overall tooth area, the second neural network can be used to output a 3D positioning frame of the single tooth based on the first 3D positioning data; the 3D positioning frame is mapped to the oral CT image to be segmented to obtain the single tooth area.

[0054] S104, using a pre-trained third neural network to perform mask segmentation on a single tooth region to obtain tooth body data of the corresponding tooth.

[0055] Specifically, after the first neural network and the second neural network are learned, the area where each tooth is located can be accurately located, and then the tooth data of each tooth can be extracted through the third neural network. The main training framework is as follows: Figure 6 As shown, the third neural network may include a 3D convolution layer and a 3D pooling layer.

[0056] As an example, using a pre-trained third neural network to perform mask segmentation on a single tooth area may include: performing three-dimensional Otsu threshold segmentation, adaptive threshold segmentation and morphological processing on the single tooth area in sequence to obtain second prior knowledge data; using the third neural network to perform mask segmentation of the single tooth area based on the single tooth area and the second prior knowledge data.

[0057] In one embodiment of the present invention, a pre-trained third neural network may be used to perform mask segmentation on the root canals in a single tooth region to obtain root canal volume data.

[0058] Specifically, if the root canal data of each tooth needs to be segmented during tooth segmentation, the single tooth area can be subjected to adaptive threshold segmentation and morphological processing in turn to obtain the root canal data; and then the third neural network is used to perform mask segmentation of the single tooth area and the root canal area according to the single tooth area, the root canal data and the second prior knowledge data to obtain the tooth body data and the root canal data.

[0059] Specifically, the third neural network completes the mask segmentation of the single tooth area and the root canal area based on the first neural network and the second neural network, and uses one network model to complete multiple tasks at the same time. The input of the third neural network includes the CT three-dimensional data of the area where the single tooth is located output by the second neural network, the root canal data obtained by adaptive threshold segmentation and morphological processing, and the prior knowledge data (i.e., the second prior knowledge data mentioned above), and each data acquisition method is as follows:

[0060] CT three-dimensional data of the area where a single tooth is located: The 3D positioning frame output by the second neural network is mapped to the oral CT image to be segmented, and the 3D data of the corresponding part is extracted. The numerical distribution normalization and size normalization operations are completed to obtain the 3D data of the single tooth area;

[0061] Root canal data: Adaptive threshold segmentation is performed on the extracted 3D data of the single tooth area, and then morphological processing of corrosion and expansion is performed to extract the 3D data of the root canal area in the middle of the tooth;

[0062] Second prior knowledge data: Since the 3D data of the single tooth area extracted above includes air, soft tissue and other parts, the 3D data of the single tooth area is first subjected to three-dimensional Otsu threshold segmentation to remove the interference of the air part, and then a second adaptive three-dimensional threshold segmentation is performed to separate the bone part including the tooth area. The separated bone data is morphologically processed as prior knowledge data, with the aim of reducing the learning difficulty of the network and accelerating convergence.

[0063] By combining the above three types of data and using the labeled real data as supervision, the third neural network can effectively and quickly extract the 3D mask of the tooth and the 3D root canal mask at the same time, completing multiple tasks with a single model.

[0064] Therefore, the teeth segmentation method of the embodiment of the present invention can achieve accurate and effective segmentation of teeth in oral CT images.

[0065] In an embodiment of the present invention, the first neural network, the second neural network and the third neural network may be pre-trained, and the data used to train the first neural network, the second neural network and the third neural network are labeled in the following manner:

[0066] A1, perform three-dimensional Otsu threshold segmentation, adaptive threshold segmentation and morphological processing on the oral CT training image in sequence to obtain the tooth image to be labeled;

[0067] A2, using the trained tooth region model to perform mask segmentation on the to-be-annotated tooth image to obtain the oral CT training image of the pre-marked tooth, wherein the tooth region model is trained by a residual-based 3D semantic segmentation artificial neural network;

[0068] A3, perform tooth marking correction on the oral CT training images of pre-marked teeth;

[0069] A4, determine the three-dimensional frame of a single tooth based on tooth markings to mark the area of ​​a single tooth;

[0070] A5, determining a three-dimensional frame for marking the entire tooth area according to the three-dimensional frames of all single teeth, so as to mark the entire tooth area;

[0071] A6, obtaining data for training the first neural network, the second neural network and the third neural network according to the markers in the oral CT training image.

[0072] Specifically, the training images are three-dimensional data, and their labeling has two major problems. First, the amount of labeling is large. Compared with the amount of two-dimensional data, the amount of labeling of three-dimensional data increases exponentially. Second, the labeling is difficult. Due to the addition of a dimension, the traditional labeling method on two-dimensional slices is inaccurate. Because the teeth are densely arranged and the occlusal relationship of the occlusal surface is relatively complex, labeling only on two-dimensional slices cannot accurately indicate which part a certain area belongs to.

[0073] In order to overcome the above two major problems and obtain high-quality and high-accuracy annotation data, the present invention combines the three-dimensional information of oral CT images for annotation, annotates each tooth individually, and uses preprocessing methods to improve annotation efficiency.

[0074] For more accurate labeling, existing 3D labeling software can be used to view oral CT images from three angles: cross-section, sagittal and coronal. The software may include a 3D reconstruction window. The specific labeling process is as follows: for a tooth to be labeled, each layer of slices is labeled in a selected direction, that is, the outer edge of the tooth area is marked on the cut surface. When all slice images of the tooth are labeled, the 3D labeling (i.e. 3D labeling) of the tooth is completed.

[0075] When marking, the marking is realized on the 3D volume data. Combining different perspectives and reconstructed skull data, the marked area can be clearly seen, so that each tooth can be accurately marked, solving the problem of inaccurate 2D marking. Figure 7 As shown, the annotation perspective is Figure 8 shown.

[0076] As an example, after completing the 3D annotation of a single tooth region in the training image, the method may further include: iteratively training the tooth region model according to the obtained annotation data.

[0077] Specifically, in order to reduce the workload of annotation and improve the annotation efficiency, the present invention proposes the following two preprocessing methods:

[0078] The first method: using two threshold segmentation algorithms to extract bone data

[0079] Step 1: Since oral CT images include air, soft tissue and other parts, and in order to achieve tooth segmentation, only the tooth part needs to be marked, so the oral CT images are first segmented by the three-dimensional Otsu method to remove the interference of the air part. Among them, the three-dimensional Otsu method segments the oral CT images by maximizing the inter-class variance to find the appropriate threshold. The inter-class variance of the threshold T in the histogram is defined as:

[0080]

[0081] The threshold T corresponding to the minimum variance is the segmentation threshold. The oral CT images before and after segmentation are as follows: Fig. 9 shown.

[0082] Step 2: Perform a second adaptive 3D threshold segmentation on the data obtained in Step 1 to remove soft tissue and separate the bone part including the tooth area in the oral CT image. Fig.10 shown.

[0083] Step 3: Perform morphological processing on the skeleton data separated in Step 2 to clearly separate and extract the teeth, and then annotate the teeth. Since the teeth are clearly morphologically separated, the annotation difficulty is greatly reduced and the efficiency is significantly improved.

[0084] The second method: using artificial neural networks to obtain prior knowledge to assist in labeling

[0085] Step 1: After accumulating and labeling a small amount of tooth segmentation data, a residual-based 3D semantic segmentation artificial neural network was built to quickly train the labeled tooth area using the labeled tooth segmentation data in the database;

[0086] Step 2: Use the trained tooth region model to automatically segment the labeled data and obtain the segmentation mask;

[0087] Step 3: The data in Step 2 is preprocessed, such as dilation, to fill the small black holes in the segmented tooth area, and then corrosion and connected domain threshold are used to remove the mis-positioned area to obtain more complete three-dimensional tooth mask information;

[0088] Step 4: The annotator makes a simple and quick partial correction to the data in Step 3, completing the annotation of the current oral CT image and storing it in the database;

[0089] Step 5: Since the data in the database has been updated, the constructed 3D semantic segmentation artificial neural network can be quickly trained again, and the weight parameters in the model can be automatically fine-tuned and optimized to train a model with better performance;

[0090] Step6: Repeat the above steps.

[0091] After the data annotation is completed, the present invention builds a first neural network, a second neural network and a third neural network for training a tooth instance segmentation model of oral CT images. In order to quickly train the first neural network, the second neural network and the third neural network, the present invention designs a fast training method that does not require a pre-trained model on a large data set.

[0092] In order to quickly train the first neural network, the second neural network, and the third neural network without a pre-trained model, the teeth in each oral CT image in the database are first numbered. Under normal circumstances, the number of teeth is between 28 and 32, but there may be missing teeth, supernumerary teeth, deciduous teeth, etc. The number of teeth varies greatly in different situations. We number the teeth in each oral CT image according to the tooth position, respectively ys1, ys2, ys3, ..., zs1, zs2, zs2, ..., dsy1, dsy2, dsy3..., representing the first tooth on the upper right, the second tooth on the upper right, the third tooth on the upper right, ..., the first tooth on the upper left, the second tooth on the upper left, the third tooth on the upper left, ..., the first supernumerary tooth, the second supernumerary tooth, the third supernumerary tooth, etc., and so on. By default, the number of all labels does not exceed 50. The following takes the training of the first neural network and the second neural network as an example to illustrate the training process of the neural network:

[0093] As an example, the training steps for the first neural network are as follows:

[0094] B1, obtaining a first training database including a plurality of training images, wherein the entire tooth region has been 3D labeled in each training image;

[0095] The 3D annotation of the whole tooth area can be obtained based on the annotation of a single tooth, and preferably, the smallest circumscribed frame obtained by annotating all single teeth is annotated as the whole tooth area.

[0096] Specifically, the training images can be oral CT images of various patients taken by different models, different fields of view, and different photographers, which are obtained from the database of the established medical image management system, thereby ensuring the reliability, richness, and data volume of the data.

[0097] B2, select data from the first training database as the first training set and the first validation set respectively;

[0098] Specifically, before step B2, the weight parameters of the first neural network may be initialized using normal distribution.

[0099] B3, iteratively training the first neural network model according to the first training set, and verifying the model obtained in each round of iteration according to the first verification set;

[0100] The first neural network model is an initial neural network model used to train the first neural network.

[0101] B4. When the verification result satisfies the first preset condition, or the number of iterations satisfies the second preset condition, select data from the remaining training data in the first training database to update the first training set and the first verification set respectively, and return to step B3, wherein the number of currently selected training data is k multiplied by the number of training data selected last time, and k is the update coefficient; otherwise, the training is terminated.

[0102] Among them, k can be selected according to the difficulty of the task, for example, it can be 2; the above iteration cutoff condition can be that all training data in the first training database participate in the iterative training.

[0103] Therefore, a small amount of data sets can be used to quickly train the parameters of the underlying network, which can efficiently learn the underlying detailed features and then gradually improve them to speed up convergence.

[0104] As an example, the training steps for the second neural network are as follows:

[0105] C1, obtaining a second training database including a plurality of training images, wherein a single tooth region has been 3D labeled in each training image;

[0106] C2, select data from the second training database as the second training set and the second validation set;

[0107] The 3D frame of a single tooth area can be composed of the minimum circumscribed rectangle of a single tooth annotation.

[0108] Specifically, before step C2, the weight parameters of the second neural network can be initialized using normal distribution. The tooth numbers are randomly divided into n1 groups, where the value of n1 can be 8. For example, the first group corresponds to teeth numbered 1, 2, 5, and 7, the second group corresponds to teeth numbered 3, 4, 6, and 8, and so on, and all groups cover all numbers. Among them, the first time, the numbered teeth corresponding to the first group can be selected.

[0109] C3, iteratively training the second neural network model according to the second training set, and verifying the model obtained in each round of iteration according to the second verification set;

[0110] The second neural network model is an initial neural network model used to train the second neural network.

[0111] C4, when the verification result satisfies the third preset condition, or the number of iterations satisfies the fourth preset condition, respectively select data from the remaining training data in the second training database to respectively update the second training set and the second verification set, and return to step C3; otherwise, terminate the training;

[0112] The above iteration cutoff condition may be that all training data in the second training database participate in the iterative training.

[0113] Therefore, the network can learn the basic features of teeth simply and quickly without pre-training weights, reducing the learning difficulty and accelerating convergence.

[0114] In one embodiment of the present invention, the tooth segmentation method may further include: determining whether there is mispositioning or missed positioning in a single tooth area; if so, correcting the single tooth area according to the 3D positioning frame corresponding to the mispositioning or missed positioning to correct the tooth data.

[0115] Among them, correcting the tooth data may include: reusing the pre-trained third neural network to perform mask segmentation of the single tooth area according to the corrected single tooth area, to obtain the tooth data of each tooth, and to replace the previously obtained tooth data.

[0116] Optionally, if the root canal data is also corrected, the pre-trained third neural network can be reused to perform mask segmentation of the root canal area according to the corrected single tooth area to obtain the root canal data of each tooth and replace the previously obtained root canal data.

[0117] Specifically, the first neural network, the second neural network, and the third neural network trained by big data have good performance in 3D tooth positioning and 3D segmentation of tooth root canals, but the output of the neural network is often uncontrollable. For very extreme data, there may be missed or false detection of teeth, and the output results are very dependent on the settings of thresholds such as Nms (Non-maximum suppression). In order to enhance the robustness of the model and make it insensitive to threshold settings, the output of the neural network can be further processed to accurately locate and identify the tooth parts in the three-dimensional oral CT images for any data. The following is an explanation from the two processing directions of mislocation and missed detection. Adjusting the Nms threshold improves the detection rate. In this case, combined with the following settings of mislocation and missed positioning, the recognition accuracy is improved under the condition of a higher detection rate.

[0118] In the case of mispositioning, these rare mispositionings can be removed through effective post-processing, which is very necessary in the oral field, because a mispositioning in the oral field may bring a greater impact.

[0119] Due to the effect of the first neural network, the algorithm model of the present invention will not have mispositioning outside the tooth area, and will only have a very small number of mispositioning when the teeth are arranged very closely or disorderly. Therefore, by combining the tooth segmentation results to assist in eliminating mispositioning in the positioning results of the first neural network and the second neural network, the accuracy of the results of the first neural network and the second neural network is improved, thereby forming a mutually reinforcing cycle. The specific process can be as follows: Fig.11 shown.

[0120] As an example, Fig.12 As shown, for the case of an erroneous positioning frame positioned between two teeth. Determining whether there is erroneous positioning in a single tooth region may include: counting the segmentation faults according to the tooth data to obtain the number of segmentation faults; if the number of segmentation faults is greater than a first threshold, it is determined that there is erroneous positioning in the single tooth region; wherein, correcting the single tooth region according to the 3D positioning frame corresponding to the erroneous positioning to correct the tooth data may include: removing the 3D positioning frame corresponding to the segmentation fault, and performing mask segmentation on the corrected single tooth region using a pre-trained third neural network to obtain tooth data of the corresponding tooth.

[0121] Specifically, for this type of mispositioning, the teeth in the 3D positioning frame can be segmented. Since there is a bite bar (made of silicone with a small CT value) between the two teeth, there will be a gap between the teeth, so there will be a non-tooth area in the middle of the segmentation result, that is, there will be a fault in the segmentation, such as Fig.13As shown. The middle segmentation fault (corresponding to the non-tooth area mentioned above) can be counted. For example, in a single positioning frame, whether the tooth area on each slice changes from existence to non-existence and then to existence in the vertical direction. If so, there is a segmentation fault and it is counted once. If the count value exceeds the threshold value n, it is judged as a mispositioning. In practice, n can be set to 1 to ensure that all positioning frames with segmentation faults are found. At the same time, the detection threshold can also be lowered to ensure that almost all teeth are detected, but at the same time the mispositioning rate will be increased. By setting the n threshold to 1, that is, greater than or equal to 1, it means that it is a mispositioning frame. At this time, the positioning frame can be eliminated, thereby ensuring the effectiveness and strictness of the elimination, while ensuring the detection rate. Among them, the detection threshold is the threshold that controls the detection process. Within a certain range, the larger the detection threshold, the lower the detection rate and the lower the mispositioning rate; conversely, the higher the detection rate, the higher the mispositioning rate.

[0122] As another example, Fig.14 As shown, for the case of a mispositioning frame on a single tooth region. Determining whether there is mispositioning in a single tooth region may include: determining whether there is a 3D positioning frame that meets the mispositioning condition based on tooth data, wherein if there are some slices of the 3D positioning frame that do not have tooth segmentation results, then it is determined that the 3D positioning frame meets the mispositioning condition; if so, the 3D positioning frame is adjusted to obtain a correct 3D positioning frame. Of course, if the segmentation result includes root canal data, mispositioning may also be determined based on the root canal data.

[0123] In the tooth positioning of the present invention, the following may occur: Fig.14 Mispositioning shown: Part of the tooth is framed.

[0124] For the 3D tooth root canal segmentation algorithm of the present invention, the current mis-positioned frame is first treated as the correct 3D positioning frame for segmentation. Since the correct 3D positioning frame will by default just frame the current tooth, that is, the minimum circumscribed 3D rectangular frame, the segmentation result has the segmentation result of the tooth part in each slice in the height direction, such as Fig.15 As shown in the figure: the box on the lower left is a correctly identified box, and each slice of the box has segmentation results in the height direction, while the box on the upper right is a mis-positioned box, and only has segmentation and recognition results in some areas. Therefore, the ratio of the segmentation result area of ​​each box to the height of the entire box can be calculated, and the above-mentioned mis-positioned boxes can be effectively removed by setting a threshold. Specifically, the height of the tooth area in the box is divided by the height of the positioning box to get the ratio, and the threshold is set according to experience or test results to exclude the positioning box with a relatively small ratio. A small ratio can be considered as a mis-positioning.

[0125] As another example, determining whether there is a missed location in a single tooth area may include: calculating the gap between two adjacent teeth based on the single tooth area (if the distance between the two adjacent teeth is too large, it means that there may be other teeth in the middle that have not been identified and need to be searched again), and determining whether a missed location search is needed based on the gap and size threshold; if necessary, dividing the gap position into grids, such as into 2×3 grids, and searching with each grid as the center of a search box, wherein the size of the search box is the size mean; using a third neural network to segment the area corresponding to the search box, and determining whether there is a missed location based on the segmentation result, specifically: if the tooth area is segmented, it is considered that there is a missed identification.

[0126] In this example, the size average of all 3D positioning boxes corresponding to a single tooth region may also be calculated; and the size of the search box may be set to the size average.

[0127] Specifically, for some extreme data, such as very small or only a little crown data, the above algorithm may cause missed detection. Taking Figure 16 (a) and Figure 16 (b) as examples, in order to find out whether there is missed detection on the right, first obtain the center position of the rightmost positioning frame, and then use the horizontal axis coordinate of the center position as a reference to perform a greedy search of different scale frames on its right side. In order to ensure the search efficiency, the horizontal coordinate does not exceed the upper and lower edges of the positioning frame, and the vertical coordinate does not exceed 1.5 times the width of the positioning frame. The scale type can be set to multiple types. Then the search is performed. The search method is to draw a grid in a certain area, and search with the center point of each grid as the center point of the search box. Each time a box is searched, the 3D data of the corresponding area is taken, and the segmentation recognition of the third neural network is performed. The recognition result is used to determine whether there is a tooth. If so, the position relationship between the search box and the rightmost positioning box is determined to confirm whether it is retained, otherwise continue the search. The specific process is as described above. Among them, the rightmost positioning box is the right boundary, and the search box should search on its left. The position relationship between the search box and the right positioning box is determined as follows: calculate the iou (i.e. intersection-over-union) value of the search box and the positioning box. If the value is less than a certain threshold (generally set to 0.5), it means that there are two teeth, and the search box is retained.

[0128] Therefore, the existing extreme missed detection situations can be effectively searched out, which greatly improves the robustness of the detection of the first neural network and the second neural network. This method is also applicable to the missed detection judgment of the middle large teeth.

[0129] In summary, the tooth segmentation method of the embodiment of the present invention first uses a pre-trained first neural network to locate the overall tooth area in the oral CT image to be segmented, and then uses a pre-trained second neural network to locate a single tooth area within a preset range in the overall tooth area, and finally uses a pre-trained third neural network to perform mask segmentation on the single tooth area to obtain the tooth data of the corresponding tooth. In this process, the overall tooth area is first identified, which can not only reduce the data input amount of the second neural network, but also reduce the misidentification of areas outside the overall tooth area. At the same time, compared with the segmentation of a single tooth by inputting the entire oral CT image into one network, inputting a single tooth area into the third neural network for segmentation processing can greatly reduce the data volume and complexity of the input data, and improve the accuracy of tooth segmentation. In this way, effective and accurate segmentation of teeth in oral CT images can be achieved. In addition, the entire tooth segmentation process is carried out in stages using different neural networks, which can reduce the complexity of the network at each stage and the difficulty of achieving the corresponding task.

[0130] Furthermore, after obtaining the tooth body data of each tooth, tooth point cloud surface data can be extracted based on the tooth body data of each tooth to obtain a better tooth segmentation effect, which is convenient for subsequent tooth arrangement processing.

[0131] To this end, the present invention proposes a method for acquiring tooth surface data. Fig.17 As shown, the tooth surface data acquisition method includes:

[0132] S01, acquiring tooth data according to the above-mentioned tooth segmentation method.

[0133] S02, perform three-dimensional Gaussian smoothing on the tooth data.

[0134] S03, extracting tooth mesh data from the smoothed tooth data using the Marching Cube algorithm.

[0135] S04, performing three-dimensional Laplace smoothing on the extracted tooth mesh data.

[0136] S05, determining and removing redundant points based on the smoothed tooth mesh data and the sharpness of the mesh boundary, and splicing the remaining points into a surface to obtain tooth point cloud surface data.

[0137] Specifically, the corresponding tooth voxel data can be obtained according to the tooth body data of each tooth; the tooth voxel data can be three-dimensionally Gaussian smoothed; the tooth grayscale threshold is calculated according to the oral CT image to be segmented, and the tooth mesh data is extracted from the smoothed tooth body data through the Marching Cube algorithm; the extracted tooth mesh data is three-dimensionally Laplace smoothed; redundant points are determined and removed according to the smoothed tooth mesh data, and the remaining points are spliced ​​into a surface to obtain the tooth point cloud surface data.

[0138] Optionally, before performing 3D Laplace smoothing on the tooth mesh data, the tooth mesh data may be downsampled, and then 3D Laplace smoothing may be performed on the downsampled tooth mesh data, so as to reduce the signal sampling rate, reduce the computational complexity, and improve the processing speed.

[0139] As an example, calculating the tooth grayscale threshold based on the oral CT image to be segmented may include: using the Otsu method to calculate the first grayscale threshold of the boundary between air and facial soft tissue in the oral CT image to be segmented; based on the first grayscale threshold, using the Otsu method to calculate the tooth grayscale threshold of the boundary between soft tissue and bone in the oral CT image to be segmented.

[0140] As an example, determining redundant points based on smoothed tooth mesh data may include: for each vertex of each boundary of the smoothed tooth mesh data, calculating the angle between the normal vectors of each two adjacent mesh faces containing the corresponding vertex; calculating the maximum angle corresponding to the corresponding vertex based on the angle of each vertex; determining the adjustment item of the edge folding cost corresponding to the corresponding vertex based on the maximum angle corresponding to each vertex; and performing simplification operations using the edge folding algorithm and the adjustment items of the edge folding costs of each vertex to obtain redundant points.

[0141] In this example, determining the adjustment item of the edge folding cost corresponding to each vertex according to the maximum angle corresponding to each vertex may include: calculating the number of adjacent vertices of each vertex; determining the adjustment item of the edge folding cost corresponding to the corresponding vertex according to the maximum angle corresponding to each vertex and the number of adjacent vertices of each adjacent vertex of the corresponding vertex. For example, the maximum angle θ corresponding to each vertex and the number of adjacent vertices m of each adjacent vertex i of the corresponding vertex may be used. i , determine the adjustment term Δ″(v)=θ*∑cosθ / m of the edge collapse cost corresponding to the corresponding vertex i It should be noted that it is also possible to transform cosθ into a series of combinations of sine and / or cosine of θ or its multiples. For example, according to cos 2 θ+sin 2 θ=1, and the cosine of θ in the formula is converted to the sine of θ.

[0142] Among them, the adjacent vertex of a vertex is another vertex that contains all the boundaries of the vertex. The number of adjacent vertices of a vertex is equal to the number of all the boundaries containing the vertex.

[0143] For all the angles corresponding to each vertex, the maximum angle corresponding to the vertex can be determined by building a minimum heap of the angles corresponding to the corresponding vertices.

[0144] The edge folding algorithm and the adjustment item of the edge folding cost of each vertex are used to simplify the operation. Specifically, the sum of the preset quadratic error measure and the adjustment item of the edge folding cost of each vertex can be used as the edge folding cost of the corresponding vertex for simplifying the operation. Of course, the adjustment basis of the edge folding cost can also be determined according to other existing algorithms, which will not be elaborated here.

[0145] The default quadratic error metric can be obtained from the paper Surface simplification using quadric error metrics, which was published in Proc. of the Computer Graphics by Garland M and Heckbert PS in 1997. Specifically,

[0146]

[0147] Where Δ′(v) is the quadratic error measure, v = [v x v y v z 1] T , p represents a plane in three-dimensional space

[0148] In summary, the tooth surface data acquisition method of the embodiment of the present invention first acquires tooth data according to the above-mentioned tooth segmentation method, then performs three-dimensional Gaussian smoothing on the tooth data, extracts tooth mesh data from the smoothed tooth data through the Marching Cube algorithm, and performs three-dimensional Laplace smoothing on the extracted tooth mesh data; finally, redundant points are determined and removed based on the smoothed tooth mesh data and the sharpness of the mesh boundary, and the remaining points are spliced ​​into surfaces to obtain tooth point cloud surface data. This process acquires tooth surface data based on an accurate tooth segmentation method, and accurate tooth surface data can be obtained. At the same time, before and after the tooth mesh data is extracted using the Marching Cube algorithm, different forms of smoothing are performed on the volume data and the surface data, respectively, to ensure the accuracy of data processing at each stage.

[0149] Please refer to the following Figure 18-25 The invention describes a teeth arrangement method, a tooth orthodontic simulation method, a storage medium and an oral CT device according to an embodiment of the invention.

[0150] Fig.18 FIG. 1 is a flow chart of a tooth arrangement method according to an embodiment of the present invention. Fig.18 As shown, the tooth arrangement method includes the following steps:

[0151] S11, obtaining oral CT images.

[0152] Specifically, the oral CT image can be an oral CT image obtained from a database of a medical image management system, such as Figure 2 shown.

[0153] S12, segmenting the oral CT image to obtain a first positioning frame of each tooth.

[0154] Specifically, the existing tooth segmentation processing method can be used to obtain the first positioning frame of each tooth, which can be a 2D positioning frame; the above-mentioned tooth segmentation method can also be used to obtain the first positioning frame of each tooth, which can be a 3D positioning frame.

[0155] S13, determining a plurality of tooth arrangement reference points according to the first positioning frame.

[0156] Specifically, the center coordinates of each first positioning frame can be calculated to obtain the corresponding first center coordinates, and the second positioning frame of the entire tooth area can be obtained based on the first positioning frame; and then multiple tooth row reference points can be determined based on the first center coordinates and the second positioning frame. For example, a positioning frame that can just include all the first positioning frames can be obtained and used as the second positioning frame.

[0157] As a feasible implementation, the second positioning frame can be as follows Fig.19 As shown, the number of the above-mentioned tooth arrangement reference points can be 3. In this case, the multiple tooth arrangement reference points include a center reference point, a left reference point, and a right reference point. In this example, the specific steps of determining the multiple tooth arrangement reference points according to the first positioning frame can be as follows:

[0158] Step 1: According to the first center coordinates and the second positioning frame, determine the upper left first positioning frame and the upper right first positioning frame.

[0159] Specifically, taking the first positioning frame as a 3D positioning frame as an example, a three-dimensional orthogonal coordinate system (such as Fig. 20 As shown, Fig. 20Only the x and y axes in the three-dimensional coordinate system are shown, and the z axis is perpendicular to the x and y axes and inwards), and the positional relationship between the center coordinates of each first positioning frame and the second positioning frame in the coordinate system is obtained, and the upper left first positioning frame (the y coordinate value in the first center coordinate is the smallest and is closest to the left boundary of the second positioning frame) and the upper right first positioning frame (the y coordinate value in the first center coordinate is the smallest and is closest to the right boundary of the second positioning frame) are determined according to the positional relationship, such as Fig. 20 shown.

[0160] It should be noted that Fig.19 and Fig. 20 What is shown is a panoramic view of the teeth, which is illustrated in the panoramic view of the teeth. In actual calculation, the spatial coordinates of the three-dimensional tooth model can be referred to for calculation. In one embodiment, the ordinate direction can be the vertical direction, and the abscissa direction can be Fig.10 In the left and right directions, the third coordinate direction can be perpendicular to the horizontal and vertical coordinate directions.

[0161] Step 2: Calculate the center coordinates of the second positioning frame to obtain the second center coordinates.

[0162] Step 3: Determine the center reference point based on the second center coordinates.

[0163] Specifically, a first positioning frame that meets the preset conditions can be selected based on the second center coordinates. The method includes: determining the minimum and maximum values ​​of the horizontal coordinates of each first positioning frame; comparing the minimum and maximum values ​​of the horizontal coordinates with the horizontal coordinates in the second center coordinates; selecting one from the first positioning frames that meet the preset conditions, wherein the preset conditions are that the minimum value of the horizontal coordinate is less than or equal to the horizontal coordinate in the second center coordinates, and the maximum value of the horizontal coordinate is greater than or equal to the horizontal coordinate in the second center coordinates; and determining the center reference point based on the selected first positioning frame and the second center coordinates.

[0164] Wherein, according to the selected first positioning frame and the second center coordinate, determining the center reference point may include any one of the following methods 1, 2 and 3:

[0165] Method 1: Determine the minimum and maximum values ​​of the ordinate of the selected first positioning frame; compare the minimum and maximum values ​​of the ordinate with the ordinate in the second center coordinate; use the value of the minimum and maximum values ​​of the ordinate that is closer to the ordinate in the second center coordinate as the ordinate of the center reference point; use the abscissa in the second center coordinate as the abscissa of the center reference point, i.e., the center reference point Pc (see Fig.21) is (the horizontal coordinate in the second center coordinate, the value of the minimum and maximum values ​​of the vertical coordinate that is closer to the vertical coordinate in the second center coordinate, and the vertical coordinate in the second center coordinate). Method 2: directly determine the center reference point as the center point of the second positioning frame; Method 3: when the center reference point cannot be determined in method 1, determine the center reference point as the center point of the second positioning frame.

[0166] Step 4: Extend the lower edge point of the first positioning frame at the upper left to the left along the horizontal axis direction and intersect with the second positioning frame, and use the intersection as the left reference point; and extend the lower edge point of the first positioning frame at the upper right to the right along the horizontal axis direction and intersect with the second positioning frame, and use the intersection as the right reference point. Fig. 22 As shown, the left reference point is point P1, and the right reference point is point P2.

[0167] As another feasible implementation manner, multiple tooth row reference points include a center reference point, a left reference point, a right reference point, and a left reference point and / or a right reference point. According to the first center coordinate and the second positioning frame, multiple tooth row reference points are determined, and it may also include: determining a longitudinal straight line between the center reference point and the left reference point, and determining an intersection point from the intersection points of the longitudinal straight line and the first positioning frame as the left reference point; and / or determining a longitudinal straight line between the center reference point and the right reference point, and determining an intersection point from the intersection points of the longitudinal straight line and the first positioning frame as the left reference point.

[0168] Specifically, the method for determining the center reference point, the left reference point, and the right reference point can refer to the above-mentioned implementation method. After obtaining the center reference point and the left reference point, a point between the center reference point and the left reference point and on the line connecting the two points (such as the midpoint of the line) can be randomly selected according to the horizontal coordinates of the center reference point and the left reference point, and then a vertical straight line passing through the point is determined, and the vertical straight line has an intersection with the first positioning frame. If the number of intersections is two, any one of the two intersections can be selected as the left reference point; if the number of intersections is more than two, the distance between the intersection and the corresponding center point of the first positioning frame can be further calculated, and the intersection corresponding to the minimum distance is used as the left reference point to improve the sorting effect. Similarly, the right reference point can be obtained.

[0169] S14, sorting all first positioning frames according to multiple tooth arrangement reference points to obtain a sorting result.

[0170] Specifically, the target equation of the straight line where every two adjacent row of teeth reference points are located can be determined, and the horizontal coordinate of the first positioning frame between the corresponding two row of teeth reference points can be substituted into the corresponding target equation to obtain the vertical coordinate reference value; based on the size of the vertical coordinate reference value and the vertical coordinate of the center reference point, determine whether the tooth corresponding to the corresponding first positioning frame is the upper row of teeth or the lower row of teeth.

[0171] The upper and lower teeth are sorted from left to right and along the dental arch line, and the tooth position numbers are marked. Specifically, the upper and lower teeth can be sorted according to the horizontal coordinate of the first positioning frame and the coordinate of the third dimension, and the tooth position numbers are marked.

[0172] For ease of understanding, the three tooth arrangement reference points, namely the center reference point Pc, the left reference point P1 and the right reference point P2, are taken as an example to illustrate step S14. Specifically, determine the first equation of the straight line where the center reference point and the left reference point are located, and the second equation of the straight line where the center reference point and the right reference point are located; substitute the horizontal coordinate in each first center coordinate on the left side of the center reference point into the first equation to obtain the first vertical coordinate reference value, and substitute the horizontal coordinate in each first center coordinate on the right side of the center reference point into the second equation to obtain the second vertical coordinate reference value; respectively compare the first vertical coordinate reference value and the second vertical coordinate reference value with the vertical coordinate of the center reference point; if the first vertical coordinate reference value is less than the vertical coordinate of the center reference point, determine that the corresponding first positioning frame is the upper row of teeth positioning frame, otherwise it is the lower row of teeth positioning frame, and if the second vertical coordinate reference value is less than the vertical coordinate of the center reference point, determine that the corresponding first positioning frame is the upper row of teeth positioning frame, otherwise it is the lower row of teeth positioning frame; sort all upper row of teeth positioning frames according to the horizontal coordinate in the first center coordinate of the upper row of teeth positioning frame, and sort all lower row of teeth positioning frames according to the horizontal coordinate in the first center coordinate of the lower row of teeth positioning frame. For example, the first equation of the straight line where the center reference point and the left reference point are located can be determined to be y=k1x+b1, and the second equation of the straight line where the center reference point and the right reference point are located can be determined to be y=k2x+b2; the horizontal coordinate x in each first center coordinate on the left side of the center reference point is i Substitute into the first equation and get the corresponding first ordinate reference value y i , and the horizontal coordinate x in each first center coordinate on the right side of the center reference point j Substitute into the second equation and get the second ordinate reference value y j ; Then, the first ordinate reference value, the second ordinate reference value and the ordinate of the center reference point are compared respectively. Thus, the upper and lower teeth can be accurately segmented, thereby obtaining a sorting result, such as Fig.23 shown.

[0173] S15, identifying the tooth type of each tooth using a pre-trained classification model.

[0174] It should be noted that before using the pre-trained classification model to identify the tooth type of each tooth, it is also necessary to determine whether there are any missing teeth in the above sorting results; if so, the above sorting results need to be corrected. The specific steps are as follows:

[0175] A1, determine whether there are missing teeth based on the sorting results.

[0176] Specifically, it is possible to directly determine whether there is a missing tooth in the sorting result based on the center coordinates of each first positioning frame in the sorting result. For example, the horizontal coordinates of the center coordinates of each first positioning frame can be obtained, and then if it is determined that there are two first definition frames, and the difference between the horizontal coordinates of the center coordinates corresponding to the two first definition frames is greater than the first preset distance threshold and less than the second preset distance threshold, it is determined that there is a missing tooth.

[0177] A2: If it exists, the central incisor position is determined according to the sorting result, and the missing tooth positioning frame is determined according to the first positioning frames on both sides of the missing tooth position.

[0178] Specifically, the horizontal coordinates of the left and right boundaries of the first positioning frame on both sides of the missing tooth position can be obtained, and then the horizontal coordinate difference of the left and right boundaries of the first positioning frame on the left side of the missing tooth position and the horizontal coordinate difference of the left and right boundaries of the first positioning frame on the right side of the missing tooth position can be obtained, so as to determine the size of the missing tooth positioning frame according to the average of the above two horizontal coordinate differences.

[0179] A3, determine the position of the missing teeth positioning frame according to the position of the central incisor, and modify the sorting result according to the position and the missing teeth positioning frame.

[0180] S16, determining the tooth arrangement result according to the sorting result and the tooth type.

[0181] In summary, the tooth arrangement method of the embodiment of the present invention can realize fully automatic tooth arrangement based on oral CT images. The tooth arrangement results identify various types of teeth on the upper left, lower left, upper right, and lower right, such as central incisors, lateral incisors, canines, first premolars, second premolars, first permanent molars, second permanent molars, and third permanent molars, such as upper left central incisors, upper left lateral incisors, etc.

[0182] Fig.24 It is a flow chart of a tooth orthodontic simulation method according to an embodiment of the present invention.

[0183] like Fig.24 As shown, the orthodontic simulation method may include the following steps:

[0184] S241, determining the dental arch line of the model to be corrected, and generating a target model according to the dental arch line and a pre-stored standard tooth model.

[0185] The corresponding teeth in the target model and the model to be corrected are identified according to the teeth arrangement result of the model to be corrected, and the model to be corrected is obtained by teeth arrangement using the above-mentioned teeth arrangement method.

[0186] Specifically, since each person's oral condition is different, there may be situations such as missing teeth and supernumerary teeth. Therefore, in order to further accurately locate the tooth position of each tooth (such as the upper left central incisor, the lower right first molar, etc.), the three-dimensional tooth model corresponding to the tooth arrangement result can be input as the model to be corrected into the pre-trained neural network model, and the tooth body data and root canal data of each tooth mentioned above can be input into the above neural network model; thereby, the neural network model can be used for recognition to obtain a recognition result. Among them, the recognition result includes the correspondence between the target model and the teeth in the model to be corrected, so as to determine the tooth position of each tooth and whether each tooth is a tooth to be corrected and the type of deformation of the tooth to be corrected; the above pre-trained classification model is a neural network model, including a convolutional layer, a pooling layer, a normalization layer and a fully connected layer; the neural network model architecture can be as follows Fig.25 shown.

[0187] S242: According to each tooth of the target model, a first adjustment is made to the corresponding tooth in the model to be corrected, so as to achieve orthodontic simulation of the dentition in the model to be corrected.

[0188] The first adjustment may include translation, rotation and twisting. Moving a tooth in a non-tooth position to a corresponding tooth position is a translation operation; rotating a tooth in a tooth position from an inclined position to a position perpendicular to the corresponding tooth position is a rotation operation; rotating a tooth in a corresponding tooth position and perpendicular to the tooth position but at an angle to the positive direction to the positive direction is a twisting operation, which can be understood as a rotation to ensure that the tooth position remains unchanged.

[0189] In order to improve the orthodontic effect, a second adjustment may be performed on the corresponding tooth according to the distance between the corresponding tooth and its adjacent teeth after the first adjustment.

[0190] The second adjustment may include translation.

[0191] In order to further improve the orthodontic effect, the minimum distance between the root of the tooth to be adjusted and the jaw surface can also be calculated; based on the minimum distance, it is determined whether there is an intersection between the root and the jaw surface; if there is an intersection, a fourth adjustment is performed on the corresponding tooth within the first preset position range.

[0192] In one embodiment of the present invention, after the adjustment of all single teeth is completed, the first distance between the crown surfaces of the upper and lower teeth of the mandibles can also be calculated; according to the first distance and the upper and lower relative positions of the crown surfaces of the teeth of the mandibles, the vertical positions of the teeth of the mandibles can be adjusted within the second preset position range. In this way, a better orthodontic effect can be obtained. If the first distance exceeds or is equal to a certain threshold, and the crown surface of the upper teeth of the teeth of the mandibles is located below the crown surface of the lower teeth, the vertical position of the teeth of the mandibles needs to be adjusted within the second preset position range. In the specific calculation, the ordinate of the crown surface of the lower teeth can be calculated minus the ordinate of the crown surface of the upper teeth. If it is a negative value and exceeds or is equal to a threshold value such as 0, the vertical position of the teeth of the mandibles can be adjusted within the second preset position range.

[0193] Specifically, the long axis of each tooth can be obtained according to the recognition result, and the corresponding impedance center can be obtained according to the long axis of the tooth; then the tooth-jaw data of the tooth in the oral CT image can be measured; thus, the teeth can be aligned according to the recognition result, the impedance center and the tooth-jaw data. The above-mentioned tooth-jaw data includes at least one of the tooth width, Bolton index, dental arch width, Spee curve depth, and midline deviation. In this way, it is possible to correct the tooth in the oral CT image according to the type of distortion of the tooth to be corrected.

[0194] The above-mentioned pre-stored standard tooth model is a normal and neat dentition model, and the above-mentioned tooth types include central incisors, lateral incisors, canines, first premolars, second premolars, first permanent molars, second permanent molars and third permanent molars. The central incisor is used as the dividing line, and the jaw is divided into right maxillary, left maxillary, left mandibular and right mandibular. The steps of tooth orthodontics specifically include:

[0195] Step 1: When aligning the teeth of the upper right jaw, align them in the order of central incisor, lateral incisor, canine, first premolar, second premolar, first permanent molar, second permanent molar and third permanent molar.

[0196] Step 2: When the tooth axis is located on the dental arch line in the dental data, determine the teeth to be corrected and their adjustment parameters according to the impedance center and the inclination law of the corresponding teeth arrangement in the pre-stored standard tooth model, and adjust the teeth to be corrected according to the adjustment parameters to align the teeth of the right maxilla. The adjustment parameters include at least one of the translation distance, rotation angle and torsion angle.

[0197] Specifically, the above translation distance includes horizontal translation distance, anterior-posterior translation distance, and vertical translation distance; the above rotation angle includes buccal-lingual rotation angle and mesio-distal rotation angle. It should be noted that if the angle between the tangent of the anterior teeth and the tangent of the dental arch line is greater than a preset threshold (such as greater than 2-3°), it is considered that the torsion angle needs to be calculated.

[0198] Step 3: Align the dentition of the left maxillary, right mandibular and left mandibular in the same way as the dentition of the right maxillary.

[0199] Step 4: Calculate the first distance between the crown surfaces of the upper and lower teeth.

[0200] Step 5: If there is an opposing tooth whose first distance is a negative value, fine-tune the vertical position of the opposing tooth so that the first distance of the opposing tooth is a positive value.

[0201] Step 6: Calculate the minimum distance between the tooth root and the jaw surface.

[0202] Step 7: If the minimum distance is a negative value, fine-tune the overall tooth torque to make the minimum distance a positive value while ensuring that the tooth axis is on the dental arch line and the occlusal relationship with the opposing teeth is normal.

[0203] As a feasible implementation, the third adjustment can be made to the corresponding teeth of the orthodontic model directly according to the adjustment parameters input by the user. The user can be an experienced dentist, and the adjustment parameters input by the dentist can be determined by the dentist based on experience; the third adjustment can include translation, rotation and twist.

[0204] Optionally, after aligning the dentition, the above-mentioned adjustments to the teeth can also be obtained, and the adjusted teeth and adjustment parameters can be printed in the form of a report, so that the actual treatment strategy can be determined according to the teeth alignment results.

[0205] It should be noted that, for ease of understanding, the identification frames used in the drawings of the embodiments of the present invention are all two-dimensional identification frames, but are actually three-dimensional identification frames.

[0206] In summary, the orthodontic simulation method of the embodiment of the present invention can adjust the teeth based on the tooth arrangement results of the oral CT images, thereby providing a reference for actual treatment.

[0207] Furthermore, the present invention provides a computer-readable storage medium.

[0208] In an embodiment of the present invention, a computer program is stored on a computer-readable storage medium, and when the computer program is executed by a processor, the above-mentioned teeth arrangement method or tooth orthodontic simulation method is implemented.

[0209] The computer readable storage medium of the embodiment of the present invention can realize fully automatic tooth arrangement based on oral CT images when the computer program stored thereon is executed by a processor. Furthermore, the adjustment of teeth for tooth arrangement in the oral CT images can be obtained, thereby providing a reference for actual treatment.

[0210] To achieve the above objectives, a fourth aspect of the present invention provides an oral CT device.

[0211] In an embodiment of the present invention, the oral CT device includes a memory, a processor, and a computer program stored in the memory. When the computer program is executed by the processor, the above-mentioned teeth arrangement method or tooth orthodontic simulation method is implemented.

[0212] The oral CT device of the embodiment of the present invention can realize fully automatic tooth arrangement based on oral CT images by implementing the above-mentioned tooth arrangement method, and can also obtain the adjustment of teeth for tooth arrangement in the oral CT images, thereby providing a reference for actual treatment.

[0213] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0214] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0215] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0216] In the description of the present invention, it is to be understood that the terms “center”, “longitudinal”, “lateral”, “length”, “width”, “thickness”, “up”, “down”, “front”, “back”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, “clockwise”, “counterclockwise”, “axial”, “radial”, “circumferential”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0217] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0218] In the present invention, unless otherwise clearly specified and limited, the terms "installed", "connected", "connected", "fixed" and the like should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements, unless otherwise clearly defined. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0219] In the present invention, unless otherwise clearly specified and limited, a first feature being "above" or "below" a second feature may mean that the first and second features are in direct contact, or the first and second features are in indirect contact through an intermediate medium. Moreover, a first feature being "above", "above" or "above" a second feature may mean that the first feature is directly above or obliquely above the second feature, or simply means that the first feature is higher in level than the second feature. A first feature being "below", "below" or "below" a second feature may mean that the first feature is directly below or obliquely below the second feature, or simply means that the first feature is lower in level than the second feature.

[0220] Although the embodiments of the present invention have been shown and described above, it is to be understood that the above embodiments are exemplary and are not to be construed as limitations of the present invention. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present invention.

Claims

1. A method for arranging teeth, characterized in that: The following steps are involved: Obtain oral CT images; Segmenting the oral CT image to obtain a first positioning frame of each tooth; Determine a plurality of tooth arrangement reference points according to the first positioning frame; Sorting all first positioning frames according to the multiple tooth arrangement reference points to obtain a sorting result; Identify the tooth type of each tooth using a pre-trained classification model; Determine a tooth arrangement result according to the sorting result and the tooth type; The step of determining a plurality of tooth arrangement reference points according to the first positioning frame includes: Calculate the center coordinates of each of the first positioning frames to obtain corresponding first center coordinates, and obtain a second positioning frame of the entire tooth area according to the first positioning frames; Determining the plurality of tooth arrangement reference points according to the first center coordinates and the second positioning frame; The multiple tooth arrangement reference points include a center reference point, a left reference point, and a right reference point, wherein determining the multiple tooth arrangement reference points according to the first center coordinates and the second positioning frame includes: Determine a first upper leftmost positioning frame and a first upper rightmost positioning frame according to the first center coordinates and the second positioning frame; Calculating the center coordinates of the second positioning frame to obtain second center coordinates; Determining the central reference point according to the second central coordinates; Extend the lower edge point of the upper leftmost first positioning frame to the left along the horizontal axis direction and intersect with the second positioning frame, and use the intersection as the left reference point; and extend the lower edge point of the upper rightmost first positioning frame to the right along the horizontal axis direction and intersect with the second positioning frame, and use the intersection as the right reference point.

2. The method for arranging teeth according to claim 1, characterized in that: The plurality of tooth row reference points include a center reference point, a left reference point, a right reference point, and a left reference point and / or a right reference point. The step of determining the plurality of tooth arrangement reference points according to the first center coordinates and the second positioning frame further includes: Determine a longitudinal straight line between the central reference point and the left reference point, and determine an intersection point from the intersection points of the longitudinal straight line and the first positioning frame as the left reference point; and / or, A longitudinal straight line between the center reference point and the right reference point is determined, and an intersection point is determined from the intersection points of the longitudinal straight line and the first positioning frame as the right reference point.

3. The method for arranging teeth according to claim 1, characterized in that: The step of determining the central reference point according to the second central coordinates includes: Determine the minimum and maximum values ​​of the horizontal coordinates of each of the first positioning frames; comparing the minimum and maximum values ​​of the abscissa with the abscissa in the second center coordinate; Select one from the first positioning box that meets the preset condition, wherein the preset condition is that the minimum value of the horizontal coordinate is less than or equal to the horizontal coordinate in the second center coordinate, and the maximum value of the horizontal coordinate is greater than or equal to the horizontal coordinate in the second center coordinate; The center reference point is determined according to the selected first positioning frame and the second center coordinates.

4. The method for arranging teeth according to claim 1, characterized in that: Determining the center reference point according to the selected first positioning frame and the second center coordinates includes any one of the following methods 1, 2, and 3: Method 1: Determine the minimum and maximum values ​​of the ordinates of the selected first positioning frame; The value of the minimum value and the maximum value of the longitudinal coordinate that is closer to the longitudinal coordinate in the second central coordinate is used as the longitudinal coordinate of the central reference point; Using the abscissa in the second center coordinate as the abscissa of the center reference point; Method 2: Directly determine the central reference point as the central point of the second positioning frame; Method 3: When the center reference point cannot be determined in the first method, the center reference point is determined to be the center point of the second positioning frame.

5. The method for arranging teeth according to any one of claims 2 to 4, characterized in that: The step of sorting the positions of all first positioning frames according to the multiple tooth arrangement reference points includes: Determine the target equation of the straight line where every two adjacent tooth row reference points are located, substitute the horizontal coordinate of the first positioning frame between the corresponding two tooth row reference points into the corresponding target equation, and obtain the vertical coordinate reference value; Determine whether the tooth corresponding to the corresponding first positioning frame is an upper row of teeth or a lower row of teeth according to the size of the ordinate reference value and the ordinate of the central reference point; The upper and lower rows of teeth are sorted according to the coordinates of the first positioning frame.

6. The method for arranging teeth according to claim 1, characterized in that: The method further comprises: Determine whether there is missing tooth according to the sorting result; If it exists, determining the position of the central incisor according to the sorting result, and determining the missing tooth positioning frame according to the first positioning frames on both sides of the missing tooth position; The position of the missing tooth positioning frame is determined according to the position of the central incisor, and the sorting result is corrected according to the position and the missing tooth positioning frame.

7. The method for arranging teeth according to claim 1, characterized in that: The segmenting process of the oral CT image to obtain a first positioning frame of each tooth includes: Using a pre-trained first neural network to locate the entire tooth area in the oral CT image; A single tooth region within a preset range in the entire tooth region is located using a pre-trained second neural network to obtain a first positioning frame.

8. A tooth orthodontic simulation method, characterized in that: The method comprises the following steps: Determine the dental arch line of the model to be corrected, and generate a target model according to the dental arch line and a pre-stored standard tooth model; According to each tooth of the target model, a first adjustment is made to the corresponding tooth in the model to be corrected to achieve orthodontic simulation of the dentition in the model to be corrected, wherein the target model and the corresponding teeth in the model to be corrected are identified based on the tooth arrangement result of the model to be corrected, and the model to be corrected is obtained by tooth arrangement according to the tooth arrangement method described in any one of claims 1-7.

9. The tooth orthodontic simulation method according to claim 8, characterized in that: The method further comprises: A second adjustment is performed on the corresponding tooth according to the distance between the corresponding tooth and its adjacent teeth after the first adjustment.

10. The orthodontic simulation method according to claim 8, characterized in that: The method further comprises: According to the adjustment parameters input by the user, a third adjustment is performed on the teeth corresponding to the model to be corrected.

11. The tooth orthodontic simulation method according to claim 8, characterized in that: The method further comprises: Calculate the minimum distance between the root of the tooth to be adjusted and the jaw surface; Determining whether the tooth root intersects with the jaw surface according to the minimum distance; If there is an intersection, a fourth adjustment is performed on the corresponding teeth within the first preset position range.

12. The tooth orthodontic simulation method according to any one of claims 8 to 11, characterized in that: After the adjustment of all individual teeth is completed, the method further includes: Calculate the first distance between the crown surfaces of the upper and lower teeth; According to the first distance and the upper and lower relative position of the crown surface of the opposing tooth, the vertical position of the opposing tooth is adjusted within the second preset position range.

13. The tooth orthodontic simulation method according to any one of claims 8 to 11, characterized in that: The first adjustment includes: translation, rotation and torsion; The second adjustment includes: translation; The third adjustment includes: translation, rotation and torsion.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the teeth arrangement method as described in any one of claims 1 to 7, or implements the teeth orthodontic simulation method as described in any one of claims 8 to 13.

15. An oral CT device, comprising a memory, a processor and a computer program stored in the memory, characterized in that: When the computer program is executed by a processor, it implements the teeth arrangement method as described in any one of claims 1 to 7, or implements the teeth orthodontic simulation method as described in any one of claims 8 to 13.

Images