Tooth segmentation method, tooth surface data acquisition method, and oral ct apparatus

Abstract

The application discloses a tooth segmentation method, a tooth surface data acquisition method, a storage medium and an oral CT device. The tooth segmentation method comprises the following steps: acquiring an oral CT image to be segmented; positioning a whole tooth region in the oral CT image to be segmented by using a first neural network which is pre-trained; positioning a single tooth region of a preset range in the whole tooth region by using a second neural network which is pre-trained; and performing mask segmentation on the single tooth region by using a third neural network which is pre-trained, so as to obtain tooth data of a corresponding tooth. Therefore, the tooth segmentation method can realize effective and accurate segmentation of teeth in the oral CT image.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of stomatology, and in particular to a tooth segmentation method, a tooth surface data acquisition method, a storage medium and an oral CT device. BACKGROUND

[0002] In the field of oral medicine, an oral CT (Computed Tomography) device can truly reflect the three-dimensional structure of the maxillofacial region, which helps doctors to develop more reasonable orthodontic plans. However, the CT tooth slice image has low brightness, poor contrast, many noise points, unclear boundaries, and complex structures, and individuality is diverse. Moreover, the CT is affected by artifacts caused by noise and non-uniform intensity, which makes the tooth image gray uneven, and thus the automatic segmentation of teeth is very difficult. Therefore, how to effectively segment the teeth in the tooth image is a key step to obtain tooth information.

[0003] At present, the tooth segmentation methods mainly include manual segmentation, adaptive threshold segmentation, morphological segmentation, and level set segmentation based on an active contour model. The manual segmentation has the disadvantages of requiring multiple people to participate, long segmentation time, and measurement accuracy being affected by the operator's operation. The adaptive threshold segmentation has problems of many noise points, blurred boundaries of adjacent teeth, and uneven tooth gray values, and is easily disturbed by non-effective information. The morphological segmentation is too sensitive to edges and often over-segmented. The level set segmentation based on the active contour model relies on relatively accurate prior knowledge and lacks local control of the evolving surface. SUMMARY

[0004] The present application aims to at least partially solve one of the problems in the related art. To this end, one object of the present application is to provide a tooth segmentation method to effectively and accurately segment teeth in an oral CT image.

[0005] A second object of the present application is to provide a computer-readable storage medium.

[0006] A third object of the present application is to provide an oral CT device.

[0007] To achieve the above objects, the first aspect of the present application provides a tooth segmentation method, comprising the following steps: acquiring an oral CT image to be segmented; locating a whole tooth region in the oral CT image to be segmented by using a pre-trained first neural network; locating a single tooth region of a preset range in the whole tooth region by using a pre-trained second neural network; and performing mask segmentation on the single tooth region by using a pre-trained third neural network to obtain tooth data of the corresponding tooth.

[0008] The tooth segmentation method of the embodiment of the present application uses the first neural network trained in advance to locate the overall tooth region in the to-be-segmented oral CT image, then uses the second neural network trained in advance to locate the single-tooth region of the preset range in the overall tooth region, and finally uses the third neural network trained in advance to perform mask segmentation on the single-tooth region to obtain the tooth data of the corresponding tooth. In this process, the overall tooth region is first identified, which not only reduces the data input amount of the second neural network, but also reduces the misidentification in the region outside the overall tooth region. At the same time, compared with the segmentation of a single tooth by inputting the entire oral CT image into one network, inputting the single-tooth region into the third neural network for segmentation processing can greatly reduce the data amount and complexity of the input data, and improve the accuracy of tooth segmentation. Thus, effective and accurate segmentation of teeth in the oral CT image can be realized. In addition, the entire tooth segmentation process is performed in stages by using different neural networks, which can reduce the complexity of the network at each stage and the difficulty of the corresponding task implementation.

[0009] To achieve the above object, the second embodiment of the present application proposes a tooth surface data acquisition method, which comprises: acquiring tooth data according to the above tooth segmentation method; performing three-dimensional Gaussian smoothing on the tooth data; extracting tooth mesh data from the smoothed tooth data by the Marching Cube algorithm; performing three-dimensional Laplace smoothing on the extracted tooth mesh data; determining and removing redundant points according to the smoothed tooth mesh data and the sharpness of the mesh boundary, and splicing the remaining points into a surface to obtain the tooth point cloud surface data.

[0010] The tooth surface data acquisition method of the embodiment of the present application first acquires tooth data according to the above tooth segmentation method, then performs three-dimensional Gaussian smoothing on the tooth data, extracts tooth mesh data from the smoothed tooth data by the Marching Cube algorithm, and performs three-dimensional Laplace smoothing on the extracted tooth mesh data; finally, determines and removes redundant points according to the smoothed tooth mesh data and the sharpness of the mesh boundary, and splices the remaining points into a surface to obtain tooth point cloud surface data. This process acquires tooth surface data based on an accurate tooth segmentation method, and can acquire accurate tooth surface data. At the same time, different forms of smoothing processing are performed on the volume data and the surface data before and after the tooth mesh data extraction by the Marching Cube algorithm, which ensures the accuracy of data processing at each stage.

[0011] Thus, a smoother and clearer tooth segmentation result can be obtained.

[0012] To achieve the above object, the third aspect of the present application provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the tooth segmentation method or the tooth surface data acquisition method.

[0013] To achieve the above object, the fourth aspect of the present application provides an oral CT device, which comprises a memory, a processor and a computer program stored in the memory, and the computer program is executed by the processor to implement the tooth segmentation method or the tooth surface data acquisition method.

[0014] Additional aspects and advantages of the present application will be made apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0015] Figure 1 is a flow chart of the tooth segmentation method of the embodiment of the present application;

[0016] Figure 2 is a schematic diagram of the oral CT image to be segmented of an example of the present application;

[0017] Figure 3 is a schematic diagram of the framework of the first neural network of an example of the present application;

[0018] Figure 4 is a schematic diagram of the tooth overall region positioning box of an example of the present application;

[0019] Figure 5 is a schematic diagram of the framework of the second neural network of an example of the present application;

[0020] Figure 6 is a schematic diagram of the framework of the third neural network of an example of the present application;

[0021] Figure 7 is a schematic diagram of the software perspective of an example of the present application;

[0022] Figure 8 is a schematic diagram of the labeling perspective of an example of the present application;

[0023] Figure 9 is an effect diagram of the Otsu threshold segmentation of an example of the present application;

[0024] Figure 10 is an effect diagram of the adaptive three-dimensional threshold segmentation of an example of the present application;

[0025] Figure 11 is a flow chart of the positioning box false detection or missed detection of an example of the present application;

[0026] Figure 12 , Figure 13 This is a schematic diagram of mislocation in the first example of the present invention;

[0027] Figure 14 , Figure 15 This is a schematic diagram of mislocation in the second example of the present invention;

[0028] Figures 16(a) and 16(b) are schematic diagrams of a leak location example of the present invention;

[0029] Figure 17 This is a flowchart of a method for acquiring tooth surface data according to an embodiment of the present invention;

[0030] Figure 18 This is a flowchart of the tooth arrangement method according to an embodiment of the present invention;

[0031] Figure 19 This is a schematic diagram of a tooth region positioning frame as an example of the present invention;

[0032] Figures 20-22 This is a schematic diagram illustrating an example of the process of determining multiple tooth alignment reference points according to the present invention;

[0033] Figure 23 This is a schematic diagram of the tooth arrangement result of an example of the present invention;

[0034] Figure 24 This is a flowchart of the orthodontic simulation method according to an embodiment of the present invention;

[0035] Figure 25 This is a schematic diagram of the framework of a neural network model of an example of the present invention. Detailed Implementation

[0036] Embodiments of the present invention are described in detail below, examples of which are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0037] The following is a reference appendix. Figures 1-25 This invention describes a tooth segmentation method, a tooth surface data acquisition method, a storage medium, and an oral CT device according to embodiments of the present invention.

[0038] Figure 1 This is a flowchart of a tooth segmentation method according to an embodiment of the present invention.

[0039] like Figure 1 As shown, the tooth segmentation method includes the following steps:

[0040] S101, acquire the oral cavity CT image to be segmented.

[0041] Specifically, the to-be-segmented oral CT image can be directly obtained from an oral CT device, or can be obtained from a database of a built medical image management system, such as Figure 2 as shown.

[0042] In S102, a first neural network is used to locate a tooth overall region in the to-be-segmented oral CT image.

[0043] The tooth overall region is a region smaller than the overall region of the oral CT image and covering all teeth in the oral CT image, for example, can be a region framed by a minimum circumscribed three-dimensional box covering all teeth, or a region corresponding to a three-dimensional box slightly larger than the minimum three-dimensional box. Of course, a region corresponding to a three-dimensional box larger than the minimum three-dimensional box within a preset range is also possible, such as the preset range being a range below the nose. The located tooth overall region can be represented by first 3D positioning data, which can be image data corresponding to the three-dimensional box in the oral CT image.

[0044] Specifically, the first neural network can include a 3D convolution layer, a 3D pooling layer, and a normalization layer, which can locate a three-dimensional tooth overall region in the oral CT image, and further locate a single tooth region through a second neural network. The network framework of the first neural network can be as shown in Figure 3 Through the first neural network, the tooth overall region in the oral CT image can be quickly located, as shown in Figure 4 Thus, it can be quickly judged whether there is a tooth in the oral CT image, and if not, subsequent operations are not performed; at the same time, misidentification in the nasal cavity and other parts can be avoided, the amount of input data of the second neural network is reduced, and thus the overall speed and accuracy of tooth segmentation can be improved.

[0045] In S103, a second neural network is used to locate a single tooth region of a preset range in the tooth overall region.

[0046] The preset range is a self-defined tooth type, which can be all teeth or partial teeth. That is, the second neural network is not limited to identifying the region of each tooth in all teeth, but can also be used to identify a single tooth region of a specific type (such as all central incisors) or several types (such as all lateral incisors and canines) in all teeth, or can also be used to identify a single tooth region of a certain region (such as the left side of the upper central incisor) or several regions (such as the left side of the upper central incisor and the right side of the lower central incisor) in all teeth.

[0047] Of course, the tooth types can include central incisors, lateral incisors, canines, first premolars, second premolars, first permanent molars, second permanent molars, and third permanent molars. The single-tooth region is a region corresponding to a single tooth framed by a three-dimensional frame of minimum size or slightly larger than the minimum size, and the single-tooth region can be a single-tooth image framed by the three-dimensional frame in the oral CT image or the first 3D positioning data.

[0048] Specifically, if it is determined in step S102 that there is a tooth in the oral CT image, the recognized tooth overall region is input to the second neural network for positioning of the single-tooth region to obtain the second 3D positioning data. In this way, the false positioning in the nasal cavity and the like can be avoided, and the amount of input data of the second neural network is reduced, thereby improving the overall speed and the accuracy of recognition. As shown in FIG. 6, the second neural network can include a plurality of 3D detection layers (for example, 3). The 3D detection layer includes a 3D convolution layer and a 3D pooling layer. Figure 5

[0049] As an example, the positioning of the single-tooth region of a preset range in the tooth overall region by using the pre-trained second neural network can include: sequentially performing three-dimensional Otsu threshold segmentation, adaptive threshold segmentation, and morphological processing on the tooth overall region to obtain first prior knowledge data; and positioning the single-tooth region of a preset range in the tooth overall region by using the pre-trained second neural network according to the first prior knowledge data and the tooth overall region. The influence of soft tissue and the like is removed in the first prior knowledge data. In combination with the tooth overall region, the recognition accuracy of the single-tooth region can be reduced by referring to the first prior knowledge.

[0050] Specifically, the second neural network mainly completes the positioning of the single-tooth region on the basis of the first neural network. In order to reduce the missed detection area, the tooth overall region data (i.e., the first 3D positioning data) obtained by the first neural network input to the input end of the second neural network can be preprocessed by denoising (such as the three-dimensional Otsu threshold segmentation, adaptive threshold segmentation, and morphological processing described above) to obtain the first prior knowledge data described above. In this way, the learning difficulty of the second neural network can be reduced, and the convergence can be accelerated. The acquisition step of the prior knowledge data can be as follows:

[0051] Step 1: Since the oral CT image includes air, soft tissue, and the like, and only the tooth part needs to be labeled in order to realize tooth segmentation, the three-dimensional Otsu threshold segmentation is first performed on the oral CT image to remove the interference of the air part.

[0052] Step 2: The data in Step 1 is subjected to a second adaptive three-dimensional threshold segmentation to separate the bone part including the tooth region in the oral CT image.

[0053] ​Step3: Morphological processing is performed on the separated bone data, and the overall tooth part can be clearly separated and extracted.

[0054] Due to the prior knowledge of the bone region, combined with the labeled real data as supervision, the second neural network can quickly locate and identify the region of each tooth. In order to better cope with teeth of different sizes, three detection layers can be set in the second neural network to detect teeth of different shapes and sizes.

[0055] As an example, when the preset range of the single tooth region in the overall tooth region is located by using the pre-trained second neural network, the 3D positioning frame of the single tooth can be output by the second neural network according to the first 3D positioning data; the 3D positioning frame is mapped to the to-be-segmented oral CT image to obtain the single tooth region.

[0056] S104, using a pre-trained third neural network to perform mask segmentation on the single tooth region to obtain tooth data of the corresponding tooth.

[0057] Specifically, after the learning of the first neural network and the second neural network is completed, the region of each tooth can be accurately located, and then the tooth data of each tooth can be extracted by the third neural network. The main training framework of the third neural network is as shown in Figure 6 The third neural network can include a 3D convolution layer and a 3D pooling layer.

[0058] As an example, using the pre-trained third neural network to perform mask segmentation on the single tooth region can include: sequentially performing three-dimensional Otsu threshold segmentation, adaptive threshold segmentation and morphological processing on the single tooth region to obtain second prior knowledge data; using the third neural network to perform mask segmentation on the single tooth region according to the single tooth region and the second prior knowledge data.

[0059] In an embodiment of the present application, the pre-trained third neural network can also be used to perform mask segmentation on the root canal in the single tooth region to obtain root canal data.

[0060] Specifically, if the tooth segmentation also needs to segment the root canal data of each tooth, the single tooth region can also be sequentially subjected to adaptive threshold segmentation and morphological processing to obtain root canal data; and then the third neural network is used to perform mask segmentation on the single tooth region and the root canal region according to the single tooth region, the root canal data and the second prior knowledge data, to obtain tooth data and root canal data.

[0061] Specifically, the third neural network completes the mask segmentation of the single tooth region and the root canal region on the basis of the first neural network and the second neural network, and simultaneously completes multiple tasks by using one network model. The input of the third neural network includes the CT three-dimensional data of the region where the single tooth is located which is output by the second neural network, the root canal data obtained by using adaptive threshold segmentation and morphological processing, and prior knowledge data (i.e., the second prior knowledge data described above), and each data acquisition method is as follows:

[0062] The CT three-dimensional data of the region where the single tooth is located: the 3D data of the corresponding part is extracted after the 3D positioning frame output by the second neural network is mapped to the to-be-segmented oral CT image, the numerical distribution normalization and size normalization operations are completed, and the single tooth region 3D data is obtained;

[0063] The root canal data: adaptive threshold segmentation is performed on the single tooth region 3D data extracted above, and then morphological processing of corrosion and expansion is performed to extract the root canal region 3D data in the tooth;

[0064] The second prior knowledge data: since the single tooth region 3D data extracted above includes air, soft tissue and other parts, the three-dimensional Otsu threshold segmentation is first performed on the single tooth region 3D data to remove the interference of the air part, and then the second adaptive three-dimensional threshold segmentation is performed to separate the bone part including the tooth region, and the morphological processing is performed on the separated bone data as the prior knowledge data, so as to reduce the learning difficulty of the network and accelerate the convergence.

[0065] Through the combination of the above three kinds of data and the combination of the labeled real data as supervision, the third neural network can simultaneously and effectively extract the 3D mask and 3D root canal mask of the tooth, and complete the single model multi-task.

[0066] Therefore, the tooth segmentation method of the embodiment of the application can realize accurate and effective segmentation of the tooth in the oral CT image.

[0067] In the embodiment of the application, the first neural network, the second neural network and the third neural network can be pre-trained, and the data used for training the first neural network, the second neural network and the third neural network is labeled in the following manner:

[0068] A1, the three-dimensional Otsu threshold segmentation, adaptive threshold segmentation and morphological processing are sequentially performed on the oral CT training image to obtain a to-be-labeled tooth image;

[0069] A2, the mask segmentation is performed on the to-be-labeled tooth image by using the trained tooth region model to obtain a pre-labeled oral CT training image, wherein the tooth region model is trained by a residual-based 3D semantic segmentation artificial neural network;

[0070] A3, correcting tooth marking of the oral CT training image of the pre-marked tooth body;

[0071] A4, determining a three-dimensional frame of a single tooth according to the tooth marking, to mark a single tooth region;

[0072] A5, determining a three-dimensional frame of a tooth whole region according to the three-dimensional frames of all single teeth, to mark the tooth whole region;

[0073] A6, obtaining data for training the first neural network, the second neural network and the third neural network according to the markings in the oral CT training image.

[0074] Specifically, the training image is three-dimensional data, and there are two big problems in labeling, one is that the labeling amount is large, and compared with the two-dimensional data amount, the three-dimensional data labeling amount increases exponentially; the second is that the labeling difficulty is large, because one dimension is added, so the traditional labeling method on the two-dimensional slice is inaccurate, because the teeth are densely arranged, and the occlusal surface has a complex occlusal relationship, so the labeling on the two-dimensional slice cannot accurately mark which part a certain region belongs to.

[0075] In order to overcome the above two big problems and obtain high-quality and high-accuracy labeled data, the present application labels in combination with three-dimensional information of the oral CT image, and labels each tooth separately, and improves the labeling efficiency by using a preprocessing method.

[0076] In order to more accurately label, the existing three-dimensional labeling software can be applied to view the oral CT image from three angles of transverse, sagittal and coronal, and the software can include a three-dimensional reconstruction window. The specific labeling process is as follows: for a tooth to be marked, the outer edge of the tooth region is marked on the section in a selected direction by labeling each slice, and when the labeling of all slice images of the tooth is completed, the three-dimensional labeling (3D labeling) of the tooth is completed.

[0077] During labeling, labeling on three-dimensional body data is realized, different viewing angles and reconstructed head data are combined, the labeled region can be clearly seen, and each tooth can be accurately labeled, solving the problem of inaccurate two-dimensional labeling. The software viewing angle can be as shown in Figure 7 , and the labeling viewing angle can be as shown in Figure 8 .

[0078] As an example, after completing the 3D labeling of the single tooth region in the training image, the following can also be included: iteratively training the tooth region model according to the obtained labeled data.

[0079] Specifically, in order to reduce the labeling workload and improve the labeling efficiency, the present application proposes the following two preprocessing methods:

[0080] The first kind: using twice threshold segmentation algorithm to extract bone data

[0081] Step1: Since the oral CT image includes air, soft tissue and other parts, and in order to realize tooth segmentation, only the tooth part needs to be labeled, so first a three-dimensional Otsu threshold segmentation is performed on the oral CT image to remove the interference of the air part. Among them, the three-dimensional Otsu method is to find the adaptive threshold by maximizing the inter-class variance to segment the oral CT image. The inter-class variance of the threshold T in the histogram is defined as:

[0082]

[0083] The threshold T corresponding to the minimum variance is the segmentation threshold. The oral CT image before and after segmentation is shown in Figure 9

[0084] Step2: The data obtained in Step1 is subjected to a second adaptive three-dimensional threshold segmentation to remove soft tissue and separate the bone part including the tooth region in the oral CT image, as shown in Figure 10

[0085] Step3: Morphological processing is performed on the bone data separated in Step2 to clearly separate and extract the tooth part, and then the tooth is labeled. Since the tooth morphology is clear and separated, the labeling difficulty is greatly reduced, and the efficiency is obviously improved.

[0086] The second kind: using artificial neural network to obtain prior knowledge to assist labeling

[0087] Step1: After accumulating a small amount of tooth segmentation data, a residual-based 3D semantic segmentation artificial neural network is built, and the tooth region is quickly labeled by using the labeled tooth segmentation data in the database;

[0088] Step2: The tooth region model is used to automatically segment the data to be labeled, and the segmentation mask is obtained;

[0089] Step3: The data in Step2 is preprocessed, such as inflation, filling small black holes in the segmented tooth region, and then eroded and connected domain threshold to remove mispositioning area, to obtain more perfect three-dimensional tooth mask information;

[0090] Step4: The labeling personnel simply and quickly correct the data in Step3, that is, the current oral CT image labeling is completed, and stored in the database;

[0091] Step5: Since the data in the database has been updated, the 3D semantic segmentation artificial neural network can be quickly trained again to automatically fine-tune and optimize the weight parameters in the model, and a better model is trained.​​

[0092] Step6: Repeat the above steps.

[0093] After the data labeling is completed, the first neural network, the second neural network and the third neural network are built for training the tooth instance segmentation model of the oral CT image. In order to quickly train the first neural network, the second neural network and the third neural network, a fast training method is designed, and the pre-training model on a large data set is not required.

[0094] In order to quickly train the first neural network, the second neural network and the third neural network in the context without a pre-training model, first, the teeth in each oral CT image in the database are numbered. Normally, the number of teeth is between 28-32, but there may be cases of missing teeth, supernumerary teeth, deciduous teeth, etc. The number of teeth is quite different in different cases. We number the teeth in each oral CT image according to the tooth position, which are ys1, ys2, ys3,..., zs1, zs2, zs2,..., dsy1, dsy2, dsy3,..., respectively, representing the first tooth on the right upper side, the second tooth on the right upper side, the third tooth on the right upper side,..., the first tooth on the left upper side, the second tooth on the left upper side, the third tooth on the left upper side,..., the first supernumerary tooth, the second supernumerary tooth, the third supernumerary tooth, and so on. By default, the number of all labels does not exceed 50. The following takes the training of the first neural network and the second neural network as an example to illustrate the training process of the neural network:

[0095] As an example, the training steps of the first neural network are as follows:

[0096] B1, obtaining a first training database containing a plurality of training images, wherein the teeth in each training image have been 3D labeled as a whole region;

[0097] The 3D labeling of the tooth as a whole region can be obtained based on the labeling of a single tooth, and preferably the smallest circumscribed frame obtained by labeling all single teeth is labeled as the tooth as a whole region.

[0098] Specifically, the training images can be various types of oral CT images of patients taken by different models, different fields of view, and different photographers from the database of the built medical image management system, which can ensure the reliability, richness and data volume of the data.

[0099] B2, selecting data from the first training database as a first training set and a first validation set, respectively;

[0100] Specifically, before step B2, the weight parameters of the first neural network can be initialized by using a normal distribution.

[0101] B3, iteratively train the first neural network model according to the first training set, and verify the model obtained in each iteration according to the first verification set;

[0102] The first neural network model is an initial neural network model used for training the first neural network.

[0103] B4, when the verification result meets the first preset condition or the number of iterations meets the second preset condition, respectively select data from the remaining training data in the first training database to update the first training set and the first verification set, and return to step B3, wherein the number of currently selected training data is k times the number of training data selected last time, and k is an update coefficient; otherwise, end the training.

[0104] The k can be selected according to the task difficulty, for example, it can be 2; and the iteration stopping condition can be that all training data in the first training database participate in the iterative training.

[0105] Therefore, a small amount of data set can be used to quickly train the parameters of the bottom network, which can efficiently learn the bottom detail features and then gradually improve and accelerate the convergence speed.

[0106] As an example, the training steps of the second neural network are as follows:

[0107] C1, obtain a second training database containing a plurality of training images, wherein each training image has been 3D labeled in a single tooth region;

[0108] C2, respectively select data from the second training database as a second training set and a second verification set;

[0109] The 3D frame of the single tooth region can be composed of the minimum circumscribed rectangle of the single tooth label.

[0110] Specifically, before step C2, the weight parameters of the second neural network can be initialized by using a normal distribution. The tooth numbers are randomly divided into n1 groups, wherein the value of n1 can be 8. For example, the first group corresponds to the teeth numbered 1, 2, 5 and 7, the second group corresponds to the teeth numbered 3, 4, 6 and 8, and so on, all groups cover all numbers. Wherein, the first time can select the numbered teeth corresponding to the first group.

[0111] C3, iteratively train the second neural network model according to the second training set, and verify the model obtained in each iteration according to the second verification set;

[0112] The second neural network model is an initial neural network model used for training the second neural network.

[0113] C4, when the verification result meets a third preset condition or the number of iterations meets a fourth preset condition, selecting data from the remaining training data of the second training database to update the second training set and the second verification set respectively, and returning to step C3; otherwise, ending the training;

[0114] The iteration stopping condition can be that all training data in the second training database participates in the iterative training.

[0115] Therefore, the network can learn the basic features of teeth simply and quickly without pre-training weights, reduce the learning difficulty, and accelerate convergence.

[0116] In an embodiment of the present application, the tooth segmentation method can further include: determining whether there is mispositioning or missed positioning in the single tooth region; if there is, correcting the single tooth region according to the 3D positioning frame corresponding to the mispositioning or missed positioning to correct the dental data.

[0117] The correction of the dental data can include: performing mask segmentation of the single tooth region according to the corrected single tooth region by using the third neural network trained in advance to obtain dental data of each tooth, and replacing the previously obtained dental data.

[0118] Optionally, if the root canal data is also corrected, the third neural network trained in advance can be used to perform mask segmentation of the root canal region according to the corrected single tooth region to obtain root canal data of each tooth, and replace the previously obtained root canal data.

[0119] Specifically, the first neural network, the second neural network and the third neural network trained by big data have good 3D tooth positioning and tooth root canal 3D segmentation performance, but the output of the neural network often has certain uncontrollability. For very extreme data, there can be tooth missed detection or misdetected, and the output result is very dependent on the setting of Nms (Non-maximum suppression, Non-maximum suppression algorithm) threshold, which is also a common problem in the current deep learning field. In order to enhance the robustness of the model and make it less sensitive to threshold setting, accurate positioning and identification of tooth parts in three-dimensional oral CT images can be achieved for any data, the output of the neural network can be further processed. The following will be described from the two processing directions of mispositioning and missed detection. Adjust the Nms threshold to improve the detection rate. In this case, combined with the following mispositioning and missed positioning settings, the recognition accuracy is improved under the condition of high detection rate.

[0120] For mispositioning cases, these few mispositioning can be removed by effective post-processing, which is very necessary in the oral field because one mispositioning in the oral field can bring greater impact.

[0121] Due to the effect of the first neural network, the algorithm model of the present application will not have mispositioning outside the tooth area, and only a few mispositioning will exist in the case of very close or chaotic tooth arrangement. Therefore, by combining the tooth segmentation result to assist in removing the mispositioning in the positioning result of the first neural network and the second neural network, the accuracy of the results of the first neural network and the second neural network is improved, thereby forming a mutual promotion cycle, and the specific process can be as shown in Figure 11

[0122] As an example, as shown in Figure 12 , for the case of a mispositioning box positioned in the middle of two teeth. To determine whether there is mispositioning in a single tooth region, the number of segmented slices can be counted according to the dental data, and if the number of segmented slices is greater than a first threshold, it is determined that there is mispositioning in the single tooth region. Wherein, the single tooth region is corrected according to the 3D positioning box corresponding to the mispositioning, and the dental data is corrected by using a third neural network trained in advance to perform mask segmentation on the corrected single tooth region to obtain the dental data of the corresponding tooth.

[0123] Specifically, for such mispositioning, the tooth in the 3D positioning box can be segmented. Since there will be an occlusal bar (silicone material, CT value is small) between the two teeth, there will be a gap in the middle of the tooth, so there will be a non-tooth region in the middle of the segmentation result, that is, the segmentation has a fault, as shown in Figure 13 The middle segmentation fault (corresponding to the above non-tooth region) can be counted, for example, whether the tooth region on each slice in a single positioning box changes from having to not having to having in the vertical direction, and if so, the segmentation fault exists and is counted once. If the count value exceeds the threshold n, it is determined to be mispositioning, and in practice, n can be set to 1 to ensure that all positioning boxes with segmentation faults are found. At the same time, the detection threshold can also be reduced to ensure that almost all teeth are detected, but at the same time the mispositioning rate will increase. By setting the n threshold to 1, that is, greater than or equal to 1, it means that it is a mispositioning box, which can be removed at this time, thereby ensuring the effectiveness and strictness of the removal, and at the same time ensuring the detection rate. The detection threshold is a threshold for controlling the detection process. Within a certain range, the larger the detection threshold, the lower the detection rate and the mispositioning rate; on the contrary, the higher the detection rate, the higher the mispositioning rate.

[0124] As another example, as shown in Figure 14 ​As shown, for the case of mispositioning frame on single tooth region. The judgment of whether there is mispositioning in the single tooth region can include: judging whether there is a 3D positioning frame satisfying the mispositioning condition according to the tooth data, wherein if part of the slice of the 3D positioning frame does not exist the tooth segmentation result, it is determined that the 3D positioning frame satisfies the mispositioning condition; if so, adjust the 3D positioning frame to obtain the correct 3D positioning frame. Of course, if the segmentation result includes the tooth root canal data, the mispositioning can also be judged according to the tooth root canal data.

[0125] In the tooth positioning of the present application, mispositioning as shown can occur: that is, part of the region of the tooth is boxed. Figure 14

[0126] For the 3D tooth root canal segmentation algorithm of the present application, first, the current mispositioning frame is treated as a correct 3D positioning frame for segmentation processing. Since the correct 3D positioning frame will default to exactly box the current tooth, that is, the minimum circumscribed 3D rectangular frame, its segmentation result has tooth part segmentation result in each slice in the height direction, as shown: Figure 15 The lower left frame is the correctly recognized frame, and each slice in the height direction of the frame has segmentation result, while the upper right frame is the mispositioning frame, which only has segmentation recognition result in part of the region. Therefore, the proportion of the region with segmentation result to the entire frame height can be calculated, and by setting a threshold, the above mispositioning frame can be effectively removed. Specifically, the height of the tooth region in the frame is divided by the height of the positioning frame to obtain the proportion, and the threshold is set according to experience or test results to exclude positioning frames with relatively small proportions. A smaller proportion can be considered as mispositioning.

[0127] As another example, the judgment of whether there is a missing positioning in the single tooth region can include: calculating the gap between the adjacent two teeth according to the single tooth region (if the distance between the adjacent two teeth is too large, it means that there may be other teeth in the middle that have not been identified, which needs to be searched again), judging whether missing positioning search is needed according to the gap and size threshold; if so, the gap position is divided into a grid, such as a 2x3 grid, and the center of each grid is taken as the center of the search frame to search, wherein the size of the search frame is the size average; using a third neural network to segment the region corresponding to the search frame, and judging whether there is a missing positioning according to the segmentation result, which can be specifically: if a tooth region is segmented, it is considered that there is a missing identification.

[0128] In this example, the size average of all 3D positioning frames corresponding to the single tooth region can also be calculated; and the size of the search frame is set to the size average.

[0129] ​Specifically, for some extreme data, such as very small or only a little crown data, the above algorithm can cause missed detection. For example, as shown in FIG. 16(a) and FIG. 16(b), in order to find out whether there is missed detection on the right side, first, the center position of the rightmost positioning frame is obtained, and then a greedy search of frames of different scales is performed on the right side of the horizontal axis coordinate of the center position. In order to ensure search efficiency, the horizontal coordinate does not exceed the upper and lower edges of the positioning frame, the vertical coordinate does not exceed 1.5 times the width of the positioning frame, and the scale type can be set to multiple. Then, the search is performed, the search mode is to draw a grid in a certain area, and the center point of each grid is taken as the center point of the search frame. After each search frame, the 3D data of the corresponding area is taken, and the segmentation and identification of the third neural network are performed. Whether there is a tooth is determined through the identification result. If there is, the positional relationship between the search frame and the rightmost positioning frame is determined to confirm whether to retain or not, otherwise, the search is continued, and the specific process is as described above. The rightmost positioning frame is the right boundary, and the search frame should be searched on the left side of the rightmost positioning frame. The positional relationship between the search frame and the rightmost positioning frame is determined as follows: the iou (intersection over union) value of the search frame and the positioning frame is calculated. If the value is less than a certain threshold (generally set to 0.5), it means that there are two teeth, and the search frame is retained.

[0130] Therefore, the extreme missed detection situation can be effectively searched out, and the robustness of the first neural network and the second neural network detection is greatly improved. The method is also applicable to the missed detection judgment of middle large teeth.

[0131] In summary, the tooth segmentation method of the embodiment of the present application first locates the overall tooth region in the to-be-segmented oral CT image by using the pre-trained first neural network, then locates the single tooth region of the preset range in the overall tooth region by using the pre-trained second neural network, and finally performs mask segmentation on the single tooth region by using the pre-trained third neural network to obtain the tooth data of the corresponding tooth. In this process, the overall tooth region is first identified, which not only reduces the data input amount of the second neural network, but also reduces the misidentification in the region outside the overall tooth region. At the same time, compared with the segmentation of a single tooth by inputting the entire oral CT image into one network, inputting the single tooth region into the third neural network for segmentation processing can greatly reduce the data amount and complexity of the input data, and improve the accuracy of tooth segmentation. Therefore, the effective and accurate segmentation of teeth in the oral CT image can be realized. In addition, the entire tooth segmentation process is performed in stages by using different neural networks, which can reduce the complexity of each stage of the network and the difficulty of the corresponding task implementation.

[0132] Further, after obtaining the tooth data of each tooth, tooth point cloud surface data extraction can be performed according to the tooth data of each tooth to obtain better tooth segmentation effect and facilitate subsequent tooth arrangement processing.

[0133] To this end, the present application provides a tooth surface data acquisition method. As shown in Figure 17 The tooth surface data acquisition method comprises the following steps.

[0134] S01, acquiring tooth data according to the tooth segmentation method described above.

[0135] S02, performing three-dimensional Gaussian smoothing on the tooth data.

[0136] S03, extracting tooth mesh data from the smoothed tooth data by using a Marching Cube algorithm.

[0137] S04, performing three-dimensional Laplace smoothing on the extracted tooth mesh data.

[0138] S05, determining and removing redundant points according to the smoothed tooth mesh data and the sharpness of the mesh boundary, and splicing the remaining points into a surface to obtain tooth point cloud surface data.

[0139] Specifically, the corresponding tooth voxel data can be obtained according to the tooth data of each tooth; three-dimensional Gaussian smoothing is performed on the tooth voxel data; a tooth gray scale threshold is calculated according to the to-be-segmented oral CT image, and tooth mesh data is extracted from the smoothed tooth data by using a Marching Cube algorithm; three-dimensional Laplace smoothing is performed on the extracted tooth mesh data; redundant points are determined and removed according to the smoothed tooth mesh data, and tooth point cloud surface data is obtained by splicing the remaining points into a surface.

[0140] Optionally, before the three-dimensional Laplace smoothing is performed on the tooth mesh data, the tooth mesh data can also be down-sampled, and then the three-dimensional Laplace smoothing is performed on the down-sampled tooth mesh data, so as to reduce the signal sampling rate, reduce the computational complexity, and improve the processing speed.

[0141] As an example, the tooth gray scale threshold can be calculated according to the to-be-segmented oral CT image, which can include: calculating a first gray scale threshold of the boundary between air and facial soft tissue in the to-be-segmented oral CT image by using the Otsu method; and calculating a tooth gray scale threshold of the boundary between soft tissue and bone in the to-be-segmented oral CT image by using the Otsu method based on the first gray scale threshold.

[0142] As an example, the redundant points can be determined according to the smoothed tooth mesh data, which can include: calculating the included angle of the normal vectors of each two adjacent mesh surfaces containing each vertex of each boundary of the smoothed tooth mesh data; calculating the maximum included angle corresponding to each vertex according to the included angle of each vertex; determining an adjustment term of the edge collapse cost corresponding to each vertex according to the maximum included angle corresponding to each vertex; and performing a simplification operation by using an edge collapse algorithm and the adjustment term of the edge collapse cost of each vertex to obtain the redundant points.

[0143] In this example, determining the adjustment term of the edge collapse cost corresponding to each vertex according to the maximum included angle corresponding to the vertex can include: calculating the number of adjacent vertices of each vertex; and determining the adjustment term of the edge collapse cost corresponding to each vertex according to the maximum included angle corresponding to the vertex and the number of adjacent vertices of each adjacent vertex of the vertex. For example, the adjustment term of the edge collapse cost corresponding to each vertex can be determined according to the maximum included angle θ corresponding to the vertex and the number of adjacent vertices m of each adjacent vertex i of the vertex as follows: Δ''(v) = θ * ∑cosθ / m i i It should be noted that it is also possible to express cosθ by a combination of sine and / or cosine of θ or multiples of θ, for example, according to cos 2 θ + sin 2 θ = 1, the cosine of θ in the formula is converted to the sine of θ.

[0144] where the adjacent vertex of a vertex is another vertex containing all boundaries of the vertex, and the number of adjacent vertices of a vertex is equal to the number of all boundaries containing the vertex.

[0145] The maximum included angle corresponding to each vertex can be determined by building a minimum heap of the included angles corresponding to the vertex for all included angles corresponding to the vertex.

[0146] The edge collapse algorithm and the adjustment term of the edge collapse cost of each vertex are used for simplified operation. Specifically, the sum of the preset quadratic error metric and the adjustment term of the edge collapse cost of each vertex can be used as the edge collapse cost of the corresponding vertex for simplified operation. Of course, the adjustment basis of the edge collapse cost can also be determined according to other existing algorithms, which will not be described here.

[0147] The preset quadratic error metric can be obtained from the paper Surface simplification using quadric error metrics, which was published by authors Garland M and Heckbert PS in 1997 on Proc. of the Computer Graphics. Specifically,

[0148]

[0149] where Δ'(v) is the quadratic error metric, v = [v x v y v z 1] T , and p represents a plane in a three-dimensional space

[0150] ​In summary, the tooth surface data acquisition method of the embodiment of the present application firstly acquires tooth data according to the above tooth segmentation method, then performs three-dimensional Gaussian smoothing on the tooth data, extracts tooth grid data from the smoothed tooth data through the Marching Cube algorithm, performs three-dimensional Laplace smoothing on the extracted tooth grid data, finally determines and removes redundant points according to the smoothed tooth grid data and the sharpness of the grid boundary, and splices the remaining points into a surface to obtain tooth point cloud surface data. This process acquires tooth surface data based on an accurate tooth segmentation method, and can acquire accurate tooth surface data. Meanwhile, different forms of smoothing processing are performed on the volume data and the surface data before and after the tooth grid data extraction through the Marching Cube algorithm, thereby ensuring the accuracy of data processing at each stage.

[0151] Further, the present application provides a computer readable storage medium having a computer program stored thereon, and the computer program is executed by a processor to implement the above tooth segmentation method or the above tooth surface data acquisition method.

[0152] Further, the present application also provides an oral CT device. The oral CT device comprises a memory, a processor and a computer program stored on the memory, and the computer program is executed by the processor to implement the above tooth segmentation method or the above tooth surface data acquisition method.

[0153] Figure 18 is a flowchart of the tooth arrangement method of an embodiment of the present application.

[0154] As shown in Figure 18 , the tooth arrangement method comprises the following steps:

[0155] S11, acquiring an oral CT image.

[0156] Specifically, the oral CT image can be an oral CT image acquired from a database of a built medical image management system, as shown in Figure 2 .

[0157] S12, performing segmentation processing on the oral CT image to obtain a first positioning frame of each tooth.

[0158] Specifically, an existing tooth segmentation processing method can be used to obtain the first positioning frame of each tooth, and the first positioning frame can be a 2D positioning frame; or the above tooth segmentation method can be used to obtain the first positioning frame of each tooth, and the first positioning frame can be a 3D positioning frame.

[0159] S13, determining a plurality of tooth arrangement reference points according to the first positioning frame.

[0160] Specifically, the center coordinates of each first positioning frame can be calculated to obtain the corresponding first center coordinates, and a second positioning frame for the entire tooth area can be obtained based on the first positioning frames; then, multiple tooth alignment reference points can be determined based on the first center coordinates and the second positioning frames. For example, a positioning frame that exactly includes all the first positioning frames can be obtained and used as the second positioning frame.

[0161] As a feasible implementation, the aforementioned second positioning frame can be as follows: Figure 19 As shown, the number of tooth alignment reference points can be three. In this case, the multiple tooth alignment reference points include a center reference point, a left reference point, and a right reference point. In this example, the specific steps for determining multiple tooth alignment reference points based on the first positioning frame are as follows:

[0162] Step 1: Based on the first center coordinates and the second positioning frame, determine the first positioning frame at the top left and the first positioning frame at the top right.

[0163] Specifically, taking the first positioning box as a 3D positioning box as an example, a three-dimensional orthogonal coordinate system can be established (e.g., Figure 20 As shown, Figure 20 The diagram only shows the x and y axes in a three-dimensional coordinate system (the z-axis is perpendicular to the x and y axes and points inward). It obtains the positional relationship between the center coordinates of each first positioning frame and the second positioning frame within this coordinate system. Based on this positional relationship, it determines the top left first positioning frame (with the smallest y-coordinate value in the first center coordinate system and closest to the left boundary of the second positioning frame) and the top right first positioning frame (with the smallest y-coordinate value in the first center coordinate system and closest to the right boundary of the second positioning frame). Figure 20 As shown.

[0164] It should be noted that, Figure 19 and Figure 20 This displays a panoramic view of the teeth, illustrated within the panoramic image. Actual calculations can be performed using the spatial coordinates of a 3D tooth model. In one implementation, the vertical axis can be the vertical direction, and the horizontal axis can be... Figure 10 The left and right directions in the coordinate system, and the third coordinate direction can be perpendicular to the horizontal and vertical coordinate directions.

[0165] Step 2: Calculate the center coordinates of the second positioning frame to obtain the second center coordinates.

[0166] Step 3: Determine the center reference point based on the second center coordinates.

[0167] Specifically, one first positioning frame satisfying a preset condition can be selected according to the second center coordinate. The method comprises: determining the minimum value and the maximum value of the horizontal coordinate of each first positioning frame; comparing the minimum value and the maximum value of the horizontal coordinate with the horizontal coordinate in the second center coordinate; selecting one from the first positioning frames satisfying the preset condition, wherein the preset condition is that the minimum value of the horizontal coordinate is less than or equal to the horizontal coordinate in the second center coordinate, and the maximum value of the horizontal coordinate is greater than or equal to the horizontal coordinate in the second center coordinate; and determining the center reference point according to the selected first positioning frame and the second center coordinate.

[0168] The method of determining the center reference point according to the selected first positioning frame and the second center coordinate can comprise any one of the following modes one, two and three:

[0169] Mode one: determining the minimum value and the maximum value of the vertical coordinate of the selected first positioning frame; comparing the minimum value and the maximum value of the vertical coordinate with the vertical coordinate in the second center coordinate; taking the value closer to the vertical coordinate in the second center coordinate as the vertical coordinate of the center reference point; and taking the horizontal coordinate in the second center coordinate as the horizontal coordinate of the center reference point, i.e. the coordinates of the center reference point Pc (see FIG. 4) are (the horizontal coordinate in the second center coordinate, the value closer to the vertical coordinate in the second center coordinate among the minimum value and the maximum value of the vertical coordinate, the vertical coordinate in the second center coordinate). Figure 21 Mode two: directly determining the center reference point as the center point of the second positioning frame; and mode three: in the case that the center reference point cannot be determined in mode one, determining the center reference point as the center point of the second positioning frame.

[0170] Step 4: extending the lower edge point of the leftmost upper first positioning frame to the left along the horizontal coordinate direction and intersecting with the second positioning frame, taking the intersection point as the left reference point, and extending the lower edge point of the rightmost upper first positioning frame to the right along the horizontal coordinate direction and intersecting with the second positioning frame, taking the intersection point as the right reference point. As shown in FIG. 4, the left reference point is point P1, and the right reference point is point P2. Figure 22

[0171] As another feasible implementation, the plurality of tooth arrangement reference points comprises the center reference point, the left reference point, the right reference point, and the left reference point and / or the right reference point, and the method of determining the plurality of tooth arrangement reference points according to the first center coordinate and the second positioning frame further comprises: determining a vertical straight line between the center reference point and the left reference point, and taking one of the intersection points of the vertical straight line and the first positioning frame as the left reference point; and / or determining a vertical straight line between the center reference point and the right reference point, and taking one of the intersection points of the vertical straight line and the first positioning frame as the right reference point.

[0172] ​Specifically, the determination manners of the center reference point, the left reference point and the right reference point can refer to the above-mentioned embodiments. After obtaining the center reference point and the left reference point, a point between the center reference point and the left reference point and on the line connecting the two points (such as the midpoint of the line) can be randomly selected according to the horizontal coordinates of the center reference point and the left reference point, and then a vertical straight line passing through the point is determined, and the vertical straight line intersects the first positioning frame. If the number of intersection points is two, one of the two intersection points can be selected as the left reference point; if the number of intersection points is more than two, the distances between the intersection points and the center points of the corresponding first positioning frames can be further calculated, and the intersection point corresponding to the minimum distance is taken as the left reference point, so as to improve the sorting effect. Similarly, the right reference point can be obtained.

[0173] S14, sorting all the first positioning frames according to the plurality of tooth arrangement reference points to obtain a sorting result.

[0174] Specifically, a target equation of a straight line on which every two adjacent tooth arrangement reference points in the horizontal direction are located can be determined, the horizontal coordinates of the first positioning frame between the corresponding two tooth arrangement reference points in the horizontal direction are substituted into the corresponding target equation to obtain a vertical coordinate reference value, and according to the size of the vertical coordinate reference value and the vertical coordinate of the center reference point, it is determined that the corresponding tooth of the corresponding first positioning frame is an upper row of teeth or a lower row of teeth.

[0175] The upper row of teeth and the lower row of teeth are sorted from left to right and along the direction of the dental arch, and the tooth position numbers are marked. Specifically, the upper row of teeth and the lower row of teeth can be sorted according to the horizontal coordinates of the first positioning frame and the third-dimensional coordinates, and the tooth position numbers are marked.

[0176] To facilitate understanding, the above three tooth alignment reference points, namely the center reference point Pc, the left reference point P1, and the right reference point P2, will be used as examples to illustrate step S14. Specifically, the first equation of the straight line containing the central reference point and the left reference point, and the second equation of the straight line containing the central reference point and the right reference point are determined. The abscissa of each first central coordinate to the left of the central reference point is substituted into the first equation to obtain the first ordinate reference value, and the abscissa of each first central coordinate to the right of the central reference point is substituted into the second equation to obtain the second ordinate reference value. The first and second ordinate reference values ​​are compared with the ordinate of the central reference point. If the first ordinate reference value is less than the ordinate of the central reference point, the corresponding first positioning frame is determined as the upper tooth positioning frame; otherwise, it is the lower tooth positioning frame. Similarly, if the second ordinate reference value is less than the ordinate of the central reference point, the corresponding first positioning frame is determined as the upper tooth positioning frame; otherwise, it is the lower tooth positioning frame. All upper tooth positioning frames are sorted according to the abscissa of the first central coordinate of the upper tooth positioning frame, and all lower tooth positioning frames are sorted according to the abscissa of the first central coordinate of the lower tooth positioning frame. For example, the first equation of the line containing the central reference point and the left reference point can be determined as y = k1x + b1, and the second equation of the line containing the central reference point and the right reference point can be determined as y = k2x + b2; the x-coordinate of each first central coordinate to the left of the central reference point can be... i Substituting into the first equation, we obtain the corresponding first ordinate reference value y. i And the x-coordinate of each first center coordinate to the right of the center reference point j Substituting into the second equation, we obtain the second ordinate reference value y. j Then, the first and second ordinate reference values ​​are compared with the ordinate of the center reference point. This allows for accurate segmentation of the upper and lower teeth, resulting in a sorting order, such as... Figure 23 As shown.

[0177] S15 uses a pre-trained classification model to identify the tooth type of each tooth.

[0178] It should be noted that before using the pre-trained classification model to identify the tooth type of each tooth, it is necessary to determine whether any teeth are missing in the above ranking results; if so, the ranking results need to be corrected. The specific steps are as follows:

[0179] A1. Determine whether there are missing teeth based on the sorting results.

[0180] Specifically, whether there is a missing tooth in the sorting result can be determined directly according to the center coordinates of each first positioning box in the sorting result. For example, the horizontal coordinates of the center coordinates of each first positioning box can be obtained, and then if it is determined that there are two first positioning boxes, the difference between the horizontal coordinates of the center coordinates corresponding to the two first positioning boxes is greater than the first preset distance threshold and less than the second preset distance threshold, it is determined that there is a missing tooth.

[0181] A2, if there is, determining the central incisor position according to the sorting result, and determining the missing tooth positioning box according to the first positioning boxes on both sides of the missing tooth position.

[0182] Specifically, the horizontal coordinates of the left and right boundaries of the first positioning boxes on both sides of the missing tooth position can be obtained, and then the horizontal coordinate difference of the left and right boundaries of the first positioning box on the left side of the missing tooth position and the horizontal coordinate difference of the left and right boundaries of the first positioning box on the right side of the missing tooth position are obtained, so as to determine the size of the missing tooth positioning box according to the average of the above two horizontal coordinate differences.

[0183] A3, determining the position of the missing tooth positioning box according to the central incisor position, and correcting the sorting result according to the position and the missing tooth positioning box.

[0184] S16, determining the tooth arrangement result according to the sorting result and the tooth type.

[0185] In summary, the tooth arrangement method of the embodiment of the present application can realize full-automatic tooth arrangement based on oral CT images. The upper left, lower left, upper right and lower right teeth of each type of tooth recognized in the tooth arrangement result, such as central incisor, lateral incisor, canine, first premolar, second premolar, first molar, second molar and third molar, are specifically, such as upper left central incisor, upper left lateral incisor...

[0186] Figure 24 is a flowchart of the tooth orthodontic simulation method of the embodiment of the present application.

[0187] As shown in Figure 24 , the tooth orthodontic simulation method can include the following steps:

[0188] S241, determining the dental arch line of the model to be corrected, and generating a target model according to the dental arch line and the pre-stored standard tooth model.

[0189] Among them, the corresponding teeth in the target model and the model to be corrected are recognized according to the tooth arrangement result of the model to be corrected, and the model to be corrected is obtained by using the above-mentioned tooth arrangement method.

[0190] Specifically, since the oral conditions of each person are different, there can be conditions such as missing teeth, supernumerary teeth, etc., and therefore, in order to further accurately locate the tooth position of each tooth (such as the left upper central incisor, the right lower first molar, etc.), the three-dimensional tooth model corresponding to the arrangement result can be input into the pre-trained neural network model as the to-be-corrected model, and the tooth body data and root canal data of each tooth are input into the neural network model; so that the neural network model can identify to obtain an identification result. The identification result includes the correspondence between the target model and the teeth in the to-be-corrected model, so as to determine the tooth position of each tooth, whether each tooth is a to-be-corrected tooth, and the distortion category of the to-be-corrected tooth; the pre-trained classification model is a neural network model, including a convolution layer, a pooling layer, a normalization layer, and a full connection layer; the neural network model architecture can be as shown in Figure 25

[0191] S242, according to each tooth of the target model, the first adjustment is performed on the corresponding tooth in the to-be-corrected model to realize orthodontic simulation of the dentition in the to-be-corrected model.

[0192] Among them, the first adjustment can include translation, rotation and torsion. Moving the tooth out of the tooth position to the corresponding tooth position is a translation operation; rotating the tooth in the tooth position from the inclined position to the vertical position of the corresponding tooth position is a rotation operation; rotating the tooth in the corresponding tooth position and perpendicular to the tooth position but having an angle with the positive direction to the positive direction is a torsion operation, which can be understood as a self-rotation that ensures the tooth position unchanged.

[0193] In order to improve the orthodontic effect, the second adjustment can be performed on the corresponding tooth according to the distance between the corresponding tooth and its adjacent teeth after the first adjustment.

[0194] Among them, the second adjustment can include translation.

[0195] In order to further improve the orthodontic effect, the minimum distance between the tooth root of the to-be-adjusted tooth and the jaw surface can also be calculated; the minimum distance is used to determine whether the tooth root and the jaw surface intersect; if there is an intersection, the fourth adjustment is performed on the corresponding tooth within the first preset position range.

[0196] ​In one embodiment of the present application, after the adjustment of all single teeth is completed, a first distance between the crown surfaces of each pair of teeth in the upper and lower jaws can also be calculated; and the vertical position of the aligned teeth is adjusted within a second preset position range according to the first distance and the relative position of the crown surfaces of the aligned teeth. Thus, a better orthodontic effect can be obtained. If the first distance exceeds or equals a certain threshold value, and the crown surface of the upper tooth of the aligned teeth is below the crown surface of the lower tooth, the vertical position of the aligned teeth needs to be adjusted within the second preset position range. In the specific calculation, the longitudinal coordinate of the crown surface of the lower tooth can be calculated, and the longitudinal coordinate of the crown surface of the upper tooth is subtracted. If the result is negative and exceeds or equals the threshold value, such as 0, the vertical position of the aligned teeth is adjusted within the second preset position range.

[0197] Specifically, the dental long axis of each tooth can be obtained according to the recognition result, and the corresponding impedance center can be obtained according to the dental long axis; then the dental jaw data of the teeth in the oral CT image is measured; and the teeth are aligned according to the recognition result, the impedance center and the dental jaw data. The above-mentioned dental jaw data includes at least one of tooth width, Bolton index, dental arch width, Spee curve depth, and midline deviation. Thus, the teeth to be corrected can be corrected in the oral CT image according to the type of the distortion of the teeth to be corrected.

[0198] The pre-stored standard tooth model is a normal and neat teeth model, and the tooth types include central incisors, lateral incisors, canines, first premolars, second premolars, first permanent molars, second permanent molars and third permanent molars. The teeth are divided into right upper jaw, left upper jaw, left lower jaw and right lower jaw according to the central incisors as the dividing line. The steps of orthodontic treatment specifically include:

[0199] Step 1: When aligning the teeth in the right upper jaw, the teeth are aligned in order of central incisors, lateral incisors, canines, first premolars, second premolars, first permanent molars, second permanent molars and third permanent molars.

[0200] Step 2: When the tooth axis is located on the dental arch line in the dental jaw data, the tooth to be corrected and its adjustment parameters are determined according to the impedance center and the inclination rule of the corresponding teeth in the pre-stored standard tooth model when the teeth are arranged, and the tooth to be corrected is adjusted according to the adjustment parameters to align the teeth in the right upper jaw. The adjustment parameters include at least one of the translation distance, the rotation angle and the torsion angle.

[0201] Specifically, the translation distance includes horizontal translation distance, forward and backward translation distance, and vertical translation distance; and the rotation angle includes buccal and lingual rotation angle, and mesial and distal rotation angle. It should be noted that if the included angle between the tangent of the incisal line of the anterior teeth and the tangent of the dental arch line is greater than a preset threshold value (such as greater than 2-3°), it is considered that the torsion angle needs to be calculated.

[0202] Step3: align the dental arches of the left upper jaw, the right lower jaw and the left lower jaw according to the alignment mode of the right upper jaw dental arch.

[0203] Step4: calculate the first distance between the dental crown surfaces of the paired teeth of the upper and lower jaws.

[0204] Step5: If there is a paired tooth with a negative first distance, fine-tune the vertical position of the paired tooth to make the first distance of the paired tooth positive.

[0205] Step6: Calculate the minimum distance between the dental root and the jaw surface.

[0206] Step7: If the minimum distance is negative, fine-tune the torque of the teeth to make the minimum distance positive while ensuring that the tooth axis is on the dental arch line and the occlusion relationship with the paired teeth is normal.

[0207] As a feasible implementation, the third adjustment can also be directly performed on the corresponding teeth of the model to be corrected according to the adjustment parameters input by the user. The user can be a dentist with rich experience, and the adjustment parameters input by the user can be determined by the dentist according to experience. The third adjustment can include translation, rotation and torsion.

[0208] Optionally, after the dental arches are aligned, the adjustments made to the teeth can also be obtained, and the teeth to be adjusted and the adjustment parameters can be printed in the form of a report, so that the actual treatment strategy can be determined according to the results of the dental alignment.

[0209] It should be noted that, for the convenience of understanding, the recognition boxes used in the drawings of the embodiments of the present application are all two-dimensional recognition boxes, and are actually three-dimensional recognition boxes.

[0210] In summary, the orthodontic simulation method of the embodiments of the present application can be based on the results of the dental alignment of the oral CT image to adjust the teeth, thereby providing a reference for actual treatment.

[0211] It is to be appreciated that the above description and the examples that follow are intended to be illustrative only and that changes can be made to the description, either functionally or chronologically, as well as changes being made concerning which elements of the description and / or examples are employed per se, all without departing from the spirit and scope of the application. It should be further appreciated that the logic and / or steps described in the flow diagrams and / or otherwise described herein can be considered as a sequence of executable instructions or a computer program product that can be executed by a computer, such as a computer-based system, processor, or other system that can fetch instructions from a instruction execution system, apparatus or device and execute the instructions. In this regard, the "computer-readable medium" can be any medium that can contain, store, communicate, propagate or transport the program for use by or in connection with the instruction execution system, apparatus or device. The computer readable medium can comprise any one of the following: an electronic connection having one or more wires (electronic devices), a portable computer diskette (magnetic devices), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, and a portable compact disc read-only memory (CDROM). Further, the computer readable medium can even be paper or other suitable medium upon which the program is printed, as the program can be electronically captured, via, for instance, optical scanning of the paper or other medium, then compiled, interpreted or otherwise processed in a suitable manner if necessary, and then stored in a computer memory.

[0212] It should be understood that aspects of the application can be implemented in hardware, software, firmware or combinations thereof. In the above embodiments, various steps or methods can be implemented in software or firmware that is stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, any of the following techniques, which are known in the art and practice of the present application, can be used: a combination of logic gates, discrete logic, application specific integrated circuits (ASICs), programmable gate arrays (PGAs), field programmable gate arrays (FPGAs), and / or other implementations to be designed and constructed by persons of ordinary skill in the art.

[0213] In the description of the present application, reference has been made to descriptive terms such as "one embodiment", "some embodiments", "an example", "a specific example" or "some examples" means that a particular feature, structure, material or characteristic is included in at least one embodiment or example of the present application. The illustrative examples given are not necessarily mutually exclusive, and it is appreciated that the specific features, structures, materials or characteristics described can be combined in any suitable manner in one or more embodiments or examples.

[0214] In the description of the application, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", "axial", "radial", "circumferential" and the like is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the application and simplifying the description, and does not indicate or imply that the device or element referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation on the application.

[0215] In addition, the terms "first", "second" are only for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the technical features indicated. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the application, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise explicitly specified and limited.

[0216] In the present application, unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting", "fixing" and the like should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or it can be integrated; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium; it can be the internal communication of two elements or the interaction relationship between two elements, unless otherwise explicitly limited. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0217] In the present application, unless otherwise explicitly specified and limited, the first feature is "on" or "under" the second feature, which can be direct contact between the first and second features, or indirect contact between the first and second features through an intermediate medium. Moreover, the first feature "above", "above" and "above" the second feature can be directly above or obliquely above the first feature, or only indicate that the horizontal height of the first feature is higher than that of the second feature. The first feature "below", "below" and "below" the second feature can be directly below or obliquely below the first feature, or only indicate that the horizontal height of the first feature is less than that of the second feature.

[0218] Although the embodiments of the application have been shown and described above, it should be understood that the above embodiments are exemplary and cannot be understood as a limitation on the application, and those skilled in the art can make changes, modifications, replacements and variations to the above embodiments within the scope of the application.

Claims

1. A tooth segmentation method, characterized by, The method comprises the following steps: Obtaining a CT image of a tooth to be segmented; Locating a whole tooth region in the CT image of the tooth to be segmented by using a first neural network trained in advance; Locating a single tooth region in the whole tooth region by using a second neural network trained in advance; Performing mask segmentation on the single tooth region by using a third neural network trained in advance to obtain tooth data of the corresponding tooth; The method further comprises: Determining whether there is mislocation or missed location in the single tooth region; If there is, correcting the single tooth region according to a 3D positioning frame corresponding to the mislocation or missed location to correct the tooth data. The determination of whether there is mislocation in the single tooth region comprises: Counting the number of segmented slices according to the tooth data to obtain a number of segmented slices; If the number of segmented slices is greater than a first threshold, it is determined that there is mislocation in the single tooth region.

2. The tooth segmentation method of claim 1, wherein, The correction of the single tooth region according to the 3D positioning frame corresponding to the mislocation to correct the tooth data comprises: excluding the 3D positioning frame corresponding to the segmented slice, and performing mask segmentation on the corrected single tooth region by using the third neural network trained in advance to obtain tooth data of the corresponding tooth. The locating of the single tooth region in the whole tooth region by using the second neural network trained in advance comprises: Performing three-dimensional Otsu threshold segmentation, adaptive threshold segmentation and morphological processing on the whole tooth region in sequence to obtain first priori knowledge data; 3. The tooth segmentation method of claim 1, wherein, Locating the single tooth region in the whole tooth region according to the first priori knowledge data and the whole tooth region by using the second neural network trained in advance. The mask segmentation of the single tooth region by using the third neural network trained in advance comprises: Performing three-dimensional Otsu threshold segmentation, adaptive threshold segmentation and morphological processing on the single tooth region in sequence to obtain second priori knowledge data; 4. The tooth segmentation method of claim 1, wherein, Performing mask segmentation of the single tooth region according to the single tooth region and the second priori knowledge data by using the third neural network. The method further comprises:

5. The tooth segmentation method of claim 1, wherein, Performing mask segmentation on a root canal in the single tooth region by using the third neural network trained in advance to obtain root canal data. The data used to train the first neural network, the second neural network and the third neural network is labeled in the following manner: Performing three-dimensional Otsu threshold segmentation, adaptive threshold segmentation and morphological processing on the CT training image of the tooth in sequence to obtain a tooth image to be labeled; Performing mask segmentation on the tooth image to be labeled by using a tooth region model trained to obtain a CT training image of a tooth to be labeled, wherein the tooth region model is trained by a residual-based 3D semantic segmentation artificial neural network; Performing tooth labeling correction on the CT training image of the tooth to be labeled; Determining a three-dimensional frame of a single tooth according to tooth labeling to label a single tooth region; determine a three-dimensional frame of a whole tooth region according to three-dimensional frames of all single teeth, to mark the whole tooth region; acquire data for training the first neural network, the second neural network and the third neural network according to the mark in the oral CT training image.

6. The tooth segmentation method of claim 1, wherein, determine whether there is a missing positioning in the single tooth region, including: calculate a gap between two adjacent teeth according to the single tooth region, and determine whether a missing positioning search is needed according to the gap and a size threshold; if needed, divide a gap position grid, and search with each grid as a center of a search frame; segment a region corresponding to the search frame by using the third neural network, and determine whether there is a missing positioning according to a segmentation result.

7. The tooth segmentation method of claim 6, wherein, The method further includes: calculating a size average of all 3D positioning frames corresponding to the single tooth region; set a size of the search frame as the size average.

8. A dental surface data acquisition method, characterized by, The method includes: acquire tooth data according to the tooth segmentation method in any one of claims 1-7; perform three-dimensional Gaussian smoothing on the tooth data; extract tooth grid data from the smoothed tooth data by using a Marching Cube algorithm; perform three-dimensional Laplace smoothing on the extracted tooth grid data; determine redundant points according to the smoothed tooth grid data and a sharp degree of a grid boundary, and remove the redundant points, and splice remaining points into a surface to obtain the tooth point cloud surface data.

9. The tooth face data acquisition method of claim 8, wherein, The determination of the redundant points according to the smoothed tooth grid data includes: for each vertex of a boundary of the smoothed tooth grid data, calculate an included angle of normal vectors of each two adjacent grid surfaces containing the corresponding vertex; determine a maximum included angle corresponding to each vertex according to the included angle corresponding to the vertex; determine an adjustment term of an edge folding cost corresponding to each vertex according to the maximum included angle corresponding to the vertex; perform a simplification operation by using an edge folding algorithm and the adjustment term of the edge folding cost of each vertex to obtain the redundant points.

10. The tooth face data acquisition method of claim 9, wherein, The determination of the adjustment term of the edge folding cost corresponding to each vertex according to the maximum included angle corresponding to the vertex includes: calculate a number of adjacent vertices of each vertex; determine the adjustment term of the edge folding cost corresponding to each vertex according to the maximum included angle corresponding to the vertex and the number of adjacent vertices of each adjacent vertex of the corresponding vertex.

11. A computer readable storage medium having stored thereon a computer program, characterized in that The computer program, when executed by a processor, implements the tooth segmentation method in any one of claims 1-7, or the tooth surface data acquisition method in any one of claims 8-10.

12. An oral CT apparatus comprising a memory, a processor, and a computer program stored on the memory, characterized by, The computer program, when executed by a processor, implements the tooth segmentation method in any one of claims 1-7, or the tooth surface data acquisition method in any one of claims 8-10.

Images