A Progressive Three-Dimensional Tooth Automatic Segmentation Method, Device and Storage Medium

Abstract

The present invention discloses a progressive three-dimensional automatic tooth segmentation method, an electronic device, and a storage medium. The method is divided into two parts: tooth pre-segmentation and tooth secondary segmentation. Among them, the role of pre-segmentation is to remove the interference and invalid regions of teeth, and the role of tooth secondary segmentation is to separate individual teeth after the interference is removed. Specifically, it includes: preprocessing the historical tooth data and the current tooth data to be segmented to obtain the historical point cloud data and the current point cloud data to be segmented; then constructing a tooth pre-segmentation network and a tooth secondary segmentation network to obtain a tooth segmentation model with a progressive structure; training the tooth pre-segmentation network and the tooth secondary segmentation network respectively using the point cloud data; and finally inputting the current point cloud data to be segmented into the overall network with a progressive structure to achieve automatic tooth segmentation. The present invention can solve the defects of the current tooth segmentation methods, such as poor anti-interference ability, low performance, and mostly relying on manually designed operators.

Description

[0001] Technical Field

[0002] The present invention belongs to the technical field of image recognition, and particularly relates to a progressive three-dimensional tooth automatic segmentation method, an electronic device, and a storage medium. Background Art

[0003] With the development of society and the improvement of living standards, people pay more attention to tooth beauty. Although the informatization level of hospitals is constantly improving, the current automation level of the tooth beauty restoration process is still not high enough. For example, during tooth beauty restoration, it often relies on manual judgment by doctors, which has disadvantages such as strong subjectivity and large errors. Therefore, there is a practical need for automatic analysis of tooth beauty, and the premise of automatic analysis of tooth beauty is to determine which tooth the restoration corresponds to, that is, automatic segmentation of teeth is required.

[0004] Tooth segmentation belongs to the problem of three-dimensional object segmentation. Traditional methods are usually based on handcrafted features designed based on personal experience. These features are shallow features, so when the algorithm is used in a specific system, the segmentation robustness is weak and the segmentation performance is not high. In recent years, with the rapid development of deep learning, deep neural networks have great advantages in three-dimensional object segmentation. Through an end-to-end learning process, deep learning methods can automatically learn highly representative deep features from the input raw data and use the learned features for automatic segmentation.

[0005] However, current three-dimensional tooth segmentation methods based on deep learning generally do not consider the anti-interference problem of tooth data. In practice, due to design problems of oral scanners and operator usage problems, etc., the collected original tooth data inevitably contains various interference noises and invalid regions. This problem causes the performance of both traditional methods and deep methods to decrease significantly. Therefore, designing a tooth automatic segmentation algorithm for the interference problem is a current technical difficulty. On the other hand, the feature extraction ability of existing deep neural networks for tooth segmentation needs to be improved. Existing segmentation neural networks do not simultaneously focus on global and local tooth information during feature extraction, resulting in weak discriminative ability of features.

[0006] Through the above analysis, the problems and defects existing in the prior art are as follows:

[0007] (1) Many existing tooth segmentation methods are based on manually designed operators;

[0008] (2) Some methods using deep learning generally do not consider the interference problem of tooth data;

[0009] (3) The feature extraction performance of current tooth segmentation deep neural networks needs to be improved.

[0010] The difficulty in solving the above problems and defects is:

[0011] At present, most tooth segmentation methods are still based on manually designed operators, with relatively low segmentation efficiency, low accuracy, and poor robustness. While a few methods using deep learning generally do not consider the interference problem of tooth data, resulting in a significant decline in performance in practical applications. At the same time, the feature extraction performance of current tooth segmentation deep neural networks needs to be improved, and a feature extraction structure with stronger discriminative ability needs to be designed, so as to improve the efficiency of tooth segmentation and thus save the time required for tooth repair.

[0012] The significance of solving the above problems and defects is as follows:

[0013] Implementing automatic tooth segmentation is of great significance for tooth beauty. At the current stage, in practical applications, there is a lack of an automatic segmentation deep neural network specifically optimized for the specific object of three-dimensional teeth. Existing tooth segmentation methods either adopt traditional methods or directly use non-specific deep neural networks, lacking special consideration of factors such as feature extraction efficiency and anti-interference. The present invention specifically proposes a progressive tooth segmentation method for this problem. This method is based on deep learning methods, uses the idea of progressive segmentation, and combines many technologies such as hierarchical feature extraction, multi-scale combined features, encoder-decoder structure, bias attention, and neighborhood embedding. The accuracy of the method is higher than that of other methods, and it has a strong anti-interference advantage, and has good application value in tooth beauty. Summary of the Invention

[0014] Aiming at the problems existing in the prior art, the present invention provides a progressive three-dimensional tooth automatic segmentation method, device, and storage medium.

[0015] In the first aspect, the present invention provides a progressive three-dimensional tooth automatic segmentation method, and the progressive three-dimensional tooth automatic segmentation method includes:

[0016] Step 1: Obtain historical tooth data from a database and current tooth data to be segmented using a collection device, and preprocess the historical tooth data and the current tooth data to be segmented to convert the original tooth data into point cloud data; divide the point cloud data converted from the historical tooth data into a tooth segmentation training set and a tooth segmentation test set; the tooth segmentation training set is divided into a tooth pre-segmentation training set and a tooth secondary segmentation training set;

[0017] Step 2: Construct a tooth pre-segmentation network with a binary classification function;

[0018] Step 3: Construct a tooth secondary segmentation network with a multi-classification function;

[0019] Step 4: Train the tooth pre-segmentation network using the tooth segmentation training set and train the tooth secondary segmentation network using the tooth secondary segmentation training set to obtain a trained tooth segmentation model with a progressive structure;

[0020] Among them, the tooth segmentation model with a progressive structure is composed of a tooth pre-segmentation network and a tooth secondary segmentation network, and the two are connected in a cascaded manner; during the segmentation process, the input data is pre-segmented through the tooth pre-segmentation network and then secondarily segmented through the tooth secondary segmentation network to obtain the final segmentation result, realizing the connection on the data;

[0021] Step 5: Input the point cloud data converted from the currently to-be-segmented tooth data into the trained tooth segmentation model with a progressive structure to obtain the tooth segmentation result.

[0022] In a second aspect, the present invention provides an electronic device, characterized in that the electronic device includes a memory and a processor; the memory stores a computer program, and when the computer program is executed by the processor, the processor is enabled to execute a progressive three-dimensional tooth automatic segmentation method described in the first aspect.

[0023] In a third aspect, the present invention provides a computer-readable storage medium storing a computer program, and when the computer program is executed by a processor, the processor is enabled to execute a progressive three-dimensional tooth automatic segmentation method described in the first aspect.

[0024] The present invention proposes a complete set of progressive three-dimensional tooth automatic segmentation technologies, which can solve the defects of existing traditional tooth segmentation methods and deep learning methods; compared with the prior art, the present invention also has the following advantages:

[0025] (1) Aiming at the problem of interference in tooth data, the present invention introduces the idea of progressive tooth segmentation, which can efficiently remove invalid and interference regions;

[0026] (2) The present invention uses technologies such as hierarchical feature extraction, multi-scale feature combination, and neighborhood embedding strategy, which can better extract robust features and effectively combine local features and global features;

[0027] (3) The reverse interpolation and skip connection technologies designed by the present invention can obtain tooth features with stronger discriminative ability;

[0028] (4) The bias attention module designed by the present invention is superior to the original self-attention mechanism and can better extract global features. Description of the Drawings

[0029] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following will briefly introduce the accompanying drawings required for use in the embodiments of the present invention. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative efforts.

[0030] Figure 1 It is the schematic diagram of the principle of the progressive three-dimensional automatic tooth segmentation method provided by the embodiments of the present invention;

[0031] Figure 2 It is the overall framework diagram of the progressive three-dimensional automatic tooth segmentation system provided by the embodiments of the present invention;

[0032] Figure 3 It is the schematic diagram of the overall structure of the pre-segmentation network provided by the embodiments of the present invention;

[0033] Figure 4 It is the schematic diagram of the point set feature extraction structure of the pre-segmentation network provided by the embodiments of the present invention;

[0034] Figure 5 It is the schematic diagram of the MLP structure in the point set feature extraction of the pre-segmentation network provided by the embodiments of the present invention;

[0035] Figure 6 It is the schematic diagram of the overall structure of the secondary segmentation network provided by the embodiments of the present invention;

[0036] Figure 7 It is the schematic diagram of the neighborhood embedding module of the secondary segmentation network provided by the embodiments of the present invention;

[0037] Figure 8 It is the schematic diagram of the attention module of the secondary segmentation network provided by the embodiments of the present invention;

[0038] Figure 9 It is the schematic diagram of the visualization of the pre-segmentation result provided by the embodiments of the present invention;

[0039] Figure 10 It is the schematic diagram of the visualization of the tooth segmentation result provided by the embodiments of the present invention;

[0040] Figure 11 It is the schematic diagram of the computer-readable storage medium for the progressive three-dimensional automatic tooth segmentation provided by the embodiments of the present invention. Specific Embodiments

[0041] In order to make the objectives, technical solutions and advantages of the present invention clearer and more understandable, the following further details the present invention in conjunction with embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] In view of the problems existing in the prior art, the present invention provides a progressive three-dimensional tooth automatic segmentation method, an electronic device and a storage medium. The present invention will be described in detail below with reference to the accompanying drawings.

[0043] As Figure 1 described above, the progressive three-dimensional tooth automatic segmentation method provided by the embodiments of the present invention includes the following steps:

[0044] Step 1: Obtain historical tooth data from the database and current tooth data to be segmented by the acquisition device, and preprocess the historical tooth data and the current tooth data to be segmented to convert the original tooth data into point cloud data; divide the point cloud data converted from the historical tooth data into a tooth segmentation training set and a tooth segmentation test set; the tooth segmentation training set is divided into a tooth pre-segmentation training set and a tooth secondary segmentation training set;

[0045] Step 2: Construct a tooth pre-segmentation network with a binary classification function;

[0046] Step 3: Construct a tooth secondary segmentation network with a multi-classification function;

[0047] Step 4: Train the tooth pre-segmentation network using the tooth segmentation training set and train the tooth secondary segmentation network using the tooth secondary segmentation training set to obtain a trained tooth segmentation model with a progressive structure;

[0048] Among them, the tooth segmentation model with a progressive structure is composed of a tooth pre-segmentation network and a tooth secondary segmentation network, and the structures of the two are not connected; in the segmentation process, the input data is pre-segmented through the tooth pre-segmentation network and secondarily segmented through the tooth secondary segmentation network to obtain the final segmentation result, realizing the connection on the data;

[0049] Step 5: Input the point cloud data converted from the current tooth data to be segmented into the trained tooth segmentation model with a progressive structure to obtain the tooth segmentation result.

[0050] The advantages and positive effects of the present invention are as follows: The progressive three-dimensional tooth automatic segmentation method provided by the present invention is divided into two parts: tooth pre-segmentation and tooth secondary segmentation, and the two parts use different network structures. Among them, tooth pre-segmentation can effectively remove interference factors from the original tooth data, and tooth secondary segmentation performs a single tooth segmentation task.

[0051] The present invention discloses a progressive three-dimensional automatic tooth segmentation method, an electronic device, and a storage medium. The method is divided into two parts: tooth pre-segmentation and tooth secondary segmentation. The role of pre-segmentation is to remove the interference and invalid areas of the teeth, and the role of tooth secondary segmentation is to separate individual teeth after the interference is removed. Specifically, it includes: preprocessing the historical tooth data and the current tooth data to be segmented, then constructing a tooth pre-segmentation network with a binary classification function for the whole set of teeth, and then constructing a tooth secondary segmentation network; training the tooth pre-segmentation network and the tooth secondary segmentation network respectively using point cloud data; finally, inputting the current point cloud data to be segmented into the overall network with a progressive structure to achieve automatic tooth segmentation. The present invention can solve the defects of the current tooth segmentation methods, such as poor anti-interference ability, low performance, and mostly relying on manually designed operators.

[0052] The technical solution of the present invention will be further described below in conjunction with specific embodiments.

[0053] Embodiment 1

[0054] In the current traditional methods and deep methods applied to three-dimensional object segmentation, there is basically no use of tooth data as a data set. When applying tooth data to existing deep methods, the segmentation accuracy is not high. With the development of deep learning and the advent of the big data era, the demand for artificial intelligence in oral clinical medicine is also increasing. It is particularly important to automatically complete tooth segmentation by using deep methods through a computer. Constructing a deep segmentation method applied to tooth data can effectively assist clinicians in dental aesthetic restoration. Therefore, please refer to Figure 1 , Figure 1 FIG. is a schematic diagram of a progressive three-dimensional automatic tooth segmentation method provided by an embodiment of the present invention. The present embodiment provides a progressive three-dimensional automatic tooth segmentation method, which includes the following steps:

[0055] Step 1: Obtain historical tooth data from a database and current tooth data to be segmented when a collection device acquires them, and preprocess the historical tooth data and the current tooth data to be segmented to convert the original tooth data into point cloud data; divide the point cloud data converted from the historical tooth data into a tooth segmentation training set and a tooth segmentation test set; the tooth segmentation training set is divided into a tooth pre-segmentation training set and a tooth secondary segmentation training set.

[0056] Specifically, first obtain historical tooth data from a database, and collect the tooth data of a patient using an oral scanner, and use the collected tooth data as the current tooth data to be segmented; convert all tooth data in ply format into point cloud data; divide the point cloud data converted from the historical tooth data to respectively form a tooth segmentation training set and a tooth segmentation test set.

[0057] Among them, the original formats of the historical tooth data and the current tooth data to be segmented are both in ply format.

[0058] Step 2: Construct a tooth pre-segmentation network with binary classification function;

[0059] Reference Figure 3 and Figure 4 , Figure 3 is the overall structure diagram of the tooth pre-segmentation network with binary classification function. The tooth pre-segmentation network adopts an Encoder-Decoder structure. Figure 4 is the schematic diagram of the point set feature extraction module structure of the pre-segmentation network; the encoding part of the tooth pre-segmentation network consists of three parts, namely a sampling layer, a grouping layer, and a feature extraction layer.

[0060] The sampling layer uses the Iterative Farthest Point Sampling (FPS) algorithm to downsample the point set, reducing the input point set from scale N1 to a smaller scale N2. In the present invention, a point is randomly selected as the initial point as the selected sampling point, the distance between each point in the unselected sampling point set and the selected sampling point set is calculated, the point with the largest distance is added to the selected sampling point set, and then the distance is updated, and the loop iteration continues until the first preset number of sampling points is obtained. The first preset number is a preset positive integer value.

[0061] The grouping layer is used to find the second preset number K of adjacent points centered on the sampling points obtained by the sampling layer through the K-Nearest Neighbor (KNN) algorithm; the K adjacent points are formed into a local neighborhood to complete the grouping of the sampling points, and each grouping corresponds to a local neighborhood.

[0062] Among them, the second preset number is a preset positive integer value.

[0063] The feature extraction layer is used to extract features from the K groupings through the PointNet network to obtain the characterization results of the local neighborhoods; the feature extraction process includes feature embedding (Embedding) to change the number of feature channels, pooling operation (Pooling), and multi-layer perceptron (MLP) network layers.

[0064] Aiming at the possible problem of sparse point cloud, the present invention adopts a multi-scale grouping strategy. For each center point of the current layer, the query ball algorithm (that is, a certain radius is defined, and the points within the radius ball are used as adjacent points) is used, and query balls with different radii are selected to obtain multiple local neighborhoods with the same center but different scales. The local neighborhoods are characterized respectively, and all the characterizations are concatenated.

[0065] In the decoder, the present invention designs a method of inverse interpolation to implement the decoder structure. The main function is to obtain discriminative pointwise features through inverse interpolation and skip connections. When performing inverse interpolation, the present invention interpolates more point features based on the inverse distance weighting rule (i.e., the farther the point, the less contribution to the feature) to achieve upsampling. The present invention obtains local-level information through skip connections and concatenates it with the global-level information to obtain features with strong discriminative power.

[0066] The specific interpolation formula is as follows:

[0067]

[0068] where w i is the calculated point weight, which is inversely proportional to the distance. The closer the distance, the greater the influence; f (j) is the value of the unknown point, and f i (j) are the values of the known points; K represents selecting K points from the known point set for interpolation calculation; p represents the influence degree of the distance on the weight; d(x, x i ) represents the distance between the unknown point and the known point.

[0069] Reference Figure 5 , Figure 5 is the schematic diagram of the MLP structure in the point set feature extraction provided by the embodiment of the present invention. In the verification process of this embodiment, the parameter design of each layer of the MLP structure in the point set feature extraction is specifically shown in Table 1, and the padding method in the convolution process is zero padding.

[0070] In the verification process of this embodiment, the parameter design of each layer of the MLP structure in the feature propagation is specifically shown in Table 2, and the padding method in the convolution process is zero padding.

[0071] Table 1 Parameter design of each layer in point set feature extraction

[0072]

[0073] Table 2 Parameter design of each layer in feature propagation

[0074]

[0075] Step 3: Construct a tooth secondary segmentation network with multi-classification function;

[0076] The tooth secondary segmentation network adopts the structure of Transformer. The main reason is that all operations of Transformer can be executed in parallel and are order-independent. Theoretically speaking, it can replace the convolution operation in CNN calculation and has better generality. By using the inherent order invariance of Transformer, the order of the point cloud data is avoided from being defined.

[0077] The tooth secondary segmentation network with multi-classification function consists of three parts, namely the input embedding layer, the attention layer, and the segmentation layer.

[0078] Figure 6 It is a schematic diagram of the overall structure of the tooth secondary segmentation network provided by an embodiment of the present invention, where LBR combines linear operation (Linear), batch normalization (BN), and rectified linear unit activation layer (ReLU). LBRD means that a dropout layer is connected after LBR.

[0079] During the verification process of this embodiment, the specific design of the overall structure parameters of the network is shown in Table 3, N in represents the dimension of the input point number, D in represents the dimension of the input channel.

[0080] Table 3 Overall structure parameters of the network

[0081]

[0082] The input embedding layer is used to map the point cloud data from the Euclidean space to a 128-dimensional high-dimensional space by integrating local neighborhood features on the basis of the original features of the point cloud data; integrating local neighborhood features on the basis of the original features of the point cloud data means integrating local neighborhood features by searching and grouping each point based on the Euclidean distance through the KNN algorithm on the basis of the original features, so as to optimize the original features of the point cloud data and enhance the local feature extraction ability of the tooth secondary segmentation network;

[0083] Reference Figure 7 , Figure 7 It is a schematic diagram of the neighborhood embedding module of the secondary segmentation network provided by an embodiment of the present invention. During the verification process of this embodiment, the specific design of the neighborhood embedding structure parameters is shown in Table 4.

[0084] Table 4 Neighborhood embedding structure parameters

[0085]

[0086] The attention layer includes a self-attention module and a bias attention module. Among them, the self-attention module generates fine attention features based on the input features of the global context; the bias attention module uses the offset between the input of the self-attention module and the attention features to replace the attention features;

[0087] First, the self-attention module takes the sum of the input word embeddings and positional encodings as input, and calculates three vectors for each word through a trained linear layer: query, key, and value; the attention weights between any two words are obtained by matching the query and key vectors. The output attention weights for each word are related to all the input features, so it can learn the global context. Biased attention is an effective upgrade of self-attention. Its working principle is to replace the attention features with the offset between the input of the self-attention module and the attention features. The optimization process of offset attention can be approximately understood as a Laplacian process. Figure 8 It is a schematic diagram of the attention module of the secondary segmentation network provided by the embodiment of the present invention.

[0088] F out = OA(F in ) = LBR(F in - F sA ) + F in

[0089] where, OA is the abbreviation of biased attention, SA is the abbreviation of self-attention; F in is the input feature, F SA is the output feature of the attention module, F out is the final output feature.

[0090] The segmentation layer, through the pooling operation on the features after the attention layer, encodes the pooled features into a target label feature vector, cascades it with the global feature, and then predicts the segmentation score to achieve the final segmentation. Specifically, the point cloud is divided into N parts, and a label corresponding to the part where each point in the point cloud is located is predicted. The pooled features are encoded into a 64-dimensional one-hot target label feature vector, and then the global feature and the features of each point are cascaded; then the segmentation score of each point is predicted, and the category with the highest score is determined as the finally predicted category.

[0091] Step 4: Use the tooth segmentation training set to train the tooth pre-segmentation network and use the tooth secondary segmentation training set to train the tooth secondary segmentation network to obtain a trained tooth segmentation model with a progressive structure;

[0092] Use the tooth pre-segmentation training set to train the tooth pre-segmentation network to obtain a trained tooth pre-segmentation model. Please refer to Figure 9 , Figure 9It is a visualization schematic diagram of the pre-segmentation result of teeth provided by an embodiment of the present invention. Among them, figure (a) shows the teeth before pre-segmentation with interference, figure (b) shows the interference filtered out through the pre-segmentation step, and figure (c) shows the teeth after removing the interference. Teeth pre-segmentation is a binary segmentation task, aiming to separate teeth from noise points. In the training set, the teeth part and the noise part are distinguished using labels, and then the teeth pre-segmentation network is trained using the teeth pre-segmentation training set to obtain a trained teeth pre-segmentation model. The teeth pre-segmentation test set is passed through the trained teeth pre-segmentation model to achieve teeth pre-segmentation. In the pre-segmentation task, the commonly used cross-entropy classification loss is used:

[0093]

[0094] where N represents the total number of samples; y i represents the label of sample i, with the positive class being 1 and the negative class being 0; p i represents the probability that sample i is predicted as the positive class, log represents the natural logarithm, and each sample is a point cloud data annotating the interference and teeth regions;

[0095] At the start of training, the network will randomly rotate the point cloud data for data augmentation. The angle of each random rotation is set to 20. By default, no perturbation is added to the teeth data. A total of 200 epochs are set for the entire training process. Multiple batches are set according to the total number of teeth data. During each epoch, multiple batches are input in sequence for training. The size of each batch is set to 8, and the learning rate is 0.001. During the training process, the learning rate will decrease at a constant value e -4 and the Adam optimizer is used to update the parameters throughout the training process.

[0096] The teeth secondary segmentation network is trained using the teeth secondary segmentation training set to obtain a trained teeth secondary segmentation model. Please refer to Figure 10 , Figure 10 which is a visualization schematic diagram of the teeth segmentation result provided by an embodiment of the present invention. Teeth secondary segmentation is a multi-classification task (15 classes for a single jaw of teeth), aiming to distinguish 14 teeth and the gums. In the teeth secondary segmentation training set, 15 integers from 0 to 14 are used to label different teeth and gums; then the teeth pre-segmentation network is trained using the teeth segmentation training set to obtain a trained teeth segmentation model. The teeth segmentation test set is passed through the trained teeth segmentation model to achieve teeth segmentation.

[0097] The loss function for the teeth secondary segmentation network to perform secondary segmentation is:

[0098]

[0099] Among them, M represents the total number of categories; y ic represents the sign function, which is 0 or 1, taking 1 if the true category of the observed sample j is equal to c, otherwise taking 0; p jc represents the predicted probability that the observed sample j belongs to the category c, and the observed sample is a sample labeled with teeth and gums.

[0100] At the beginning of training, the network will randomly rotate the point cloud data for data augmentation. The angle of each random rotation is set to 20, and perturbations are added to the tooth data. A total of 250 epochs are set in the whole training process. Multiple batches are set according to the total number of tooth data. During each epoch, multiple batches are input in sequence for training. The size of each batch is set to 8, and the learning rate is 0.001. During the training process, the learning rate will decrease at a constant value e -4 and the parameters are updated using the Adam optimizer throughout the training process.

[0101] Specifically, in this embodiment, the Euclidean distance is used as the metric function. The Euclidean distance also known as the Euclidean metric measures the absolute distance between two points in a multi-dimensional space, that is, the true distance between two points in an n-dimensional space, or the natural length of a vector (i.e., the distance from this point to the origin). The distance formula in an n-dimensional space is:

[0102]

[0103] where x, y are any two points in the n-dimensional space; x i , y i , and a = 1, 2,..., n represent the n-dimensional coordinates of x, y.

[0104] The tooth segmentation test set is sequentially input into the trained tooth pre-segmentation model and the tooth secondary segmentation model for segmentation. The tooth data and its labels are input into the Softmax function to output the segmentation labels, and classification labels are obtained through Softmax to achieve tooth segmentation. The Softmax function, also known as the normalized exponential function, can "compress" a K-dimensional vector z = (z1, z1,..., z K ) into another K-dimensional real vector σ(z) such that the range of each element is between (0, 1) and the sum of all elements is 1. Its formula is:

[0105]

[0106] where z j is the output value of the j-th node; K is the number of output nodes, that is, the number of classification categories.

[0107] The specific steps are as follows:

[0108] (1) Using the maximum Euclidean distance D within the class max as the set threshold, compare the minimum Euclidean distance D between the predicted label and the current image bw_min with D max If it is less than D max then it is considered that the predicted classification is correct, otherwise it is considered that the predicted classification is incorrect. And count the number of correct and incorrect classification categories.

[0109] (2) According to the number of correct and incorrect classification categories counted in step (1), calculate the classification accuracy (Accuracy). The specific formula is as follows:

[0110]

[0111] Among them, TP: true positive, the number of samples that are actually positive and predicted as positive;

[0112] FP: false positive, the number of samples that are actually negative and predicted as positive;

[0113] TN: true negative, the number of samples that are actually negative and predicted as negative;

[0114] FN: false negative, the number of samples that are actually positive and predicted as negative.

[0115] According to the automatic tooth segmentation results of each observation sample in the tooth segmentation test set and the annotation situation of each observation sample, update the trained tooth segmentation model.

[0116] In summary, the introduction of the progressive three-dimensional automatic tooth segmentation technology in this embodiment improves the segmentation accuracy through the tooth pre-segmentation idea and accordingly.

[0117] Step Five: Input the point cloud data converted from the current tooth data to be segmented into the trained tooth segmentation model with a progressive structure to obtain the tooth segmentation result.

[0118] Perform tooth pre-segmentation on the current tooth data to be segmented through the trained tooth pre-segmentation model to obtain the tooth pre-segmentation result; then retain the tooth area in the tooth pre-segmentation result and discard the non-tooth area; then input the tooth area into the trained tooth secondary segmentation model to obtain the classification result of each tooth, thereby realizing the final complete tooth segmentation result.

[0119] The experiment designed in this embodiment is demonstrated from the following aspects:

[0120] To illustrate the influence of the pre - segmentation method used in the present invention on the accuracy of the segmentation results, two sets of comparative experiments were conducted. In the first group, the original tooth data was directly input into the tooth pre - segmentation network without adding tooth pre - segmentation. In the second group, tooth pre - segmentation was added. First, tooth pre - segmentation was performed, and then the tooth data was input into the tooth pre - segmentation network. The results of the two experiments were compared. Please refer to Table 5, which lists the results of different tooth segmentation methods before and after introducing pre - segmentation and the idea of pre - segmentation. The results clearly show that the segmentation effect is better after introducing tooth pre - segmentation, and it shows that the accuracy rate and intersection - over - union ratio of the method of the present invention are optimal compared to all other methods. The experimental results prove that the use of the method of the present invention can significantly improve the current tooth segmentation accuracy.

[0121] Table 5 Comparison of the results of different tooth segmentation methods

[0122]

[0123]

[0124] This embodiment proposes a progressive three - dimensional automatic tooth segmentation method, which can solve the defects of insufficient robustness and poor anti - interference ability of existing automatic tooth segmentation methods; this embodiment provides new theories and new method supports for the practical application of tooth segmentation technology, making the automatic tooth segmentation technology more practical and popular; this embodiment can be widely applied to tooth beauty.

[0125] Embodiment 2

[0126] Based on the above - mentioned Embodiment 1, an electronic device for progressive three - dimensional automatic tooth segmentation provided by an embodiment of the present invention includes a data collector, a display, a processor, a communication interface, a memory, a communication bus, and a peripheral application system. Among them, the data collector, the display, the processor, the communication interface, the memory, and the peripheral call system complete mutual communication through the communication bus;

[0127] The data collector is used to collect patient tooth data;

[0128] The display is used to display tooth data and tooth segmentation results;

[0129] The memory is used to store computer programs;

[0130] The peripheral application system is used to realize the functions of the peripheral system, act as a host computer, and call the tooth segmentation results for other application purposes based on tooth segmentation;

[0131] The processor is used to execute the computer programs stored on the memory. When the computer programs are executed by the processor, the following steps are realized:

[0132] Step 1: Obtain historical tooth data from the database and current tooth data to be segmented by the acquisition device, and preprocess the historical tooth data and the current tooth data to be segmented to convert the original tooth data into point cloud data; divide the point cloud data converted from the historical tooth data into a tooth segmentation training set and a tooth segmentation test set; the tooth segmentation training set is divided into a tooth pre-segmentation training set and a tooth secondary segmentation training set;

[0133] Specifically, in Step 1 of the embodiment of the present invention, an oral scanner is used to collect tooth data and perform format conversion.

[0134] Step 2: Construct a tooth pre-segmentation network with a binary classification function;

[0135] Step 3: Construct a tooth secondary segmentation network with a multi-classification function;

[0136] Step 4: Train the tooth pre-segmentation network using the tooth segmentation training set and train the tooth secondary segmentation network using the tooth secondary segmentation training set to obtain a trained tooth segmentation model with a progressive structure;

[0137] Step 5: Input the point cloud data converted from the current tooth data to be segmented into the trained tooth segmentation model with a progressive structure to obtain a tooth segmentation result.

[0138] An electronic device for progressive three-dimensional tooth automatic segmentation provided by an embodiment of the present invention can execute the three-dimensional tooth segmentation method embodiment described above, that is, the three-dimensional tooth segmentation embodiment. The electronic device provided in this embodiment adopts the progressive three-dimensional tooth segmentation method in Embodiment 2 above, solves the vacancy of the depth method applied to tooth segmentation, and can be widely applied to tooth beauty.

[0139] Embodiment 3

[0140] Based on the above Embodiment 1, please refer to Figure 11 , Figure 11 which is a schematic diagram of a computer-readable storage medium for progressive three-dimensional tooth segmentation provided by an embodiment of the present invention. A computer-readable storage medium provided by an embodiment of the present invention stores a computer program thereon. When the above computer program is executed by a processor, the following steps are implemented:

[0141] Step 1: The program starts and necessary initialization is performed.

[0142] Step 2: Obtain historical tooth data from the database and current tooth data to be segmented by the acquisition device, and preprocess the historical tooth data and the current tooth data to be segmented to convert the original tooth data into point cloud data; divide the point cloud data converted from the historical tooth data into a tooth segmentation training set and a tooth segmentation test set; the tooth segmentation training set is divided into a tooth pre-segmentation training set and a tooth secondary segmentation training set.

[0143] Step 3: Construct a tooth pre-segmentation network with a binary classification function;

[0144] Step 4: Construct a tooth secondary segmentation network with a multi-classification function;

[0145] Step 5: Train the tooth pre-segmentation network using the tooth segmentation training set and train the tooth secondary segmentation network using the tooth secondary segmentation training set to obtain a trained tooth segmentation model with a progressive structure;

[0146] Step 6: Input the point cloud data converted from the current tooth data to be segmented into the trained tooth segmentation model with a progressive structure to obtain a tooth segmentation result.

[0147] Step 7: Exit the program when the program needs to end.

[0148] A computer-readable storage medium provided by an embodiment of the present invention can execute the embodiment of the above progressive three-dimensional tooth automatic segmentation method. A computer program of the above progressive three-dimensional tooth automatic segmentation method is stored thereon, which can be used by a mobile terminal and an Internet of Things tooth segmentation device, thereby improving the accuracy of the tooth segmentation device and bringing a good experience to users.

[0149] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented in whole or in part in the form of a computer program product, the computer program product includes one or more computer instructions. When the computer program instructions are loaded or executed on a computer, the processes or functions described in the embodiments of the present invention are generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wire (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.). The computer-readable storage medium may be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0150] As described above, the above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention, any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be covered by the protection scope of the present invention.

Claims

1. A progressive three-dimensional automatic tooth segmentation method, characterized in that, The progressive three-dimensional automatic tooth segmentation method includes: Step 1: Obtain historical tooth data from a database and use a collection device to obtain the current tooth data to be segmented, and preprocess the historical tooth data and the current tooth data to be segmented to convert the original tooth data into point cloud data; divide the point cloud data converted from the historical tooth data into a tooth segmentation training set and a tooth segmentation test set; the tooth segmentation training set is divided into a tooth pre-segmentation training set and a tooth secondary segmentation training set; Step 2: Construct a tooth pre-segmentation network with a binary classification function; the tooth pre-segmentation network with a binary classification function in Step 2 adopts an encoder-decoder structure; the encoding part of the tooth pre-segmentation network consists of three parts, namely a sampling layer, a grouping layer, and a feature extraction layer; Step 3: Construct a tooth secondary segmentation network with a multi-classification function; the tooth secondary segmentation network with a multi-classification function in Step 3 consists of three parts, namely an input embedding layer, an attention layer, and a segmentation layer; The input embedding layer is used to map the point cloud data from the Euclidean space to a 128-dimensional high-dimensional space by integrating local neighborhood features on the basis of the original features of the point cloud data; integrating local neighborhood features on the basis of the original features of the point cloud data means integrating local neighborhood features by searching for each point grouped by the KNN algorithm based on the Euclidean distance on the basis of the original features, so as to optimize the original features of the point cloud data and enhance the local feature extraction ability of the tooth secondary segmentation network; Step 4: Use the tooth segmentation training set to train the tooth pre-segmentation network and use the tooth secondary segmentation training set to train the tooth secondary segmentation network to obtain a trained tooth segmentation model with a progressive structure; Among them, the tooth segmentation model with a progressive structure is composed of a tooth pre-segmentation network and a tooth secondary segmentation network, and the two are connected in a cascaded manner; during the segmentation process, the input data is pre-segmented through the tooth pre-segmentation network and then secondary-segmented through the tooth secondary segmentation network to obtain the final segmentation result, realizing the connection on the data; Step 5: Input the point cloud data converted from the current tooth data to be segmented into the trained tooth segmentation model with a progressive structure to obtain the tooth segmentation result.

2. The progressive three-dimensional automatic tooth segmentation method according to claim 1, wherein The Step 1 includes: Obtain historical tooth data from a database and use a three-dimensional tooth scanning device to collect the tooth data of a patient, and use the collected tooth data as the current tooth data to be segmented; Among them, the original formats of the historical tooth data and the current tooth data to be segmented are both in ply format; Convert all tooth data in ply format into point cloud data; Divide the point cloud data converted from the historical tooth data to form a tooth segmentation training set and a tooth segmentation test set respectively.

3. The progressive three-dimensional automatic tooth segmentation method according to claim 1, wherein The sampling layer is used to first downsample the input training set using the iterative farthest point sampling (FPS) algorithm to reduce the data scale of the training set; then randomly select a sampled point as the initial point; Calculate the distance between each point in the unselected sampling point set and the selected sampling point set, add the point with the largest corresponding distance to the selected sampling point set, and update the distance; iterate in a loop until the first preset number of sampling points is obtained; The grouping layer is used to find the second preset number K of adjacent points centered on the sampling points obtained by the sampling layer through the K-nearest neighbor algorithm KNN; form a local neighborhood with these K adjacent points to complete the grouping of the sampling points, and each grouping corresponds to a local neighborhood; The feature extraction layer is used to extract features of the K groupings through the PointNet network to obtain the representation result of the local neighborhood; the feature extraction process includes feature embedding Embedding to change the number of feature channels, pooling operation Pooling, and multi-layer perceptron layer MLP; The decoding part of the tooth pre-segmentation network is used to upsample the representation result through reverse interpolation, then obtain local information by using skip connections, and then splice the global information obtained by the pooling operation with the local information to obtain discriminative point-by-point features.

4. The progressive three-dimensional tooth automatic segmentation method according to claim 1, wherein The attention layer includes a self-attention module and a bias attention module. Among them, the self-attention module generates fine attention features based on the input features of the global context; the bias attention module uses the offset between the input of the self-attention module and the attention features to replace the attention features; The segmentation layer is specifically used to perform a pooling operation on the features after passing through the attention layer, encode the pooled features into a 64-dimensional one-hot target label feature vector, and concatenate it with the global features; predict the final segmentation score of each point, and determine the category with the highest score as the finally predicted category.

5. The progressive three-dimensional automatic tooth segmentation method according to claim 4, characterized in that, The self-attention module generates fine attention features based on the input features of the global context, including: The self-attention module takes the sum of the input word embedding and the position encoding as the input, and calculates 3 vectors for each word through a trained linear layer: query, key, and value; Obtain the attention weights between any two words by matching the query and key vectors; Define the attention features as the weighted sum of all value vectors and attention weights.

6. The progressive three-dimensional automatic tooth segmentation method according to claim 1, wherein The step four includes: Label different labels for the tooth area and the noise area in the tooth pre-segmentation training set to distinguish teeth and noise; Use the tooth pre-segmentation training set to train the tooth pre-segmentation network to obtain a trained tooth pre-segmentation model; Among them, the loss function used for the pre-segmentation of the tooth pre-segmentation network is the cross-entropy loss: Among them, N represents the total number of samples; represents the sample i label, where the positive class is 1 and the negative class is 0; represents the sample i probability predicted as the positive class, log represents the natural logarithm, and each sample is a point cloud data annotating the interference and tooth regions; Use 15 integers from 0 to 14 to label different teeth and gums in the tooth secondary segmentation training set; Use the tooth secondary segmentation training set to train the tooth secondary segmentation network to obtain a trained tooth segmentation model with a progressive structure; The loss function for the tooth secondary segmentation network to perform secondary segmentation is: Among them, M represents the total number of categories; represents the sign function, which is 0 or 1. If the true category of the observed sample j is equal to c then take 1, otherwise take 0; represents the predicted probability that the observed sample j belongs to the category c The observed samples are samples of labeled teeth and gums.

7. The progressive three-dimensional automatic tooth segmentation method according to claim 1, characterized in that, The step five includes: The teeth data to be currently segmented are passed through the trained teeth pre-segmentation model to achieve teeth pre-segmentation, obtaining a teeth pre-segmentation result; then the teeth regions in the teeth pre-segmentation result are retained, and the non-teeth regions are discarded; and then the teeth regions are input into the trained teeth secondary segmentation model to obtain the classification result of each tooth, thereby achieving the final complete teeth segmentation result.

8. The progressive three-dimensional automatic tooth segmentation method according to claim 1, characterized in that After step five, the progressive three-dimensional teeth automatic segmentation method further includes: The point cloud data in the teeth segmentation test set are successively passed through the trained teeth pre-segmentation network and the teeth secondary segmentation network to respectively complete teeth pre-segmentation and teeth secondary segmentation, so as to achieve automatic teeth segmentation of each sample in the teeth segmentation test set. According to the automatic teeth segmentation results of each observation sample in the teeth segmentation test set and the annotation situation of each observation sample, the trained teeth segmentation model is updated.

9. An electronic device, characterized in that, The electronic device includes a memory and a processor; the memory stores a computer program, and when the computer program is executed by the processor, the processor executes a progressive three-dimensional teeth automatic segmentation method according to any one of claims 1 to 8.

10. A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes a progressive three-dimensional teeth automatic segmentation method according to any one of claims 1 to 8.

Images