Dention crowding degree identification method based on deep learning

Abstract

The invention discloses a dentition crowding degree identification method based on deep learning, and belongs to the technical field of oral medicine. The method comprises the following steps: making three segmentation data sets with a dental arch, a tooth crown edge and a tooth crown width direction as boundaries; building a neural network based on dynamic and expanded edge convolution, and training the neural network to obtain three network models, so as to predict three types of point cloud preparations; post-processing is carried out based on the dental arch line point cloud preparation body to obtain the existing dental arch length; post-processing is carried out based on the dental crown width point cloud preparation body to obtain a due dental arch length; and finally, subtracting the existing dental arch length from the due dental arch length to calculate a crowding degree value. According to the method, the tooth oral scanning model of the patient is input, the crowding degree value can be directly recognized, the problems of long consumed time, low efficiency and the like when a doctor measures the crowding degree manually or through human-computer interaction at present are solved, and accurate and rapid recognition of the dentition crowding degree is achieved.

Description

Technical Field

[0001] The present invention belongs to the technical field of oral medicine and also relates to the field of computer-aided design, and specifically to a method for identifying dental crowding based on deep learning. Background Art

[0002] During the orthodontic treatment process, in order to facilitate the evaluation of the progress and effect of the treatment, it is necessary to regularly conduct a quantitative analysis of the patient's dental crowding, and the doctor can determine the degree of tooth misalignment or crowding based on the crowding value. Patent CN112022383A discloses a method for determining the gap required for orthodontic treatment, which is divided into four gap values, of which the first gap value is the dental crowding. In the seventh edition of the textbook "Orthodontics", the calculation content of the crowding degree is: the difference between the arc length of the dental arch and the existing arc length of the dental arch, or the difference between the required gap and the available gap, is the crowding degree of the dental arch.

[0003] Currently, traditional methods for measuring dental crowding include the copper wire method and the caliper method. Doctors can manually use a copper wire to fit the dental arch from the mesial point of the mandibular first molar along the mesial point of the adjacent teeth to the mesial point of the first molar on the opposite side to form the existing dental arch and straighten the copper wire to calculate the length. The width of each crown in front of the first molar is measured with a vernier caliper, and the sum of the crown widths is calculated to obtain the expected dental arch length. Some existing software also uses human-computer interaction to draw the existing dental arch shape and measure the crown width to obtain the dental crowding. These methods have the problems of long operation time and high labor cost, and are inefficient. Therefore, it is particularly important to propose a method that can efficiently and accurately identify the numerical value of dental crowding. Summary of the Invention

[0004] The purpose of the present invention is to solve the problem that the existing manual or human-computer interaction methods for measuring dental crowding are cumbersome and inefficient. A dental crowding identification method based on deep learning is provided. Only the patient's dental mouth scan model needs to be input to directly identify the crowding value.

[0005] In order to achieve the above objectives, the present invention provides a technical solution: a method for identifying dental crowding based on deep learning, comprising the following steps:

[0006] (1) Create three segmentation datasets based on the dental arch, tooth crown edge, and crown width, and preprocess the datasets;

[0007] (2) A neural network based on dynamic and dilated edge convolution was constructed, and three network models were trained based on three segmentation datasets to output the dental arch line point cloud preparation, the tooth crown point cloud preparation, and the crown width point cloud preparation respectively;

[0008] (3) Post-processing is performed based on the dental arch line point cloud preparation to obtain the existing dental arch length;

[0009] (4) Post-processing is performed based on the crown width point cloud preparation to obtain the expected dental arch length;

[0010] (5) The crowding value is obtained by subtracting the existing dental arch length from the expected dental arch length.

[0011] Preferably, in step (1), three segmented data sets are prepared and preprocessed, including:

[0012] (1-1) Taking the dental arch as the boundary, add label 0 to each point of the point cloud data within the dental arch line, and add label 1 to each point of the point cloud data outside the dental arch line to construct the dataset;

[0013] (1-2) Using the edge of the tooth crown as the boundary, add corresponding labels 1-16 to the point cloud data of each crown, and add label 0 to the point cloud data of the gum area to construct the dataset;

[0014] (1-3) Each crown is divided into inner and outer parts according to the crown width direction, and each point of the inner point cloud data is labeled 0, and each point of the outer point cloud data is labeled 1 to construct the data set;

[0015] (1-4) The three types of data sets are downsampled by the farthest point sampling algorithm so that the number of tooth point cloud points is controlled within the range of [β1,β2], where β1 and β2 are the maximum and minimum values of the tooth point cloud points, respectively;

[0016] (1-5) The three types of data sets are rotated and expanded, and the expanded data are filtered out of noise points and normalized. Finally, the data is stored. The data contained in each point is represented as {x, y, z, n x ,n y ,n z ,l}, where x, y, z are coordinate values, n x 、n y 、n z is the normal vector and l is the label.

[0017] Preferably, in step (2), a neural network based on dynamic and dilated edge convolution is constructed, and three network models are trained to obtain three point cloud preparation bodies, including:

[0018] (2-1) Build a dynamic and dilated edge convolutional neural network, which is divided into a tooth point cloud feature extraction module and a segmentation module;

[0019] The tooth point cloud feature extraction module performs the following operations:

[0020] The EdgeConv dynamic edge convolution module is used to construct the graph structure of each point in the tooth point cloud model using the K-nearest neighbor algorithm, and the local feature information of the point cloud is obtained through convolution and symmetric aggregation functions;

[0021] The Dilated EdgeConv module is used to expand the edge convolution module. First, the K-nearest neighbor algorithm is used to build a graph structure so that the graph structure contains neighboring point information. Then, the farthest point sampling algorithm is used to sparse the graph structure. The global feature information of the point cloud is obtained through convolution and symmetric aggregation functions.

[0022] The segmentation module performs the following operations:

[0023] The feature dimension of each point in the tooth point cloud is downsampled to the corresponding number of labels through the MLP multi-layer perceptron. The output of the MLP multi-layer perceptron is mapped to the probability value of the corresponding label through the SoftMax classifier. The label value with the maximum probability of each point in the tooth point cloud is extracted as the label of the point.

[0024] The network classification labels for the segmentation dataset trained with the tooth crown edge as the boundary are 0-16, and the network classification labels for the other two segmentation datasets are 0-1;

[0025] (2-2) The three tooth segmentation datasets were divided into training set, test set and validation set in proportion. The network was initialized by He normal distribution initialization method, hyperparameters were set, and the three types of training set data were imported into the network for training respectively.

[0026] (2-3) Perform network parameter adjustment, import the validation set data of the three tooth data into the network respectively, adjust the hyperparameters, select the optimal hyperparameters based on the loss function curve and segmentation accuracy, and output three network models with the best performance;

[0027] (2-4) Conducting network testing, importing the test set data of the three tooth data into the corresponding optimal network obtained in step (2-3), outputting the network segmentation accuracy and average intersection-over-union ratio value, and judging whether the network performance meets the requirements;

[0028] (2-5) Inputting the patient's oral scan model containing point cloud data of the dentition, crown and gums into a network model trained with a segmentation dataset bounded by the edge of the tooth crown, and outputting a point cloud preparation of the tooth crown;

[0029] (2-6) The tooth crown point cloud preparation output in step (2-5) is subjected to relevant post-processing to obtain a point cloud model containing only the dentition crown, and the model is input into two network models trained with two segmentation data sets bounded by the dental arch and crown width directions, respectively, to output the dental arch line point cloud preparation and the dental crown width point cloud preparation.

[0030] Preferably, in step (2-6), the output tooth crown point cloud preparation is subjected to relevant post-processing, including:

[0031] (2-6-1) Perform scale restoration on the tooth crown point cloud preparation;

[0032] (2-6-2) Delete the point cloud of the gum area, that is, the points with label 0, and delete the point cloud of the wisdom tooth area, that is, the points with labels 1 and 16, and filter out the noise points;

[0033] (2-6-3) Based on the labels 2-15 of each point in the tooth point cloud, each tooth crown is classified and a point cloud model containing only the tooth crowns is extracted.

[0034] Preferably, in step (3), post-processing is performed based on the dental arch line point cloud preparation to obtain the existing dental arch length, including:

[0035] (3-1) Scale restoration of the dental arch line point cloud preparation;

[0036] (3-2) extracting the complete point cloud boundary of the dental arch line point cloud preparation and the point cloud boundary of the dental arch line point cloud preparation with a label of 1 respectively by using an edge detection algorithm, extracting points in the point cloud boundary with a label of 1 that are different from the complete point cloud boundary, and constructing a dental arch line area point set;

[0037] (3-3) Reorder the dental arch line area point set along the x-axis direction and filter out noise points;

[0038] (3-4) The point set of the dental arch line area is downsampled to δ control points, where δ is an odd number. The control points are projected onto the occlusal plane, and the existing dental arch is fitted using a cubic B-spline curve to calculate the existing dental arch length.

[0039] Preferably, in step (3-4), downsampling the dental arch line area point set to δ control points includes:

[0040] (3-4-1) According to the number of points in the dental arch line area after reordering along the x-axis, uniformly downsample to δ control points;

[0041] (3-4-2) Extract the point with the negative x-axis coordinate value and the smallest y-axis coordinate value in the dental arch area point set, and replace it with the first control point among the δ control points fitting the existing dental arch;

[0042] (3-4-3) Extract the point with the positive x-axis coordinate value and the smallest y-axis coordinate value in the dental arch area point set, and replace it with the last control point among the δ control points fitting the existing dental arch;

[0043] (3-4-4) Extract the point with the smallest difference between the x-axis coordinate value of the dental arch line area point concentration and the x-axis coordinate value of the center point of the occlusal plane, and replace the first point of the dental arch fitting control points.

[0044] Preferably, in step (4), post-processing is performed based on the crown width point cloud preparation to obtain the expected dental arch length, including:

[0045] (4-1) Scale restoration of crown width point cloud preparation;

[0046] (4-2) extracting the complete point cloud boundary of the crown width point cloud preparation and the point cloud boundary labeled as 1 in the crown width point cloud preparation respectively through the edge detection algorithm, extracting the points in the point cloud boundary labeled as 1 that are different from the complete point cloud boundary, and constructing the crown width direction regional point set of the entire dentition;

[0047] (4-3) Traverse the crown width direction area point set of the entire dentition, find the same point in the point cloud model of the dentition crown extracted in step (2-6), and obtain the crown label to which the point belongs, then classify according to the label to construct the crown width direction area point set of each crown;

[0048] (4-4) Use the RANSAC algorithm to fit a straight line model of each crown in the crown width direction;

[0049] (4-5) Calculate the width of each crown in the crown width direction and sum them up to calculate the required arch length.

[0050] Preferably, in step (4-5), calculating the width value of each crown in the crown width direction includes:

[0051] (4-5-1) Calculate the angle θ between the width direction vector of each crown and the x-axis;

[0052] (4-5-2) Rotate the crown point cloud model around the z-axis with a rotation angle of θ;

[0053] (4-5-3) Obtain the axis-aligned bounding box of each tooth crown in space;

[0054] (4-5-4) Calculate the length of the axis-aligned bounding box of each crown in the x-axis direction, that is, the width of each crown in the crown width direction.

[0055] Beneficial effects: Compared with the prior art, the technical solution of the present invention has the following beneficial technical effects:

[0056] 1. Due to the adoption of step (2), the present invention builds a neural network based on dynamic and dilated edge convolution and trains three network models based on three segmentation data sets. Among them, the network model trained with the segmentation data set with the edge of the tooth crown as the boundary inputs the oral scan model data and can output the tooth crown point cloud preparation. After post-processing, the point cloud data of the gum and wisdom tooth area can be deleted, and each crown of the patient can be segmented. Compared with the previous method of segmenting the tooth crown by human-computer interaction, this method solves the problems of time-consuming, labor-intensive and inefficient process in segmenting the tooth crown.

[0057] 2. Since the present invention adopts steps (2) and (3), the existing dental arch length can be obtained by post-processing the dental arch line point cloud preparation obtained in step (2). Previously, doctors used copper wire to mesh one by one along the adjacent points of the teeth on the model, and then measured the total length after straightening it, or used human-computer interaction to draw the existing curved dental arch and measure the length. Compared with these two methods, the present invention can solve the error problem caused by subjective factors in the process of doctors measuring the existing dental arch length and reduce the time doctors spend preparing for surgery during orthodontic diagnosis and treatment.

[0058] 3. Since the present invention adopts steps (2) and (4), the expected dental arch length can be obtained by post-processing the crown width point cloud preparation obtained in step (2). Previously, doctors used a vernier caliper to measure the maximum width of each crown or used human-computer interaction to measure the maximum width of each crown, and then summed up all the crown widths to obtain the expected dental arch length. Compared with these two methods, the present invention can solve the error problem caused by subjective factors in the process of doctors measuring the expected dental arch length, as well as the problems of long operation time and high labor cost.

[0059] 4. The present invention adopts step (1), which uses a deep learning-based dental crowding recognition method, and can directly identify the crowding value of the patient's oral scan model, which will greatly reduce the time doctors spend on orthodontic diagnosis and treatment preparation, improve the comfort and efficiency of patients during treatment, and has important application value in the field of computer-assisted orthodontic diagnosis and treatment. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 This is a flowchart of the deep learning-based dental crowding identification method of the present invention.

[0061] Figure 2 This is a single case of the segmentation dataset based on the tooth crown edge in the present invention.

[0062] Figure 3 This is a single case of the segmentation dataset based on the dental arch in the present invention.

[0063] Figure 4This is a single case of the present invention where the data set is segmented based on the width of the crown.

[0064] Figure 5 This is a diagram of the neural network structure based on dynamic and dilated edge convolution of the present invention.

[0065] Figure 6 Flowchart for the training optimization of the neural network model based on dynamic and dilated edge convolution in the present invention.

[0066] Figure 7 This is a flow chart of the process of obtaining the existing curved dental arch by post-processing based on the dental arch line point cloud preparation of the present invention.

[0067] Figure 8 This is a flow chart of the process of post-processing the prepared body based on the crown width point cloud to obtain the expected curved dental arch. DETAILED DESCRIPTION

[0068] The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0069] The present invention provides a method for recognizing dental crowding based on deep learning, the process is as follows: Figure 1 As shown, the following steps are included:

[0070] (1) Create three segmentation datasets based on the dental arch, tooth crown edge, and crown width, and preprocess the datasets.

[0071] In this embodiment, the three segmentation data sets are prepared by segmenting the scanned STL model using 3-matic software, and storing the segmented parts as point cloud files in asc format. Then, a C++ program is written to read the asc format point cloud files for preprocessing, and the coordinates, normal vectors, and labels of each point are stored in a txt format file. The data contained in each point is represented as {x, y, z, n x ,n y ,n z ,l}, where x, y, z are coordinate values, n x 、n y 、n z is the normal vector and l is the label.

[0072] Specifically, this step includes the following sub-steps:

[0073] (1-1) Figure 3 As shown, a single dentition case of a segmentation dataset with the dental arch as the boundary is used. In this case, 0 and 1 labels are added to each point of the point cloud data inside and outside the dental arch line respectively to construct a tooth segmentation dataset.

[0074] (1-2) Figure 2As shown, a single dentition case of a segmentation dataset with the edge of the tooth crown as the boundary, in which corresponding labels 1-16 are added to the point cloud data of each crown, and label 0 is added to the point cloud data of the gum area to construct a tooth segmentation dataset.

[0075] (1-3) Figure 4 As shown, a single dentition case of a segmentation dataset based on the width of the crown is used as the boundary. In this case, each crown is divided into the inner and outer sides according to the width of the crown. Each point in the inner point cloud data is labeled 0, and each point in the outer point cloud data is labeled 1 to construct a tooth segmentation dataset.

[0076] (1-4) Preprocess the datasets. Downsample the three datasets using the farthest point sampling algorithm to keep the number of tooth point clouds within the range [β1, β2], where β1 and β2 are the maximum and minimum values of the tooth point cloud, respectively. In this embodiment, β1 = 50,000 and β2 = 55,000. Rotate and expand the three datasets using a 6×60° rotation factor, increasing the data volume by 6 times. The enhanced data is then filtered for noise and normalized before being stored.

[0077] Specifically, the normalization method of steps (1-4) mainly adopts mean and standard deviation normalization, which is performed by subtracting the mean and dividing by the standard deviation, so that the point cloud data obeys the standard normal distribution (mean is 0 and standard deviation is 1).

[0078] Specifically, the advantage of steps (1-4) is that the data is rotated and expanded, making the model more robust when facing dentition point clouds of different postures and angles, thereby improving the generalization ability of the network model.

[0079] (2) A neural network based on dynamic and dilated edge convolution was constructed, and three network models were trained based on three segmentation data sets respectively, which can output three types of point cloud preparations according to the corresponding tooth point cloud input data.

[0080] In this embodiment, the neural network based on dynamic and dilated edge convolution is built using the PyTorch framework, and the network structure is consistent, but the training data set is divided into three types, and three network models can be trained (the training process is as follows Figure 6 As shown), and the input data for prediction has sequence requirements.

[0081] For example, for the patient's oral scan STL model, the neural network trained based on the segmentation data set with the edge of the tooth crown as the boundary is first used to predict and output the tooth crown point cloud preparation. Post-processing can obtain a point cloud model that only contains the tooth crowns and each crown has been segmented. This model is used as the input of the other two network models to obtain the corresponding two point cloud preparations.

[0082] In this step, the trained model is saved as a pt format file and deployed to C++ through libtorch. Subsequent post-processing operations are implemented by writing subroutines in C++.

[0083] Specifically, this step includes the following sub-steps:

[0084] (2-1) Build a neural network based on dynamic and dilated edge convolution, which is divided into tooth point cloud feature extraction module and segmentation module. The neural network structure is as follows Figure 5 shown.

[0085] In the tooth point cloud feature extraction module, the EdgeConv dynamic edge convolution module is referenced, and the K-nearest neighbor algorithm is used to build a graph structure. The local feature information of the point cloud is obtained through convolution and symmetric aggregation functions. The Dilated EdgeConv expanded edge convolution module is referenced. The K-nearest neighbor algorithm is first used to build a larger graph structure so that the graph structure contains more neighboring tooth point information. The farthest point sampling algorithm is then used to sparse the graph structure. The global feature information of the point cloud is obtained through convolution and symmetric aggregation functions. This method expands the receptive field, enabling the network to learn more distant tooth features, thereby improving the network's tooth segmentation performance.

[0086] In the segmentation module, the feature dimension is downsampled to the corresponding number of labels through the MLP multi-layer perceptron. Finally, the output of the MLP multi-layer perceptron is mapped to the probability value of the corresponding label through the SoftMax classifier, and the label value with the maximum probability is extracted as the label of the point. The network classification label for the segmentation dataset trained with the edge of the tooth crown as the boundary is 0-16, and the network classification label for the other two segmentation datasets is 0-1.

[0087] The advantage of step (2-1) is that the neural network based on dynamic and dilated edge convolution significantly improves the segmentation performance of the point cloud model by introducing a dynamic graph convolution structure. In this embodiment, the dentition point cloud model is mainly segmented.

[0088] (2-2) The three tooth segmentation datasets were shuffled and divided into training set, test set and validation set in a ratio of 7:2:1. The network was initialized using the He normal distribution initialization method. The hyperparameters included the initial learning rate, number of iterations, weight loss rate, neuron mortality rate, and two momentum parameters β1 and β2 in the Adam optimizer. The cross-entropy loss function was selected, and the three types of training set data were imported into the network for training.

[0089] (2-3) Perform network parameter adjustment, import the validation set data of the three types of tooth data into the network respectively, adjust the hyperparameters multiple times, observe the loss function curve and segmentation accuracy to select the optimal hyperparameters, and output three network models with the best performance.

[0090] (2-4) Conduct network testing, import the test set data of three types of tooth data into the network respectively, output the network segmentation accuracy and average intersection-over-union value to judge whether the network performance meets the requirements.

[0091] Specifically, the segmentation accuracy formula for steps (2-4) is as follows:

[0092]

[0093] Among them TP i is the number of points correctly classified as the i-th category, N is the total number of points in the dentition point cloud, and C is the total number of categories. For example, for the segmentation dataset bounded by the crown edge, the value of C is 17, and for the segmentation dataset bounded by the dental arch line and the crown width direction, the value of C is 2.

[0094] Specifically, the average intersection-over-union ratio formula for steps (2-4) is as follows:

[0095]

[0096] where |P i ∩G i | is the number of intersection points between the predicted area and the true area, |P i ∪G i | is the number of points in the union of the predicted area and the true area, and C is the total number of categories.

[0097] (2-5) The patient's oral scan model (including point cloud data of the dentition crown and gums) is input into the network model trained with the segmentation data set bounded by the edge of the tooth crown, and the tooth crown point cloud preparation is output. The point cloud preparation is subjected to relevant post-processing to obtain a point cloud model containing only the dentition crown and each crown is segmented. The model is input into the other two network models (network models trained with two segmentation data sets bounded by the dental arch and crown width directions respectively), and the dental arch line point cloud preparation and the crown width point cloud preparation are output.

[0098] Specifically, in steps (2-5), the output tooth crown point cloud preparation is post-processed, including the following sub-steps:

[0099] (2-5-1) Perform scale restoration on the tooth crown point cloud preparation, that is, perform denormalization using the mean and standard deviation of the normalized data before training.

[0100] (2-5-2) Delete the point cloud of the gum area (points with label 0) and the point cloud of the wisdom tooth area (points with labels 1 and 16), and filter out the noise points.

[0101] Specifically, deleting the point cloud of the gingival area in step (2-5-2) can reduce the influence of the point cloud features of the gingival area on the prediction of the dental arch preparation and crown width preparation of the point cloud of the crown area, and because the method for calculating dental crowding requires that wisdom teeth are not included in the calculation, the point cloud of the wisdom tooth area is deleted.

[0102] (2-5-3) Each tooth crown is segmented according to label 2-15, thereby extracting a point cloud model containing only the tooth crowns.

[0103] (3) Figure 7 As shown, the existing dental arch length can be obtained by post-processing based on the dental arch line point cloud preparation, which includes the following sub-steps:

[0104] (3-1) Scale restoration is performed on the dental arch line point cloud preparation, that is, denormalization is performed using the mean and standard deviation of the normalized data before training.

[0105] (3-2) The complete point cloud boundary of the dental arch line point cloud preparation and the point cloud boundary with the label 1 in the dental arch line point cloud preparation are extracted respectively through the edge detection algorithm, and the points in the point cloud boundary with the label 1 that are different from the complete point cloud boundary are extracted to construct the dental arch line area point set.

[0106] Specifically, the edge detection algorithm in step (3-2) mainly sets the angle threshold between the normal vector of each point and its neighboring points to 240 degrees. Points exceeding 240 degrees are considered to be boundary points. The normal vector of the point cloud is estimated, and the covariance matrix of each point p and its 8 neighboring points {p1, p2, ..., p8} is calculated as follows:

[0107]

[0108] in, is the average position of the neighborhood points.

[0109] Calculate the eigenvalues and eigenvectors of the covariance matrix and select the eigenvector corresponding to the minimum eigenvalue as the normal vector.

[0110] (3-3) The dental arch line area point set is reordered along the x-axis direction, that is, the point set is sorted from small to large according to the x-axis coordinate of each point, so as to facilitate the subsequent extraction of the control points required for the dental arch line and finally filter out the noise points.

[0111] (3-4) Downsampling the dental arch line point set to δ control points, projecting the point set onto the occlusal plane, fitting the existing dental arch using a cubic B-spline curve, and calculating the existing dental arch length, including the following sub-steps:

[0112] (3-4-1) Based on the number of dental arch line area points after reordering along the x-axis, uniformly downsample to δ control points starting from the point with the smallest x-axis coordinate value.

[0113] (3-4-2) Extract the point with the negative x-axis coordinate value and the smallest y-axis coordinate value in the dental arch line area point set, and replace it with the first control point among the δ control points fitting the existing dental arch.

[0114] (3-4-3) Extract the point with the positive x-axis coordinate value and the smallest y-axis coordinate value in the dental arch line area point set, and replace it with the last control point among the δ control points fitting the existing dental arch.

[0115] (3-4-4) Extract the point with the smallest difference between the x-axis coordinate value of the dental arch line area point concentration and the x-axis coordinate value of the center point of the occlusal plane, and replace it with the existing dental arch point. control points.

[0116] (3-4-5) Project the δ control points onto the occlusal plane, fit the existing dental arch with a cubic B-spline curve, then discretize the existing dental arch into N points, calculate the sum of the straight-line distances between the two points, and obtain the length of the existing dental arch.

[0117] Specifically, the occlusal plane needs to be defined at the initial stage of orthodontic treatment. This is typically an imaginary plane drawn from the mesial incisor of the maxillary central incisor to the mesiobuccal cusps of the bilateral first molars. The doctor uses computer-assisted software to select three points on the maxillary surface to construct the occlusal plane. If the occlusal plane has not yet been defined when identifying crowding, the center of gravity of the dental arch model will be used as the center of the occlusal plane.

[0118] The advantage of step (3) is that the post-processing of the dental arch line point cloud preparation is implemented using C++ subroutines, and no other auxiliary operations are required in the middle. The existing dental arch length can be directly obtained, which reduces the time for doctors to draw the existing dental arch and improves the efficiency of diagnosis and treatment.

[0119] (4) Figure 8 As shown in the figure, the expected arch length can be obtained by post-processing the crown width point cloud preparation, which includes the following sub-steps:

[0120] (4-1) Scale restoration is performed on the crown width point cloud preparation, that is, denormalization is performed using the mean and standard deviation of the normalized data before training.

[0121] (4-2) Through the edge detection algorithm, the complete point cloud boundary of the crown width point cloud preparation and the point cloud boundary labeled as 1 in the crown width point cloud preparation are extracted respectively, and the points in the point cloud boundary labeled as 1 that are different from the complete point cloud boundary are extracted, thereby constructing the crown width direction area point set of the entire dentition.

[0122] (4-3) Traverse the crown width direction area point set of the entire dentition, find the same point in the point cloud model of the dentition crown extracted in step (2-5-3), and obtain the crown label to which the point belongs. Then classify according to the label to construct the crown width direction area point set of each crown.

[0123] (4-4) The RANSAC algorithm is used to fit a straight line model of each crown in the crown width direction.

[0124] Specifically, the RANSAC algorithm is an iterative method for estimating parameter models. It is particularly suitable for data sets containing a large number of outliers. Since the point set in the crown width direction of each crown is relatively discrete in the adjacent tooth area, the linear model fitted by this method is more consistent with the requirements in the tooth width direction. This step includes the following sub-steps:

[0125] (4-4-1) Set the number of iterations to e, randomly select two points to construct the initial straight line model as the best straight line model, and count the number of inliers. The inliers are determined when the distance between the straight line model and other point sets is less than the set maximum distance threshold бcm.

[0126] (4-4-2) Randomly select two points again to construct the current straight line model and calculate the number of inner points of the current straight line model.

[0127] (4-4-3) If the number of inliers in the current straight line model is greater than the best straight line model found previously, the best model is updated.

[0128] (4-4-4) Repeat the above steps until the maximum number of iterations is reached. At this time, the optimal straight line model is the vector in the width direction of the crown.

[0129] (4-5) Calculate the width of each crown in that direction and sum them up to calculate the expected arch length, including the following sub-steps:

[0130] (4-5-1) Calculate the angle θ between the width vector of each crown and the x-axis.

[0131] (4-5-2) Rotate the crown point cloud model around the z-axis with a rotation angle of θ.

[0132] (4-5-3) Get the axis-aligned bounding box of each tooth crown in space.

[0133] (4-5-4) Calculate the length of the axis-aligned bounding box of each crown in the x-axis direction, that is, the width of each crown in the crown width direction. The sum of all crown width values is the expected dental arch length.

[0134] The advantage of step (4) is that the post-processing of the crown width point cloud preparation is implemented using a C++ subroutine, and no other auxiliary operations are required in the middle. The expected dental arch length can be directly obtained, reducing the doctor's workload of manual measurement or human-computer interaction measurement, and improving the efficiency of diagnosis and treatment.

[0135] (5) The crowding value is obtained by subtracting the existing dental arch length from the expected dental arch length.

[0136] It will be easily understood by those skilled in the art that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying dental crowding based on deep learning, characterized in that: The following steps are involved: (1) Create three segmentation datasets based on the dental arch, tooth crown edge, and crown width, and preprocess the datasets; (2) A neural network based on dynamic and dilated edge convolution was constructed, and three network models were trained based on three segmentation datasets to output the dental arch line point cloud preparation, the tooth crown point cloud preparation, and the crown width point cloud preparation respectively; (3) Post-processing is performed based on the dental arch line point cloud preparation to obtain the existing dental arch length; (4) Post-processing is performed based on the crown width point cloud preparation to obtain the expected dental arch length; (5) The crowding value is obtained by subtracting the existing dental arch length from the expected dental arch length.

2. The method for identifying dental crowding based on deep learning according to claim 1, characterized in that: In step (1), three segmentation datasets are prepared and preprocessed, including: (1-1) Taking the dental arch as the boundary, add label 0 to each point of the point cloud data within the dental arch line, and add label 1 to each point of the point cloud data outside the dental arch line to construct the dataset; (1-2) Using the edge of the tooth crown as the boundary, add corresponding labels 1-16 to the point cloud data of each crown, and add label 0 to the point cloud data of the gum area to construct the dataset; (1-3) Each crown is divided into inner and outer parts according to the crown width direction, and each point of the inner point cloud data is labeled 0, and each point of the outer point cloud data is labeled 1 to construct the data set; (1-4) The three types of data sets are downsampled by the farthest point sampling algorithm so that the number of tooth point cloud points is controlled within the range of [β1,β2], where β1 and β2 are the maximum and minimum values of the tooth point cloud points, respectively; (1-5) The three types of data sets are rotated and expanded, and the expanded data are filtered out of noise points and normalized. Finally, the data is stored. The data contained in each point is represented as {x, y, z, n x ,n y ,n z ,l}, where x, y, z are coordinate values, n x 、n y 、n z is the normal vector and l is the label.

3. The method for identifying dental crowding based on deep learning according to claim 1, characterized in that: In step (2), a neural network based on dynamic and dilated edge convolution is built and trained to obtain three network models, which predict three point cloud preparations, including: (2-1) Build a dynamic and dilated edge convolutional neural network, which is divided into a tooth point cloud feature extraction module and a segmentation module; The tooth point cloud feature extraction module performs the following operations: The EdgeConv dynamic edge convolution module is used to construct the graph structure of each point in the tooth point cloud model using the K-nearest neighbor algorithm, and the local feature information of the point cloud is obtained through convolution and symmetric aggregation functions; The Dilated EdgeConv module is used to expand the edge convolution module. First, the K-nearest neighbor algorithm is used to build a graph structure so that the graph structure contains neighboring point information. Then, the farthest point sampling algorithm is used to sparse the graph structure. The global feature information of the point cloud is obtained through convolution and symmetric aggregation functions. The segmentation module performs the following operations: The feature dimension of each point in the tooth point cloud is downsampled to the corresponding number of labels through the MLP multi-layer perceptron. The output of the MLP multi-layer perceptron is mapped to the probability value of the corresponding label through the SoftMax classifier. The label value with the maximum probability of each point in the tooth point cloud is extracted as the label of the point. The network classification labels for the segmentation dataset trained with the tooth crown edge as the boundary are 0-16, and the network classification labels for the other two segmentation datasets are 0-1; (2-2) The three tooth segmentation datasets were divided into training set, test set and validation set in proportion. The network was initialized by He normal distribution initialization method, hyperparameters were set, and the three types of training set data were imported into the network for training respectively. (2-3) Perform network parameter adjustment, import the validation set data of the three tooth data into the network respectively, adjust the hyperparameters, select the optimal hyperparameters based on the loss function curve and segmentation accuracy, and output three network models with the best performance; (2-4) Conducting network testing, importing the test set data of the three tooth data into the corresponding optimal network obtained in step (2-3), outputting the network segmentation accuracy and average intersection-over-union ratio value, and judging whether the network performance meets the requirements; (2-5) Inputting the patient's oral scan model containing point cloud data of the dentition, crown and gums into a network model trained with a segmentation dataset bounded by the edge of the tooth crown, and outputting a point cloud preparation of the tooth crown; (2-6) The tooth crown point cloud preparation output in step (2-5) is subjected to relevant post-processing to obtain a point cloud model containing only the dentition crown, and the model is input into two network models trained with two segmentation data sets bounded by the dental arch and crown width directions, respectively, to output the dental arch line point cloud preparation and the dental crown width point cloud preparation.

4. The method for identifying dental crowding based on deep learning according to claim 3, characterized in that: In steps (2-6), the output tooth crown point cloud preparation is post-processed, including: (2-6-1) Perform scale restoration on the tooth crown point cloud preparation; (2-6-2) Delete the point cloud of the gum area, that is, the points with label 0, and delete the point cloud of the wisdom tooth area, that is, the points with labels 1 and 16, and filter out the noise points; (2-6-3) Based on the labels 2-15 of each point in the tooth point cloud, each tooth crown is classified and a point cloud model containing only the tooth crowns is extracted.

5. The method for identifying dental crowding based on deep learning according to claim 1, characterized in that: In step (3), the existing dental arch length is obtained by post-processing based on the dental arch line point cloud preparation, including: (3-1) Scale restoration of the dental arch line point cloud preparation; (3-2) extracting the complete point cloud boundary of the dental arch line point cloud preparation and the point cloud boundary of the dental arch line point cloud preparation with a label of 1 respectively by using an edge detection algorithm, extracting points in the point cloud boundary with a label of 1 that are different from the complete point cloud boundary, and constructing a dental arch line area point set; (3-3) Reorder the dental arch line area point set along the x-axis direction and filter out noise points; (3-4) The point set of the dental arch line area is downsampled to δ control points, where δ is an odd number. The control points are projected onto the occlusal plane, and the existing dental arch is fitted using a cubic B-spline curve to calculate the existing dental arch length.

6. The method for identifying dental crowding based on deep learning according to claim 5, characterized in that: In step (3-4), the dental arch line area point set is downsampled to δ control points, including: (3-4-1) According to the number of points in the dental arch line area after reordering along the x-axis, uniformly downsample to δ control points; (3-4-2) Extract the point with the negative x-axis coordinate value and the smallest y-axis coordinate value in the dental arch area point set, and replace it with the first control point among the δ control points fitting the existing dental arch; (3-4-3) Extract the point with the positive x-axis coordinate value and the smallest y-axis coordinate value in the dental arch area point set, and replace it with the last control point among the δ control points fitting the existing dental arch; (3-4-4) Extract the point with the smallest difference between the x-axis coordinate value of the dental arch line area point concentration and the x-axis coordinate value of the center point of the occlusal plane, and replace the first point of the dental arch fitting control points.

7. The method for identifying dental crowding based on deep learning according to claim 3 or 4, characterized in that: In step (4), the expected dental arch length is obtained by post-processing the prepared body based on the crown width point cloud, including: (4-1) Scale restoration of crown width point cloud preparation; (4-2) extracting the complete point cloud boundary of the crown width point cloud preparation and the point cloud boundary labeled as 1 in the crown width point cloud preparation respectively through the edge detection algorithm, extracting the points in the point cloud boundary labeled as 1 that are different from the complete point cloud boundary, and constructing the crown width direction regional point set of the entire dentition; (4-3) Traverse the crown width direction area point set of the entire dentition, find the same point in the point cloud model of the dentition crown extracted in step (2-6), and obtain the crown label to which the point belongs, then classify according to the label to construct the crown width direction area point set of each crown; (4-4) Use the RANSAC algorithm to fit a straight line model of each crown in the crown width direction; (4-5) Calculate the width of each crown in the crown width direction and sum them up to calculate the required arch length.

8. The method for identifying dental crowding based on deep learning according to claim 7, characterized in that: In step (4-5), the width value of each crown in the crown width direction is calculated, including: (4-5-1) Calculate the angle θ between the width direction vector of each crown and the x-axis; (4-5-2) Rotate the crown point cloud model around the z-axis with a rotation angle of θ; (4-5-3) Obtain the axis-aligned bounding box of each tooth crown in space; (4-5-4) Calculate the length of the axis-aligned bounding box of each crown in the x-axis direction, that is, the width of each crown in the crown width direction.

Images