Two-stage deep learning dental crown segmentation method using dental crown edge information
Through a two-stage deep learning method and Point Transformer network, dental segmentation is used to use crown edge information to solve the problem of difficult balance of robustness and post-processing complexity in the prior art, and achieve high robustness and simplified operation teeth Split effect.
Patent Information
- Application Number
- CN202411353106.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-08-12
- Filing Date
- 2024-09-26
- Publication Date
- 2024-12-13
AI Technical Summary
Existing dental segmentation methods are difficult to balance between robustness and data post-processing complexity. Traditional methods require manual operation by users and are not robust enough. Deep learning methods require complex parameter adjustment and post-processing steps.
Using a two-stage deep learning method, the crown edge information is used to obtain the sample point set of tooth grid data through the farthest point sampling and boundary-aware sampling, and segment it through the Point Transformer network, combining semantic labels and instance labels for final label merging. , avoid complicated post-processing steps.
It realizes that the robustness of the tooth segmentation algorithm is improved without the need for complex post-processing steps, and directly obtains the tooth instance tag, reducing the sensitivity to parameters and simplifying user operations.
Smart Images
Figure CN119131068A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of tooth segmentation, and in particular relates to a two-stage deep learning tooth crown segmentation method using tooth crown edge information. Background Art
[0002] Dental oral scan data plays an important role in oral medicine. Oral scan data is generally stored in the form of 3D meshes, which has the characteristics of large data volume and complex topological relationships. At present, the segmentation algorithms for 3D tooth mesh models can be divided into traditional algorithms and deep learning algorithms.
[0003] Traditional algorithms can be divided into curvature-based segmentation and path-based segmentation. The curvature and topological information of the tooth gap and crown-gingival connection of the three-dimensional tooth model will change. Traditional methods rely on these pre-designed geometric features to segment teeth, and the selection of these features requires users to specify parameters. The parameters largely determine the segmentation results, and the algorithm is less robust.
[0004] Segmentation algorithms based on deep learning are flourishing. Architectures such as CNN, U-Net, and Mask R-CNN are used according to different needs, and robustness is improved through training with large amounts of data. However, in order to obtain comprehensive spatial information or make the segmentation results conform to the geometric structure of the teeth, complex pre-processing or post-processing steps are often required for the data.
[0005] One of the existing technical methods is based on surface curvature. The curvature in three-dimensional space reflects the degree to which the surface deviates from the plane. The curvature changes at the junction of the crown and gums and at the tooth gap can be used for segmentation; the typical algorithm process is:
[0006] 1. Tooth data alignment: The user needs to mark four reference points on the tooth 3D model to determine the origin of the tooth model space;
[0007] 2. Calculate vertex curvature and preliminarily determine feature points;
[0008] 3. Filtering and cleaning data: Filter out the noise points introduced in step 2, specifically cleaning up the vertices in the area where the point density is lower than the user-specified threshold; in addition, after filtering, the user is required to select and erase the vertices that are located in the non-crown area but meet the density requirements;
[0009] 4. Morphological processing and boundary cleaning: The points at the crown-gingival boundary need to be morphologically processed to maintain a clear boundary;
[0010] 5. Select the characteristic line: Based on the characteristic skeleton obtained in step 4, the user needs to select a point between the tooth gaps and use the quartic equation to fit the point on the characteristic skeleton closest to the point to obtain the dental arch curve, that is, the required characteristic line;
[0011] 6. Close the feature contour: Use the Dijkstra shortest path algorithm to close the feature line obtained in step 5. For the closing of the boundary line between the teeth and gums and the closing of the boundary line between the teeth, the edge weights of the corresponding situations in the Dijkstra algorithm are specified.
[0012] The method is not robust enough: the selection of initial values largely determines the segmentation results. Inappropriate initial values may cause under-segmentation or over-segmentation. User interaction is complex: the entire cycle of the algorithm requires manual operation by the user, which is complex and requires high operation requirements. Inappropriate interaction will also greatly affect the segmentation results.
[0013] Another existing method is the segmentation method based on deep learning. Most traditional methods are based on pre-designed geometric features for segmentation, which is not robust enough. Deep learning methods can overcome this shortcoming. The proposed MeshSegNet is currently a high-precision deep learning tooth model automatic segmentation method. This method considers the spatial characteristics of tooth mesh model cells at different scales and uses graph cutting to process the segmentation results.
[0014] The disadvantage of this method is that after obtaining the predicted probability, it is necessary to use the graph segmentation method to impose restrictions on the segmentation results. In addition, the graph segmentation method itself is very sensitive to parameters, which will also reduce the robustness of the algorithm.
[0015] Therefore, a dental scan data segmentation method is needed that is robust and does not require complex data preprocessing. Summary of the invention
[0016] In view of the above-mentioned deficiencies in the prior art, the present invention provides a two-stage deep learning crown segmentation method using crown edge information, which solves the problem that the existing tooth segmentation methods cannot balance the complexity and robustness of data post-processing.
[0017] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a two-stage deep learning crown segmentation method using crown edge information, comprising the following steps:
[0018] S1, obtaining tooth data through a mouth scanning device, dividing the tooth data into grids, and obtaining tooth grid data;
[0019] S2, using the farthest point sampling to sample the tooth mesh data, to obtain a first sampling point set of the tooth mesh data;
[0020] S3, obtaining a tooth semantic label and a first tooth instance label using a tooth segmentation network according to a first sampling point set of the tooth mesh data;
[0021] S4. According to the first tooth instance label, using boundary-aware sampling, obtain a second sampling point set of the tooth mesh data;
[0022] S5, inputting a second sampling point set of the tooth mesh data into the tooth segmentation network to obtain a second tooth instance label;
[0023] S6. Merge the first tooth instance label and the second tooth instance label, and combine them with the tooth semantic label to obtain a final tooth instance label.
[0024] Furthermore, the step S2 is specifically as follows:
[0025] S201, according to the tooth mesh data, a point P0 is selected as a starting point from a point cloud composed of N points composed of vertices to obtain a sampling point set S = {P0};
[0026] S202, calculate the distance from all points in the point cloud to P0, forming an N-dimensional array L = {L i}, select the point corresponding to the maximum distance and add it to the sampling point set; where L i is the i-th point P i The shortest distance to the set of sampling points;
[0027] S203, taking the latest sampling point added to the sampling point set as the center point, calculate the distance from all points in the point cloud except the sampling points in the sampling point set to the center point, for each point P i , determine whether the distance to the center point is less than L i , if so, update L i , otherwise, no update;
[0028] S204, select the point corresponding to the maximum distance in L and add it to the sampling point set;
[0029] S205 , repeating steps S203 - S204 until the number of sampling points in the sampling point set reaches a set value N′, thereby obtaining a first sampling point set.
[0030] Furthermore, the tooth segmentation network in step S3 includes a first Point Transformer module, a point cloud offset module, a label filtering module, a clustering module, a cropping module, a second Point Transformer module and a label output module;
[0031] The first Point Transformer module is used to obtain the tooth semantic label and offset distance of each sampling point according to the input sampling point set;
[0032] The point cloud offset module is used to move the sampling points in the input sampling point set to the center point of the tooth instance according to the offset distance of each sampling point, so as to obtain a point cloud data set after the movement;
[0033] The label filtering module is used to remove the point cloud labeled as gums from the moved point cloud data set according to the tooth semantic labels of each sampling point, so as to obtain filtered sample points;
[0034] The aggregation module is used to cluster the filtered sample points using the DBSCAN method, assign a tooth instance label to each filtered sample point, and assign a tooth instance label to the point cloud in the unsampled point set according to the nearest neighbor principle based on the clustered tooth instance labels to obtain the tooth instance label of each point cloud; the unsampled point set is a sampling point set obtained by removing the point cloud in the input sampling point set from the point cloud of the tooth grid data;
[0035] The clipping module is used to obtain the center point position of each tooth instance according to the tooth instance label of each point cloud, and to set a radius threshold based on the center point position of each tooth instance to clip the sampling points in the first sampling point set to obtain the clipped sample points;
[0036] The second Point Transformer module is used to obtain a tooth-gum mask of each sampling point according to the cropped sample points;
[0037] The label output module is used to make a one-to-one correspondence between the tooth-gum mask of each sampling point and the tooth instance label of each point cloud. If the tooth-gum mask and the tooth instance label of the sampling point are both teeth or gums, no processing is performed. If the tooth-gum mask of the sampling point is displayed as teeth and the tooth instance label is displayed as gums, the tooth instance label of the sampling point is changed to the tooth instance label of the nearest point. If the tooth-gum mask of the sampling point is displayed as gums and the tooth instance label is displayed as teeth, the tooth instance label of the sampling point is changed to gums, and the changed tooth instance labels of each point are used as output data.
[0038] Furthermore, the loss function of the tooth segmentation network is:
[0039] L=L center_offset +L weighted_center_offset +L chamfer +L tooth_class +L teeth_mask
[0040]
[0041] Where L is the loss function of the tooth segmentation network; L center_offset is the center offset loss; L weighted_center_offset is the center weighted offset loss; L chamfer is the distance loss; L tooth_class Loss of tooth classification; L teeth_mask is the tooth mask loss; C is the number of categories; N i is the number of points in the i-th class; x j is the coordinate of the jth point in the i-th category; o j is the offset of the jth point in the i-th category relative to the instance center coordinate c; d() is the distance between two spatial points; D is the number of all direction vector types; N i' is the number of points in the i'th type of direction vector; d j' is the direction vector of the j'th point; d' j' is the direction vector of the j'th point pointing to the instance center point coordinate c; w j is the weight corresponding to the distance; c1 is the distance x j The coordinates of the nearest instance center point; c2 is the distance x j The coordinates of the second closest instance center point; CrossEntropy(p,y) is the cross entropy function; p is the probability of classification; y is the actual type label; N is the total number of sample points; r is the sample point number; y r,i is the label of the rth point in the i-th category; p r,i is the probability that the rth point is predicted to be the i-th category; y r is the true label of the rth point; p r is the predicted value of the rth point; σ() is the sigmoid function.
[0042] Furthermore, the step S3 is specifically as follows: taking the first sampling point set of the tooth mesh data as the input sampling point set, inputting it into the tooth segmentation network, extracting the tooth semantic label obtained by the first Point Transformer module, and outputting the output data as the first tooth instance label.
[0043] Furthermore, the step S4 is specifically as follows:
[0044] S401, obtaining the coordinates of each point in the tooth grid data, and constructing a K-dimensional search tree;
[0045] S402, searching for K neighboring points of each point based on a K-dimensional search tree;
[0046] S403, according to the first tooth instance label, calculate the proportion of points with different tooth instance labels from the corresponding point among the K neighboring points of each point as the boundary criterion:
[0047]
[0048] Among them, p1 r is the boundary criterion of the rth point; dif(K r ) is the number of points among the K neighboring points that have different tooth instance labels from the rth point; K is the total number of neighboring points;
[0049] S404: taking a set of points whose boundary criteria are greater than a set threshold as a second sampling point set of the tooth grid data.
[0050] Furthermore, the step S5 is specifically as follows: taking the second sampling point set of the tooth mesh data as the input sampling point set, inputting it into the tooth segmentation network, replacing the tooth semantic label obtained by the first Point Transformer module with the tooth semantic label obtained in step S3, and outputting the output data as the second tooth instance label.
[0051] Furthermore, the step S6 is specifically as follows:
[0052] S601, merging a first tooth instance label and a second tooth instance label to obtain a merged tooth instance label;
[0053] S602, according to the merged tooth instance labels, determine the category of each tooth instance by performing a majority vote on the tooth semantic labels in the corresponding tooth instance; the tooth semantic label is the tooth semantic label obtained in step S3;
[0054] S603: Obtain a final tooth instance label according to the merged tooth instance label and the category of each tooth instance.
[0055] The beneficial effects of the present invention are as follows: the present invention proposes a network architecture based on Point Transformer, which can have the robustness of deep learning segmentation algorithm, and uses tooth edge information to perform two-stage segmentation, without complex post-processing steps, and directly obtains the label of the tooth instance. It does not require complex and parameter-sensitive post-processing steps, and directly obtains the segmented tooth instance label; it does not require the design of complex geometric features, and has robustness. BRIEF DESCRIPTION OF THE DRAWINGS
[0056] Figure 1 The figure is a flow chart of the method of the present invention. DETAILED DESCRIPTION
[0057] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.
[0058] like Figure 1 As shown, in one embodiment of the present invention, a two-stage deep learning crown segmentation method using crown edge information includes the following steps:
[0059] S1, obtaining tooth data through a mouth scanning device, dividing the tooth data into grids, and obtaining tooth grid data;
[0060] S2, using the farthest point sampling to sample the tooth mesh data, to obtain a first sampling point set of the tooth mesh data;
[0061] S3, obtaining a tooth semantic label and a first tooth instance label using a tooth segmentation network according to a first sampling point set of the tooth mesh data;
[0062] S4. According to the first tooth instance label, using boundary-aware sampling, obtain a second sampling point set of the tooth mesh data;
[0063] S5, inputting a second sampling point set of the tooth mesh data into the tooth segmentation network to obtain a second tooth instance label;
[0064] S6. Merge the first tooth instance label and the second tooth instance label, and combine them with the tooth semantic label to obtain a final tooth instance label.
[0065] In this embodiment, the traditional tooth segmentation algorithm is not robust enough because of the need to use specially designed geometric features as the basis for segmentation, but the results produced are in line with the geometric constraints and can be used directly. At present, some deep learning algorithms have made some progress in algorithm robustness, but often require complex data post-processing (such as graph segmentation) to ensure that the segmentation results meet the geometric structure characteristics of the teeth.
[0066] The present invention proposes a network architecture based on Point Transformer, which can have the robustness of deep learning segmentation algorithms while using tooth edge information to perform two-stage segmentation, without the need for complex post-processing steps, and directly obtain the label of the tooth instance. Specifically, in the first stage, the farthest point sampling is used to obtain the sampling points, which are used as the input of the tooth segmentation network to obtain preliminary tooth instance labels. Based on the tooth labels obtained for the first time, the boundary-aware sampling method is used to obtain boundary point sampling, which is also used as the input of the tooth segmentation network to obtain the tooth instance labels corresponding to the boundary points. The labels obtained in the two steps are taken as the union to obtain the final tooth instance labels.
[0067] The tooth segmentation network parameters corresponding to the farthest sampling point in the first stage and the boundary sampling point in the second stage of the present invention are not shared, that is, two trainings are required to complete the final segmentation.
[0068] In this embodiment, each step is described in more detail as follows:
[0069] The step S2 is specifically as follows:
[0070] S201, according to the tooth mesh data, a point P0 is selected as a starting point from a point cloud composed of N points composed of vertices to obtain a sampling point set S = {P0};
[0071] S202, calculate the distance from all points in the point cloud to P0, forming an N-dimensional array L = {L i}, select the point corresponding to the maximum distance and add it to the sampling point set; where L i is the i-th point P i The shortest distance to the set of sampling points;
[0072] S203, taking the latest sampling point added to the sampling point set as the center point, calculate the distance from all points in the point cloud except the sampling points in the sampling point set to the center point, for each point P i , determine whether the distance to the center point is less than L i , if so, update L i , otherwise, no update;
[0073] S204, select the point corresponding to the maximum distance in L and add it to the sampling point set;
[0074] S205 , repeating steps S203 - S204 until the number of sampling points in the sampling point set reaches a set value N′, thereby obtaining a first sampling point set.
[0075] In this embodiment, a point P0 is selected as the starting point from the point cloud composed of N points composed of the vertices of the input grid, and a sampling point set S = {P0} is obtained; the distances from all points in the point cloud to P0 are calculated to form an N-dimensional array L, from which the point corresponding to the maximum distance is selected as P1, and the sampling point set is updated to S = {P0, P1}; the distances from all points outside the sampling point set to P1 are calculated, and for each point P i , if its distance to P1 is less than L i , then update L i is this new smaller distance. That is, L i What is stored in is the shortest distance from each point to the sampling point set S; select the point corresponding to the maximum distance in L as P2, update the sampling point set S = {P0, P1, P2}; and continue sampling until the specified N' sampling points are reached.
[0076] The tooth segmentation network in step S3 includes a first Point Transformer module, a point cloud offset module, a label filtering module, a clustering module, a cropping module, a second Point Transformer module and a label output module;
[0077] The first Point Transformer module is used to obtain the tooth semantic label and offset distance of each sampling point according to the input sampling point set;
[0078] The point cloud offset module is used to move the sampling points in the input sampling point set to the center point of the tooth instance according to the offset distance of each sampling point, so as to obtain a point cloud data set after the movement;
[0079] The label filtering module is used to remove the point cloud labeled as gums from the moved point cloud data set according to the tooth semantic labels of each sampling point, so as to obtain filtered sample points;
[0080] The aggregation module is used to cluster the filtered sample points using the DBSCAN method, assign a tooth instance label to each filtered sample point, and assign a tooth instance label to the point cloud in the unsampled point set according to the nearest neighbor principle based on the clustered tooth instance labels to obtain the tooth instance label of each point cloud; the unsampled point set is a sampling point set obtained by removing the point cloud in the input sampling point set from the point cloud of the tooth grid data;
[0081] The clipping module is used to obtain the center point position of each tooth instance according to the tooth instance label of each point cloud, and to set a radius threshold based on the center point position of each tooth instance to clip the sampling points in the first sampling point set to obtain the clipped sample points;
[0082] The second Point Transformer module is used to obtain a tooth-gum mask of each sampling point according to the cropped sample points;
[0083] The label output module is used to make a one-to-one correspondence between the tooth-gum mask of each sampling point and the tooth instance label of each point cloud. If the tooth-gum mask and the tooth instance label of the sampling point are both teeth or gums, no processing is performed. If the tooth-gum mask of the sampling point is displayed as teeth and the tooth instance label is displayed as gums, the tooth instance label of the sampling point is changed to the tooth instance label of the nearest point. If the tooth-gum mask of the sampling point is displayed as gums and the tooth instance label is displayed as teeth, the tooth instance label of the sampling point is changed to gums, and the changed tooth instance labels of each point are used as output data.
[0084] The loss function of the tooth segmentation network is:
[0085] L=L center_offset +L weighted_center_offset +L chamfer +L tooth_class +L teeth_mask
[0086]
[0087] Where L is the loss function of the tooth segmentation network; L center_offset is the center offset loss; L weighted_center_offset is the center weighted offset loss; L chamfer is the distance loss; L tooth_class Loss of tooth classification; L teeth_mask is the tooth mask loss; C is the number of categories; N i is the number of points in the i-th class; x j is the coordinate of the jth point in the i-th category; o j is the offset of the jth point in the i-th category relative to the instance center coordinate c; d() is the distance between two spatial points; D is the number of all direction vector types; N i' is the number of points in the i'th type of direction vector; d j' is the direction vector of the j'th point; d' j' is the direction vector of the j'th point pointing to the instance center point coordinate c; w j is the weight corresponding to the distance; c1 is the distance x j The coordinates of the nearest instance center point; c2 is the distance x j The coordinates of the second closest instance center point; CrossEntropy(p,y) is the cross entropy function; p is the probability of classification; y is the actual type label; N is the total number of sample points; r is the sample point number; y r,i is the label of the rth point in the i-th category; pr,i is the probability that the rth point is predicted to be the i-th category; y r is the true label of the rth point; p r is the predicted value of the rth point; σ() is the sigmoid function.
[0088] The step S3 specifically includes: taking the first sampling point set of the tooth mesh data as the input sampling point set, inputting it into the tooth segmentation network, extracting the tooth semantic label obtained by the first Point Transformer module, and outputting the output data as the first tooth instance label.
[0089] In this embodiment, the coordinates and normal vectors of the input N' sampling points are passed to the first PointTransformer, which has an attention head for offset prediction and an attention head for tooth semantic label prediction, to obtain the predicted tooth semantic label and offset distance; by moving each point to its center point according to the obtained offset, the point cloud after the overall movement is obtained; the part of the point cloud labeled as gums is filtered out; the remaining point cloud is clustered using the DBSCAN method, and labels are assigned to unsampled points based on the clustered labels according to the nearest neighbor principle to obtain tooth instance labels; according to the obtained tooth instance labels, the center point position of each tooth instance is obtained, and a radius threshold is set based on the center position to crop the input N' sampling points. The cropped sampling points are input to the second PointTransformer, which has only one attention head for inferring the tooth-gum mask, to obtain the tooth-gum mask to distinguish whether the sampling point is a tooth or gums; the obtained mask is used to limit the tooth instance label in S303. If the instance label shows that the point cloud is a tooth, and the mask shows that the point cloud is gingiva, then its label is changed to gingiva; if the instance shows that the point cloud is gingiva, and the mask shows that the point cloud is a tooth, then its label is changed to the tooth label of the nearest point in the instance label.
[0090] The loss function is defined as the weighted sum of the center shift loss (between the model prediction and the ground truth), the Chamfer distance loss, the cross entropy loss for tooth classification, and the cross entropy loss for tooth-gingiva mask. The Adam optimization method is used to minimize the loss function until the error converges.
[0091] The step S4 is specifically as follows:
[0092] S401, obtaining the coordinates of each point in the tooth grid data, and constructing a K-dimensional search tree;
[0093] S402, searching for K neighboring points of each point based on a K-dimensional search tree;
[0094] S403, according to the first tooth instance label, calculate the proportion of points with different tooth instance labels from the corresponding point among the K neighboring points of each point as the boundary criterion:
[0095]
[0096] Among them, p1 r is the boundary criterion of the rth point; dif(K r ) is the number of points among the K neighboring points that have different tooth instance labels from the rth point; K is the total number of neighboring points;
[0097] S404: taking a set of points whose boundary criteria are greater than a set threshold as a second sampling point set of the tooth grid data.
[0098] In this embodiment, a KDTree is constructed using the coordinates of each point to quickly query the neighboring points of each point. The label of each neighboring point is determined according to the tooth instance label obtained in S3. The label ratio of each point is calculated. Among the 40 neighboring points, the ratio of points with different labels from the current one is calculated. When the ratio is greater than 0.7, it is considered to be a boundary point and added to the boundary sampling point set.
[0099] The step S5 specifically includes: taking the second sampling point set of the tooth mesh data as the input sampling point set, inputting it into the tooth segmentation network, replacing the tooth semantic label obtained by the first Point Transformer module with the tooth semantic label obtained in step S3, and outputting the output data as the second tooth instance label.
[0100] In this embodiment, the coordinates and normal vector of the input sampling point are passed to the first Point Transformer, which has an attention head for offset prediction and a contrastive boundary learning (CBL) attention head. In addition to predicting the offset distance, the introduced CBL head allows points with the same label at the boundary to have the same features. Conversely, points with different labels at the boundary will output different features. The output additional feature vector is helpful for determining the degree of offset of the points at the boundary. The tooth semantic label uses the tooth semantic label obtained in step S301. By moving each point to its center point according to the obtained offset, the point cloud after the overall movement is obtained. The part of the point cloud labeled as gums is filtered out. The remaining point cloud is clustered using the DBSCAN method, and labels are assigned to the unsampled points in S1 based on the clustered labels according to the nearest neighbor principle to obtain tooth instance labels. According to the obtained tooth instance labels, the center point position of each tooth instance is obtained, and a radius threshold is set based on the center position to crop the sampling points input in S501. The cropped sample points are input into the Point Transformer, which has only one attention head for inferring the tooth-gum mask. The tooth-gum mask is obtained to distinguish whether the sample point is a tooth or a gum. The obtained mask is used to restrict the tooth instance label. The label update rule is the same as in S3. Compared with S3, an additional CBL error is introduced, and the remaining errors are consistent with the training method and S3.
[0101] The step S6 is specifically as follows:
[0102] S601, merging a first tooth instance label and a second tooth instance label to obtain a merged tooth instance label;
[0103] S602, according to the merged tooth instance labels, determine the category of each tooth instance by performing a majority vote on the tooth semantic labels in the corresponding tooth instance; the tooth semantic label is the tooth semantic label obtained in step S3;
[0104] S603: Obtain a final tooth instance label according to the merged tooth instance label and the category of each tooth instance.
[0105] In this embodiment, the category of each tooth instance in step S6 is determined by majority voting of the tooth semantic labels in the corresponding tooth instance. The tooth semantic labels are generated by the farthest sampling point in the process S3 through PointTransformer.
Claims
1. A two-stage deep learning crown segmentation method using crown edge information, It is characterized in that The following steps are involved: S1, obtaining tooth data through a mouth scanning device, dividing the tooth data into grids, and obtaining tooth grid data; S2, using the farthest point sampling to sample the tooth mesh data, to obtain a first sampling point set of the tooth mesh data; S3, obtaining a tooth semantic label and a first tooth instance label using a tooth segmentation network according to a first sampling point set of the tooth mesh data; S4. According to the first tooth instance label, using boundary-aware sampling, obtain a second sampling point set of the tooth mesh data; S5, inputting a second sampling point set of the tooth mesh data into the tooth segmentation network to obtain a second tooth instance label; S6. Merge the first tooth instance label and the second tooth instance label, and combine them with the tooth semantic label to obtain a final tooth instance label.
2. The two-stage deep learning crown segmentation method using crown edge information according to claim 1, It is characterized in that The step S2 is specifically as follows: S201, according to the tooth mesh data, select a point P from the point cloud composed of N points composed of vertices 0 As the starting point, we get the sampling point set S = {P 0 }; S202, calculate all points in the point cloud to P 0 The distance constitutes an N-dimensional array L = {L i }, select the point corresponding to the maximum distance and add it to the sampling point set; where L i is the i-th point P i The shortest distance to the set of sampling points; S203, taking the latest sampling point added to the sampling point set as the center point, calculate the distance from all points in the point cloud except the sampling points in the sampling point set to the center point, for each point P i , determine whether the distance to the center point is less than L i , if so, update L i , otherwise, no update; S204, select the point corresponding to the maximum distance in L and add it to the sampling point set; S205 , repeating steps S203 - S204 until the number of sampling points in the sampling point set reaches a set value N′, thereby obtaining a first sampling point set.
3. The two-stage deep learning crown segmentation method using crown edge information according to claim 1, It is characterized in that The tooth segmentation network in step S3 includes a first Point Transformer module, a point cloud offset module, a label filtering module, a clustering module, a cropping module, a second Point Transformer module and a label output module; The first Point Transformer module is used to obtain the tooth semantic label and offset distance of each sampling point according to the input sampling point set; The point cloud offset module is used to move the sampling points in the input sampling point set to the center point of the tooth instance according to the offset distance of each sampling point, so as to obtain a point cloud data set after the movement; The label filtering module is used to remove the point cloud labeled as gums from the moved point cloud data set according to the tooth semantic labels of each sampling point, so as to obtain filtered sample points; The aggregation module is used to cluster the filtered sample points using the DBSCAN method, assign a tooth instance label to each filtered sample point, and assign a tooth instance label to the point cloud in the unsampled point set according to the nearest neighbor principle based on the clustered tooth instance labels to obtain the tooth instance label of each point cloud; The unsampled point set is a sampling point set obtained by removing the point cloud in the input sampling point set from the point cloud of the tooth mesh data; The clipping module is used to obtain the center point position of each tooth instance according to the tooth instance label of each point cloud, and to set a radius threshold based on the center point position of each tooth instance to clip the sampling points in the first sampling point set to obtain the clipped sample points; The second Point Transformer module is used to obtain a tooth-gum mask of each sampling point according to the cropped sample points; The label output module is used to make a one-to-one correspondence between the tooth-gum mask of each sampling point and the tooth instance label of each point cloud. If the tooth-gum mask and the tooth instance label of the sampling point are both teeth or gums, no processing is performed. If the tooth-gum mask of the sampling point is displayed as teeth and the tooth instance label is displayed as gums, the tooth instance label of the sampling point is changed to the tooth instance label of the nearest point. If the tooth-gum mask of the sampling point is displayed as gums and the tooth instance label is displayed as teeth, the tooth instance label of the sampling point is changed to gums, and the changed tooth instance labels of each point are used as output data.
4. The two-stage deep learning crown segmentation method using crown edge information according to claim 3, It is characterized in that The loss function of the tooth segmentation network is: L=L center_offset +L weighted_center_offset +L chamfer +L tooth_class +L teeth_mask Where L is the loss function of the tooth segmentation network; L center_offset is the center offset loss; L weighted_center_offset is the center weighted offset loss; L chamfer is the distance loss; L tooth_class Loss of tooth classification; L teeth_mask is the tooth mask loss; C is the number of categories; N i is the number of points in the i-th class; x j is the coordinate of the jth point in the i-th category; o j is the offset of the jth point in the i-th category relative to the instance center coordinate c; d() is the distance between two spatial points; D is the number of all direction vector types; N i' is the number of points in the i'th type of direction vector; d j' is the direction vector of the j'th point; d' j' is the direction vector of the j'th point pointing to the instance center point coordinate c; w j is the weight of the corresponding distance; c 1 is the distance x j The coordinates of the nearest instance center point; c 2 is the distance x j The coordinates of the second closest instance center point; CrossEntropy(p,y) is the cross entropy function; p is the probability of classification; y is the actual type label; N is the total number of sample points; r is the sample point number; y r,i is the label of the rth point in the i-th category; p r,i is the probability that the rth point is predicted to be the i-th category; y r is the true label of the rth point; p r is the predicted value of the rth point; σ() is the sigmoid function.
5. The two-stage deep learning crown segmentation method using crown edge information according to claim 3, It is characterized in that The step S3 specifically includes: taking the first sampling point set of the tooth mesh data as the input sampling point set, inputting it into the tooth segmentation network, extracting the tooth semantic label obtained by the first Point Transformer module, and outputting the output data as the first tooth instance label.
6. The two-stage deep learning crown segmentation method using crown edge information according to claim 1, It is characterized in that The step S4 is specifically as follows: S401, obtaining the coordinates of each point in the tooth grid data, and constructing a K-dimensional search tree; S402, searching for K neighboring points of each point based on a K-dimensional search tree; S403, according to the first tooth instance label, calculate the proportion of points with different tooth instance labels from the corresponding point among the K neighboring points of each point as the boundary criterion: Among them, p1 r is the boundary criterion of the rth point; dif(K r ) is the number of points among the K neighboring points that have different tooth instance labels from the rth point; K is the total number of neighboring points; S404: taking a set of points whose boundary criteria are greater than a set threshold as a second sampling point set of the tooth grid data.
7. The two-stage deep learning crown segmentation method using crown edge information according to claim 3, It is characterized in that The step S5 specifically includes: taking the second sampling point set of the tooth mesh data as the input sampling point set, inputting it into the tooth segmentation network, replacing the tooth semantic label obtained by the first Point Transformer module with the tooth semantic label obtained in step S3, and outputting the output data as the second tooth instance label.
8. The two-stage deep learning crown segmentation method using crown edge information according to claim 1, It is characterized in that The step S6 is specifically as follows: S601, merging a first tooth instance label and a second tooth instance label to obtain a merged tooth instance label; S602, according to the merged tooth instance labels, determine the category of each tooth instance by performing a majority vote on the tooth semantic labels in the corresponding tooth instance; the tooth semantic label is the tooth semantic label obtained in step S3; S603: Obtain a final tooth instance label according to the merged tooth instance label and the category of each tooth instance.