A three-dimensional tooth point cloud segmentation and tooth type recognition method based on deep learning
By introducing normal vector and curvature features and boundary-aware attention mechanism into the PointNet++ network, the problem of insufficient accuracy in 3D tooth point cloud segmentation is solved, and stable conversion between single tooth instance extraction and tooth type recognition is achieved, thereby improving segmentation performance and recognition effect.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- UNIV OF ELECTRONICS SCI & TECH OF CHINA
- Filing Date
- 2026-04-07
- Publication Date
- 2026-07-10
AI Technical Summary
Existing 3D tooth point cloud segmentation methods lack sufficient segmentation accuracy in complex dental arch scenes, especially in adjacent tooth regions and areas with blurred boundaries, which are prone to missegmentation and instance adhesion, affecting the accuracy of single tooth instance extraction and tooth type identification.
A deep learning-based method for 3D tooth point cloud segmentation and tooth type recognition is adopted. By constructing a PointNet++ network, introducing normal vectors and curvature geometric features, and combining a boundary-aware attention mechanism, multi-level probabilistic smoothing and interpolation recovery are performed, and density-adaptive connectivity analysis is executed to achieve single tooth instance extraction and tooth type recognition.
It improves the ability to perceive tooth boundaries, enhances the segmentation accuracy of complex adjacent regions, achieves a stable transition from point-level semantic prediction to single-tooth instance extraction, and improves the stability and accuracy of tooth type recognition. Experimental results show an average intersection-union ratio of 89.11% and an average class accuracy of 94.69% on the 3DTeethSeg dataset.
Smart Images
Figure CN122368979A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of medical image processing technology, and in particular to a method for three-dimensional tooth point cloud segmentation and tooth type recognition based on deep learning. Background Technology
[0002] With the development of digital dentistry, intraoral scanning technology has been widely used in orthodontic treatment, dental arch modeling, and digital analysis. Intraoral scanning devices typically output three-dimensional point cloud or mesh data of the dental arch, including teeth, gums, and local soft tissues. This type of data can reflect the spatial structure of the dental arch relatively completely and is an important foundation for automatic analysis of orthodontic parameters and intelligent assisted treatment. However, this type of three-dimensional point cloud data is usually an unordered discrete set of points and does not directly possess anatomical semantic information about teeth, making it difficult to use directly for clinical analysis.
[0003] Existing 3D tooth segmentation methods, particularly traditional geometric rule-based methods, typically rely on strategies such as curvature, normal vectors, geodesic distance, or region growing. However, these methods are susceptible to threshold settings and point cloud noise in complex dental arch scenarios, resulting in insufficient automation and robustness. While deep learning methods have been applied to 3D tooth point cloud segmentation in recent years, they still tend to suffer from segmentation boundary overlap and instance adhesion issues in adjacent tooth regions, areas with blurred boundaries, and regions with complex local geometric variations. Especially in cases of crowded, misaligned, or unstable scan quality, the local geometric differences between adjacent teeth are not significant, reducing the network's ability to discriminate boundary regions and further impacting the accuracy of single-tooth instance extraction and tooth type identification.
[0004] Therefore, there is an urgent need to propose an automatic 3D tooth point cloud processing method that can enhance tooth boundary perception, improve the segmentation accuracy of complex adjacent regions, and enable single tooth instance extraction and tooth type identification. Summary of the Invention
[0005] The purpose of this invention is to provide a method for three-dimensional tooth point cloud segmentation and tooth type recognition based on deep learning, so as to solve the problems of insufficient accuracy of three-dimensional tooth point cloud segmentation, easy missegmentation in complex boundary regions, and difficulty in stably converting point-level semantic prediction into single tooth instance and tooth type recognition results in the prior art.
[0006] The technical solution adopted by this invention to solve its technical problem is as follows: A deep learning-based method for 3D tooth point cloud segmentation and tooth type recognition includes the following steps: Acquire 3D tooth point cloud data and construct a 3D tooth point cloud dataset; Data preprocessing operations are performed on the 3D tooth point cloud data in the 3D tooth point cloud dataset, and the dataset is divided into training set, validation set and test set. A three-dimensional tooth point cloud semantic segmentation network is constructed and a loss function is determined. The three-dimensional tooth point cloud semantic segmentation network is then trained based on the loss function. The trained 3D tooth point cloud semantic segmentation network is used to perform point-by-point semantic prediction of the tooth row point cloud under test to obtain the point-level category probability distribution. Multi-level probability smoothing and interpolation recovery are performed on the point-level category probability distribution to improve the spatial continuity of the prediction results and restore the original point cloud resolution; Density-adaptive connectivity analysis and denoising are performed on the restored point cloud segmentation results to extract single tooth instances; For each single tooth instance, an instance-level category statistical method is used to determine the final semantic category, thereby achieving tooth type recognition.
[0007] In some embodiments, the data preprocessing operation on the 3D tooth point cloud data in the 3D tooth point cloud dataset includes: The original FDI tooth position numbers corresponding to the 3D tooth point cloud data are classified and mapped, the 3D tooth point cloud data is standardized, and the 3D tooth point cloud data is uniformly sampled.
[0008] In some embodiments, the ratio of the training set, validation set, and test set is 8:1:1.
[0009] In some embodiments, the constructed 3D tooth point cloud semantic segmentation network uses PointNet++ as the backbone network and includes a geometric feature enhancement module, an encoder module, a decoder module, and a boundary-aware attention module. The geometric feature enhancement module is used to extract normal vectors and curvature information from the input point cloud and concatenate them with the original three-dimensional coordinates to form enhanced features; The encoder module consists of multiple layers of abstraction layers, used to perform point cloud downsampling and local feature extraction layer by layer; The decoder module consists of multiple feature propagation layers, which are used to upsample deep features step by step to restore the original point cloud resolution and output the semantic category probability of each point; The boundary-aware attention module is set in the local neighborhood feature aggregation stage of the encoder. It is used to adaptively allocate attention weights and complete the boundary enhancement feature aggregation based on the spatial relative position and feature differences between the center point and the neighboring points.
[0010] In some embodiments, the geometric feature enhancement module includes: For each point in the input point cloud, a local neighborhood is constructed based on K-nearest neighbor search; Construct the covariance matrix based on the local neighborhood point set and perform eigenvalue decomposition; Use the eigenvector corresponding to the smallest eigenvalue as the normal vector of that point; Calculate the local curvature of the point based on the sum of the minimum eigenvalue and all eigenvalues; The three-dimensional coordinates, normal vectors, and curvature are concatenated to form enhanced input features.
[0011] In some embodiments, the boundary-aware attention module includes: Extract the three-dimensional coordinates and feature vectors from the center point and its neighboring points respectively; Calculate the spatial location and feature differences of neighboring points relative to the center point; The features of the center point, the features of the neighboring points, the spatial location differences, and the feature differences are concatenated and then input into the multilayer perceptron to calculate the correlation score. The relevance score is normalized using the Softmax function to obtain the attention weights of the neighborhood points. The neighborhood point features are weighted and summed according to the attention weights to obtain the local aggregated features with enhanced boundaries.
[0012] In some embodiments, the loss function is a weighted cross-entropy loss function.
[0013] In some embodiments, performing density-adaptive connectivity analysis on the recovered point cloud segmentation results includes: Randomly sample the entire point cloud and calculate the average spatial distance between the sampled point and its nearest neighbor. The search radius for growing connected regions is dynamically determined based on the average spatial spacing. Perform Euclidean space connectivity analysis within the same set of semantically labeled points to extract all connected regions; The connected region with the most points is taken as the main connected component, and isolated pseudo-connected components are filtered out according to the ratio of the number of points of the remaining connected regions to the main connected component. The isolated regions after filtering are semantically reassigned based on the distribution of neighborhood categories.
[0014] The beneficial effects of this invention are as follows: First, by introducing normal vectors and curvature geometric features at the input of the PointNet++ network, this invention effectively enhances the network's ability to express local geometric changes on the tooth surface. Second, by introducing a boundary-aware attention mechanism in the local neighborhood feature aggregation stage, this invention enables the network to pay more attention to key feature points in the tooth boundary region and adjacent regions, reducing feature aliasing between adjacent teeth. Third, through multi-level probabilistic smoothing, interpolation recovery, and density adaptive connectivity analysis, this invention achieves a stable transition from point-level semantic prediction to single-tooth instance extraction. Fourth, through an instance-level majority voting strategy, this invention improves the stability and clinical usability of tooth type recognition results. Finally, the method of this invention has been experimentally verified, achieving an average intersection-union ratio of 89.11% and an average class accuracy of 94.69% on the 3DTeethSeg dataset, indicating that the method has good segmentation performance and recognition effect. Attached Figure Description
[0015] Figure 1 This is a flowchart of a method for three-dimensional tooth point cloud segmentation and tooth type recognition based on deep learning in Embodiment 1 of the present invention; Figure 2 This is a diagram of the semantic segmentation network structure of the three-dimensional tooth point cloud in Embodiment 2 of the present invention; Figure 3 This is a flowchart of the local geometric feature enhancement module in Embodiment 3 of the present invention; Figure 4 This is a structural diagram of the boundary-aware attention module in Embodiment 4 of the present invention; Figure 5 This is a flowchart of the multi-level probability smoothing and interpolation recovery process in Embodiment 5 of the present invention. Detailed Implementation
[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.
[0017] Example 1
[0018] This embodiment provides a method for 3D tooth point cloud segmentation and tooth type recognition based on deep learning. See the flowchart below. Figure 1 The method may include the following steps: S1. Obtain 3D tooth point cloud data and construct a 3D tooth point cloud dataset; S2. Perform data preprocessing on the 3D tooth point cloud data in the 3D tooth point cloud dataset and divide it into training set, validation set and test set. S3. Construct a three-dimensional tooth point cloud semantic segmentation network and determine the loss function, and train the three-dimensional tooth point cloud semantic segmentation network based on the loss function; S4. Use the trained 3D tooth point cloud semantic segmentation network to perform point-by-point semantic prediction of the tooth row point cloud to obtain the point-level category probability distribution. S5. Perform multi-level probability smoothing and interpolation recovery on the point-level category probability distribution to improve the spatial continuity of the prediction results and restore the original point cloud resolution; S6. Perform density-adaptive connectivity analysis and denoising on the restored point cloud segmentation results to extract single tooth instances; S7. For each single tooth instance, the final semantic category is determined by instance-level category statistics to achieve tooth type recognition.
[0019] In this embodiment, the first step is to acquire raw intraoral scan point cloud data. This can be achieved by acquiring 3D dental arch point cloud data collected by an intraoral scanning device, which serves as the input for subsequent segmentation and recognition. In practical applications, the publicly available 3DTeethSeg dataset can be used. This dataset contains complete geometric surface information of the teeth and gingival regions and provides point-by-point semantic annotations. To ensure the quality of the training samples, only samples with a single jaw, 14 complete permanent teeth, no obvious missing teeth, and no severe scanning defects are retained to form the final experimental dataset.
[0020] In this embodiment, the data preprocessing operation of the three-dimensional tooth point cloud data in the three-dimensional tooth point cloud dataset may include: class mapping of the original FDI tooth position numbers corresponding to the three-dimensional tooth point cloud data, standardization processing of the three-dimensional tooth point cloud data, and unified sampling of the three-dimensional tooth point cloud data.
[0021] Specifically, in this embodiment, the category mapping of the original FDI tooth position numbers corresponding to the 3D tooth point cloud data refers to: merging the corresponding teeth of the upper and lower jaws, preserving the differences between the left and right sides, and uniformly mapping the gingiva / background to category 0, ultimately constructing a 15-category label system to establish a unified semantic label foundation for subsequent network training; standardizing the 3D tooth point cloud data, including centering and translational normalization, to eliminate differences in scanning coordinate system, position, and scale among different samples, making the model focus more on the relative geometric structure of the teeth themselves; when uniformly sampling the 3D tooth point cloud data, since the number of points in the original oral scan samples is approximately between 8×10^4 and 1.4×10^5, in order to reduce training overhead and maintain a consistent input scale, each sample is uniformly processed into a fixed number of points input. When the number of original points is large, the farthest point sampling is used; when the number of original points is insufficient, random sampling with replacement is used to supplement them.
[0022] After preprocessing, all samples are divided into training set, validation set and test set. In this embodiment, they are divided in a ratio of 8:1:1, with 789 samples in the training set, 99 samples in the validation set and 99 samples in the test set. The training set is used for model learning, the validation set is used for parameter adjustment and training monitoring, and the test set is used for final performance evaluation.
[0023] After the dataset is divided, a 3D tooth point cloud semantic segmentation network can be constructed. In this embodiment, the model is based on PointNet++, with a geometric feature enhancement module added at the input end and a boundary awareness attention module added at the local feature aggregation stage to improve the ability to recognize changes in tooth surface morphology and adjacent boundary regions.
[0024] Specifically, the constructed 3D tooth point cloud semantic segmentation network uses PointNet++ as its backbone network and includes a geometric feature enhancement module, an encoder module, a decoder module, and a boundary-aware attention module. The geometric feature enhancement module extracts normal vectors and curvature information from the input point cloud and concatenates them with the original 3D coordinates to form enhanced features. The geometric feature enhancement module includes: For each point in the input point cloud, a local neighborhood is constructed based on K-nearest neighbor search; Construct the covariance matrix based on the local neighborhood point set and perform eigenvalue decomposition; Use the eigenvector corresponding to the smallest eigenvalue as the normal vector of that point; Calculate the local curvature of the point based on the sum of the minimum eigenvalue and all eigenvalues; The three-dimensional coordinates, normal vectors, and curvature are concatenated to form enhanced input features.
[0025] The encoder module consists of multiple layers of abstraction, which are used to perform point cloud downsampling and local feature extraction layer by layer.
[0026] The decoder module consists of multiple feature propagation layers, which are used to upsample deep features step by step to restore the original point cloud resolution and output the semantic category probability of each point.
[0027] The boundary-aware attention module is set in the local neighborhood feature aggregation stage of the encoder. It is used to adaptively allocate attention weights and complete boundary enhancement feature aggregation based on the spatial relative position and feature differences between the center point and the neighboring points. The boundary-aware attention module includes: extracting three-dimensional coordinates and feature vectors for the center point and its neighboring points respectively; calculating the spatial position difference and feature difference of the neighboring points relative to the center point; concatenating the center point features, neighboring point features, spatial position difference and feature difference, and inputting them into the multilayer perceptron to calculate the correlation score. The relevance score is normalized using the Softmax function to obtain the attention weights of the neighborhood points; the features of the neighborhood points are then weighted and summed according to the attention weights to obtain the local aggregated features with enhanced boundaries.
[0028] It should be noted that traditional PointNet++ typically only uses the three-dimensional coordinates (x, y, z) of a point as input. While this input method can express the spatial location of a point, it is difficult to accurately describe the local orientation and curvature changes on the tooth surface. Therefore, it is prone to feature confusion in areas with drastic geometric changes, such as interdental spaces, cusps, proximal surfaces, and gingival junctions. In this embodiment, local normal vectors and curvatures are additionally calculated for each point at the network input end and concatenated with the original three-dimensional coordinates to form a 7-dimensional enhanced feature vector before being input into the PointNet++ backbone network. The specific process is as follows: First, a K-nearest neighbor search is performed on any point in the point cloud. In this embodiment, k=20 can be set to construct the local neighborhood of the point. Second, a covariance matrix is established based on the local neighborhood point set, and eigenvalue decomposition is performed. Then, the eigenvector corresponding to the smallest eigenvalue is used as the normal vector of the point, and the local curvature of the point is calculated based on the eigenvalue relationship. Finally, the three-dimensional coordinates, normalized normal vectors, and curvatures are concatenated to obtain the 7-dimensional enhanced input for each point.
[0029] The boundary attention mechanism module extracts the coordinates of the center point, the coordinates of the neighboring points, the features of the center point, and the features of the neighboring points for any center point and its neighboring points. Then, it calculates the spatial relative position difference and feature difference between the neighboring points and the center point. Next, it concatenates the center point features, the neighboring point features, the spatial difference, and the feature difference, and inputs the result into a multilayer perceptron to calculate the relevance score of the neighboring points to the center point. Subsequently, it normalizes the above scores using Softmax to obtain the attention weights of each neighboring point. Finally, instead of using traditional max pooling, it performs a weighted summation of the neighboring features based on the attention weights to obtain local aggregated features.
[0030] In terms of overall performance comparison, under the same dataset partitioning and training conditions, the basic PointNet++ achieves an mAcc of 84.25% and an mIoU of 69.22%; while the improved model proposed in this embodiment improves the mAcc to 94.69% and the mIoU to 89.11%. In other words, compared to the basic PointNet++, the model in this embodiment improves the average intersection-union ratio by 19.89 percentage points and the average class accuracy by 10.44 percentage points, indicating that the improved network not only has more accurate overall segmentation but also less class confusion.
[0031] It should be noted that in this embodiment, the loss function is a weighted cross-entropy loss function, which can reduce the impact of class imbalance on network training.
[0032] Specifically, performing density-adaptive connectivity analysis on the recovered point cloud segmentation results may include: Randomly sample the entire point cloud and calculate the average spatial distance between the sampled point and its nearest neighbor. The search radius for growing connected regions is dynamically determined based on the average spatial spacing. Perform Euclidean space connectivity analysis within the same set of semantically labeled points to extract all connected regions; The connected region with the most points is taken as the main connected component, and isolated pseudo-connected components are filtered out according to the ratio of the number of points of the remaining connected regions to the main connected component. The isolated regions after filtering are semantically redistributed based on the distribution of neighborhood categories.
[0033] Example 2
[0034] Based on Example 1, this example uses intraoral scanning three-dimensional tooth point cloud data as an example, and includes the following steps: Step 1: Build the dataset.
[0035] First, a 3D tooth point cloud dataset is constructed for training. This step corresponds to... Figure 2 At the top is "Build Dataset". The dataset is sourced from 3DTeethSeg and has undergone sample selection, label mapping, standardization, and uniform sampling to obtain standardized input data that can be directly used for deep learning training.
[0036] Step 2: Confirm the loss function, hyperparameters, etc.
[0037] After constructing the dataset, it is necessary to determine the loss function and various training parameters required for network training. In this embodiment, to address the issues of a large number of points in the gingival region and an uneven distribution among different tooth categories, a weighted cross-entropy loss function is adopted. Class weights are constructed based on the reciprocal of the number of points for each category in the training set, and mean normalization is performed to enhance the learning effect of minority classes and maintain training stability. Simultaneously, training hyperparameters such as the number of training epochs, learning rate, and batch size also need to be set, which corresponds to... Figure 2 The "confirmation loss function, hyperparameters, etc." in the text.
[0038] Regarding training hyperparameters: the total number of training epochs was 400, the initial learning rate was 0.0003, the batch size was 2, and the input point cloud size was 20,000 points, with each point containing 7-dimensional features. During training, data augmentation methods were employed, including random rotation along the Z-axis and Gaussian coordinate perturbation, where the standard deviation of the Gaussian perturbation was 0.005 and the truncation threshold was 0.02. A Dropout rate of 0.5 was set in the fully connected layers, and an equivalent batch size of 16 was achieved using 8-step gradient accumulation.
[0039] Step 3: Add a feature enhancement module.
[0040] Figure 2 The "Add Feature Enhancement Module" mentioned corresponds to the geometric feature enhancement design proposed in this embodiment. That is, based on the original PointNet++ which only inputs three-dimensional coordinates, normal vectors and curvature features are added, expanding the network input from a simple position description to a comprehensive geometric representation that includes spatial position, surface orientation, and curvature. Experiments in this embodiment show that adding this module alone improves mIoU from 69.22% to 88.00%, indicating that this module has a significant effect on tooth boundary and local detail recognition.
[0041] Building upon the original PointNet++ which only accepts 3D coordinates as input, a feature enhancement module based on normal vectors and curvature is added. Specifically, a local neighborhood is constructed for each point in the input point cloud using K-nearest neighbor search, with the number of neighborhood points set to 20. A covariance matrix is constructed based on the local neighborhood point set, and the normal vector and local curvature of that point are extracted through eigenvalue decomposition. Then, the original 3D coordinates, normal vectors, and curvature are concatenated to form a 7D enhanced input feature. Through this design, the network input is expanded from a simple positional description to a comprehensive geometric representation that simultaneously includes spatial location, surface orientation, and curvature, thereby enhancing the model's ability to express tooth boundaries, adjacent surfaces, and local detailed structures.
[0042] Ablation experiments were conducted under the same dataset partitioning, training strategy, and hyperparameter conditions. The experiments used the original PointNet++ model with only 3D coordinates as the baseline, achieving an average class accuracy (mAcc) of 84.25% and an average intersection-over-union (mIoU) of 69.22%. When only a feature enhancement module was added, without incorporating a boundary-aware attention mechanism, the model performance improved to mAcc = 93.95% and mIoU = 88.00%. In other words, by introducing only normal vectors and curvature features, mIoU increased from 69.22% to 88.00%, an absolute improvement of 18.78 percentage points; and mAcc increased from 84.25% to 93.95%, an absolute improvement of 9.70 percentage points. These results demonstrate that the feature enhancement module itself significantly improves model performance.
[0043] Step 4: Add a boundary attention mechanism module.
[0044] After adding the feature enhancement module, a boundary attention mechanism module was further added. This module is embedded in the PointNet++ local neighborhood feature aggregation process. By learning the spatial and feature relationships between the center point and neighboring points, it adaptively enhances the feature representation of the boundary region, making it particularly suitable for complex regions such as crowded teeth, close proximal contacts, and narrow interdental spaces. Ablation experiments show that after adding this boundary attention module on top of feature enhancement, the mIoU further improved from 88.00% to 89.11%.
[0045] Specifically, for any center point and its neighboring points, the coordinates of the center point, the coordinates of the neighboring points, the features of the center point, and the features of the neighboring points are first extracted. Then, the spatial positional differences and feature differences of the neighboring points relative to the center point are calculated. The center point features, the features of the neighboring points, and the above-mentioned difference information are then concatenated and input into a multilayer perceptron to obtain the attention relevance score corresponding to the neighboring points. Subsequently, the attention weights of each neighboring point are obtained through Softmax normalization. Finally, weighted summation is used instead of traditional max pooling to complete the local feature aggregation. In this way, the network can automatically strengthen the expression of key boundary points such as interdental spaces, proximal contact areas, and cervical transition areas, while assigning lower weights to smooth non-boundary regions.
[0046] Ablation experiments were conducted under the same dataset partitioning, training strategy, and hyperparameters. Experimental results show that when using only the original PointNet++ baseline model, mAcc is 84.25% and mIoU is 69.22%. After adding the feature enhancement module, mAcc increases to 93.95% and mIoU increases to 88.00%. Furthermore, adding a boundary-aware attention mechanism further improves the complete model's mAcc to 94.69% and mIoU to 89.11%. In other words, compared to the model with only the feature enhancement module, the boundary attention mechanism further improves mAcc by 0.74 percentage points and mIoU by 1.11 percentage points.
[0047] Step 5: Deep learning model training.
[0048] After determining the network structure, the model is trained using training and validation sets. During training, the model continuously learns the feature differences between teeth and gums, and between different tooth position categories. The model convergence and parameter settings are monitored using the validation set, ultimately resulting in a trained semantic segmentation model. Figure 2 The "deep learning model training" shown below illustrates this process.
[0049] Step Six: Obtain a 3D tooth point cloud semantic segmentation model based on deep learning. After the above steps, the final result is obtained Figure 2 The image at the bottom shows a "Deep Learning-Based Semantic Segmentation Model for 3D Tooth Point Clouds." This model can perform point-by-point semantic prediction on the input dental arch point cloud and provides a foundation for subsequent probability smoothing, interpolation recovery, single-tooth instance extraction, and tooth type recognition. Experimental results show that the model has good boundary recognition ability and segmentation stability in complex dental arch scenarios.
[0050] Example 3
[0051] See Figure 3 This embodiment describes the geometric feature enhancement module. First, the original point cloud [N,3] is input, meaning each point contains only three-dimensional coordinate information. Then, a KNN neighborhood search is performed on each point to construct a local neighborhood; next, a local covariance matrix is established based on the neighborhood point set, and eigenvalue decomposition is performed to extract the normal vector and curvature information of that point. Finally, the original coordinates, normalized normal vector, and curvature are concatenated to form the enhanced feature input [N,7]. In this way, the network input not only includes position coordinates but also surface orientation and curvature information, thereby enhancing the ability to express complex morphologies of tooth boundaries, gaps, and occlusal surfaces.
[0052] Example 4
[0053] See Figure 4 This embodiment describes the boundary-aware attention mechanism module. For any center point and its neighboring points, the coordinates of the center point, the coordinates of the neighboring points, the features of the center point, and the features of the neighboring points are extracted respectively. Then, the spatial difference and feature difference between the two are calculated, and the above information is concatenated and input into a multilayer perceptron to obtain the attention score of the neighboring points relative to the center point. Then, the attention score is normalized using Softmax to obtain the attention weight of each neighboring point, and the neighboring features are weighted and summed to form local aggregated features. Compared with traditional max pooling, this module can better highlight points with drastic changes in boundary region features, thereby improving the segmentation accuracy of adjacent tooth contact areas and complex boundary regions.
[0054] Ablation experiments were conducted under the same dataset partitioning, training strategy, and hyperparameter settings. In the experiments, PointNet++(+FE) can be considered a control model that "includes a feature enhancement module but still uses traditional max pooling for local aggregation," achieving an average class accuracy (mAcc) of 93.95% and an average intersection-over-union (mIoU) of 88.00%. Based on this, by replacing traditional max pooling with boundary-aware attention-weighted aggregation, PointNet++(+FE+BAM) was obtained, with its mAcc increasing to 94.69% and mIoU increasing to 89.11%. In other words, by only changing the local aggregation method, the boundary-aware attention mechanism improved mAcc by 0.74 percentage points and mIoU by 1.11 percentage points. These results demonstrate that compared to traditional max pooling, the boundary-aware attention mechanism can more effectively preserve local boundary details, thereby improving the segmentation accuracy of 3D tooth point clouds.
[0055] Example 5
[0056] See Figure 5 This embodiment describes the post-processing optimization of the segmentation results. First, global probability smoothing is performed on the 20,000 predicted points output by the network. Mean fusion is then applied to the class probabilities of points within a larger neighborhood to reduce the influence of isolated noise points. Next, local boundary correction is performed on low-confidence points. A secondary correction is applied to the prediction results within a smaller neighborhood based on neighborhood class consistency to avoid excessive boundary blurring. Finally, an inverse distance weighted interpolation method is used to restore the optimized sampled point prediction results to the original resolution point cloud, obtaining the original-scale high-precision segmentation result. This method can simultaneously ensure overall smoothness and boundary preservation, providing a more stable segmentation foundation for subsequent single-tooth instance extraction and type recognition.
[0057] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.
Claims
1. A method for 3D tooth point cloud segmentation and tooth type recognition based on deep learning, characterized in that, Includes the following steps: Acquire 3D tooth point cloud data and construct a 3D tooth point cloud dataset; Data preprocessing operations are performed on the 3D tooth point cloud data in the 3D tooth point cloud dataset, and the dataset is divided into training set, validation set and test set. A three-dimensional tooth point cloud semantic segmentation network is constructed and a loss function is determined. The three-dimensional tooth point cloud semantic segmentation network is then trained based on the loss function. The trained 3D tooth point cloud semantic segmentation network is used to perform point-by-point semantic prediction of the tooth row point cloud under test to obtain the point-level category probability distribution. Multi-level probability smoothing and interpolation recovery are performed on the point-level category probability distribution to improve the spatial continuity of the prediction results and restore the original point cloud resolution; Density-adaptive connectivity analysis and denoising are performed on the restored point cloud segmentation results to extract single tooth instances; For each single tooth instance, an instance-level category statistical method is used to determine the final semantic category, thereby achieving tooth type recognition.
2. The method for three-dimensional tooth point cloud segmentation and tooth type recognition based on deep learning according to claim 1, characterized in that, The data preprocessing operation for the 3D tooth point cloud data in the 3D tooth point cloud dataset includes: The original FDI tooth position numbers corresponding to the 3D tooth point cloud data are classified and mapped, the 3D tooth point cloud data is standardized, and the 3D tooth point cloud data is uniformly sampled.
3. The method for three-dimensional tooth point cloud segmentation and tooth type recognition based on deep learning according to claim 1, characterized in that, The ratio of the training set, validation set, and test set is 8:1:
1.
4. The method for three-dimensional tooth point cloud segmentation and tooth type recognition based on deep learning according to claim 1, characterized in that, The constructed 3D tooth point cloud semantic segmentation network uses PointNet++ as the backbone network and includes a geometric feature enhancement module, an encoder module, a decoder module, and a boundary-aware attention module. The geometric feature enhancement module is used to extract normal vectors and curvature information from the input point cloud and concatenate them with the original three-dimensional coordinates to form enhanced features; The encoder module consists of multiple layers of abstraction layers, used to perform point cloud downsampling and local feature extraction layer by layer; The decoder module consists of multiple feature propagation layers, which are used to upsample deep features step by step to restore the original point cloud resolution and output the semantic category probability of each point; The boundary-aware attention module is set in the local neighborhood feature aggregation stage of the encoder. It is used to adaptively allocate attention weights and complete the boundary enhancement feature aggregation based on the spatial relative position and feature differences between the center point and the neighboring points.
5. The method for three-dimensional tooth point cloud segmentation and tooth type recognition based on deep learning according to claim 4, characterized in that, The geometric feature enhancement module includes: For each point in the input point cloud, a local neighborhood is constructed based on K-nearest neighbor search; Construct the covariance matrix based on the local neighborhood point set and perform eigenvalue decomposition; Use the eigenvector corresponding to the smallest eigenvalue as the normal vector of that point; Calculate the local curvature of the point based on the minimum eigenvalue and the sum of all eigenvalues; The three-dimensional coordinates, normal vectors, and curvature are concatenated to form enhanced input features.
6. The method for three-dimensional tooth point cloud segmentation and tooth type recognition based on deep learning according to claim 4, characterized in that, The boundary-aware attention module includes: Extract the three-dimensional coordinates and feature vectors from the center point and its neighboring points respectively; Calculate the spatial location and feature differences of neighboring points relative to the center point; The features of the center point, the features of the neighboring points, the spatial location differences, and the feature differences are concatenated and then input into the multilayer perceptron to calculate the correlation score. The relevance score is normalized using the Softmax function to obtain the attention weights of the neighborhood points. The neighborhood point features are weighted and summed according to the attention weights to obtain the local aggregated features with enhanced boundaries.
7. The method for three-dimensional tooth point cloud segmentation and tooth type recognition based on deep learning according to claim 1, characterized in that, The loss function used is the weighted cross-entropy loss function.
8. The method for three-dimensional tooth point cloud segmentation and tooth type recognition based on deep learning according to claim 1, characterized in that, The density-adaptive connectivity analysis performed on the restored point cloud segmentation results includes: Randomly sample the entire point cloud and calculate the average spatial distance between the sampled point and its nearest neighbor. The search radius for growing connected regions is dynamically determined based on the average spatial spacing. Perform Euclidean space connectivity analysis within the same set of semantically labeled points to extract all connected regions; The connected region with the most points is taken as the main connected component, and isolated pseudo-connected components are filtered out according to the ratio of the number of points of the remaining connected regions to the main connected component. The isolated regions after filtering are semantically redistributed based on the distribution of neighborhood categories.