Rapid and automatic tooth three-dimensional model segmentation method and equipment based on deep learning

Abstract

The invention provides a rapid and automatic tooth three-dimensional model segmentation method and equipment based on deep learning, and the method comprises the steps: carrying out a data collection process: collecting and preprocessing three-dimensional tooth point cloud data, and carrying out the reconstruction of a triangular mesh and the separation of an upper jaw and a lower jaw, preprocessing to obtain a grid model of an upper jaw and a grid model of a lower jaw; the rapid segmentation process comprises the step of segmenting the preprocessed model based on an edge refining cascade network, and the edge refining cascade network comprises a coarse segmentation branch and an inter-tooth refining branch which are connected in series; and in the post-processing process, a result obtained in the rapid segmentation process is optimized. According to the invention, accurate and rapid segmentation of the tooth grid model can be realized.

Description

Technical Field

[0001] The present invention belongs to the field of automatic segmentation of three-dimensional tooth models, and particularly relates to a fast automatic segmentation technical solution for three-dimensional tooth models based on deep learning. Background Art

[0002] With the rapid development of digital technology and artificial intelligence, orthodontics is gradually moving towards intelligence and high efficiency. Three-dimensional (3D) dental models have become an indispensable important tool in orthodontic diagnosis and treatment because they can accurately present the spatial information of teeth and oral structures. These models not only provide doctors with detailed tooth arrangements and occlusion relationships, but also lay a solid foundation for the formulation of personalized treatment plans, such as the design and production of clear aligners. However, the widespread application of 3D dental models has also brought new technical challenges, especially in tooth segmentation and anatomical landmark localization.

[0003] Currently, tooth segmentation mainly relies on manual operations and semi-automated software tools. Although manual segmentation can relatively accurately separate each tooth, the process is cumbersome and time-consuming, and requires professional operation skills, which limits its efficiency in clinical practice. In addition, although existing commercial software integrates a certain degree of automated algorithms, a large amount of manual intervention is still required when dealing with complex cases. There are significant differences in the shape and size of different types of teeth, and even for the same type of tooth, there will be slight variations between different individuals. This diversity makes it difficult for a unified segmentation algorithm to adapt to all situations, resulting in unstable accuracy of segmentation results. In addition, the tooth arrangements in patients' oral cavities are often irregular, with phenomena such as tooth crowding, overlapping, or missing. In addition, the gums, periodontal tissues, and auxiliary devices such as braces that may exist will all interfere with tooth segmentation, increasing the difficulty of segmentation. In terms of automatic segmentation methods based on deep learning, although the currently more advanced automatic segmentation methods have high accuracy in theory, they need to process a large amount of three-dimensional data in practical applications, require high computing resources, and the training process is complex. In addition, existing algorithms often have difficulty maintaining stable performance when facing individual differences and complex oral structures. Summary of the Invention

[0004] The present invention provides a fast automatic segmentation method for three-dimensional tooth models based on deep learning. This method is based on a handheld oral three-dimensional scanner to collect three-dimensional tooth models, generates a tooth segmentation data set through data annotation and preprocessing methods, then completes the preliminary segmentation of teeth based on an edge refinement cascade network model for accurate segmentation of teeth in 3D intraoral models, and finally uses a post-processing algorithm to further optimize the segmentation results.

[0005] To achieve the above object, a method for rapid automatic segmentation of a three-dimensional tooth model based on deep learning designed by the present invention is characterized by combining an efficient data preprocessing method and a result postprocessing method, and studying an edge refinement cascade network model for accurate segmentation of teeth in a 3D intraoral model.

[0006] To solve the above technical problems, the technical solution of the present invention is a method for rapid automatic segmentation of a three-dimensional tooth model based on deep learning, including the following processes: The data acquisition process, including collecting three-dimensional tooth point cloud data and performing preprocessing, where the preprocessing includes triangular mesh reconstruction and separation of the maxilla and mandible. After preprocessing, a mesh model of the maxilla and a mesh model of the mandible are obtained; The rapid segmentation process, including segmenting the preprocessed model based on an edge refinement cascade network, where the edge refinement cascade network includes a cascaded coarse segmentation branch and an interproximal refinement branch; The postprocessing process, optimizing the results obtained in the rapid segmentation process.

[0007] Moreover, in the data acquisition process, the preprocessing further includes, after triangular mesh reconstruction, decomposing the maxilla and mandible models into multiple connected components respectively, and selecting the component with the largest area as the main analysis object to obtain the main tooth models of the maxilla and the mandible; Removing un-referenced vertices and filling holes to generate a mesh model of the maxilla and a mesh model of the mandible.

[0008] Moreover, based on the mesh models of the maxilla and the mandible, standard alignment is performed. The implementation method is to extract the main axis direction of the model based on principal component analysis, including the direction of dental arch extension, the buccolingual distribution direction, and the occlusal plane vertical direction; the occlusal plane vertical direction is corrected by dynamic constraint of the bilateral centroid angle.

[0009] Moreover, the implementation method of correcting the occlusal plane vertical direction by dynamic constraint of the bilateral centroid angle is to calculate the centroids of the maxilla and mandible respectively according to the vertex and face data, and comprehensively construct a vector pointing from the full-mouth centroid to the unilateral centroid; then, for the maxilla and mandible, perform main direction matching respectively, and adjust the occlusal plane vertical direction according to the centroid vector.

[0010] Moreover, a centroid feature extraction module and a feature refinement extraction module are set in the coarse segmentation branch of the edge refinement cascade network, combined with curvature-driven downsampling.

[0011] Moreover, a centroid feature extraction module and a feature refinement extraction module are set in the interproximal refinement branch of the edge refinement cascade network, and downsampling is performed by integrating curvature and semantic labels in the coarse segmentation stage.

[0012] Moreover, when training the edge refinement cascade network, the coarse segmentation branch is trained first and then the inter-tooth refinement branch is trained. The loss functions used in training include the center offset loss and the tooth classification loss that calculate the results of the centroid feature extraction module and the inter-tooth refinement module. The center offset loss calculates the error of the predicted offset to move points to the centers of their respective tooth instances, including the center position error and the direction alignment error.

[0013] On the other hand, the present invention also provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the program, it implements the above-mentioned fast and automatic segmentation method for the three-dimensional tooth model based on deep learning.

[0014] On the other hand, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the above-mentioned fast and automatic segmentation method for the three-dimensional tooth model based on deep learning.

[0015] On the other hand, the present invention also provides a computer program product, including a computer program. When the computer program is executed by a processor, it implements the above-mentioned fast and automatic segmentation method for the three-dimensional tooth model based on deep learning.

[0016] The present invention has the following positive effects: 1) The present invention proposes a fast and automatic segmentation method for the three-dimensional tooth model based on deep learning, which combines an efficient data preprocessing method and a result postprocessing method, and realizes the accurate and fast segmentation of the tooth grid model based on an edge refinement cascade network for accurate segmentation of teeth in 3D intraoral models.

[0017] 3) The present invention introduces a dynamic constraint of the bilateral centroid angle to replace the traditional static geometric alignment and dynamically corrects the Z-axis direction.

[0018] 2) By combining coarse segmentation and inter-tooth refinement and using Point Transformer as the feature extraction backbone, the present invention significantly improves the segmentation accuracy and detail retention ability of the three-dimensional tooth model.

[0019] 3) By using curvature-driven downsampling and multi-scale voxel grid division, the present invention effectively reduces the point cloud data volume while retaining key geometric details, and improves the processing efficiency and the adaptability to the complex structure of the model.

[0020] 4) By combining two-dimensional projection correction with morphological operations and median filtering, the present invention optimizes the accuracy and integrity of the segmentation result and enhances the practicality and reliability of the model in clinical applications.

[0021] In view of the problems existing in the acquisition, segmentation, and processing of three-dimensional dental models, such as data noise, low segmentation accuracy, and low processing efficiency, the present invention provides a set of efficient and accurate automated solutions. This technology has broad application prospects in the fields of dental digital diagnosis, personalized orthodontics, virtual dental surgery planning, and oral health monitoring, promoting the intelligent and precise development of dental medicine. Brief Description of the Drawings

[0022] To make the technical solutions of the present invention clearer, the accompanying drawings included in the document will be briefly described below. It should be clear that these drawings only represent several embodiments of the present invention. For those skilled in the relevant art, without additional creative work, they can fully derive other possible drawings based on these drawings.

[0023] Figure 1 This is a rapid automatic segmentation process of a three-dimensional dental model based on deep learning according to an embodiment of the present invention.

[0024] Figure 2 This is a schematic diagram of an edge refinement cascade network structure for precise segmentation of teeth in a 3D intraoral model according to an embodiment of the present invention. Detailed Embodiments

[0025] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of them. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the protection scope of the present invention.

[0026] The present invention proposes a method for automatic segmentation of three-dimensional dental models, including, during the data acquisition process, using a handheld oral three-dimensional scanner to collect three-dimensional dental point cloud data in the patient's oral cavity, and dividing the model into the upper and lower jaws through mesh construction; then, removing duplicate patches and filling holes, and selecting the main connected components to exclude noise; then, standardizing and aligning the model, and determining the main direction using PCA. Based on the acquisition results, semantic labels can be made and the dataset can be divided. In the rapid segmentation stage based on deep learning, an edge refinement cascade network for precise segmentation of teeth in a 3D intraoral model is used as the three-dimensional dental automatic segmentation network model for rough segmentation and inter-tooth refinement, and intelligent downsampling is achieved through curvature and multi-scale voxel grids to improve segmentation accuracy. Finally, the segmentation result is optimized using two-dimensional projection correction and morphological operations, and a high-quality three-dimensional grid model is exported. This method realizes efficient and precise three-dimensional dental segmentation and has broad clinical application prospects.

[0027] See Figure 1, in the embodiments of the present invention, a fast and automatic segmentation method for three-dimensional tooth models based on an edge refinement cascade network is proposed, including the following processes: In the data acquisition process, the following steps are carried out to achieve data preprocessing and model standardization: Step 1), acquisition and preprocessing of three-dimensional tooth model data.

[0028] Data acquisition and preprocessing are the basis of the entire process, directly affecting the quality and effect of subsequent steps. Ensuring high-quality and accurate point cloud data in this stage and performing effective preprocessing are the keys.

[0029] The implementation process of Step 1 preferably adopted in the embodiments includes the following steps: Step 1.1, select a high-precision three-dimensional scanner, operate according to the device instructions in a suitable scanning environment, and ensure that the scanning path covers all teeth in the upper and lower jaws.

[0030] In specific implementation, it is preferably recommended that the accuracy of the three-dimensional scanner reaches 0.01 - 0.03 mm. A handheld oral three-dimensional scanner can be used for convenient operation, and strong environmental light should be avoided during operation.

[0031] Step 1.2, perform triangular meshing based on the point cloud data, and then divide the model into two parts: the upper jaw and the lower jaw. In the embodiments, Poisson (triangular mesh reconstruction algorithm) is preferably used to convert the point cloud into a mesh model, and then the model is divided into two parts: the upper jaw and the lower jaw and saved separately.

[0032] Step 1.3, decompose the upper jaw and lower jaw models into multiple connected components respectively, and select the component with the largest area as the main analysis object, that is, obtain the main tooth models of the upper jaw and the main tooth models of the lower jaw. The subsequent steps are processed for the upper jaw and the lower jaw respectively based on this. The purpose of this step is to focus on the main tooth structure and exclude the noisy or fragmented oral mucosa parts.

[0033] This step ensures that the subsequent processing focuses on the most important parts and avoids small stray parts from interfering with the analysis.

[0034] Step 1.4, based on the main tooth models selected in the previous step 1.3, remove the un-referenced vertices and fill the possible holes, and finally export the cleaned model as a new 3D model file, such as an STL format file.

[0035] This step processes the main tooth models of the upper jaw and the main tooth models of the lower jaw obtained in step 1.3 respectively to obtain the corresponding 3D model files of the upper jaw and the corresponding 3D model files of the lower jaw, that is, the mesh models of the upper jaw and the lower jaw obtained after the preprocessing in step 1.

[0036] Step 2), Standardization of the three-dimensional tooth model Standardization processing and dataset construction are the prerequisites for training an efficient model. Ensuring the consistency of data and the accuracy of annotation are crucial for the performance of the model.

[0037] Since the point cloud is relatively evenly distributed in the vertical direction and it is difficult to judge the positive and negative of the placement direction, in order to further improve the alignment accuracy, the present invention proposes a symmetry dynamic correction mechanism to correct the Z-axis direction (the vertical direction of the occlusal surface).

[0038] The specific implementation process of Step 2 preferably adopted in the embodiment is as follows: Step 2.1, Extract the vertex and face data from the mesh models of the upper and lower jaws obtained after the preprocessing in Step 1, and respectively extract the arch extension direction, the buccolingual distribution direction, and the vertical direction of the occlusal surface as the corresponding model main axes.

[0039] The model main axis extraction method is to use the scikit-learn library for principal component analysis PCA. By calculating the covariance matrix and solving its eigenvalues and orthogonal eigenvectors, a three-dimensional coordinate system representing the main direction of the model geometric distribution is determined. The first three principal component axes respectively correspond to the arch extension direction, the buccolingual distribution direction, and the vertical direction of the occlusal surface.

[0040] Step 2.2, Correct the vertical direction of the occlusal surface of the upper and lower jaws respectively: First, determine the direction of the real-world coordinate axes. Calculate the centroids of the upper and lower jaws based on the vertex and face data respectively, and comprehensively construct a vector from the centroid of the whole mouth to the centroid of one side

[0041]

[0042]

[0043] Among them, is the coordinate of the centroid point of the upper jaw, is the total number of points of the upper jaw, i represents the i-th upper jaw point, is the coordinate of the i-th upper jaw point, is the coordinate of the centroid point of the lower jaw, is the total number of points of the lower jaw, j represents the j-th lower jaw point, is the coordinate of the j-th lower jaw point; is the coordinate of the centroid point of the whole mouth, is the vector from the centroid of the whole mouth to the centroid of the upper jaw, is the vector from the centroid of the lower jaw to the centroid of the whole mouth.

[0044] Then, for a single-jaw model (maxilla or mandible), the following operations are performed separately: 1) Select the best-matching Z-axis:

[0045] Among them, corresponds to the main direction extracted by PCA, corresponds to the unilateral centroid vector of the single-jaw model, is the serial number of the extracted main direction, is the serial number of the best-matching Z-axis, is to take the absolute value, is to return the serial number corresponding to the maximum value.

[0046] 2) Adjust the Z-axis direction (positive or negative) according to the best-matching axis:

[0047] Among them, is the adjusted Z-axis direction, is the original Z-axis direction, is the extracted positive and negative information.

[0048] 3) Calculate the rotation matrix required to align the main axis of the model to the target coordinate axis, and finally apply the rotation matrix to all vertices of the model to complete the alignment.

[0049] In specific implementation, according to the data acquisition results obtained in advance according to the above process, labels corresponding to tooth data can be made, and the training set, validation set, and test set can be divided. In specific implementation, it is preferably recommended to use the 3D mesh processing software MeshLaber to assign tooth number labels to each vertex or face according to the tooth anatomical structure (it is preferably recommended to use the FDI number commonly used in medicine), and divide the data set into the training set, validation set, and test set according to the ratio of 8:1:1 for training, validation, and testing of the three-dimensional tooth automatic segmentation network model. The trained and validated network can be used for rapid segmentation of real-time acquired data. For the implemented acquired data, after consistent processing according to the above process and input into the network, the rapid segmentation result can be obtained.

[0050] In the rapid segmentation process based on deep learning, for the established three-dimensional tooth automatic segmentation network model, the three-dimensional tooth automatic segmentation network model training is carried out for the maxilla and mandible respectively, and real-time segmentation is realized based on the trained three-dimensional tooth automatic segmentation network model. The specific implementation method is as follows: As Figure 2 shown, in order to implement the three-dimensional tooth automatic segmentation network model, the present invention proposes an edge refinement cascade network, which includes two cascaded branches, a coarse segmentation branch and an inter-tooth refinement branch.

[0051] The working process of the edge refinement cascade network is as follows: First, perform voxel optimization downsampling on the oral three-dimensional model, and input the sampled point cloud data into the coarse segmentation branch; in the coarse segmentation branch, based on the centroid feature extraction module, obtain the offset of each original tooth point to the centroid, and based on this, extract the tooth centroid in the coarse segmentation stage. For each predicted tooth and the point cloud data in its neighborhood, input it into the feature refinement extraction module to obtain the segmentation result of the coarse segmentation branch; Combined with the original model data, perform voxel optimization edge downsampling to obtain the edge-optimized downsampled point cloud, and input it into the inter-tooth refinement branch; in the inter-tooth refinement branch, based on the centroid feature extraction module, obtain the offset of each original tooth point to the centroid, and based on this, extract the tooth centroid in the inter-tooth refinement stage, and combine the prediction result of the coarse segmentation stage (take the union of the tooth centroid predicted in the inter-tooth refinement stage and the tooth centroid predicted in the coarse segmentation stage), extract the point cloud data of each predicted tooth and its neighborhood, input it into the feature refinement extraction module to obtain the segmentation result of the inter-tooth refinement branch. Finally, combine the segmentation results of the coarse segmentation branch to obtain the overall segmentation result.

[0052] The present invention further proposes to use Point Transformer as the backbone network and design a centroid feature extraction module and an inter-tooth refinement module.

[0053] The preferred implementation method adopted in the embodiment is that the implementation method of the centroid feature extraction module is PointTransformer + offset prediction head (Offset Head), and the implementation method of the feature refinement extraction module is PointTransformer + segmentation head (Segmentation Head).

[0054] Among them, the centroid feature extraction module receives the point cloud containing coordinates and normals, and predicts the semantic label and offset of the tooth. Move the points to the center point according to the offset, and filter out the gingival points. Cluster the translated point cloud through DBSCAN to obtain the tooth instance label. Since each tooth is in a compact cylindrical shape, the clustering-based labeling method has high robustness.

[0055] In the inter-tooth refinement module, the sampled point cloud is cropped near the center point of the predicted tooth instance and input into the PointTransformer backbone network to generate a tooth-gingiva mask. This mask is used to optimize the tooth instance label: if the centroid feature extraction module predicts a tooth while the inter-tooth refinement module predicts gingiva, change the point label to gingiva; if the centroid feature extraction module predicts gingiva while the inter-tooth refinement module predicts a tooth, determine the tooth instance label of this point according to the label of the nearest neighbor point.

[0056] The present invention proposes to first train the rough segmentation stage of the three-dimensional tooth automatic segmentation network model for the rough segmentation branch. The rough segmentation stage provides a basis for the subsequent fine segmentation, and it is necessary to ensure the accuracy and stability of the preliminary segmentation.

[0057] The implementation process of the rough segmentation stage preferably adopted in the embodiment is specifically as follows: First, perform curvature downsampling: 1) Estimate the normal for each vertex of the three-dimensional tooth model in the training set based on the Open3D library. For each vertex, calculate the covariance matrix of the normals within its neighborhood, and calculate and store the curvature value of each point through eigenvalue analysis of the covariance matrix.

[0058] 2) Divide the vertices in 1) into voxel grids of a specified size (preferably recommended to set the value within the range of 1-3 mm). For the points within each voxel, calculate the average value of their curvatures, sort the voxels according to the average curvature, retain more points in the high-curvature voxels to retain the details of the model, and combine the retained points to generate the downsampled point cloud for input to the model.

[0059] In specific implementation, it is preferably recommended that after sorting the voxels according to the average curvature, they are divided into high curvature (the first 30%) and low curvature (the last 70%), and 60% of the original points are retained for the high-curvature part.

[0060] Then, train the rough segmentation branch based on the loss function. The loss function includes the center offset loss and the tooth classification loss for calculating the results of the centroid feature extraction module and the inter-tooth refinement module. Among them, the center offset loss calculates the error of the predicted offset to move the points to the centers of their respective tooth instances, including the center position error and the direction alignment error. Center position loss The calculation formula is:

[0061] Where is the number of tooth instances, is the number of points in the th instance, is the i th original point coordinate, is the predicted offset, is the th instance center coordinate.

[0062] Direction alignment loss The calculation formula is:

[0063] Where, is the vector norm, , representing the iThe unit moving direction vector of the original points, , representing the unit true direction vector of the i th original point, is the number of points that satisfy . is the threshold of the offset modulus length.

[0064] After the training in the coarse segmentation stage is completed, for the inter-tooth refinement branch, train the inter-tooth refinement segmentation stage of the three-dimensional tooth automatic segmentation network model: The fine segmentation stage further improves the segmentation accuracy on the basis of the coarse segmentation, especially in the inter-tooth region. Ensure that the implementation of this stage can effectively improve the overall segmentation quality.

[0065] The implementation process of the inter-tooth refinement segmentation stage preferably adopted in the embodiment is specifically as follows: First, perform comprehensive downsampling: 1) Estimate the normal for each vertex based on the Open3D library. For each point, calculate the covariance matrix of the normals in its neighborhood. Through eigenvalue analysis of the covariance matrix, calculate the curvature value of each point and store it. This step is implemented in the same way as part 1) of the curvature downsampling in the coarse segmentation stage.

[0066] 2) Divide the point cloud into voxel grids of multiple scales. For the points in each voxel, calculate their average curvature values. Using the semantic labels in the coarse segmentation stage, analyze the label diversity in each voxel and determine whether the voxel contains multiple semantic labels. Calculate the proportion of the point cloud of the semantic label with the largest proportion in the voxel. The lower the proportion, the closer the voxel is to the edge. Divide the voxels into boundary voxels and other voxels, and retain 70% of the original points in the boundary voxels.

[0067] Then, as Figure 2 shown, input the point cloud data into the inter-tooth refinement segmentation branch, including the centroid feature extraction module and the inter-tooth refinement module. The loss function calculation is similar to that in the coarse segmentation stage. Calculate the center offset loss and tooth classification loss of the results of the centroid feature extraction module and the inter-tooth refinement module to obtain the segmentation model of the inter-tooth refinement.

[0068] Based on the edge refinement cascade network for accurate segmentation of teeth in the 3D intraoral model, combining the segmentation results of the two stages, an optimized segmentation result can be output.

[0069] To improve the segmentation result, the present invention further proposes to input the real-time collected data into the trained three-dimensional tooth automatic segmentation network model, and post-process the obtained segmentation result output by the network. The preferred implementation suggestions are as follows: The post-processing steps aim to optimize the segmentation result and improve the practicality and accuracy of the final model. Ensure the smoothness and consistency of the segmentation result, remove noise and fill defects. The implementation process is specifically as follows: 1) Select the occlusal surface for projection, project the centroid position and semantic labels predicted by the trained 3D tooth automatic segmentation network model onto the selected 2D plane, sort along the dental arch path according to the projected tooth centroids, and the labels of adjacent teeth should conform to the anatomical order. Identify obvious errors in the semantic labels based on the relative positional relationship of the teeth, and update the corrected label information back to the 3D model.

[0070] 2) Optimize the segmentation result through morphological operations (erosion, dilation, opening, closing). Erosion checks each patch. If its label dominates in its neighborhood (i.e., most of the patch labels in the neighbors are the same), the label is retained; if its label has a very low proportion in the neighborhood (a specific implementation can preset a threshold, for example, preferably set below 30%), its label is changed to the background label. Dilation is that if the label of a patch is the background and there are other labels in the surrounding neighbor regions and their occurrence times exceed the preset threshold (a specific implementation can preset a threshold, for example, preferably set to 60%), then the patch is updated to the neighbor label. Through opening (erosion first and then dilation) and closing (dilation and then erosion), remove the small-area mispredicted parts in the model.

[0071] 3) Use median filtering to further smooth the labels. For each patch, take the labels of its surrounding neighbors and its own label, and then calculate their median as the updated label. Repeat this smoothing process multiple times (a specific implementation can preset the number of times, and the recommended number of times is 3) to gradually smooth the labels.

[0072] 4) Correlate the vertex and face data of the 3D model with the optimized labels, and assign different colors to different semantic labels to enhance the visual effect.

[0073] To facilitate the description of the technical effects of the present invention, after the rapid automatic segmentation of the tooth 3D model using the method of the present invention, the main accuracy evaluation indicators include IOU and CLS, which represent the tooth segmentation accuracy and tooth recognition rate respectively. Both indicators are expressed as percentages, and the larger the value, the better the segmentation effect. Select 30 full-mouth tooth models, which are manually marked and extracted (true values) in advance by professional orthodontists, repeat the method provided in the above embodiments 5 times, and compare the predicted results obtained each time with the true values to obtain the statistical results as shown in the following table:

[0074] It can be seen that the rapid segmentation and extraction effect of the present invention has sufficiently achieved the expectation.

[0075] In specific implementation, those skilled in the art can use software technology to realize the automatic operation of the above process. Correspondingly, if a fast automatic segmentation solution for three-dimensional tooth models based on deep learning is provided, including a computer or a server, and the above process is executed on the computer or the server for fast automatic segmentation of three-dimensional tooth models based on deep learning, it should also be within the protection scope of the present invention.

[0076] In another embodiment, an electronic device is further involved, including at least one processor; and a memory communicatively connected to the at least one processor; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the above-mentioned fast automatic segmentation method for three-dimensional tooth models based on deep learning.

[0077] In another embodiment, a non-transitory computer-readable storage medium storing computer instructions is further involved, wherein the computer instructions are used to cause the computer to execute the above-mentioned fast automatic segmentation method for three-dimensional tooth models based on deep learning.

[0078] In another embodiment, a computer program product is further involved, including a computer program, characterized in that when the computer program is executed by a processor, the above-mentioned fast automatic segmentation method for three-dimensional tooth models based on deep learning is realized.

[0079] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts. Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on such an understanding, the above technical solution, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments. The above is only the specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art within the technical scope disclosed by the present invention can easily think of changes or substitutions, which should all be covered by the protection scope of the present invention. Therefore, the protection scope of the present invention should be subject to the protection scope of the claims.

Claims

1. A fast and automatic segmentation method for three-dimensional tooth models based on deep learning, characterized in that, It includes the following processes: The data acquisition process, including acquiring three-dimensional dental point cloud data and performing preprocessing, where the preprocessing includes triangular mesh reconstruction and separation of the maxilla and mandible. After preprocessing, a mesh model of the maxilla and a mesh model of the mandible are obtained; The fast segmentation process, including segmenting the preprocessed model based on an edge refinement cascade network, where the edge refinement cascade network includes a cascaded coarse segmentation branch and an interproximal refinement branch; The post-processing process, which optimizes the results obtained from the fast segmentation process.

2. The method according to claim 1, wherein: During the data acquisition process, the preprocessing further includes, after triangular mesh reconstruction, decomposing the maxilla and mandible models into multiple connected components respectively, and selecting the component with the largest area as the main analysis object, to obtain the main dental models of the maxilla and the mandible; Removing un-referenced vertices and filling holes to generate a mesh model of the maxilla and a mesh model of the mandible.

3. The method according to claim 1, characterized in that: Based on the mesh models of the maxilla and the mandible, standard alignment is performed. The implementation method is to extract the main axis directions of the models based on principal component analysis, including the dental arch extension direction, the buccolingual distribution direction, and the occlusal plane vertical direction; the occlusal plane vertical direction is corrected by the dynamic constraint of the bilateral centroid angle.

4. The method according to claim 3, characterized in that: The implementation method of correcting the occlusal plane vertical direction by the dynamic constraint of the bilateral centroid angle is to calculate the centroids of the maxilla and mandible respectively according to the vertex and face data, and comprehensively construct a vector pointing from the full-mouth centroid to the unilateral centroid; then, for the maxilla and mandible, the main direction matching is performed respectively, and the occlusal plane vertical direction is adjusted according to the centroid vector.

5. The method according to claim 3, characterized in that: A centroid feature extraction module and a feature refinement extraction module are set in the coarse segmentation branch of the edge refinement cascade network, combined with curvature-driven downsampling.

6. The method according to claim 3, wherein: A centroid feature extraction module and a feature refinement extraction module are set in the interproximal refinement branch of the edge refinement cascade network, and downsampling is performed by integrating curvature and the semantic labels in the coarse segmentation stage.

7. The method according to claim 3, wherein: When training the edge refinement cascade network, the coarse segmentation branch is trained first and then the interproximal refinement branch. The loss function used for training includes the center offset loss and the tooth classification loss for calculating the results of the centroid feature extraction module and the interproximal refinement module. The center offset loss calculates the error of predicting the offset amount to move the points to the centers of their respective tooth instances, including the center position error and the direction alignment error.

8. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, characterized in that: When the processor executes the program, it implements the method for fast automatic segmentation of a three-dimensional dental model based on deep learning as described in any one of claims 1 to 7.

9. A non-transitory computer-readable storage medium storing a computer program thereon, characterized in that: When the computer program is executed by the processor, it implements the method for fast automatic segmentation of a three-dimensional dental model based on deep learning as described in any one of claims 1 to 7.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by the processor, it implements the method for fast automatic segmentation of a three-dimensional dental model based on deep learning as described in any one of claims 1 to 7.

Images