Method for removing surface air bubbles from a three-dimensional digital model of a tooth based on an artificial neural network

Abstract

An aspect of the present application provides a computer-implemented method for removing surface bubbles from a three-dimensional digital model of a tooth based on an artificial neural network, comprising: obtaining a three-dimensional digital model of a first tooth; performing feature extraction on the three-dimensional digital model of the first tooth; identifying surface bubbles of the three-dimensional digital model of the first tooth based on the extracted features using a trained artificial neural network; and removing the identified surface bubbles.

Description

Technical Field

[0001] This application generally relates to a computer-executed method for removing surface bubbles from a three-dimensional digital model of teeth based on an artificial neural network. Background Technology

[0002] With the rapid development of computer technology, dental treatment increasingly relies on computer technology. For example, computers are used to develop orthodontic treatment plans and design dentures. When using computer technology in dental treatment, three-dimensional digital models of teeth are frequently required. These models are typically obtained by scanning existing dental models or through intraoral scanning. Due to scanning errors, air bubbles are often present in the resulting three-dimensional digital models. The presence of these air bubbles hinders subsequent use of the model; therefore, they usually need to be removed beforehand. Currently, the removal of air bubbles from the three-dimensional digital models is primarily done manually, but this method is inefficient and labor-intensive.

[0003] In view of the above, it is necessary to provide a new method for bubble removal in three-dimensional digital models of teeth. Summary of the Invention

[0004] One aspect of this application provides a computer-executed method for removing surface bubbles from a three-dimensional digital model of a tooth based on an artificial neural network, comprising: acquiring a three-dimensional digital model of a first tooth; extracting features from the three-dimensional digital model of the first tooth; identifying surface bubbles in the three-dimensional digital model of the first tooth using a trained artificial neural network based on the extracted features; and removing the identified surface bubbles.

[0005] In some implementations, the artificial neural network may be a convolutional neural network.

[0006] In some implementations, the convolutional neural network may sequentially include the following layers: an input layer, a convolutional layer, a pooling layer, a fully connected layer, a dropout layer, a fully connected layer, and an output layer.

[0007] In some implementations, the features may include: Curvature Feature, PCA Feature, Shape Diameter Feature, Distance from Medial Surface, Average Geodesic Distance, Shape Contexts, and Spin Images.

[0008] In some implementations, the artificial neural network can be trained using a three-dimensional model of teeth with resampled surface bubbles.

[0009] In some implementations, the identified surface bubbles can be removed based on the Laplace smoothing method.

[0010] In some implementations, the artificial neural network is a surface bubble of a three-dimensional digital model of a tooth that is used only to identify the tooth at the position of the first tooth, that is, each tooth position corresponds to a dedicated artificial neural network. Attached Figure Description

[0011] The above and other features of this application will be further described below with reference to the accompanying drawings and their detailed description. It should be understood that these drawings only illustrate several exemplary embodiments according to this application and should not be considered as limiting the scope of protection of this application. Unless otherwise specified, the drawings are not necessarily to scale, and similar reference numerals denote similar parts.

[0012] Figure 1 A schematic flowchart illustrating a computer-executed method for removing surface bubbles from a 3D digital model of teeth based on an artificial neural network, as described in one embodiment of this application; and

[0013] Figure 2 The structure of a CNN network in one embodiment of this application is illustrated schematically. Detailed Implementation

[0014] The following detailed description incorporates the accompanying drawings, which form part of this specification. The illustrative embodiments mentioned in the specification and drawings are for illustrative purposes only and are not intended to limit the scope of this application. Those skilled in the art will understand, based on the teachings of this application, that many other embodiments can be employed, and various changes can be made to the described embodiments without departing from the spirit and scope of this application. It should be understood that the various aspects of this application illustrated herein can be arranged, substituted, combined, separated, and designed in many different configurations, all of which are within the scope of this application.

[0015] One aspect of this application provides a computer-executed method for removing air bubbles from the surface of a three-dimensional digital model of teeth based on an artificial neural network.

[0016] Based on the teachings of this application, it is understood that in some computer-aided dental treatment applications, only a three-dimensional digital model of a single tooth is required; while in other computer-aided dental treatment applications, three-dimensional digital models of multiple teeth, such as a three-dimensional digital model of the entire dentition, are required. The following describes in detail, using a single tooth as an example, a computer-executed method based on an artificial neural network for removing air bubbles from the surface of a three-dimensional digital model of a tooth, according to one embodiment of this application.

[0017] Please refer to Figure 1 This is a schematic flowchart of a computer-executed method 100 for removing air bubbles from the surface of a three-dimensional digital model of teeth based on an artificial neural network, according to one embodiment of this application.

[0018] In step 101, obtain a three-dimensional digital model of the first tooth.

[0019] In one embodiment, a three-dimensional digital model can be constructed based on a triangular mesh, and the specific embodiments described below are all based on this three-dimensional digital model. It is understood that, in addition to triangular meshes, three-dimensional digital models can also be constructed based on other types of meshes, such as quadrilateral meshes, pentagonal meshes, hexagonal meshes, etc., which will not be described in detail here.

[0020] In step 103, feature extraction was performed on the three-dimensional digital model of the first tooth.

[0021] Feature extraction was performed on the three-dimensional digital model of the first tooth, which was then used as input to a trained artificial neural network.

[0022] Through extensive research and verification, the inventors of this application have found that the following feature combination is effective in identifying air bubbles on the surface of a 3D digital tooth model: Curvature Feature, PCA Feature (Principal Components Analysis), Shape Diameter Feature, Distance from Medial Surface, Average Geodesic Distance, Shape Contexts, and Spin Images. For details on the extraction of these features, please refer to "Learning 3D Mesh Segmentation and Labeling" published by Evangelos Kalogerakis, Aaron Hertzmann, and Karan Singh in ACM Transactions on Graphics, 29(3), 2010. A brief explanation of the extraction of these features follows.

[0023] 1) Curvature Feature

[0024] In one embodiment, the Curvature Feature can be extracted using the following method.

[0025] First, select N vertices that are adjacent to point P. In one embodiment, connectivity information can be used to reduce the search space and improve computational efficiency.

[0026] Next, based on the quadratic matrix block that satisfies the following conditions and all its neighboring vertices, a quadratic surface F(x,y,z) is fitted.

[0027] F(x,y,z)=ax 2 +by 2 +cz 2 +2exy+2fyz+2gzx+2lx+2my+2nz+ d=0 Equation (1)

[0028] Calculate the mapping P0 of point P on the matrix block, satisfying the following condition:

[0029] F(P0)=0 Equation (2)

[0030] The curvature characteristics at point P0, calculated based on the matrix block, can be used as the curvature of point P.

[0031] If for all neighboring vertices

[0032]

[0033] Where a~n are the coefficients of the quasi-quadratic surface function F(x,y,z),

[0034] Then, matrix B can be... -1 The two eigenvalues ​​k1 and k2 of A are taken as the principal curvatures, where,

[0035]

[0036]

[0037] Among them, the coefficients E, F, and G are the first-order partial derivatives of F(x, y, z), and the coefficients L, M, and N are the second-order partial derivatives of F(x, y, z).

[0038] 2) PCA Feature

[0039] In one embodiment, covariance matrices of local patch centers with different ranges can be constructed, and three singular values ​​s1, s2, and s3 can be calculated using area weights. The range can be determined by various geodesic distance radii, such as 5%, 10%, 20%, etc. Each group can have the following characteristic descriptions: s1 / (s1+s2+s3), s2 / (s1+s2+s3), s3 / (s1+s2+s3), (s1+s2) / (s1+s2+s3), (s1+s3) / (s1+s2+s3), (s2+s3) / (s1+s2+s3), s1 / s2, s1 / s3, s2 / s3, s1 / s2+s1 / s3, s1 / s2+s2 / s3, and s1 / s3+s2 / s3.

[0040] 3) Shape Diameter Feature

[0041] In one embodiment, the Shape Diameter Feature (SDF) can be defined as follows: A certain number of rays are emitted from the center point of the face within a certain angle opposite to the normal vector of that point. These rays intersect with another face to form line segments. Using the line segment with the median length as the standard, the weighted average, median, and mean square of all line segments with lengths within a specified standard deviation are calculated as the SDF value of that point.

[0042] In one embodiment, multiple logarithmic versions of the calculated SDF (corresponding to normalization terms α = 1, 2, 4, 8) can be added, as shown in the following equation (6).

[0043]

[0044] 4)Distance from Medial Surface

[0045] In one embodiment, the average coordinates of the three vertices of the facet can be used as the coordinates of the facet's center point. Then, within the shape defined by the 3D digital model of the tooth, the largest inscribed circle with the facet's center point as the tangent point is found. Rays are evenly emitted from the center of this inscribed circle, intersecting the curved surface (i.e., the closed shape defined by the 3D digital model of the jaw), and the lengths of all line segments are calculated. The weighted average, median, and mean square of these line segment lengths can be calculated as features, and normalized and logarithmic versions can be added.

[0046] 5) Average Geodesic Distance

[0047] This feature can be used to describe the degree of dispersion between patches. In one embodiment, the average geodesic distance between the centers of all patches on the surface can be calculated. The mean square distance and its percentage range across different distances can also be used as features. Then, normalization is performed.

[0048] 6) Shape Contexts

[0049] In one embodiment, for each patch, the distribution of other patches (with area weights) can be calculated, described by the angle of the patch normal vector and the logarithmic geodesic distance. In one embodiment, six geodesic distance intervals and six angle intervals can be established.

[0050] 7) Spin Images

[0051] In one embodiment, the calculation can be performed as follows: First, a cylindrical coordinate system is established with the vertex P normal vector as the central axis. Next, the resolution (i.e., the image width and height) and size (i.e., the number of grids) of the SpinImage are defined. Then, the 3D points are projected onto the 2D Spin Image according to the following equation (7). The 2D projection result is then interpolated to distribute the value of one point to the four surrounding points.

[0052]

[0053] Where α represents the radial distance to the normal vector n, β represents the axial distance to the tangent plane, n represents the normal vector passing through point P, and X represents the coordinates (x, y, z) of the 3D point.

[0054] In one embodiment, 593-dimensional features can be extracted, as shown in Table 1.

[0055] feature CUR PCA SC AGD SDF DIS SI Total dimension Dimension 64 48 270 15 72 24 100 593

[0056] Table 1

[0057] Based on the teachings of this application, those skilled in the art can adjust the above-mentioned feature types and / or feature dimensions, for example, by adding, deleting, or substituting features. It is understood that these changes are all within the scope of this application.

[0058] In 105, surface bubbles of the three-dimensional digital model of the first tooth were identified using a trained artificial neural network based on the extracted features.

[0059] The features extracted from 103 are input into a trained artificial neural network to identify the facets belonging to the surface bubbles.

[0060] In one embodiment, convolutional neural networks (CNNs) can be used.

[0061] In one embodiment, a CNN network can be built using PyTorch.

[0062] Please refer to Figure 2 This diagram schematically illustrates the structure of a CNN network in one embodiment of this application. In this embodiment, the CNN network has 10 layers: 4 convolutional layers, 2 pooling layers, 2 fully connected layers, 1 dropout layer, and 1 output layer (outputting the predicted probability that the current patch belongs to a bubble patch or a non-bubble patch). In this embodiment, the input to the CNN network can be a 20*30 feature matrix. Since there are only 593 features in this embodiment, the remaining 7 bits in the feature matrix can be padded with 0s. In one embodiment, it can be done as follows... Figure 2 The settings for each layer of the CNN network are shown (including template size, number, and calling method, etc.).

[0063] In this embodiment, ReLU is used as the activation function, which can avoid the vanishing gradient phenomenon to some extent. Four convolutional layers can fully analyze the feature data, obtain local information, and derive optimal weights. Two pooling layers can retain some global information and have a certain effect in preventing overfitting. One dropout layer can be used to resist overfitting.

[0064] Because the number of non-bubble pieces is much greater than the number of bubble pieces, the following methods can be used to obtain better training results in the case of this imbalanced data sample: First, BCELoss can be used as the loss function during training; Second, bubble pieces can be resampled to increase the number of bubble piece samples.

[0065] In one embodiment, the following training parameters can be used: learning rate 0.01, class_weight 5, sample10, solver_mode:GPU.

[0066] In one embodiment, the training process of an artificial neural network may include backpropagation, gradient descent, and error calculation, while the prediction process using the trained artificial neural network may not include these three processes.

[0067] Due to the morphological differences between different teeth, in order to ensure the accuracy of the artificial neural network prediction, in one embodiment, a separate artificial neural network can be set up for each tooth; in another embodiment, since symmetrical teeth have relatively consistent morphology, a separate artificial neural network can be set up for every two symmetrical teeth; in yet another embodiment, teeth can be divided into several categories according to the similarity of their morphology, and a separate artificial neural network can be set up for each category, so that teeth with similar morphology can share a single artificial neural network.

[0068] For cases involving multiple teeth, such as debubbling the three-dimensional digital model of the entire dentition, the three-dimensional digital model of the entire dentition can be segmented, meaning that each tooth is independent of the others, so that a corresponding artificial neural network can be used to identify surface bubbles on different teeth.

[0069] In one embodiment, the three-dimensional digital model of the teeth can be segmented manually. In another embodiment, the three-dimensional digital model of the teeth can be segmented automatically using a computer, as detailed in "3D Tooth Segmentation and Labeling using Deep Convolutional Neural Networks" published by Xiaojie Xu, Chang Liu, and Youyi Zheng et al. on May 22, 2018 in IEEE Transactions on Visualization and Computer Graphics.

[0070] Based on the teachings of this application, it is understood that, in addition to convolutional neural networks, other artificial neural networks can also be used to identify surface bubbles in three-dimensional tooth models, such as fully connected neural networks and graph-based autoencoders, all of which are within the scope of this application.

[0071] In step 107, surface bubbles in the three-dimensional digital model of the first identified tooth are removed.

[0072] In one embodiment, the Laplacian smoothing algorithm can be used to smooth the identified surface bubble patches to remove surface bubbles and obtain a three-dimensional digital model of the first tooth with surface bubbles removed. Its basic principle is to move each vertex to the weighted average position of its adjacent vertices, i.e., using an umbrella-shaped operator.

[0073] Once the bubble facets are identified, a point set of all the vertices of the bubble facets can be constructed. All points in this point set are then subjected to multiple Laplacian smoothing operations until the sum of the displacement distances of all points in the point set before and after a single smoothing operation is less than a preset threshold. This threshold can be set according to specific circumstances.

[0074] For details on the Laplacian algorithm, please refer to "Laplacian Mesh Optimization" published in GRAPHITE in 2006 by Andrew Nealen, Takeo Igarashi, and Marc Alexa.

[0075] Based on the teachings of this application, it is understood that, in addition to the Laplace algorithm, any applicable method can be used to remove identified surface bubbles, such as the Mesh Fairing algorithm.

[0076] Based on the inspiration of this application, it is understood that most of the operations in the method for removing surface bubbles from a 3D digital model of teeth can be performed by a computer, such as feature extraction from the 3D digital model of teeth; identification of surface bubbles on the 3D digital model of teeth using a trained artificial neural network based on the extracted features; and removal of the identified surface bubbles on the 3D digital model of teeth. Compared with traditional manual bubble removal methods, this method significantly improves efficiency and reduces labor costs.

[0077] Although various aspects and embodiments of this application have been disclosed herein, other aspects and embodiments of this application will be apparent to those skilled in the art upon inspiration from this application. The various aspects and embodiments disclosed herein are for illustrative purposes only and not for limiting purposes. The scope and spirit of this application are determined solely by the appended claims.

[0078] Similarly, the diagrams may illustrate exemplary architectures or other configurations of the disclosed methods and systems, which aid in understanding the features and functions that may be included in the disclosed methods and systems. The claims are not limited to the exemplary architectures or configurations shown, and the desired features may be implemented with various alternative architectures and configurations. Furthermore, the order of the blocks given herein with respect to flowcharts, functional descriptions, and method claims should not be limited to various embodiments implemented in the same order to perform the said functions, unless explicitly indicated in the context.

[0079] Unless otherwise expressly stated, the terms and phrases used herein, and their variations thereof, should be interpreted as open-ended rather than restrictive. In some instances, the appearance of extended words and phrases such as “one or more,” “at least,” “but not limited to,” or other similar expressions should not be construed as an intention or necessity to indicate a narrower scope in examples where such extended expressions might not exist.

Claims

1. A computer-implemented method of removing surface bubbles from a three-dimensional digital model of a tooth based on an artificial neural network, comprising: obtaining a three-dimensional digital model of a first tooth, which is a mesh three-dimensional digital model; performing feature extraction on the three-dimensional digital model of the first tooth; based on the extracted features, identifying surface bubbles of the three-dimensional digital model of the first tooth using a trained artificial neural network; and removing the identified surface bubbles.

2. The computer-implemented method of removing surface air bubbles from a three- dimensional digital model of a tooth based on an artificial neural network of claim 1, wherein, The artificial neural network is a convolutional neural network.

3. The computer-implemented method of removing surface air bubbles from a three- dimensional digital model of a tooth based on an artificial neural network of claim 2, wherein, The convolutional neural network comprises, in sequence, an input layer, a convolutional layer, a convolutional layer, a pooling layer, a convolutional layer, a convolutional layer, a pooling layer, a fully connected layer, a dropout layer, a fully connected layer, and an output layer. 4.The computer-implemented method of removing surface air bubbles from a three-dimensional digital model of a tooth based on an artificial neural network according to claim 1, wherein, The features include curvature features, principal component analysis features, shape diameter features, distance from medial surface, average geodesic distance, shape contexts, and spin images.

5. The computer-implemented method of removing surface air bubbles from a three- dimensional digital model of a tooth based on an artificial neural network of claim 1, wherein, The artificial neural network is trained on a tooth three-dimensional model with surface bubbles resampled.

6. The computer-implemented method of removing surface air bubbles from a three- dimensional digital model of a tooth based on an artificial neural network of claim 1, wherein, The identified surface bubbles are removed based on a Laplacian smoothing method.

7. The computer-implemented method of removing surface air bubbles from a three- dimensional digital model of a tooth based on an artificial neural network of claim 1, wherein, The artificial neural network is trained to identify surface bubbles of a three-dimensional digital model of a tooth of the same type as the first tooth. The artificial neural network is trained to identify surface bubbles of a three-dimensional digital model of a tooth of the same type as the first tooth.

Images