A three-dimensional tooth registration method, system, computer device and storage medium

The construction of a three-dimensional tooth registration network through deep learning methods solves the problems of time-consuming, laborious and low robustness of tooth registration, and achieves efficient and automated tooth registration, reducing the dependence on doctors' experience.

CN114612532BActive Publication Date: 2024-12-06XIDIAN UNIV
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210176751.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-02-24
Publication Date
2024-12-06
Estimated Expiration
2042-02-24

AI Technical Summary

Technical Problem

The existing dental registration methods are time-consuming and labor-intensive, have low automation, rely heavily on doctor experience, and are less robust in processing complex three-dimensional dental data. Deep learning is less common in three-dimensional dental registration.

Method used

Deep learning method is adopted to build a point cloud feature extraction network module with strict rotation invariance, a parameter estimation network module and a differentiable Gaussian model parameter calculation network module. Through training, a three-dimensional tooth registration network is obtained to realize automatic registration of tooth point cloud.

Benefits of technology

It greatly reduces the time-consuming process of tooth registration, improves the degree of automation, reduces the dependence on doctors' professional knowledge, improves the robustness and response speed of registration, and achieves efficient three-dimensional tooth registration.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN114612532B_ABST
    Figure CN114612532B_ABST
Patent Text Reader

Abstract

The present invention belongs to the field of pattern recognition and machine learning technology, and discloses a three-dimensional tooth registration method, system, computer equipment and storage medium, including: collecting original tooth three-dimensional oral scan image data, and preprocessing to construct a three-dimensional tooth data set; constructing a tooth point cloud feature extraction network module with strict rotation invariance; constructing a parameter estimation network module to establish a mapping relationship between the tooth point cloud and its Gaussian model parameters; constructing a differentiable Gaussian model parameter calculation network module; using a tooth training set to train the entire network to obtain a trained three-dimensional tooth registration network, thereby performing a three-dimensional tooth registration operation. The present invention designs a three-dimensional tooth registration neural network based on technologies such as Gaussian mixture models and deep learning, which can solve the problems of low automation of tooth registration tasks in the current field of dental plastic surgery, reliance on traditional manually designed operators, low registration performance and slow speed.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the technical field of pattern recognition and machine learning, and in particular relates to a three-dimensional tooth registration method, system, computer equipment and storage medium. Background Art

[0002] At present, orthodontics and dental beauty have received little attention due to their high prices, but with the increase in residents' income and the popularization of oral health knowledge, people's acceptance of dental beauty is increasing year by year. Especially in the still developing Chinese market, the dental correction and beauty industry has maintained a rapid growth trend in recent years. Orthodontics refers to the treatment of problems such as irregular tooth arrangement, abnormal tooth shape, abnormal tooth gloss, caries, malocclusion, etc. through orthodontics or oral surgery. According to a report released by the World Health Organization, malocclusion or crooked teeth is the third most popular dental disease in the world, second only to tooth decay and periodontal disease. According to data released by an authoritative survey agency, the overall scale of China's large oral industry reached 195.2 billion yuan in 2020, and it is in a rapid growth stage.

[0003] Caries, dental trauma, periodontal disease, etc. can cause natural tooth defects or loss, affecting the chewing, pronunciation and aesthetics of natural teeth. In order to restore the physiological functions of natural teeth, restorations are generally used to restore the morphology of defective or missing tooth tissue. Clinical application results show that when artificial crowns are tried on clinically, the occlusal surface is usually about 300μm higher than the actual required occlusal surface, and the convexity of the axial surface of the artificial crown of the anterior teeth is often too prominent compared to the adjacent teeth. Even after adjustment and adhesive wear, patients often experience discomfort symptoms such as abutment tooth pain and changes in chewing patterns while adapting to the new artificial crown shape. Therefore, how to better restore the morphology of defective or missing natural teeth through artificial crowns and make them close to the effect of natural teeth is a long-term concern of experts and scholars in oral rehabilitation.

[0004] In computer-assisted orthodontic treatment, in order to develop a treatment plan for the patient (for example, for the manufacture of transparent aligners), the doctor needs to design a suitable restoration. After the actual production is completed, in order to compare the difference between the restoration and the design model, a registration operation will be performed. Although this step is assisted by some commercial software, the methods used by these commercial software are all traditional algorithms with low performance and low degree of automation. Manual or semi-automatic tasks need to be performed during registration, which is very time-consuming, laborious and affects the experience. Compared with traditional algorithms, deep learning can more effectively represent the original signal and has outstanding advantages in autonomous learning and feature abstract representation, making classification, recognition and prediction easier and more accurate. In recent years, it has achieved success in many pattern recognition fields. Therefore, studying how to effectively apply deep learning methods to the problem of three-dimensional tooth registration not only provides a new method and perspective for the study of three-dimensional tooth registration, but also expands the application field of deep learning.

[0005] However, the existing deep learning-based registration methods are generally used in some simple and rigid object registration problems. There are few studies on automatic registration of more complex three-dimensional dental scan images.

[0006] Through the above analysis, the problems and defects of the prior art are as follows:

[0007] (1) The current tooth alignment process is time-consuming, laborious, has a low degree of automation, and is heavily dependent on the doctor's experience;

[0008] (2) Most existing automatic or semi-automatic tooth registration methods use algorithms based on traditional manual operators, while registration methods based on deep learning are relatively rare;

[0009] (3) The performance and robustness of existing traditional registration methods and deep registration methods need to be improved.

[0010] The difficulty in solving the above problems and defects is: on the one hand, the current general target registration methods are mostly proposed for two-dimensional target registration or three-dimensional target registration with relatively simple contours, while three-dimensional teeth are three-dimensional targets with complex contours. Therefore, the existing general target registration methods have the difficulty of low robustness when processing more complex three-dimensional tooth data; on the other hand, the existing tooth registration methods mostly rely on the idea of ​​manually designed operators, and how to apply deep learning to the three-dimensional tooth registration problem still needs to be discussed and tried. In summary, how to design an efficient deep neural network for the specific target registration problem of three-dimensional tooth point cloud registration has become a huge challenge that needs to be addressed at present.

[0011] The significance of solving the above problems and defects is: in computer-assisted orthodontic treatment, in order to formulate a treatment plan for the patient (for example, for the manufacture of transparent aligners), the doctor needs to design a suitable restoration. After the actual production is completed, in order to compare the difference between the restoration and the design model, a registration operation will be performed. Although this step is assisted by semi-automatic commercial software, it is time-consuming, laborious and heavily dependent on experience to perform this task. Therefore, it is very important to design an efficient, automated and intelligent three-dimensional tooth registration network. The present invention proposes a three-dimensional tooth registration method, which can effectively deal with the three-dimensional tooth registration problem, solve the defects of the current commercial registration software, reduce the workload of orthodontists, and promote the automation and intelligence of the treatment process of orthodontists. Summary of the invention

[0012] In view of the problems existing in the prior art, the present invention provides a three-dimensional tooth registration method, system, computer equipment and storage medium.

[0013] The present invention is implemented as follows: a three-dimensional tooth registration method, the three-dimensional tooth registration method collects original tooth three-dimensional oral scan image data, and performs preprocessing to construct a three-dimensional tooth data set; constructs a tooth point cloud feature extraction network module with strict rotation invariance; constructs a parameter estimation network module to establish a mapping relationship between the tooth point cloud and its Gaussian model parameters; constructs a differentiable Gaussian model parameter calculation network module; uses a tooth training set to train the entire network to obtain a trained three-dimensional tooth registration network, thereby performing a three-dimensional tooth registration operation.

[0014] Furthermore, the three-dimensional tooth registration method comprises the following steps:

[0015] Step 1: Collect the original 3D dental scan image data and perform preprocessing to build a 3D dental dataset. This step is based on a large number of real and reliable medical datasets to expand the data, which not only protects the privacy of patients, but also ensures that there is enough authentic data to verify the network;

[0016] Step 2: Construct a point cloud feature extraction module F with strict rotation invariance (RRI). The module constructed in this step makes the network rotation invariant, and at the same time enhances and transforms the features of each point, ultimately enhancing the robustness of the point cloud features;

[0017] Step 3: Use the Gaussian mixture model (GMM) in probability theory to construct the parameter estimation network module E ψ This step estimates the mapping relationship between the three-dimensional space points of the tooth point cloud and the Gaussian model parameters;

[0018] Step 4: Build a differentiable parameter calculation network module R Θ and RT This step calculates the parameters and rotation and translation matrices of the Gaussian model;

[0019] Step 5: Use the paired training set and test set of tooth point cloud data to train and test the three-dimensional tooth registration network. After this step, a trained three-dimensional tooth registration model can be obtained.

[0020] Furthermore, the three-dimensional tooth registration method further comprises: inputting the three-dimensional space points of the tooth point cloud sequentially through E ψ , R Θ and R T Module, obtain the optimal transformation matrix T, and realize three-dimensional tooth alignment based on deep learning.

[0021] Furthermore, the acquisition of the original three-dimensional dental oral scan image data in step 1 and preprocessing to construct a three-dimensional dental data set specifically includes: obtaining the original three-dimensional spatial points of the dental point cloud through a medical oral scanning device, performing necessary dental point cloud background removal processing, and then amplifying the dental point cloud data row data using translation, rotation, and adding disturbance operations to expand the scale of the three-dimensional dental registration data set. Finally, it is divided into a training set and a test set to construct a three-dimensional dental point cloud data set.

[0022] Furthermore, the strictly rotation-invariant point cloud feature extraction module F constructed in the step 2 converts the N×3 point cloud coordinates into an N×C point cloud feature vector using an operator with rotation invariance, a modulo operation ||·||, an angle operation between two points, a SIFT transformation, and a k-ary operator.

[0023] Further, the parameter estimation network module E constructed based on the Gaussian mixture model in step 3 ψ , which is used to estimate the mapping relationship H between the three-dimensional space points of the tooth point cloud and the Gaussian model parameters, including:

[0024] Input: Point cloud features Where N is the number of points in the point cloud, and C is the feature dimension of the point cloud;

[0025] Output: N×M matrix H = [h nm ],and n=1,2,…,N;each element h of matrix H nm Indicates point P n and the implicit association probability between the components m of the Gaussian mixture model (GMM);

[0026] Parameter estimation network: MLP (3, 64, 128, 256, 1024) → Concat layer → MLP (2048, 512, 256, 128, M = 16) → Softmax; MLP stands for Multi-Layer Perceptron, Concat stands for feature concatenation; the last fully connected layer does not contain BN layer and ReLU activation.

[0027] Furthermore, the differentiable parameter calculation module R constructed in step 4 Θ and R T , used to calculate the parameters Θ and transformation T of GMM;

[0028] In step 5, the training set corresponding to the dental scan data is used to train the 3D dental registration network to obtain a trained high-performance 3D dental registration model, which specifically includes: inputting the 3D spatial points of the dental point cloud into the registration network for training to obtain a trained 3D dental registration model based on deep learning; sequentially passing the 3D spatial point pairs of the dental point cloud to be registered through E ψ , R Θ and R T Module, obtain the optimal transformation matrix T, thereby realizing three-dimensional tooth alignment based on deep learning.

[0029] Another object of the present invention is to provide a three-dimensional tooth registration system for implementing the three-dimensional tooth registration method, the three-dimensional tooth registration system comprising:

[0030] A three-dimensional tooth data set construction module is used to collect original three-dimensional tooth oral scan image data and perform preprocessing to construct a three-dimensional tooth data set;

[0031] Point cloud feature extraction module, used to extract and construct tooth point cloud features with strict rotation invariance;

[0032] The mapping relationship estimation module uses the Gaussian mixture model in probability theory to construct a parameter estimation network module to estimate the mapping relationship between the three-dimensional space points of the tooth point cloud and the Gaussian model parameters;

[0033] Differentiable parameter calculation module, used to calculate the parameters of the Gaussian mixture model and the corresponding transformation matrix;

[0034] The three-dimensional tooth registration model training module is used to train the three-dimensional tooth registration network using the tooth point cloud three-dimensional space point training set to obtain a trained three-dimensional tooth registration model.

[0035] Another object of the present invention is to provide a computer device, which includes a dental mouth scanner, a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the following steps: controlling the dental mouth scanner to collect three-dimensional dental image data and construct a three-dimensional dental data set; constructing a strictly rotationally invariant point cloud feature extraction module; constructing a parameter estimation network module based on a Gaussian mixture model to estimate the mapping relationship between three-dimensional spatial points of a dental point cloud and Gaussian model parameters; constructing a differentiable parameter calculation module; and using a dental point cloud training set to train the three-dimensional dental registration network to obtain a trained three-dimensional dental registration model.

[0036] Another object of the present invention is to provide a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor executes the following steps: constructing a three-dimensional tooth point cloud data set; constructing a strictly rotationally invariant point cloud feature extraction module; using a Gaussian mixture model to build a parameter estimation network module to estimate the mapping relationship between the three-dimensional spatial points of the tooth point cloud and the Gaussian model parameters; constructing a differentiable parameter calculation module; and using a tooth training set to train the three-dimensional tooth registration network to obtain a trained three-dimensional tooth registration model.

[0037] Combining all the above technical solutions, the advantages and positive effects of the present invention are as follows:

[0038] First, in view of the technical problems and difficulties existing in the above-mentioned existing tooth alignment methods, the present invention proposes a complete set of three-dimensional tooth alignment technology based on deep learning, which can solve the many shortcomings of the current tooth alignment technology, such as cumbersome steps, slow response, low intelligence, tedious operation and heavy reliance on professional knowledge. The specific description is as follows:

[0039] (1) Traditional dental registration software such as Geomagic Control and Geomagic Wrap are not special registration software for dental point clouds. The following steps need to be performed during the registration task: importing the point cloud model, preprocessing the model format, selecting the registration mode, and selecting the alignment points. In the process of selecting the alignment points, in order to ensure the registration effect, dozens of point pairs at the same position need to be selected for multiple dental point cloud models. When selecting point pairs, the model needs to be translated and rotated frequently. Therefore, the steps to complete a registration task are extremely cumbersome and time-consuming. The present invention addresses the problem of cumbersome steps. After the network model is trained, the user only needs to upload the model to be registered, and the registration task can be completed with one click without any other additional complex operation steps.

[0040] (2) The principles of traditional registration software are based on traditional machine learning algorithms. When processing large-scale point cloud model registration tasks, the response speed is slower than that of existing deep learning methods. At the same time, traditional methods have the problem that the response speed and registration accuracy cannot be achieved at the same time. The neural network structure designed by the present invention is simple. After the model training is completed, the time required to complete a registration task is about 30 to 40 ms. While ensuring the registration performance, the response speed is greatly improved.

[0041] (3) Traditional registration methods rely on the professional knowledge of doctors during registration. Only experienced doctors can select appropriate corresponding point pairs of tooth point cloud models. The process of selecting corresponding points requires doctors to have a deep understanding of tooth structure, but experienced doctors cannot devote all their energy to registration tasks. The number of experienced doctors cannot meet the needs of a large number of tooth data registration tasks. To address this problem, the present invention realizes the intelligentization of registration tasks based on neural networks. The network model trained on real and reliable tooth data sets can replace doctors to efficiently complete registration tasks, greatly reducing the dependence on doctors' professional knowledge.

[0042] Second, the technical solution to be protected by the present invention introduces a series of core technologies and theories, such as deep neural network, Gaussian mixture model, differentiable parameter calculation module, etc., so that the corresponding method of the present invention has technical advantages such as high performance, noise resistance, and high robustness, which are specifically described as follows:

[0043] The three-dimensional tooth alignment method provided by the present invention constructs a strictly rotationally invariant point cloud feature extraction module, which enables the network to have a rotationally invariant property and at the same time enhances and transforms the features of each point; the present invention constructs a parameter estimation network module based on theories and tools such as neural networks and Gaussian mixture models, and constructs a differentiable parameter calculation module, which has higher robustness than traditional manual operators; the present invention introduces a probabilistic algorithm, so the method can tolerate noise and input point clouds of different sizes, and can also restore the correct transformation in the absence of an accurate point-pair correspondence, and is more suitable for actual situations; the embodiments of the present invention provide new theoretical and methodological support for how to use deep learning for three-dimensional tooth alignment, making the three-dimensional tooth alignment technology based on deep learning more practical, reliable and popular.

[0044] Third, as auxiliary evidence of the inventiveness of the claims of the present invention, it is also reflected in the following important aspects:

[0045] (1) The expected benefits and commercial value of the technical solution of the present invention after transformation are as follows: The technical solution of the present invention can eventually be implemented as a corresponding software and hardware platform for sale, with broad sales prospects. It can be deployed in a large number of hospitals and other medical institutions, and the demand is huge. The product value-added is obvious, and the expected benefits are more than 50,000 yuan per unit.

[0046] (2) The technical solution of the present invention fills the gap in the field of high-end intelligent orthodontic auxiliary instruments in my country, greatly promotes the competitiveness of my country's high-end intelligent orthodontic auxiliary instruments in this field, and contributes to the promotion of the concept of smart dentists in the new era in my country.

[0047] (3) The present invention combines probabilistic algorithms with the theory of deep learning to realize a complete set of three-dimensional tooth registration technologies, solving the long-standing technical pain points and difficulties of three-dimensional tooth registration, efficiently completing the three-dimensional tooth registration task, and solving the difficult problem of automated and intelligent tooth correction auxiliary technology that dentistry and oral science have always been eager to solve. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of a three-dimensional tooth alignment method provided by an embodiment of the present invention.

[0049] Figure 2 It is an overall architecture diagram of the three-dimensional tooth alignment method provided by an embodiment of the present invention.

[0050] Figure 3 The parameter estimation network module E provided in the embodiment of the present invention ψ Schematic diagram of the structure.

[0051] Figure 4 The parameter estimation network module E provided in the embodiment of the present invention ψ Schematic diagram of the structure of MLP.

[0052] Figure 5 This is a visualization result diagram of three-dimensional tooth alignment provided by an embodiment of the present invention.

[0053] Figure 6 It is a system composition block diagram of a three-dimensional tooth alignment system provided by an embodiment of the present invention.

[0054] Figure 7 It is a schematic diagram of a three-dimensional tooth registration computer device provided by an embodiment of the present invention.

[0055] Figure 8 It is a schematic diagram of a readable storage medium for three-dimensional tooth registration provided by an embodiment of the present invention.

[0056] In the figure: 1. 3D tooth data set construction module; 2. Point cloud feature extraction module; 3. Mapping relationship estimation module; 4. Differentiable parameter calculation module; 5. 3D tooth registration model training module; 6. Teeth; 7. Dental mouth scanner; 8. Display; 9. Host; 10. Read-only memory; 11. Router; 12. Power supply; 13. External hard disk. DETAILED DESCRIPTION

[0057] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0058] In view of the problems existing in the prior art, the present invention provides a three-dimensional tooth registration method, system, computer device and storage medium. The present invention is described in detail below with reference to the accompanying drawings.

[0059] like Figure 1 As shown, the three-dimensional tooth alignment method provided by the present invention comprises the following steps:

[0060] S101, collecting original three-dimensional dental oral scan image data, and performing preprocessing to construct a three-dimensional dental data set;

[0061] S102, constructing a tooth point cloud feature extraction network module with strict rotation invariance;

[0062] S103, using a Gaussian mixture model in probability theory to construct a parameter estimation network module to estimate the mapping relationship between the three-dimensional space points of the tooth point cloud and the Gaussian model parameters;

[0063] S104, constructing a differentiable Gaussian model parameter calculation network module;

[0064] S105, training the three-dimensional tooth registration network using the tooth point cloud three-dimensional space point training set to obtain a trained three-dimensional tooth registration model.

[0065] The three-dimensional tooth alignment method provided by the present invention can also be implemented by other steps by ordinary technicians in the industry. Figure 1 The three-dimensional tooth alignment method provided by the present invention is only a specific embodiment.

[0066] The technical solution of the present invention is further described below in conjunction with specific embodiments.

[0067] Embodiment 1:

[0068] Most of the current commercial 3D tooth registration technologies use traditional methods. Therefore, this technology is mostly limited to good backgrounds and user cooperation conditions. It usually performs poorly in complex backgrounds and environments with large time spans of tooth models, and this method has low security. With the development of deep learning, since deep neural networks have outstanding advantages in autonomous learning and feature abstract representation, studying how to effectively apply deep learning algorithms to 3D teeth not only provides a new method and perspective for 3D tooth registration research, but also expands the application field of deep learning. Therefore, the present invention uses deep neural network technology, introduces the probabilistic Gaussian mixture model theory, and designs a 3D tooth registration method.

[0069] The embodiment of the present invention provides a three-dimensional tooth registration method. Figure 2 : is the overall architecture diagram of the three-dimensional tooth alignment method provided by the embodiment of the present invention. The method includes the following five steps:

[0070] Step 1: Obtain the original tooth point cloud data through a medical oral scanning device, perform background removal processing, divide it into a training set and a test set, and construct a three-dimensional tooth registration training set and a test set; amplify the tooth point cloud data through operations such as translation, rotation, and adding disturbances to expand the scale of the three-dimensional tooth registration data set.

[0071] Specifically, the embodiment of the present invention uses an advanced optical oral scanning instrument to collect three-dimensional points of tooth point cloud of an individual, and uses a public segmentation algorithm (such as a tooth and background two-segmentation method based on the PointNet model) to remove irrelevant background, thereby obtaining a data set composed of three-dimensional points of tooth point cloud. Then, the tooth point cloud data set is amplified by standardization processing such as translation, rotation, and adding disturbance, thereby obtaining the final tooth point cloud data set.

[0072] Step 2, construct a point cloud feature extraction module that is strictly rotation invariant (RRI). Specifically, in order to achieve reliable guarantee of three-dimensional rotation invariance and retain all necessary information of the point cloud except direction information, an embodiment of the present invention constructs a point cloud feature extraction module that is strictly rotation invariant.

[0073] Specifically, in the process of constructing a strictly rotationally invariant point cloud feature extraction module in an embodiment of the present invention, for any point set S, a K nearest neighbor graph G = (S, Δ) is established, where Δ = {(x, y) ∈ S × S and y is one of the K nearest neighbors of x}.

[0074] Every point P in the point set i It can be expressed as it and the adjacent point P ik Relationship:

[0075] [r i ,(r i1 ,θ i1 ,φ i1 ),(r i2 ,θ i2 ,φ i2 ),...,(r ik ,θ ik ,φ ik )] (1)

[0076] Among them, r i is the vector norm, that is, from the three-dimensional real number set To the one-dimensional real number set The unary RRI operator of θ ik is the relative angle between point i and point k; φ ik is the relative angle between their corresponding orthogonal projections.

[0077] Furthermore, the variables in formula (1) are defined as follows:

[0078]

[0079] Among them, Γ P is from Orthogonal projection operator to plane L, point P is orthogonal to plane L. ψ j yes and The relative angle of is j The minimum value corresponding to the K nearest neighbors of φ. ik yes and The relative angle in the counterclockwise direction. arctan2(y,x) is the inverse tangent value of the output y / x.

[0080] Finally, every point P in the point set i It can be represented as a vector of four tuples:

[0081] ((r i ,r i1 ,θ i1 ,φ i1 ),(r i ,r i2 ,θ i2 ,φ i2 ),...,(r i ,r iK ,θ iK ,φ iK )) (3)

[0082] For each point in any neighborhood, these rotationally invariant operators are used to describe the relationship between the point and the center point, thereby obtaining a local representation of each point. i Assign to each neighboring point, and then use a four-tuple to represent the relationship between each neighboring point and the center point, and then regard the four-tuple vector as a new point cloud, each point cloud describes a neighborhood.

[0083] Step 3: Use deep neural network and Gaussian mixture model to construct parameter estimation network E ψ The module is used to estimate the mapping relationship H between the three-dimensional space points of the tooth point cloud and the Gaussian model parameters.

[0084] Specifically, the embodiment of the present invention proposes a feature extraction module based on a probabilistic algorithm, which uses a deep neural network to extract the input point cloud features as a Gaussian mixture model representation. Specifically, the constructed parameter estimation network E ψ The module includes MLP layer, pooling layer, Concat layer and fully connected layer.

[0085] Input: Point cloud features N is the number of points in the point cloud, and C is the feature dimension of the point cloud;

[0086] Output: N×M matrix H = [h nm ],and n=1,2,…,N;each element h of matrix H nm Indicates point P n and the implicit association probability between the components m of the Gaussian mixture model (GMM);

[0087] Parameter estimation network module based on deep neural structure design: MLP (3, 64, 128, 256, 1024) → Concat layer → MLP (2048, 512, 256, 128, M = 16) → Softmax. MLP stands for Multi-Layer Perceptron, and Concat stands for feature concatenation. The last fully connected layer does not have a BN layer (i.e., batch normalization layer) and ReLU activation. Figure 3 It is a structural diagram of a parameter estimation network module provided by an embodiment of the present invention.

[0088] The MLP layer is composed of multiple convolutional layers. For example, in MLP (3, 64, 128, 256, 1024), the convolution kernel size is [1×1] except for the first layer which is [1×3]. Figure 4 The parameter estimation network module E provided in the embodiment of the present invention ψ Schematic diagram of the structure of MLP.

[0089] Table 1 Parameter design of each layer in the parameter estimation network module

[0090]

[0091] The parameter design of each layer in the parameter estimation network during the verification process of the embodiment of the present invention is specifically shown in Table 1, and the filling method during the convolution process is 0 filling.

[0092] Step 4: construct a differentiable parameter calculation module R in a three-dimensional tooth registration method provided by an embodiment of the present invention. Θ and R T ,include:

[0093] (1) Calculate the parameter space Θ of the Gaussian mixture model (GMM), which consists of M triplets (λ m ,μ m ,∑ m ),m=1,2,…,M. GMM is a generative model, which can be expressed as the weighted sum of M Gaussian models:

[0094]

[0095] where λ m is the weight, μ m is the 3×1 mean vector, ∑ m is a 3×3 covariance matrix, is the Gaussian distribution function, and p(·) is the probability symbol.

[0096] (2) Calculate the transformation matrix

[0097]

[0098] That is, the optimal closed-form solution T of transformation T can be solved * .

[0099] Step 5, using paired training sets and test sets consisting of three-dimensional spatial points of tooth point clouds to train and test the three-dimensional tooth registration network to obtain a trained high-performance three-dimensional tooth registration model.

[0100] Specifically, the data set composed of the three-dimensional space points of the tooth point cloud is input into the neural network for feature learning and training to obtain a trained three-dimensional tooth registration model, and the performance of the trained model is tested and verified by the three-dimensional space point test set of the tooth point cloud. The unknown paired three-dimensional space point pairs of the tooth point cloud are input into the neural network, and the neural network converts the model pairs into Gaussian mixture models and calculates the parameters of the Gaussian mixture model to obtain the final change matrix. The transformation matrix is ​​applied to the model pair to be registered to complete the registration task.

[0101] In summary, the three-dimensional tooth alignment method provided in the embodiment of the present invention can achieve accurate alignment of three-dimensional spatial points of tooth point cloud by combining the probabilistic Gaussian mixture model and deep neural network. Figure 5 3D tooth alignment results provided by an embodiment of the present invention are visualized in FIG. Figure 5 (a) shows two unregistered three-dimensional images of teeth (note: black represents the background, light and dark represent the target teeth and the teeth to be registered, respectively). Figure 5 (b) is the image after the two images are registered (the two images have overlapped after registration).

[0102] The embodiment of the present invention designs multiple groups of experiments to comprehensively and comprehensively demonstrate its actual registration performance and registration time. Specifically, in order to prove the high performance of the three-dimensional tooth registration model of the present invention, it is compared with other traditional registration methods and deep learning-based registration methods, and the following four representative performance indicators are selected and calculated: (1) Root Mean Square Error (RMSE) and recall rate (Recall), (2) Largest Common Pointset (LCP), (3) Chamfer Distance (CD) and Hausdorff Distance (HD) and other indicators, (4) registration time.

[0103] Table 2, Table 3, and Table 4 are the comparison results of the above performance indicators of different registration methods provided in this example. The specific comparison registration methods selected in the experiment include the following four: Normal Distribution Transform (NDT), Point Feature Histogram (PFH), Fast Point Feature Histogram (FPFH), and Deep Closest Point (DCP).

[0104] Table 2 Comparison results of performance indicators such as RMSE and Recall of different registration methods

[0105]

[0106] Table 3 Comparison results of LCP performance indicators of different registration methods

[0107]

[0108] Table 4 Comparison results of CD distance and Hausdorff distance and other performance indicators of different registration methods

[0109]

[0110] Table 5 Comparison results of registration time of different registration methods (unit: milliseconds)

[0111]

[0112] At present, deep learning-based registration methods are rarely used in single-modal three-dimensional tooth registration problems, and the existing traditional and deep registration methods are less robust to model registration over a long time span. Therefore, the present invention combines the Gaussian mixture model in probability theory and proposes a deep learning-based tooth point cloud three-dimensional space point registration method, which can achieve efficient automatic registration of tooth point cloud three-dimensional space points, perform global matching, and do not require initialization. At the same time, it can tolerate noise and input point clouds of different sizes, and can also restore the correct transformation in the absence of precise point-pair correspondence, which is suitable for actual situations.

[0113] The embodiment of the present invention proposes a complete set of deep learning-based 3D tooth registration methods, which can solve the defects of low robustness of traditional registration methods, poor generalization ability of general deep learning methods, poor robustness to noisy interference and complex models, and can be effectively applied to actual situations; the embodiment of the present invention provides new theoretical and methodological support for the practical application of 3D tooth registration technology, making 3D tooth registration technology more practical, reliable and popular; the embodiment of the present invention can be widely used in dental fields such as orthodontics and tooth restoration.

[0114] Embodiment 2:

[0115] Based on the above examples, see Figure 6 , Figure 6 : is a system composition block diagram of a three-dimensional tooth alignment system provided by an embodiment of the present invention. The three-dimensional tooth alignment system provided by the present invention includes:

[0116] (I), a three-dimensional tooth data set construction module 1, used for collecting original three-dimensional tooth oral scan image data and performing preprocessing to construct a three-dimensional tooth data set;

[0117] Specifically, the three-dimensional tooth data set construction module 1 uses a model scanner to collect three-dimensional spatial points of the tooth point cloud; the data set composed of the three-dimensional spatial points of the tooth point cloud is rotated, translated, and perturbed to obtain the final tooth point cloud three-dimensional spatial point data set.

[0118] (II), point cloud feature extraction module 2, used to extract and construct tooth point cloud features with strict rotation invariance;

[0119] Specifically, the point cloud feature extraction module construction module 2 of the embodiment of the present invention uses an operator with rotation invariance, a modulus operation ||·||, an angle operation between two points, a SIFT transformation, etc. to convert the N×3 point cloud coordinates into an N×C feature vector.

[0120] (III), a mapping relationship estimation module 3, which is used to construct a parameter estimation network module using a Gaussian mixture model in probability theory, and is used to estimate the mapping relationship between the three-dimensional space points of the tooth point cloud and the Gaussian model parameters;

[0121] Specifically, the mapping relationship estimation module 3 of the embodiment of the present invention includes:

[0122] Input: Point cloud features N is the number of points in the point cloud, and C is the feature dimension of the point cloud;

[0123] Output: N×M matrix H = [h nm ],and n=1,2,…,N;each element h of matrix H nm Indicates point P n and the implicit association probability between the components m of the Gaussian mixture model (GMM);

[0124] Parameter estimation network module: MLP(3,64,128,256,1024)→Concat→MLP(2048,512,256,128,J=16)→Softmax. The last fully connected layer does not contain BN layer and ReLU.

[0125] (IV), a differentiable parameter calculation module 4, used to calculate the parameters of the Gaussian mixture model and the corresponding transformation matrix;

[0126] Specifically, the differentiable parameter calculation module 4 of the embodiment of the present invention includes:

[0127] ① Calculate the parameter space Θ of GMM, which contains M triplets (λ m ,μ m ,∑ m ),m=1,2,…,M.

[0128] ②Calculate the transformation matrix T.

[0129] The specific formulas for the parameter space Θ and the calculation transformation matrix T are provided in formulas (4) and (5) in Example 1.

[0130] (V) A three-dimensional tooth registration model training module 5 is used to train the three-dimensional tooth registration network using the tooth point cloud three-dimensional space point training set to obtain a trained three-dimensional tooth registration model.

[0131] Specifically, the three-dimensional tooth registration model training module 5 of the embodiment of the present invention includes:

[0132] ① Divide the data set corresponding to the three-dimensional spatial points of the tooth point cloud into two parts: a training set and a test set, with the training set accounting for 3 / 4 and the test set accounting for 1 / 4.

[0133] ② Use the designed three-dimensional tooth registration network to train the training set to obtain a trained three-dimensional tooth registration network model.

[0134] Embodiment 3:

[0135] Based on the above examples, see Figure 7 , Figure 7 Schematic diagram of a three-dimensional tooth registration computer device provided by an embodiment of the present invention. A three-dimensional tooth registration computer device provided by an embodiment of the present invention includes a tooth 6, a tooth mouth scanner 7, a display 8, a host (including a processor CPU) 9, a read-only memory (ROM) 10, a router 11, a power supply 12 and an external hard disk 13. Among them, the host (processor) 9 and the tooth mouth scanner 7, the display 8, the memory 10, the router 11, and the external hard disk 13 communicate with each other through a communication interface;

[0136] The dental mouth scanner 7 is used to collect dental mouth scan image data;

[0137] The display 8 is used to display the dental scan image data and the three-dimensional registration result thereof;

[0138] The read-only memory 10 is used to store computer program codes;

[0139] The router 11 is used for networking the computer device and external devices;

[0140] The external hard disk 13 is used to store the registration results;

[0141] The power supply 12 is used to supply power to the computer device;

[0142] The host (processor) 9 is used to execute the computer program stored in the memory. When the computer program is executed by the processor, the following steps are implemented:

[0143] Step 1: Open the model scanner to collect three-dimensional dental mouth scan data.

[0144] Specifically, in step 1 of the embodiment of the present invention, a model scanner is used to collect tooth point cloud data; and the collected data is converted into a data set composed of three-dimensional space points of the tooth point cloud. The three-dimensional space points of the tooth point cloud are rotated, translated, and perturbed to obtain a final tooth point cloud three-dimensional space point data set.

[0145] Step 2: Construct a point cloud feature extraction module with strict rotation invariance.

[0146] Specifically, step 2 of the embodiment of the present invention utilizes an operator with rotation invariance, a modulo operation ||·||, an angle operation between two points, a SIFT transformation, etc. to transform the N×3 point cloud coordinates into an N×C tooth feature vector.

[0147] Step 3: Use deep neural network and Gaussian mixture model to build parameter estimation network E ψ The module is used to estimate the mapping relationship H between the three-dimensional space points of the tooth point cloud and the Gaussian model parameters.

[0148] Specifically, step three of the embodiment of the present invention includes:

[0149] Input: Point cloud features N is the number of points in the point cloud, and C is the feature dimension of the point cloud.

[0150] Output: N×M matrix H = [h nm ],and n=1,2,…,N;each element h of matrix H nm Indicates point P n and the implicit association probability between the components m of the Gaussian mixture model (GMM);

[0151] The parameter estimation network module is designed using deep neural network: MLP (3, 64, 128, 256, 1024) → Concat layer → MLP (2048, 512, 256, 128, M = 16) → Softmax. The last fully connected layer has no BN layer and ReLU activation.

[0152] Step 4: Construct the differentiable parameter calculation module R Θ and R T Specifically, step three of the embodiment of the present invention includes:

[0153] (1) Calculate the parameter space Θ of GMM, which contains M triplets (λ m ,μ m ,∑ m ),m=1,2,…,M.

[0154] (2) Calculate the transformation matrix T.

[0155] For specific formulas, see formulas (4) and (5) in Example 1.

[0156] Step five: construct an overall network model, use the tooth point cloud training set to train the network, and obtain a trained and efficient three-dimensional tooth registration model.

[0157] Specifically, step five of the embodiment of the present invention includes:

[0158] (1) The data set composed of three-dimensional points of the tooth point cloud is divided into two subsets: a training set and a test set. The training set accounts for 3 / 4 and the test set accounts for 1 / 4.

[0159] (2) using the training set to train the designed three-dimensional tooth registration network to obtain a trained three-dimensional tooth registration model;

[0160] Step 6: In response to the request of the peripheral application system through the router, the 3D tooth alignment result of step 5 is sent to the peripheral system to achieve other peripheral application purposes based on the 3D tooth alignment result. The peripheral application system is not limited in actual implementation, and the following are just examples: (1) A restoration quality verification system based on 3D tooth alignment, which can be used to check the made restorations to ensure the quality of the restorations; (2) A tooth change analysis system based on 3D tooth alignment, which is used by doctors to analyze the changes of teeth after orthodontic treatment.

[0161] A three-dimensional tooth registration computer device provided in an embodiment of the present invention can execute the embodiment corresponding to the above three-dimensional tooth registration method. The computer device provided in an embodiment of the present invention adopts the above three-dimensional tooth registration method, which solves the shortcomings of low automation of existing diagnosis and treatment methods, and solves the shortcomings of low robustness and poor adaptability to actual conditions of existing tooth registration methods, and can be widely used in tooth correction and oral beauty.

[0162] Embodiment 4:

[0163] Based on the above Example 3, see Figure 8 , Figure 8 Schematic diagram of a computer-readable storage medium for three-dimensional tooth registration provided by an embodiment of the present invention. A computer-readable storage medium for three-dimensional tooth registration provided by an embodiment of the present invention stores a computer program, which implements the following steps when executed by a processor:

[0164] Step 1: The program starts and necessary initialization is performed.

[0165] Step 2: Open the model scanner to collect tooth point cloud data.

[0166] Specifically, in step 1 of the embodiment of the present invention, a model scanner is used to collect tooth point cloud data, and the data set consisting of three-dimensional points of the tooth point cloud is converted. The tooth point cloud three-dimensional point data set is rotated, translated, and perturbed to be amplified to obtain the final tooth point cloud three-dimensional point data set.

[0167] Step three: construct a strictly rotation-invariant point cloud feature extraction module.

[0168] Specifically, step 2 of the embodiment of the present invention utilizes an operator with rotation invariance, a modulo operation ||·||, an angle operation between two points, a SIFT transformation, etc. to transform the N×3 point cloud coordinates into an N×C feature vector.

[0169] Step 4: Use deep neural network and Gaussian mixture model to build parameter estimation network E ψ The module is used to estimate the mapping relationship H between the three-dimensional space points of the tooth point cloud and the Gaussian model parameters.

[0170] Specifically, step three of the embodiment of the present invention includes:

[0171] Input: Point cloud features N is the number of points in the point cloud, and C is the feature dimension of the point cloud;

[0172] Output: N×M matrix H = [h nm ],and n=1,2,…,N;each element h of matrix H nm Indicates point P n and the implicit association probability between the components m of the Gaussian mixture model (GMM);

[0173] Parameter estimation network module: MLP (3, 64, 128, 256, 1024) → Concat → MLP (2048, 512, 256, 128, J = 16) → Softmax. The last fully connected layer does not contain BN layer and ReLU.

[0174] Step 5: Construct the differentiable parameter calculation module R Θ and R T .

[0175] Specifically, step five of the embodiment of the present invention includes:

[0176] (1) Calculate the parameter space Θ of GMM, which contains M triplets (λ m ,μ m ,∑ m ),m=1,2,…,M.

[0177] (2) Calculate the transformation matrix T.

[0178] For specific formulas, see formulas (4) and (5) in Example 1.

[0179] Step six, construct the overall network model, use the tooth point cloud three-dimensional space point training set to train the network, and obtain a trained efficient three-dimensional tooth registration model.

[0180] Specifically, step six of the embodiment of the present invention includes:

[0181] (1) The data set corresponding to the three-dimensional spatial points of the tooth point cloud is divided into two parts: a training set and a test set, where the training set accounts for 3 / 4 and the test set accounts for 1 / 4.

[0182] (2) using the designed three-dimensional tooth registration network to train the training set and obtain a trained three-dimensional tooth registration network model;

[0183] Step seven, exit the program when it needs to end.

[0184] A computer-readable storage medium provided in an embodiment of the present invention can execute the above-mentioned three-dimensional teeth alignment method embodiment, and stores the above-mentioned three-dimensional teeth alignment computer program thereon, which can be used by various teeth alignment computer devices, thereby improving the safety and accuracy of the three-dimensional teeth alignment equipment and bringing a good experience to users.

[0185] It should be noted that the embodiments of the present invention can be implemented by hardware, software, or a combination of software and hardware. The hardware part can be implemented using dedicated logic; the software part can be stored in a memory and executed by an appropriate instruction execution system, such as a microprocessor or dedicated design hardware. It can be understood by a person of ordinary skill in the art that the above-mentioned devices and methods can be implemented using computer executable instructions and / or contained in a processor control code, such as a carrier medium such as a disk, CD or DVD-ROM, a programmable memory such as a read-only memory (firmware), or a data carrier such as an optical or electronic signal carrier. Such code is provided on the carrier medium. The device and its modules of the present invention can be implemented by hardware circuits such as very large-scale integrated circuits or gate arrays, semiconductors such as logic chips, transistors, etc., or programmable hardware devices such as field programmable gate arrays, programmable logic devices, etc., can also be implemented by software executed by various types of processors, and can also be implemented by a combination of the above-mentioned hardware circuits and software, such as firmware.

[0186] The above description is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Any modifications, equivalent substitutions and improvements made by any technician familiar with the technical field within the technical scope disclosed by the present invention and within the spirit and principle of the present invention should be covered by the protection scope of the present invention.

Claims

1. A three-dimensional tooth registration method, characterized in that: The three-dimensional tooth registration method collects original three-dimensional tooth scan image data and performs preprocessing to construct a three-dimensional tooth data set; constructs a tooth point cloud feature extraction network module with strict rotation invariance; Construct a parameter estimation network module to establish a mapping relationship between tooth point cloud and its Gaussian model parameters; construct a differentiable Gaussian model parameter calculation network module; The whole network is trained using the tooth training set to obtain a trained three-dimensional tooth registration network, thereby performing a three-dimensional tooth registration operation; The three-dimensional tooth registration method comprises the following steps: Step 1: collect original three-dimensional dental oral scan image data and perform preprocessing to construct a three-dimensional dental data set; Step 2: construct a point cloud feature extraction module F with strict rotation invariance; Step 3: Use the Gaussian mixture model (GMM) in probability theory to construct the parameter estimation network module E ψ , used to estimate the mapping relationship between the three-dimensional space points of the tooth point cloud and the Gaussian model parameters; Step 4: Build a differentiable parameter calculation network module R Θ and R T ; Step 5, using the tooth point cloud three-dimensional space point pair training and test set to train and test the three-dimensional tooth registration network to obtain a trained three-dimensional tooth registration model; The parameter estimation network module E constructed based on the Gaussian mixture model in step 3 ψ , which is used to estimate the mapping relationship H between the three-dimensional space points of the tooth point cloud and the Gaussian model parameters, including: Input: Point cloud features Where N is the number of points in the point cloud, and C is the feature dimension of the point cloud; Output: N×M matrix H = [h nm ],and Each element h of the matrix H nm Indicates point P n and the implicit association probability between the components m of the Gaussian mixture model (GMM); Parameter estimation network: MLP→Concat layer→MLP→Softmax; MLP stands for multi-layer perceptron, Concat stands for feature concatenation; the last fully connected layer does not contain BN layer and ReLU activation; The differentiable parameter calculation module R constructed in step 4 Θ and R T , used to calculate the parameters Θ and transformation T of GMM; In step 5, the training set corresponding to the dental scan data is used to train the 3D dental registration network to obtain a trained high-performance 3D dental registration model, which specifically includes: inputting the 3D spatial points of the dental point cloud into the registration network for training to obtain a trained 3D dental registration model; sequentially passing the 3D spatial point pairs of the dental point cloud to be registered through E ψ , R Θ and R T Module, obtain the optimal transformation matrix T, thereby realizing three-dimensional tooth alignment based on deep learning.

2. The three-dimensional tooth registration method according to claim 1, characterized in that: The three-dimensional tooth registration method further comprises: inputting the three-dimensional space points of the tooth point cloud and sequentially passing through E ψ , R Θ and R T Module, obtain the optimal transformation matrix T, and realize three-dimensional tooth alignment based on deep learning.

3. The three-dimensional tooth registration method according to claim 1, characterized in that: The step 1 involves collecting the original three-dimensional dental oral scan image data and performing preprocessing to construct a three-dimensional dental data set, specifically including: obtaining the original three-dimensional spatial points of the dental point cloud through a medical oral scanning device, performing background removal processing on the dental point cloud, and then amplifying the dental point cloud data rows using translation, rotation, and adding disturbance operations to expand the scale of the three-dimensional dental registration data set; and finally dividing the data into a training set and a test set to construct a three-dimensional dental point cloud data set.

4. The three-dimensional tooth registration method according to claim 1, characterized in that: The strictly rotation-invariant point cloud feature extraction module F constructed in the step 2 converts the N×3 point cloud coordinates into an N×C point cloud feature vector using an operator with rotation invariance, a modulo operation ||·||, an angle operation between two points, a SIFT transformation, and a k-ary operator.

5. A three-dimensional tooth registration system for implementing the three-dimensional tooth registration method according to any one of claims 1 to 4, characterized in that: The three-dimensional teeth registration system comprises: A three-dimensional tooth data set construction module is used to collect original three-dimensional tooth oral scan image data and perform preprocessing to construct a three-dimensional tooth data set; Point cloud feature extraction module, used to extract and construct tooth point cloud features with strict rotation invariance; The mapping relationship estimation module uses the Gaussian mixture model in probability theory to construct a parameter estimation network module to estimate the mapping relationship between the three-dimensional space points of the tooth point cloud and the Gaussian model parameters; Differentiable parameter calculation module, used to calculate the parameters of the Gaussian mixture model and the corresponding transformation matrix; The three-dimensional tooth registration model training module is used to train the three-dimensional tooth registration network using the tooth point cloud three-dimensional space point training set to obtain a trained three-dimensional tooth registration model.

6. A computer device, characterized in that: The computer device includes a dental mouth scanner, a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the processor executes the steps of the three-dimensional tooth alignment method according to any one of claims 1 to 4.

7. A computer-readable storage medium storing a computer program, wherein when the computer program is executed by a processor, the processor executes the steps of the three-dimensional tooth registration method according to any one of claims 1 to 4.

Citation Information

Patent Citations

  • Teeth modeling method based on three-dimensional model retrieval

    CN106504331A

  • Dental CAD automation using deep learning

    US20190282344A1