A tooth position recognition method, medium and device for a three-dimensional oral scan tooth separation model based on deep learning

Abstract

The present invention provides a method, medium, and device for tooth position recognition based on a three-dimensional oral scan tooth separation model based on deep learning. The method comprises: obtaining a training set and a test set of the three-dimensional oral scan tooth separation model; performing data preprocessing on the training set and the test set; constructing a convolutional neural network model for the three-dimensional oral scan tooth separation model; training the constructed convolutional neural network model using the preprocessed training set; testing the trained convolutional neural network model using the preprocessed test set; inputting the three-dimensional oral scan tooth separation model to be recognized into the tested convolutional neural network model to obtain tooth position classification information and tooth classification information; and using the tooth classification information to assist in correcting the tooth position classification information. Without the need for repeated manual labeling of tooth classification data, the present invention introduces a fully connected layer for tooth classification and, in combination with tooth position classification, can correct some relatively obvious tooth position classification errors.

Description

Technical Field

[0001] The present invention relates to the technical field of tooth position recognition, and in particular to a tooth position recognition method, medium and device based on a three-dimensional oral scanning tooth separation model based on deep learning. Background Art

[0002] The three-dimensional oral scan tooth separation model refers to a regular model in which the irregular triangular faces in the original three-dimensional oral scan model are automatically or manually separated and the triangular faces are sequentially stored. The tooth model and the alveolar bone model can be separated from the three-dimensional oral scan tooth separation model by sequentially reading the triangular faces and performing closure detection.

[0003] After the original 3D oral scan model is manually separated by teeth using teeth separation software or 3D software, auxiliary diagnosis is performed, and tooth position recognition is the basis and the top priority. Currently, there is no relatively complete tooth position recognition solution for the 3D oral scan tooth separation model. The tooth position recognition method of the tooth separation model in the existing technology is: a large amount of data needs to be labeled to train a deep classification model, and then the model is used to recognize the tooth position. The problems with this method are: a large amount of data needs to be labeled, and the deep classification model cannot perform inter-class constraints well, resulting in a certain probability of tooth position duplication, as well as problems such as tooth positions being covered and missed. Data preprocessing generally adopts the farthest point sampling algorithm, which increases the time for preprocessing data and the instability of tooth position prediction. Summary of the Invention

[0004] The present invention aims to provide a method, medium and device for tooth position recognition of a three-dimensional oral scanning tooth separation model based on deep learning, so as to solve the problems existing in the tooth position recognition method of the tooth separation model in the above-mentioned prior art.

[0005] The present invention provides a method for tooth position recognition based on a three-dimensional oral scan tooth separation model based on deep learning, comprising the following steps:

[0006] Obtain the training set and test set of the 3D oral scan tooth separation model;

[0007] Perform data preprocessing on training and test sets;

[0008] Constructing a convolutional neural network model for a three-dimensional oral scan tooth separation model; the convolutional neural network model includes a feature extraction network and a tooth position classification fully connected layer and a tooth classification fully connected layer connected to the feature extraction network;

[0009] Use the preprocessed training set to train the constructed convolutional neural network model;

[0010] Use the preprocessed test set to test the trained convolutional neural network model;

[0011] The three-dimensional oral scan tooth separation model to be identified is input into the tested convolutional neural network model to obtain tooth position classification information and tooth classification information;

[0012] The tooth classification information is used to assist in correcting the tooth position classification information.

[0013] Furthermore, the method for obtaining a training set and a test set of a three-dimensional oral scan tooth separation model includes:

[0014] Obtain a three-dimensional oral scan tooth separation model;

[0015] Annotate the three-dimensional oral scan tooth separation model to obtain annotated data;

[0016] The labeled data is divided into training set and test set in proportion.

[0017] Furthermore, the method for data labeling of the three-dimensional oral scan tooth separation model includes:

[0018] The 3D dental scan model is divided into an upper dental model and a lower dental model. Each upper dental model and lower dental model is a binary STL format file. The first 80 bytes of the STL format file are the file header, followed by a 4-byte integer describing the number of triangular facets in the upper dental model or the lower dental model, followed by the geometric information of each triangular facet. The geometric information of the triangular facet refers to the three vertices of the triangular facet in 3D space, and each vertex is determined by the coordinates (x, y, z).

[0019] Read the triangles sequentially, and use the product of x, y, and z in the coordinates (x, y, z) of each vertex of the triangle as the unique identifier of each vertex, and the product of the unique identifiers of two adjacent vertices as the unique identifier of the edge between the two vertices; create a key-value pair object with the unique identifier of the edge as the key and the value 1. If the unique identifier of the edge does not exist in the key-value pair object, the unique identifier of the edge is added to the key-value pair object. If the unique identifier of the edge is the same, the unique identifier of the edge is deleted from the key-value pair object;

[0020] Since the triangular faces of the teeth or alveolar bones in the three-dimensional oral scan tooth separation model are stored in sequence and are closed, when the key-value pair object is empty during the sequential traversal, it indicates that the triangular faces during the sequential traversal are triangular faces that make up a tooth or alveolar bone. In this case, data labeling is completed by recording the position of the tooth or alveolar bone and the corresponding relationship between the start and end subscripts of the triangular faces of the tooth or alveolar bone in the stl format file; among them, the upper tooth model and the lower tooth model are labeled in pairs and placed in the same json format file as labeled data.

[0021] Furthermore, the method for performing data preprocessing on the training set and the test set includes:

[0022] (1) For the labeled data in the training set and the test set, the lower tooth model in the labeled data is preprocessed as follows:

[0023] The triangular facets corresponding to each tooth in the lower tooth model are arranged in vertex order to form an (m, 3) matrix, where m represents the number of vertices of each tooth and 3 represents the three dimensions of x, y, and z. Since vertices may be repeated, duplicate vertices are filtered out to form a vertex matrix, and then 1024 vertices are randomly selected from the vertex matrix to form a (1024, 3) matrix to represent a tooth or alveolar bone. One of the teeth is randomly lost in the lower tooth model, and each tooth or alveolar bone is sorted counterclockwise by the center point and the matrix is ​​spliced, with the alveolar bone ranked first, to obtain a single lower tooth. The dimension of the tooth model is a splicing matrix of (((number of teeth - 1 (one tooth is missing)) + 1 (alveolar bone)) * 1024, 3); find the vertex farthest from the coordinate origin in the lower tooth model, and use the distance between the farthest vertex and the coordinate origin as the radius. Divide the splicing matrix by the radius to achieve proportional scaling, so that x, y, and z are normalized to (0, 1); while splicing the matrix, the labeled tooth position classification data is spliced ​​as the training and testing targets, thereby obtaining the lower tooth model preprocessing data; among which, the tooth classification target is mapped according to the tooth position classification;

[0024] (2) For the labeled data in the training set and the test set, the upper tooth model in the labeled data is preprocessed as follows:

[0025] The triangular facets corresponding to each tooth in the upper dental model are arranged in vertex order to form an (m, 3) matrix, where m represents the number of vertices of each tooth and 3 represents the three dimensions of x, y, and z. Since vertices are repeated, duplicate vertices are filtered out to form a vertex matrix, and then 1024 vertices are randomly selected from the vertex matrix to form a (1024, 3) matrix to represent a tooth or alveolar bone. One of the teeth is randomly lost in the upper dental model, and each tooth or alveolar bone is sorted counterclockwise according to the center point and the matrix is ​​spliced, with the alveolar bone ranked first, to obtain a single upper tooth. The dimension of the tooth model is a splicing matrix of (((number of teeth - 1 (one tooth is missing)) + 1 (alveolar bone)) * 1024, 3); find the vertex farthest from the coordinate origin in the upper tooth model, and use the distance between the farthest vertex and the coordinate origin as the radius. Divide the splicing matrix by the radius to achieve proportional scaling, so that x, y, and z are normalized to (0, 1); while splicing the matrix, the labeled tooth position classification data is spliced ​​as the training and testing targets, thereby obtaining the upper tooth model preprocessing data; among which, the tooth classification target is mapped according to the tooth position classification;

[0026] Furthermore, the method for constructing a convolutional neural network model for a three-dimensional oral scan tooth separation model includes:

[0027] Constructing a feature extraction network; the feature extraction network includes an STN3d module of a PointNet network model, a first deformation module, a coordinate axis transformation module, three one-dimensional convolutional layers, a Max layer, and a second deformation module connected in sequence;

[0028] Constructing a tooth position classification fully connected layer; the tooth position classification fully connected layer adopts three fully connected layers and is connected to the output end of the second deformation module;

[0029] Construct a tooth classification fully connected layer; the tooth classification fully connected layer adopts three fully connected layers and is connected to the output end of the second deformation module.

[0030] Furthermore, the method of training the constructed convolutional neural network model using the preprocessed training set includes:

[0031] The preprocessed training set is input into the constructed convolutional neural network model to output tooth position classification information and tooth classification information;

[0032] The cross entropy loss of the tooth position classification information output by the convolutional neural network model and the tooth position classification target in the preprocessed training set is calculated, and the cross entropy loss of the output tooth classification information and the tooth classification target in the preprocessed training set is calculated; the sum of the cross entropy losses of the two is used as the total loss function; the Adam optimizer is then used to perform backpropagation optimization on the total loss function; training is stopped after the maximum number of iterations is reached or the total loss tends to be stable.

[0033] Furthermore, the method of testing the trained convolutional neural network model using the preprocessed test set includes:

[0034] The preprocessed test set is input into the trained convolutional neural network model to output tooth position classification information and tooth classification information;

[0035] The tooth position classification information and tooth classification information output by the convolutional neural network model are compared with the tooth position classification target and tooth classification target in the preprocessed test set to evaluate the test effect:

[0036] If the test results are not satisfactory, adjust the structure or hyperparameters of the convolutional neural network model and repeat the training process;

[0037] If the test results meet the requirements, the tested convolutional neural network model is used for tooth position recognition.

[0038] Furthermore, the method for assisting in correcting the tooth position classification information using the tooth classification information includes:

[0039] Identify incisors based on tooth classification information;

[0040] Correct the tooth position from the incisor to the left;

[0041] Correct the tooth position from the incisor to the right.

[0042] The present invention also provides a computer terminal storage medium storing computer terminal executable instructions, characterized in that the computer terminal executable instructions are used to execute the above-mentioned three-dimensional oral scanning tooth separation model tooth position recognition method based on deep learning.

[0043] The present invention further provides a computing device, comprising:

[0044] At least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the above-mentioned three-dimensional oral scanning tooth separation model tooth position recognition method based on deep learning.

[0045] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0046] The present invention introduces a fully connected layer for tooth classification and combines it with tooth position classification to correct some obvious tooth position classification errors without the need for repeated manual labeling of tooth classification data. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings in the embodiments will be briefly introduced below. It should be understood that the following drawings only illustrate certain embodiments of the present invention and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without paying any creative work.

[0048] Figure 1 This is a flowchart of a method for tooth position recognition based on a three-dimensional oral scanning tooth separation model based on deep learning in an embodiment of the present invention.

[0049] Figure 2 Schematic diagram of the upper teeth model and the lower teeth model in the three-dimensional oral scanning tooth separation model in an embodiment of the present invention.

[0050] Figure 3 This is a schematic diagram of the tooth position marking of the lower teeth model in an embodiment of the present invention.

[0051] Figure 4 This is a schematic diagram of the annotation data of a JSON format file in an embodiment of the present invention.

[0052] Figure 5 Schematic diagram of a splicing matrix in an embodiment of the present invention.

[0053] Figure 6Schematic diagram of the structure of the convolutional neural network model constructed in an embodiment of the present invention.

[0054] Figure 7 This is a flowchart for training and testing a convolutional neural network model in an embodiment of the present invention.

[0055] Figure 8 This is a schematic diagram of using tooth classification information to assist in correcting tooth position classification information in an embodiment of the present invention. DETAILED DESCRIPTION

[0056] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions of the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Generally, the components of the embodiments of the present invention described and shown in the drawings herein can be arranged and designed in various different configurations.

[0057] Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the invention as claimed, but rather merely represents selected embodiments of the present invention. All other embodiments derived by persons of ordinary skill in the art based on the embodiments of the present invention without creative effort are intended to fall within the scope of protection of the present invention.

[0058] Example

[0059] like Figure 1 As shown, this embodiment proposes a tooth position recognition method based on a three-dimensional oral scan tooth separation model based on deep learning, comprising the following steps:

[0060] S100, obtaining a training set and a test set of a three-dimensional oral scan tooth separation model;

[0061] S200, data preprocessing for training set and test set;

[0062] S300, building a convolutional neural network model for 3D oral scan tooth separation model;

[0063] S400, training the constructed convolutional neural network model using the preprocessed training set;

[0064] S500, testing the trained convolutional neural network model using the preprocessed test set;

[0065] S600: Input the three-dimensional oral scan tooth separation model to be identified into the tested convolutional neural network model to obtain tooth position classification information and tooth classification information;

[0066] S700: Assist in correcting the tooth position classification information using the tooth classification information.

[0067] Specifically:

[0068] S100, obtain the training set and test set of the 3D oral scan tooth separation model:

[0069] S110, obtaining a three-dimensional oral scan tooth separation model;

[0070] S120, annotate the 3D oral scan tooth separation model to obtain annotated data:

[0071] S121, the three-dimensional oral scan tooth separation model is divided into an upper tooth model and a lower tooth model, such as Figure 2 As shown in the figure, the upper and lower dental models are each a binary STL format file; the first 80 bytes in the STL format file are the file header, followed by a 4-byte integer describing the number of triangular facets of the upper or lower dental model, followed by the geometric information of each triangular facet one by one; the geometric information of the triangular facet refers to the three vertices of the triangular facet in three-dimensional space, and each vertex is determined by the coordinates (x, y, z);

[0072] S122, sequentially read the triangular facets, use the product of x, y, and z in the coordinates (x, y, z) of each vertex of the triangular facet as the unique identifier of each vertex, and use the product of the unique identifiers of two adjacent vertices as the unique identifier of the edge between the two vertices; create a key-value pair object, with the unique identifier of the edge as the key and the value 1. If the unique identifier of the edge does not exist in the key-value pair object, the unique identifier of the edge is added to the key-value pair object. If the unique identifier of the edge exists, the unique identifier of the edge is deleted from the key-value pair object;

[0073] S123, since the triangular facets of the teeth or alveolar bones in the three-dimensional oral scan tooth separation model are stored in sequence and are closed, when the key-value pair object is empty during the sequential traversal, it indicates that the triangular facets during the sequential traversal are triangular facets that constitute a tooth or alveolar bone, and the data annotation is completed by recording the corresponding relationship between the position of the tooth or alveolar bone and the start and end subscripts of the triangular facets of the tooth or alveolar bone in the stl format file; taking the following tooth model as an example, data annotation is performed from left to right, and the alveolar bone is 0: [632520, 714492], the wisdom tooth is 1: [489654, 572760], the second molar is 2: [572760, 632520], and so on. Figure 3 As shown, 0, 1, 2...16 represent the tooth position classification, and "[]" represents the starting subscript; the upper tooth model is also annotated from left to right; thus, the upper tooth model and the lower tooth model are annotated in pairs and placed in the same json format file as the annotation data, as shown Figure 4 shown.

[0074] S130 , dividing the labeled data into a training set and a test set in proportion. In this embodiment, 90% of the labeled data is used as the training set, and 10% of the labeled data is used as the test set.

[0075] S200, perform data preprocessing on the training set and test set:

[0076] S210: For the labeled data in the training set and the test set, perform the following data preprocessing on the lower teeth model in the labeled data:

[0077] The triangular facets corresponding to each tooth in the lower tooth model are arranged in vertex order to form an (m,3) matrix, where m represents the number of vertices of each tooth and 3 represents the three dimensions of x, y, and z. Since vertices may be repeated, duplicate vertices are filtered out to form a vertex matrix, and then 1024 vertices are randomly selected from the vertex matrix to form a (1024,3) matrix to represent a tooth or alveolar bone. One tooth is randomly lost in the lower tooth model, and each tooth or alveolar bone is sorted counterclockwise by the center point and the matrix is ​​spliced, with the alveolar bone ranked first. The dimension of the single lower tooth model is (((number of teeth - 1 (one tooth is lost)) + 1 (alveolar bone)) * 1024, 3) splicing matrix, as shown in the following example: Figure 5 As shown in the figure, find the vertex farthest from the coordinate origin in the lower tooth model, and use the distance between the farthest vertex and the coordinate origin as the radius. Divide the splicing matrix by the radius to achieve proportional scaling, so that x, y, and z are normalized to (0, 1). While splicing the matrix, the labeled tooth position classification data is spliced ​​as the training and testing targets, thereby obtaining the lower tooth model preprocessing data. Among them, the tooth classification target is mapped according to the tooth position classification, such as 0 represents alveolar bone, 1 incisor, 2 canine, 3 premolar, and 4 molar.

[0078] S220: For the labeled data in the training set and the test set, perform the following data preprocessing on the upper teeth model in the labeled data:

[0079] The triangular facets corresponding to each tooth in the upper dental model are arranged in vertex order to form an (m, 3) matrix, where m represents the number of vertices of each tooth and 3 represents the three dimensions of x, y, and z. Since vertices are repeated, duplicate vertices are filtered out to form a vertex matrix, and then 1024 vertices are randomly selected from the vertex matrix to form a (1024, 3) matrix to represent a tooth or alveolar bone. One of the teeth is randomly lost in the upper dental model, and each tooth or alveolar bone is sorted counterclockwise according to the center point and the matrix is ​​spliced, with the alveolar bone ranked first, to obtain a single upper tooth. The dimension of the tooth model is a splicing matrix of (((number of teeth - 1 (one tooth is missing)) + 1 (alveolar bone)) * 1024, 3); find the vertex farthest from the coordinate origin in the upper tooth model, and use the distance between the farthest vertex and the coordinate origin as the radius. Divide the splicing matrix by the radius to achieve proportional scaling, so that x, y, and z are normalized to (0, 1); while splicing the matrix, the labeled tooth position classification data is spliced ​​as the training and testing targets, thereby obtaining the upper tooth model preprocessing data; among which, the tooth classification target is mapped according to the tooth position classification.

[0080] It can be seen that the data preprocessing of the lower teeth model and the upper teeth model is similar.

[0081] S300, build a convolutional neural network model for 3D oral scan tooth separation model, such as Figure 6 As shown:

[0082] S310, constructing a feature extraction network; the feature extraction network includes an STN3d module, a first deformation module, a coordinate axis transformation module, three one-dimensional convolution layers, a Max layer, and a second deformation module of a PointNet network model connected in sequence; first, the STN3d module in the PointNet network model is used to transform the splicing matrix (1, 3, n*1024), and then the first deformation module Reshape is used to transform it into (3, n, 1024), and then the coordinate axis is transformed into (n, 3, 1024) by the coordinate axis transformation module, and then three one-dimensional convolutions are used to increase the transformed three channels to 1024 channels, and then 1024 vertices are converted into 1 feature by 1 Max layer, and finally the second deformation module Reshape is used to transform it into (1, n, 1024);

[0083] S320, construct a tooth position classification fully connected layer; the tooth position classification fully connected layer adopts three fully connected layers and is connected to the output end of the second deformation module; the tooth position classification fully connected layer converts 1024 features into (16+1)==17 categories (where 16 represents 16 tooth positions and 1 represents alveolar bone).

[0084] S330, construct a tooth classification fully connected layer; the tooth classification fully connected layer uses three fully connected layers and is connected to the output end of the second deformation module; the tooth classification fully connected layer converts 1024 features into (4+1)==5 categories (where 4 represents incisors, canines, premolars, and molars; 1 represents alveolar bone).

[0085] In this embodiment, a fully connected layer is added after the feature extraction network of the convolutional neural network model. Due to the characteristics of the fully connected layer, the number of input rows (number of teeth) and the number of output rows are the same. In this way, the order of the teeth will correspond one-to-one with the output of the convolutional neural network model.

[0086] S400, using the preprocessed training set to train the constructed convolutional neural network model, such as Figure 7 As shown:

[0087] S410, inputting the preprocessed training set into the constructed convolutional neural network model, outputting tooth position classification information and tooth classification information; since batch training cannot be performed due to different numbers of teeth, the batch size is 1;

[0088] S420, calculating a cross entropy loss between the tooth position classification information output by the convolutional neural network model and the tooth position classification target in the preprocessed training set, and calculating a cross entropy loss between the output tooth classification information and the tooth classification target in the preprocessed training set; and taking the sum of the cross entropy losses as the total loss function; the total loss function is expressed as follows:

[0089] L = loss 牙位分类 +loss 牙齿分类 ;

[0090] Among them, L represents the total loss function; loss 牙位分类 Represents the cross entropy loss of tooth position classification; loss 牙齿分类 represents the cross entropy loss for tooth classification.

[0091] loss 牙位分类 The formula is as follows:

[0092]

[0093] Among them, y 牙位分类 Indicates the output tooth position classification information; Represents the tooth position classification target in the labeled data.

[0094] loss 牙齿分类 The formula is as follows:

[0095]

[0096] Among them, y牙齿分类 Indicates the output tooth classification information; Represents the tooth classification target in the labeled data.

[0097] Finally, the Adam optimizer is used to perform backpropagation optimization on the total loss function. The learning rate is set according to the requirements. In this embodiment, the learning rate is set to 0.0001. The training is stopped after the maximum number of iterations (epoch, set to 500 in this embodiment) or the total loss tends to be stable.

[0098] S500, use the preprocessed test set to test the trained convolutional neural network model:

[0099] S510, inputting the preprocessed test set into the trained convolutional neural network model, and outputting tooth position classification information and tooth classification information;

[0100] S520, comparing the tooth position classification information and tooth classification information output by the convolutional neural network model with the preprocessed tooth position classification target and tooth classification target in the test set to evaluate the test effect:

[0101] If the test results are not satisfactory, adjust the structure or hyperparameters of the convolutional neural network model and repeat the training process;

[0102] If the test results meet the requirements, the tested convolutional neural network model is used for tooth position recognition.

[0103] During the training and testing of the convolutional neural network model, 1024 vertices are randomly sampled from the separated teeth or alveolar bones. In production, 1024 vertices are sampled from the separated teeth or alveolar bones at equal intervals according to the subscripts, instead of using the farthest point sampling algorithm, which can greatly save sampling time.

[0104] S600: Input the three-dimensional oral scan tooth separation model to be identified into the tested convolutional neural network model to obtain tooth position classification information and tooth classification information;

[0105] S700: Utilize the tooth classification information to assist in correcting the tooth position classification information.

[0106] S710, determining the incisor according to the tooth classification information:

[0107] Under normal circumstances, according to the tooth classification, there are four incisors, so the order of these four incisors can be obtained in sequence; if there are five incisors, one of them is a canine, and it is necessary to combine the classification of the canines on the left and right sides to judge whether the left or right side of the five incisors is a canine (the probability of this is relatively low); if there are fewer than four, the tooth position classification result is used without any adjustment (in actual application: tooth position classification sometimes repeats in the molar position, but the effect is very good in the incisor, and the probability of misjudgment is low).

[0108] S720, correct the tooth position from the incisor to the left:

[0109] S721, determine the canine: Check from the incisor to the left to see if there is a canine. If there is, the tooth position is determined. If not, check the distance between the current tooth and the leftmost incisor. If it is less than 3mm (set value), and there are two premolars on the left side in addition to the current tooth, then it can be determined that the current tooth is a canine. Figure 8 If there are two canines on the left side according to the dental classification, the one closest to the incisor is the canine, and the other is the premolar and its classification is revised.

[0110] S722, determine the premolars: If there are two premolars, the order of the premolars is determined; if there are three, the leftmost one is the molar and its classification is modified; if there is only one, it is determined that the distance to the previous one is less than 3mm, then it is the first premolar, otherwise it is the second premolar.

[0111] S723, Determine the posterior molars: There are up to three posterior molars on the left side. If the distance between the first posterior molar and the preceding tooth (usually the second premolar) is less than 3mm, the position of the first posterior molar is determined. The tooth position is then determined by determining whether the distance between the first and preceding teeth is less than 3mm.

[0112] S724: If other situations are not determined, the original tooth position classification result is used for determination.

[0113] S730, correcting the tooth position from the incisor to the right side, which is similar to the step S720 of correcting the tooth position from the incisor to the left side, and will not be repeated here.

[0114] In addition, in some embodiments, a computer terminal storage medium is proposed, which stores computer terminal executable instructions, and the computer terminal executable instructions are used to execute the three-dimensional oral scanning tooth separation model tooth position recognition method based on deep learning as described in the above embodiments. Examples of computer storage media include magnetic storage media (e.g., floppy disks, hard disks, etc.), optical recording media (e.g., CD-ROMs, DVDs, etc.), or memories such as memory cards, ROMs, or RAMs. Computer storage media can also be distributed on network-connected computer systems, such as application stores.

[0115] In addition, some embodiments provide a computing device comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the deep learning-based tooth position recognition method for a three-dimensional oral scan tooth separation model as described in the above embodiments. Examples of computing devices include a PC, tablet computer, smartphone, or PDA.

[0116] The foregoing description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention are intended to be within the scope of protection of the present invention.

Claims

1. A tooth position recognition method based on a three-dimensional oral scan tooth separation model based on deep learning, characterized in that: The steps include: Obtain the training set and test set of the 3D oral scan tooth separation model; Perform data preprocessing on training and test sets; Constructing a convolutional neural network model for a three-dimensional oral scan tooth separation model; the convolutional neural network model includes a feature extraction network and a tooth position classification fully connected layer and a tooth classification fully connected layer connected to the feature extraction network; Use the preprocessed training set to train the constructed convolutional neural network model; Use the preprocessed test set to test the trained convolutional neural network model; The three-dimensional oral scan tooth separation model to be identified is input into the tested convolutional neural network model to obtain tooth position classification information and tooth classification information; Using tooth classification information to assist in correcting tooth position classification information; The method for obtaining a training set and a test set of a three-dimensional oral scanning tooth separation model includes: Obtain a three-dimensional oral scan tooth separation model; Annotate the three-dimensional oral scan tooth separation model to obtain annotated data; Divide the labeled data into training set and test set in proportion; The method for data labeling of a three-dimensional oral scan tooth separation model comprises: The 3D dental scan model is divided into an upper dental model and a lower dental model. Each upper dental model and lower dental model is a binary STL format file. The first 80 bytes of the STL format file are the file header, followed by a 4-byte integer describing the number of triangular facets in the upper dental model or the lower dental model, followed by the geometric information of each triangular facet. The geometric information of the triangular facet refers to the three vertices of the triangular facet in 3D space, and each vertex is determined by the coordinates (x, y, z). Read the triangles sequentially, and use the product of x, y, and z in the coordinates (x, y, z) of each vertex of the triangle as the unique identifier of each vertex, and the product of the unique identifiers of two adjacent vertices as the unique identifier of the edge between the two vertices; create a key-value pair object with the unique identifier of the edge as the key and the value 1. If the unique identifier of the edge does not exist in the key-value pair object, the unique identifier of the edge is added to the key-value pair object. If the unique identifier of the edge is the same, the unique identifier of the edge is deleted from the key-value pair object; Since the triangular faces of the teeth or alveolar bones in the three-dimensional oral scan tooth separation model are stored in sequence and are closed, when the key-value pair object is empty during the sequential traversal, it indicates that the triangular faces during the sequential traversal are triangular faces that make up a tooth or alveolar bone. In this case, data labeling is completed by recording the position of the tooth or alveolar bone and the corresponding relationship between the start and end subscripts of the triangular faces of the tooth or alveolar bone in the stl format file; among them, the upper tooth model and the lower tooth model are labeled in pairs and placed in the same json format file as labeled data.

2. The tooth position recognition method based on a three-dimensional oral scan tooth separation model based on deep learning according to claim 1 is characterized in that: The method for performing data preprocessing on the training set and the test set includes: (1) For the labeled data in the training set and the test set, the lower tooth model in the labeled data is preprocessed as follows: The triangular facets corresponding to each tooth in the lower tooth model are arranged in vertex order to form an (m, 3) matrix, where m represents the number of vertices of each tooth and 3 represents the three dimensions of x, y, and z. Since vertices may be repeated, duplicate vertices are filtered out to form a vertex matrix, and then 1024 vertices are randomly selected from the vertex matrix to form a (1024, 3) matrix to represent a tooth or alveolar bone. One of the teeth is randomly lost in the lower tooth model, and each tooth or alveolar bone is sorted counterclockwise by the center point and the matrix is ​​spliced, with the alveolar bone ranked first, to obtain a single lower tooth. The dimension of the tooth model is a splicing matrix of (((number of teeth - 1 (one tooth is missing)) + 1 (alveolar bone)) * 1024, 3); find the vertex farthest from the coordinate origin in the lower tooth model, and use the distance between the farthest vertex and the coordinate origin as the radius. Divide the splicing matrix by the radius to achieve proportional scaling, so that x, y, and z are normalized to (0, 1); while splicing the matrix, the labeled tooth position classification data is spliced ​​as the training and testing targets, thereby obtaining the lower tooth model preprocessing data; among which, the tooth classification target is mapped according to the tooth position classification; (2) For the labeled data in the training set and the test set, the upper tooth model in the labeled data is preprocessed as follows: The triangular facets corresponding to each tooth in the upper dental model are arranged in vertex order to form an (m, 3) matrix, where m represents the number of vertices of each tooth and 3 represents the three dimensions of x, y, and z. Since vertices are repeated, duplicate vertices are filtered out to form a vertex matrix, and then 1024 vertices are randomly selected from the vertex matrix to form a (1024, 3) matrix to represent a tooth or alveolar bone. One of the teeth is randomly lost in the upper dental model, and each tooth or alveolar bone is sorted counterclockwise according to the center point and the matrix is ​​spliced, with the alveolar bone ranked first, to obtain a single upper tooth. The dimension of the tooth model is a splicing matrix of (((number of teeth - 1 (one tooth is missing)) + 1 (alveolar bone)) * 1024, 3); find the vertex farthest from the coordinate origin in the upper tooth model, and use the distance between the farthest vertex and the coordinate origin as the radius. Divide the splicing matrix by the radius to achieve proportional scaling, so that x, y, and z are normalized to (0, 1); while splicing the matrix, the labeled tooth position classification data is spliced ​​as the training and testing targets, thereby obtaining the upper tooth model preprocessing data; among which, the tooth classification target is mapped according to the tooth position classification.

3. The tooth position recognition method based on a three-dimensional oral scan tooth separation model based on deep learning according to claim 2, characterized in that: The method for constructing a convolutional neural network model for a three-dimensional oral scanning tooth separation model includes: Constructing a feature extraction network; the feature extraction network includes an STN3d module of a PointNet network model, a first deformation module, a coordinate axis transformation module, three one-dimensional convolutional layers, a Max layer, and a second deformation module connected in sequence; Constructing a tooth position classification fully connected layer; the tooth position classification fully connected layer adopts three fully connected layers and is connected to the output end of the second deformation module; Construct a tooth classification fully connected layer; the tooth classification fully connected layer adopts three fully connected layers and is connected to the output end of the second deformation module.

4. The tooth position recognition method based on a three-dimensional oral scan tooth separation model based on deep learning according to claim 3, characterized in that: The method of training the constructed convolutional neural network model using the preprocessed training set includes: The preprocessed training set is input into the constructed convolutional neural network model to output tooth position classification information and tooth classification information; The cross entropy loss of the tooth position classification information output by the convolutional neural network model and the tooth position classification target in the preprocessed training set is calculated, and the cross entropy loss of the output tooth classification information and the tooth classification target in the preprocessed training set is calculated; the sum of the cross entropy losses of the two is used as the total loss function; the Adam optimizer is then used to perform backpropagation optimization on the total loss function; training is stopped after the maximum number of iterations is reached or the total loss tends to be stable.

5. The tooth position recognition method based on a three-dimensional oral scan tooth separation model based on deep learning according to claim 4 is characterized in that: The method for testing the trained convolutional neural network model using the preprocessed test set includes: The preprocessed test set is input into the trained convolutional neural network model to output tooth position classification information and tooth classification information; The tooth position classification information and tooth classification information output by the convolutional neural network model are compared with the tooth position classification target and tooth classification target in the preprocessed test set to evaluate the test effect: If the test results are not satisfactory, adjust the structure or hyperparameters of the convolutional neural network model and repeat the training process; If the test results meet the requirements, the tested convolutional neural network model is used for tooth position recognition.

6. The tooth position recognition method based on a three-dimensional oral scan tooth separation model based on deep learning according to claim 5, characterized in that: The method for assisting in correcting tooth position classification information by using tooth classification information includes: Identify incisors based on tooth classification information; Correct the tooth position from the incisor to the left; Correct the tooth position from the incisor to the right.

7. A computer terminal storage medium storing computer terminal executable instructions, characterized in that: The computer terminal executable instructions are used to execute the tooth position recognition method of the three-dimensional oral scanning tooth separation model based on deep learning as described in any one of claims 1 to 6.

8. A computing device, characterized in that: include: at least one processor; And a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the tooth position recognition method based on deep learning of a three-dimensional oral scanning tooth separation model as described in any one of claims 1 to 6.

Images