Tooth model establishment method, correction scheme design method, correction appliance manufacturing method and correction appliance

Abstract

The embodiment of the invention discloses a tooth model establishment method, a correction scheme design method, a correction appliance manufacturing method and a correction appliance. The tooth model building method comprises the following steps: acquiring a multi-layer image of an oral cavity by adopting a CBCT (cone beam computed tomography) technology; identifying a tooth pixel point region corresponding to each type of tooth tissue in each layer of image; stacking the multiple layers of images in sequence and acquiring three-dimensional point cloud data of tooth pixel point areas corresponding to various types of tooth tissues; and establishing a three-dimensional tooth model according to the three-dimensional point cloud data of the tooth pixel point area corresponding to each type of tooth tissue. According to the embodiment of the invention, each layer of the multi-layer image is identified, so that the division result is more accurate than that of directly performing division on the complete CBCT image; and finally, stacking the multiple layers of images in sequence to obtain complete three-dimensional data corresponding to each tooth. And the three-dimensional tooth model is established according to the three-dimensional data corresponding to each tooth, so that the division result is more accurate, and the established three-dimensional model is more accurate.

Description

TECHNICAL FIELD

[0001] The embodiment of the present application relates to the field of model establishment, in particular to a tooth model establishment, a treatment scheme design, a treatment appliance manufacturing method and a treatment appliance. BACKGROUND

[0002] In order to perform medical correction on the teeth of a patient, a variety of tooth treatment appliances have been developed. In comparison with the traditional fixed bracket treatment technology, the new type of invisible treatment technology does not require brackets and steel wires, but uses a series of invisible treatment appliances (also referred to as shell-shaped treatment appliances). The invisible tooth treatment appliance is made of a high-performance elastic polymer material that meets the requirements of biocompatibility, so that the treatment process is almost completed without being noticed by others, and does not affect daily life and social interaction.

[0003] The production of the invisible treatment appliance first requires modeling of the teeth of the patient, so that a suitable invisible treatment appliance can be produced according to the shape and arrangement of the teeth of the patient. Generally, the cone beam computed tomography (CBCT) technology can be used to scan and image the teeth of the patient, and then the CBCT image of the scanned teeth is segmented. The segmented CBCT image of the teeth can clearly distinguish the teeth and the surrounding tissues (such as the gums, alveolar bone, etc.), so as to facilitate three-dimensional image modeling of the teeth according to the same.

[0004] However, the existing tooth modeling method based on the CBCT image is usually based on CBCT grayscale value reconstruction. This method divides the whole dentition and alveolar bone in the CBCT image of the oral cavity by adjusting the CT grayscale value of the whole multi-layer reconstructed CBCT image, and then obtains each tooth by using connected domain processing on the whole dentition. Since the existing technology only divides the whole CBCT image into teeth and alveolar bone based on the CT grayscale value of the image, when the imaging quality of the whole image is low, the accuracy of the three-dimensional model established will also be affected. SUMMARY

[0005] The purpose of the embodiment of the present application is to provide a tooth model establishment, a treatment scheme design, a treatment appliance manufacturing method and a treatment appliance, so that different teeth can be clearly divided even when the quality of the CBCT whole image is poor, and an accurate three-dimensional model can be established.

[0006] To solve the above technical problems, the embodiment of the present application provides a tooth model establishing method, comprising: acquiring multi-layer images of an oral cavity by using a CBCT technology; identifying tooth pixel point regions corresponding to each category of tooth tissue in each layer of images; wherein the each category of tooth tissue at least includes each category of tooth body and alveolar bone; stacking the multi-layer images in sequence and acquiring three-dimensional point cloud data of the tooth pixel point regions corresponding to each category of tooth tissue; and establishing a three-dimensional tooth model according to the three-dimensional point cloud data of the tooth pixel point regions corresponding to each category of tooth tissue.

[0007] The embodiment of the present application also provides an electronic device, comprising at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the tooth model establishing method as described above.

[0008] The embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to implement the tooth model establishing method as described above.

[0009] The embodiment of the present application also provides a treatment plan designing method, comprising: establishing a three-dimensional tooth model according to the tooth model establishing method as described above; and designing a treatment plan by taking the three-dimensional tooth model as an initial tooth model.

[0010] The embodiment of the present application also provides a treatment appliance manufacturing method, comprising: manufacturing a tooth treatment appliance according to the treatment plan designed by the treatment plan designing method as described above.

[0011] The embodiment of the present application also provides a treatment appliance, comprising: a tooth treatment appliance manufactured by the treatment appliance manufacturing method as described above.

[0012] In the embodiment of the present application, instead of directly dividing teeth on a complete CBCT image, i.e. a recombined CBCT image, each layer of multi-layer images of an oral cavity acquired by using a CBCT technology is identified to identify tooth pixel point regions corresponding to each category of tooth in each layer of images; since the teeth in each layer of CBCT images are divided, the division result is more accurate than that of directly dividing on a complete CBCT image; finally, the multi-layer images are stacked in sequence to acquire complete three-dimensional data corresponding to each tooth; and a three-dimensional tooth model is established according to the three-dimensional data corresponding to each tooth, and since the division result is more accurate, the three-dimensional model established is also more accurate.

[0013] In addition, in an example, after the three-dimensional tooth model is established according to the three-dimensional point cloud data of the tooth pixel point region corresponding to each category of tooth tissue, the method further comprises: registering the crown part in the calibration tooth model with the crown part in the three-dimensional tooth model; wherein the calibration tooth model is obtained by oral cavity scanning; and after the registration is completed, replacing the crown part in the three-dimensional tooth model with the crown part in the calibration tooth model. Because the tooth model obtained by oral cavity scanning is more true and more consistent with the actual structure of the target oral crown than the tooth model established by using the CBCT image, the tooth model can be more accurate and true by using the tooth model established by using the CBCT image and replacing the crown part of the tooth model obtained by oral cavity scanning.

[0014] In addition, in an example, before the tooth pixel point region corresponding to each category of tooth tissue in each layer of image is identified, the method further comprises: pre-processing each layer of image; wherein the pre-processing comprises adjusting the window width and window level of each layer of image, and then normalizing the brightness data of each layer of image after the window width and window level are adjusted. By adjusting the window width and window level of the image, the contrast and brightness of the picture can be changed to further make the tooth and alveolar bone clearer. Because the brightness of the picture is usually represented by the values of the three channels of RGB, the intensity of RGB is valued in the range of 0-255, so normalizing the image array to the range of 0-255 can facilitate the acquisition of the brightness of the picture.

[0015] In addition, in an example, the identification of the tooth pixel point region corresponding to each category of tooth tissue in each layer of image is realized by a neural network model, and the neural network model is trained by a plurality of training samples obtained in advance; each training sample comprises an input sample; the input sample comprises a plurality of layers of images in which each category of tooth corresponding region and the tooth tissue to which each pixel point in each region belongs are labeled. The identification of each tooth in each layer of image by the pre-trained neural network model can make the tooth model establishment method more widely applicable, and the corresponding tooth model can be established faster for different oral cavities.

[0016] In addition, in an example, the tooth tissue to which each pixel point in each region belongs is labeled based on the alignment of the input sample with the corresponding tooth three-dimensional grid, and is labeled based on the aligned tooth three-dimensional grid by using a ray detection method. By labeling the training sample in the neural network model by using the ray method, full-automatic labeling can be realized, the training process of the neural network model can be accelerated, and the waste of manpower can be reduced.

[0017] In addition, in one example, the neural network model comprises a first neural network, a second neural network, and a third neural network; identifying the tooth pixel point region corresponding to each category of tooth in each layer of image comprises: inputting each layer of image into the first neural network for feature extraction, and the first neural network outputs a feature map of each layer of image; inputting the feature map of each layer of image into the second neural network for region selection, and the second neural network outputs a candidate pixel point region corresponding to each category of tooth in each layer of image; performing alignment processing on the candidate pixel point region corresponding to each category of tooth tissue, and outputting the aligned candidate pixel point region corresponding to each category of tooth tissue in each layer of image; inputting the aligned candidate pixel point region corresponding to each category of tooth into the third neural network for classification processing, and the third neural network outputs a tooth pixel point region corresponding to each category of tooth in each layer of image. Since the neural network model first determines the region of the tooth to be identified before classifying the pixel points, and then classifies the pixel points in the target region, it is only necessary to determine which pixel points correspond to the tooth and which pixel points are the background at this time, which can further reduce the calculation amount and make the classification of pixel points more accurate, so that the finally established tooth model is more complete and clear. BRIEF DESCRIPTION OF DRAWINGS

[0018] One or more embodiments are illustrated by way of example in the figures that are part of this disclosure and which are included to further provide explanatory aspects of the present embodiments. In the drawings: like reference numerals demonstrate like elements, unless otherwise specified, and the figures are not necessarily to scale as proportions can have been exaggerated to illustrate details.

[0019] Figure 1 is a flow chart of a tooth model establishment method according to an embodiment of the present application;

[0020] Figure 2 is an image of a layer of CBCT technology according to an embodiment of the present application;

[0021] Figure 3 is a layer of CBCT image according to an embodiment of the present application, which is labeled with each category of tooth and tooth tissue;

[0022] Figure 4 is a schematic diagram of finding intersection points in XOY plane by ray method according to an embodiment of the present application;

[0023] Figure 5 is a schematic diagram of finding intersection points in XOY, XOZ, YOZ planes by ray method according to an embodiment of the present application;

[0024] Figure 6 is a schematic diagram of neural network model network layer structure according to an embodiment of the present application;

[0025] Figure 7 is a diagram of the classification of each pixel in each image according to an embodiment of the present application;

[0026] Figure 8 is a diagram of the volume elements of each tooth class after stacking according to an embodiment of the present application;

[0027] Figure 9 is a diagram of a three-dimensional tooth model according to an embodiment of the present application;

[0028] Figure 10 is a flow chart of a method of creating a more accurate and realistic complete tooth model according to an embodiment of the present application;

[0029] Figure 11 is a diagram of an electronic device according to another embodiment of the present application. DETAILED DESCRIPTION

[0030] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the embodiments of the present application will be described in detail below with reference to the drawings. However, those skilled in the art can understand that, in the embodiments of the present application, many technical details are presented in order to make the reader better understand the present application. However, the technical solutions claimed by the present application can be implemented even without these technical details and based on various changes and modifications of the following embodiments. The division of the following embodiments is for the convenience of description, and should not constitute any limitation on the specific embodiments of the present application, and the embodiments can be combined and referenced with each other without contradiction.

[0031] The FDI notation mentioned in the embodiments of the present application is defined according to the classification of teeth in the 2nd edition of the book "Introduction to Oral Medicine" published by Peking University Medical Press, pages 36-38. The posterior region includes premolars and molars, which are shown as teeth 4-8 in the FDI notation. The anterior region is shown as teeth 1-3 in the FDI notation.

[0032] One embodiment of the present invention relates to a method for establishing a tooth model, which can be applied in a computer device. In this embodiment, teeth are not directly segmented on a complete, reconstructed CBCT image. Instead, each layer of a multi-layered oral cavity image acquired using CBCT technology is identified, and the corresponding tooth pixel regions for each category of teeth in each layer are identified. Because the teeth are segmented within each layer of the CBCT image, the segmentation result is more accurate compared to segmenting directly on a complete CBCT image. Finally, the multi-layered images are stacked sequentially to obtain complete three-dimensional data corresponding to each tooth. A three-dimensional tooth model is then established based on the three-dimensional data corresponding to each tooth. Because the segmentation result is more accurate, the established three-dimensional model is also more accurate. The implementation details of the tooth model establishment method of this embodiment are described below. The following content is only for ease of understanding and is not essential for implementing this solution.

[0033] like Figure 1 As shown, in step 101, CBCT technology is used to acquire CBCT images of the oral cavity. These CBCT images are multi-slice image data, such as... Figure 2 The image shown is a slice of the acquired CBCT image.

[0034] In one example, multi-slice images of a user's oral cavity can be obtained using a CBCT machine in a dental radiology department.

[0035] In step 102, the tooth pixel regions corresponding to each category of tooth tissue in each layer of the image are identified; wherein, the tooth pixel region refers to the pixel region that includes tooth tissue. The tooth tissue also includes the tooth body and alveolar bone. In addition to identifying the tooth body, a more comprehensive oral cavity and tooth model can be built by identifying other tooth-related tissues.

[0036] In one example, before step 102, the method further comprises pre-processing each layer of images; the pre-processing of the CBCT images comprises adjusting the window width and window level of the CBCT images, and normalizing the CBCT images, so that the teeth and alveolar bone in the CBCT images are clearer, wherein the window width refers to the range of CT values displayed on the CBCT image, which directly affects the contrast of the CBCT image, when the window width is narrow, the range of CT values displayed is small, the contrast is large, which is conducive to observing the details of specific tissues, when the window width is wide, the range of CT values displayed is large, the contrast is small, which is suitable for observing tissues with large density difference, the window level refers to the average of the upper and lower limit CT values of the window width, which affects the brightness of the CBCT image, a lower window level can make the image brightness higher, and a higher window level makes the image brightness lower, in some preferred examples, the parameters and image data of each layer of images are read (for example, SimpleITK, a program interface for processing medical images, can be used to read), then the image data is converted into an array format, the window width and window level of each layer of images are adjusted to adjust the contrast and brightness of the images, then the image array is normalized to the range of 0-255, and finally each layer of images is converted into a picture in a specified format (such as PNG format). By adjusting the adjusted window width and window level, the contrast and brightness of the picture can be changed to further make the teeth and alveolar bone clearer, because the brightness of the picture is usually represented by the values of the three channels of RGB, the intensity of RGB is valued in the range of 0-255, so normalizing the image array to the range of 0-255 can facilitate obtaining the brightness of the picture.

[0037] In one example, step 102 can be implemented by a neural network model trained by a plurality of training samples obtained in advance; each training sample comprises an input sample; the input sample comprises a plurality of layers of images in which each tooth body, alveolar bone and corresponding tooth pixel point region is labeled. The identification of each type of tooth in each layer of images by the pre-trained neural network model can make the tooth model establishment method more widely applicable, and the corresponding tooth model can be established faster for different oral cavities, such as Figure 3 Fig. 4 shows a layer of images of a CBCT image in which each type of tooth and tooth tissue is labeled, and Fig. 5 shows a layer of images of a CBCT image in which each type of tooth and tooth tissue is labeled. Figure 3The contour lines are used to mark the corresponding tooth pixel point region of the tooth body of different categories. In this application, each input sample includes at least tooth bodies of multiple tooth categories, such as permanent teeth, milk teeth, and different tooth positions corresponding to each tooth. Therefore, the categories marked in each input sample include at least alveolar bone, permanent teeth, milk teeth, and background. In some examples, the total number of categories marked in each input sample is 55, including at least background, maxilla, mandible, 4*8 permanent teeth, and 4*5 milk teeth. For 4*8 permanent teeth and 4*5 milk teeth, FDI can be used to distinguish their categories. Different regions of permanent teeth 1-4 quadrants represent, for example, 11 represents the first quadrant of the maxilla 1 permanent tooth. Different regions of milk teeth use 5-8 quadrants, for example, 51 represents the first quadrant of the maxilla 1 milk tooth. Of course, other identifiers can be used to mark each tooth category in the training sample, as long as different tooth positions, permanent teeth, and milk teeth can be distinguished based on the identifier. In some embodiments, the input sample can be manually labeled, that is, the categories and pixel point regions of teeth and tooth tissues in the training sample are manually identified and labeled. In other embodiments, the training sample can also be automatically labeled based on the existing three-dimensional mesh of the training sample corresponding to the existing three-dimensional mesh based on the ray detection method.

[0038] Specifically, each tooth pixel point region corresponding to each tooth tissue can be labeled based on the ray detection method. The specific implementation of automatic labeling using the ray detection method is as follows: first, obtain the CBCT multi-layer image of each oral cavity in the input sample and the corresponding three-dimensional mesh of the teeth. Then, the selected three-dimensional mesh of the teeth is converted to the coordinate system of the multi-layer image obtained by the CBCT technology through the second conversion matrix. The corresponding three-dimensional mesh of the teeth refers to the known three-dimensional mesh data of the patient corresponding to the CBCT multi-layer image.

[0039] Specifically, the second conversion matrix can be a matrix containing rotation, translation, and scaling transformations:

[0040]

[0041] wherein, is a rotation matrix, r 11 ~r 33 is each component of the rotation matrix, t x , t y , t z represent the translation amount in the three-axis direction, respectively; s x , s y , s z are the scaling ratios in the three-axis direction.

[0042] The CBCT multi-layer images and the tooth three-dimensional mesh are transformed into the same coordinate system by using a second conversion matrix to align the tooth three-dimensional mesh with the CBCT multi-layer images, and then a coordinate (x i , y j , z k ) is calculated in the coordinate system along a set vector direction and the intersection point of a triangular facet on the tooth three-dimensional mesh. Taking the three-dimensional mesh of incisor 11 (11 is the FDI number of the incisor) as an example, as shown in FIG. 11, it is assumed that the XOY plane is selected as the reference plane, the ray origin is selected as a point with integer coordinates on the XOY plane, the XOY plane does not intersect the three-dimensional mesh, and the ray direction vector is (0, 0, 1). Using the Raycasting method, a set of distances can be obtained: Figure 4

[0043] ts={t1,t2,t3,...,t n |n∈N +}

[0044] where t n is a positive real number representing the distance from the ray origin to the triangular facet intersection point, n represents the number of intersection points, n∈N + , and n can only take positive integers. The intersection point coordinates are calculated as C n =(x i , y i , z i +t n ), and ts is inf, indicating that there is no intersection point with the three-dimensional mesh. In some embodiments, all intersection points with the three-dimensional mesh can be directly obtained by the ray origin along the (0, 0, 1) direction; in other embodiments, the first intersection point C1 is returned by taking the ray origin as the starting point and the ray direction as (0, 0, 1), and then C1 is taken as the ray origin and (0, 0, 1) as the ray direction to continue calculating the intersection points with the three-dimensional mesh until there is no new intersection point. The coordinates of the points between two points are calculated from the intersection points generated by the same ray origin, for example, two intersection points are generated by the origin (x m , y n , 0) and the three-dimensional mesh with distances c1 and c2, respectively, and c2>c1, then the intersection point coordinates are (x m , y n , c1) and (x m , y n , c2), and the set of points between the two intersection points, i.e., the points inside the tooth, is:

[0045] {(x m , y n , c1), (x m , y n , c1+1), (x​m , y n , c1+2),..., (x m , y n , c2-2), (x m , y n , c2-1), (x m , y n , c2)}

[0046] Then change the ray origin, also based on the ray direction of (0, 0, 1) to obtain the intersection points of the ray origin and the three-dimensional grid and the set of integer points between the intersection points by the ray detection method.

[0047] Integrate the integer point sets generated by the ray detection method of all ray origins, which is the coordinate set of the corresponding label data of the three-dimensional grid of a certain category (for example, the incisor 11). In a two-dimensional array of the same size as the CBCT image of a certain layer, the value of the corresponding position coordinates in the coordinate set is set to the value corresponding to the category. After setting the values for all categories, the two-dimensional array obtained is the annotation data in each layer of the CBCT image. As shown in FIG. 16A, the intersection points of the rays emitted by the ray origins on the XOY plane and the three-dimensional grid of the tooth are shown, and the lowermost part of the figure is the XOY plane. Each red straight line perpendicular to the XOY plane direction is a ray, and there are two intersection points (green intersection points in the figure) between each ray and the three-dimensional grid of the tooth. In some preferred embodiments, in order to obtain a more accurate contour, the ray detection method can also be used to obtain the tooth, tooth tissue and the corresponding tooth pixel point area of each category from the YOZ plane and the XOZ plane. The process is similar to the ray detection method based on the XOY plane, and is not described here. Figure 4 Figure 5 For the intersection points obtained by the ray detection method based on the XOY, XOZ and YOZ planes as the reference planes in this application, it can be seen that the annotation of the training samples in the neural network model by the ray detection method can realize full-automatic annotation, which can speed up the training process of the neural network model and reduce the waste of manpower.

[0048] In one example, the neural network model in step 102 can be a series of models with picture instance segmentation function, such as Mask-RCNN model, Point Rend model, mask scoring rcnn model, etc. In this embodiment, the Mask-RCNN model is specifically used, which does not directly segment the entire input picture, but first finds the area where the tooth to be identified is located through target recognition, and then segments the pixel points in this area into two categories (1. Tooth 2. Background), which can save computing resources.

[0049] ​Specifically, the Mask-RCNN model mainly includes the following network layers: a feature extraction layer, a region selection layer, an alignment network layer, and a classification layer, and the network layer structure is as shown in Figure 6 The feature extraction layer extracts features of each layer of the input image and outputs a feature map of each layer of the image. Specifically, five layers of features of each layer of the image can be extracted by using a deep residual network as a backbone network as a feature extractor, and the extracted features are fused by 1x1 convolution connection FPN feature pyramid; the five layers of features are the same features of the same layer of the image in different sizes, and the features of each layer in the five layers are the same, only the sizes of the images are different, and the different size images are obtained by convolution. Since the features of different sizes are fused, small objects can be detected in large size images, large objects can be detected in small size images, and more complete features can be extracted.

[0050] The region selection layer selects regions of candidate pixel points corresponding to each category of dental tissue in each layer of the input image and outputs the regions of candidate pixel points corresponding to each category of dental tissue in each layer of the image. Specifically, a region generation network RPN can be used to generate multiple candidate boxes for the feature map of each layer of the image after feature fusion, and the optimal candidate box, i.e., the region of candidate pixel points, in each layer of the image is selected.

[0051] The alignment network layer aligns the input regions of candidate pixel points corresponding to each category of dental tissue and outputs the aligned regions of candidate pixel points corresponding to each category of dental tissue. Specifically, the ROI Align layer can be used to interpolate the optimal candidate box region of the feature map of each layer of the image using bilinear interpolation to output a fixed-size optimal candidate box region. The sizes of the output optimal candidate box regions are the same. The bilinear interpolation is used here to enlarge the resolution while ensuring the integrity of the image, so that the results of the image processing operations including the alignment operation can be more accurate.

[0052] The classification layer classifies the aligned regions of candidate pixel points corresponding to each category of dental tissue and outputs the regions of dental pixel points corresponding to each category of dental tissue in each layer of the image. Specifically, convolution and classification network layers can be used to realize the classification of the regions of candidate pixel points. In the classification of the regions of candidate pixel points, a softmax function can be used to classify the pixel points in the regions of candidate pixel points. In the classification process, a loss function can be used to further reduce the classification error. Specifically, the use of the loss function to further reduce the classification error includes first calculating the error between the optimal candidate box and the box of the labeled data, using Smooth L1 as the loss function to calculate the error,

[0053]

[0054] where x is the numerical difference between the optimal candidate box and the labeled data; yi parameters of the annotation data frame, f(x i ) represents the parameters of the best candidate frame. At the same time, the error of the class of the best candidate frame and the class of the annotation data is calculated, and the cross entropy is used as the loss function for calculating the error:

[0055]

[0056] where M represents the number of classes; y ic is a symbol function of 0 or 1, taking 1 when the true class of sample i is equal to c, and 0 otherwise; p ic is the probability that the best candidate frame i belongs to class c. When segmenting the pixel points in the best candidate frame, only the foreground (tooth, alveolar bone) and background two parts are segmented out, and the loss function for calculating this part of the error is binary cross entropy, which is used to calculate the loss under two classification conditions:

[0057]

[0058] where y i represents the current class of the current segmented pixel point, foreground (1) or background (0), and p i represents the probability that the current pixel is divided into the foreground.

[0059] In one example, the neural network model further comprises: an optimization network layer, which predicts the pixel point region corresponding to each category of tooth of the input, and outputs the predicted finer tooth pixel point region corresponding to each category of tooth. Specifically, it can be realized by upsampling network Point Head, and the processing process of Point Head includes: performing two times bilinear interpolation upsampling on the small size segmentation feature map to obtain a large size coarse segmentation feature map; randomly picking N difficult points (points different from the surrounding points, object edge points) from the low layer feature, combining the difficult points with the coarse segmentation coordinates to form a combined feature; using a multi-layer perception MLP to calculate the combined feature to obtain a finer segmentation feature map; continue to perform the process of upsampling, picking difficult points, combining, and predicting on the finer large size segmentation feature map until the resolution is greater than the required resolution, and then output the result, so as to establish a more realistic tooth model.

[0060] After the CBCT multi-layer image is segmented by the neural network model, the teeth of different tooth positions and the corresponding tooth types (such as permanent teeth and deciduous teeth) can be obtained. By classifying the permanent teeth and deciduous teeth as different categories of teeth, each tooth in the mixed dentition can be quickly distinguished in the three-dimensional tooth model containing permanent teeth and deciduous teeth established.

[0061] In one example, step 102 can also be implemented by identifying the tooth pixel point region in each layer of image based on preset tooth tissue corresponding pixel point classification conditions; wherein the pixel point classification conditions include the pixel value of the pixel point; obtaining the tooth three-dimensional segmentation result by using a three-dimensional variable scale region fitting segmentation method according to the tooth pixel point region; and obtaining the tooth pixel point region corresponding to each type of tooth according to the tooth pixel point region and the tooth three-dimensional segmentation result. Specifically, the tooth pixel point region in each layer of image can be identified by obtaining the gray histogram of the image, calculating a best threshold value between the maximum inter-class variance and the minimum inter-class variance of the gray histogram according to the threshold value law, and segmenting the tooth region and the soft tissue region in the background image by the best threshold value. The tooth three-dimensional segmentation result obtained by using the three-dimensional variable scale region fitting segmentation method includes segmenting the tooth according to the energy function of each point in the image.

[0062] In step 103, the multi-layer images are stacked in sequence, and the three-dimensional point cloud data of the tooth pixel point region corresponding to each type of tooth tissue is obtained.

[0063] In one example, the multi-layer images can also be stacked in sequence, and the three-dimensional grid data of the tooth pixel point region corresponding to each type of tooth tissue is obtained. A set of CBCT multi-layer images can represent K CBCT cross-sectional images with m*n resolution, and the prediction results of the K CBCT cross-sectional images are stacked to become m*n*K voxel data, and each cube in the voxel grid represents a segmentation category. As shown in Figure 7 For example, as shown in Figure 8 each cube in the voxel grid represents a segmentation category, and the categories include 55 categories such as alveolar bone, permanent tooth, and deciduous tooth.

[0064] In step 104, a three-dimensional tooth model is established according to the three-dimensional point cloud data of the tooth pixel point region corresponding to each type of tooth using a three-dimensional reconstruction algorithm.

[0065] In some embodiments, a voxel-level reconstruction algorithm (Marching cubes) is used to judge each vertex of each voxel grid with an isosurface, to obtain whether each vertex is inside or outside the object, and finally the grid surface is formed between the external vertices and the internal vertices, the grid vertices are points on the connecting line between the internal and external points, the voxel information of each classified tooth is processed into a three-dimensional grid, and a three-dimensional model is established according to the three-dimensional grid.

[0066] In one example, after step 104, further comprising registering the crown part in the calibration tooth model (the calibration tooth model can be obtained by oral scanning) with the crown part in the three-dimensional tooth model; specifically, the registration comprises sampling the crown part in the calibration tooth model to obtain a first point cloud, sampling the vertices of the three-dimensional mesh to obtain a second point cloud, constantly finding the nearest corresponding points under the condition that the corresponding relationship of each point in the two point clouds is known, and then iteratively obtaining the optimal transformation matrix R and t, where R is a rotation matrix and t is a translation matrix. respectively represent the centroids of the first point cloud and the second point cloud, and let wherein, is the coordinate of the i-th point in the first point cloud, is the coordinate of the i-th point after being converted to the centroid coordinate, is the coordinate of the corresponding i-th point in the second point cloud, is the coordinate of the i-th point in the second point cloud after being converted to the centroid coordinate, i.e., both the first point cloud and the second point cloud are converted to the centroid coordinate, and let H is obtained by singular value decomposition (SVD) of the matrix, i.e., H = U∑V T and the optimal translation matrix: * = VU T and the optimal translation matrix: The two are used as the optimal transformation matrix, i.e., the first transformation matrix, and after the registration is completed by using the first transformation matrix, the crown part in the calibration tooth model is used to replace the crown part in the three-dimensional tooth model. Because the tooth model obtained by oral scanning is more true and more consistent with the actual structure of the target oral crown than the tooth model established by using the CBCT image in the crown part, the more accurate and true three-dimensional tooth model can be obtained by using the CBCT image to establish the tooth model while replacing the crown part of the tooth model obtained by oral scanning, as shown in Figure 9

[0067] In one example, the overall flow of using the above steps to establish a more accurate and true tooth model can be as shown in Figure 10 ​The shown. In the overall process, first, the CBCT image of the corresponding oral cavity to be modeled is input; then the input CBCT image is preprocessed, and the specific preprocessing method is the same as in the above embodiment; then the features of the preprocessed image are extracted, and the optimal anchor box (i.e., the optimal candidate box) is extracted from the image after feature extraction; then the image features in the optimal anchor box are extracted, and the classification operation (classification of the pixel points belonging to the teeth) and the upsampling segmentation operation (segmentation of the region belonging to the tooth category) are performed on the image features in the optimal anchor box, and finally the tooth pixel point region corresponding to each category of tooth tissue is obtained, including preprocessing image feature extraction, optimal anchor box extraction, optimal anchor box feature extraction, classification, and segmentation. These steps can be implemented through the neural network model in the above embodiment; after obtaining the tooth pixel point region corresponding to each category of tooth tissue, the voxel information of each category of tooth tissue is generated according to the tooth pixel point region corresponding to each category of tooth tissue, and then the surface reconstruction of the tooth grid is performed according to the voxel information. After the surface reconstruction is completed, the point cloud registration is performed on the crown part of the reconstructed tooth grid and the crown part of the tooth model obtained by the oral scanning, and then the crown part of the tooth grid is replaced with the crown part of the tooth model obtained by the oral scanning, and finally the replaced tooth grid is output. The tooth grid model obtained through the process of this embodiment has high accuracy and is closer to the real tooth tissue structure.

[0068] In this embodiment, instead of directly dividing the teeth on the complete CBCT image (i.e., the reconstructed CBCT image), each layer of the multi-layer image of the oral cavity collected by the CBCT technology is identified, and the tooth pixel point region corresponding to each category of tooth in each layer of image is identified. Since the teeth in each layer of CBCT image are divided, the division result is more accurate than directly dividing on the complete CBCT image. Finally, the multi-layer image is stacked to obtain complete three-dimensional data corresponding to each tooth; then a three-dimensional tooth model is established according to the three-dimensional data corresponding to each tooth. Because the division result is more accurate, the established three-dimensional model is also more accurate.

[0069] The step division of the above method is only for clarity, and in implementation, some steps can be combined into one step or some steps can be split and decomposed into multiple steps, as long as the same logical relationship is included, and all are within the protection scope of the patent. Irrelevant modifications or irrelevant designs can be added to the algorithm or process, but the core design of the algorithm and process is within the protection scope of the patent.

[0070] Another embodiment of the present application relates to a treatment plan designing method, comprising: establishing a three-dimensional tooth model according to the tooth model establishing method described above; and designing a treatment plan with the three-dimensional tooth model as an initial tooth model. In one example, a design movement can be designed according to the difference between the three-dimensional tooth model and a target tooth treatment model, and then a treatment plan in different stages can be designed according to the design movement.

[0071] Another embodiment of the present application also provides a treatment appliance manufacturing method, comprising: manufacturing a tooth treatment appliance according to the treatment plan designed by the treatment plan designing method described above. In one example, the required correction force of each tooth in each stage can be calculated according to the design movement of each tooth in each stage, the thickness of the inner and outer treatment layers of each treatment appliance at each tooth can be calculated according to the required correction force of each tooth in each stage, and each treatment appliance can be 3D printed.

[0072] Another embodiment of the present application also provides a treatment appliance, comprising: a tooth treatment appliance manufactured by the treatment appliance manufacturing method described above.

[0073] Another embodiment of the present application relates to an electronic device, such as Figure 11 as shown, comprising at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the tooth model establishing method described above.

[0074] The memory and the processor are connected in a bus mode, and the bus can include any number of interconnected buses and bridges, which connect various circuits of one or more processors and memories together. The bus can also connect various other circuits such as peripheral devices, voltage regulators and power management circuits, etc., which are well known in the art, and thus, will not be further described herein. The bus interface provides an interface between the bus and the transceiver. The transceiver can be one element or multiple elements, such as multiple receivers and transmitters, which provide units for communicating with various other devices on the transmission medium. The data processed by the processor is transmitted on the wireless medium through the antenna, and further, the antenna also receives data and transmits the data to the processor.

[0075] The processor is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interface, voltage regulation, power management and other control functions. And the memory can be used to store the data used by the processor in the execution of the operation.

[0076] Another embodiment of the present application relates to a computer readable storage medium storing a computer program. The computer program, when executed by a processor, implements the tooth model establishing method.

[0077] That is, those skilled in the art can understand that all or part of the steps in the methods of the above embodiments can be completed by programs instructing relevant hardware, the programs are stored in a storage medium, and the programs include a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media capable of storing program codes.

[0078] Those skilled in the art can understand that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes can be made in form and details without departing from the spirit and scope of the present application.

Claims

1. A dental model establishing method characterized by, The method comprises the following steps: obtaining multi-layer images of the oral cavity by CBCT technology; identifying the tooth pixel point region corresponding to each category of tooth tissue in each layer image; wherein the each category of tooth tissue at least includes each category of tooth body and alveolar bone; stacking the multi-layer images in turn and obtaining three-dimensional point cloud data of the tooth pixel point region corresponding to each category of tooth tissue; establishing a three-dimensional tooth model according to the three-dimensional point cloud data of the tooth pixel point region corresponding to each category of tooth tissue.

2. The dental model establishing method according to claim 1, characterized in that, After the three-dimensional tooth model is established according to the three-dimensional point cloud data of the tooth pixel point region corresponding to each category of tooth tissue, the method further comprises the following steps: registering the crown part in the calibration tooth model with the crown part in the three-dimensional tooth model; the calibration tooth model is obtained by oral scanning of the oral cavity; after registration is completed, replacing the crown part in the three-dimensional tooth model with the crown part in the calibration tooth model.

3. The dental model establishing method according to claim 2, characterized in that, The registration of the crown part in the calibration tooth model with the crown part in the three-dimensional tooth model comprises the following steps: sampling the crown part in the calibration tooth model to obtain a first point cloud, and sampling the crown part in the three-dimensional tooth model to obtain a second point cloud; using a first transformation matrix to register the first point cloud with the second point cloud.

4. The dental model establishing method according to claim 3, characterized in that, The first transformation matrix is calculated according to the matching relationship of the nearest neighbor points of each point in the first point cloud in the second point cloud by iterative searching.

5. The dental model establishing method according to claim 1, wherein, Before identifying the tooth pixel point region corresponding to each category of tooth tissue in each layer image, the method further comprises the following steps: preprocessing each layer image; wherein the preprocessing comprises adjusting the window width and window level of each layer image, and then normalizing the brightness data of each layer image after adjusting the window width and window level.

6. The dental model establishing method according to claim 1, wherein, The identification of the tooth pixel point region corresponding to each category of tooth tissue in each layer image is realized by a neural network model, and the neural network model is trained by a plurality of training samples obtained in advance. Each training sample comprises an input sample; the input sample comprises a plurality of layer images in which each category of tooth tissue corresponding region and each pixel point belonging to tooth tissue are labeled.

7. The dental model establishing method according to claim 6, characterized in that, The neural network model comprises a first neural network, a second neural network and a third neural network; the identification of the tooth pixel point region corresponding to each category of tooth tissue in each layer image comprises the following steps: inputting each layer image into the first neural network for feature extraction, and the first neural network outputs a feature map of each layer image; inputting the feature map of each layer image into the second neural network for region selection, and the second neural network outputs a candidate pixel point region corresponding to each category of tooth tissue in each layer image; aligning the candidate pixel point region corresponding to each category of tooth tissue, and outputting the aligned candidate pixel point region corresponding to each category of tooth tissue in each layer image; The aligned candidate pixel point region corresponding to each category of dental tissue is input into a third neural network for classification processing, and the third neural network outputs a tooth pixel point region corresponding to each category of dental tissue in each layer of image.

8. The dental model establishing method according to claim 7, characterized in that, The neural network model further comprises: A fourth neural network configured to perform prediction on the input pixel point region corresponding to each category of dental tissue, and output a predicted finer tooth pixel point region corresponding to each category of dental tissue.

9. The dental model establishing method according to claim 8, characterized in that, The first neural network, the second neural network, the third neural network, and the fourth neural network satisfy at least one of the following conditions: The first neural network comprises a deep residual network, the second neural network comprises a region generation network, the third neural network comprises a convolution and classification network, and the fourth neural network comprises an up-sampling network.

10. The dental model establishing method according to claim 6, wherein, The dental tissue to which each pixel point in each region belongs is obtained based on aligning the input sample with the corresponding three-dimensional dental mesh, and is obtained based on the aligned three-dimensional dental mesh using a ray detection method.

11. The dental model establishing method according to claim 10, characterized in that, The aligning of the input sample with the corresponding three-dimensional dental mesh and the labeling based on the aligned three-dimensional dental mesh using the ray detection method comprise: aligning the multi-layer image of the input sample with the corresponding three-dimensional dental mesh in the same coordinate system through a second transformation matrix; selecting a reference plane in the same coordinate system, selecting a plurality of different ray origins on the reference plane, and emitting a ray along a preset direction to obtain a set of intersection points of the ray passing through the three-dimensional mesh; and labeling the points in the set as tooth pixel point regions corresponding to the dental tissue in the multi-layer image.

12. The dental cast establishment method according to claim 1, characterized by, The identifying of the tooth pixel point region in each layer of image, comprises, identifying the tooth pixel point region in each layer of image based on a preset pixel point classification condition corresponding to the dental tissue; obtaining a three-dimensional segmentation result of teeth by using a three-dimensional variable scale region fitting segmentation method according to the tooth pixel point region; obtaining a tooth pixel point region corresponding to each category of tooth according to the tooth pixel point region and the three-dimensional segmentation result of teeth; wherein the pixel point classification condition comprises a pixel value of the pixel point.

13. The dental cast establishment method according to claim 1, wherein, The categories of each dental tissue at least include category information of alveolar bone and category information corresponding to each tooth body.

14. The dental cast establishment method according to claim 1, wherein, The category information corresponding to each tooth body at least distinguishes between permanent teeth and deciduous teeth in the tooth body.

15. The dental cast establishment method according to claim 1, wherein, The establishing of a three-dimensional tooth model according to the three-dimensional point cloud data of the tooth pixel point region corresponding to each category of tooth comprises: reconstructing the three-dimensional point cloud data corresponding to each category of tooth into the three-dimensional tooth model by using a three-dimensional reconstruction algorithm.

16. An electronic device, comprising: at least one processor; and a memory connected in communication with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the tooth model establishing method according to any one of claims 1 to 15.

17. A computer readable storage medium storing a computer program, wherein the computer program comprises program instructions configured to cause a processor to perform the method according to any one of claims 1 to 16. The computer program, when executed by a processor, implements the tooth model establishing method of any one of claims 1 to 15.

18. A method of orthodontic treatment plan design, characterized by, Comprising: establishing a three-dimensional tooth model according to the tooth model establishing method of any one of claims 1 to 15; designing a treatment plan with the three-dimensional tooth model as an initial tooth model.

19. A method of manufacturing an appliance, characterized by, Comprising: manufacturing a dental appliance according to the treatment plan designed by the treatment plan designing method of claim 18.

20. An appliance characterized by, Comprising: the dental appliance manufactured by the appliance manufacturing method of claim 19.

Images