Teeth group segmentation using neural networks

Abstract

To provide a computer-implemented method and system of digitally segmenting a teeth group in a digital model.SOLUTION: A method comprises: (2302) receiving a 3D digital model of patient scan data of at least a portion of a patient's dentition; (2304) generating a panoramic image from the 3D digital model of the patient's dentition; (2306) labeling, using a first trained neural network, the panoramic image, to provide a labeled panoramic image; (2308) mapping the labeled panoramic image to corresponding coarse digital surface triangle labels in the 3D digital model, to provide a labeled 3D digital model; and (2310) segmenting the labeled 3D digital model to provide a segmented 3D digital model.SELECTED DRAWING: Figure 25(a)

Description

[Technical field]

[0001] This application is a continuation of U.S. Utility Patent Application No. 16 / 778,4 filed on January 31, 2020. No. 06, and U.S. Utility Patent Application No. 17 / 14, filed January 4, 2021. No. 6,239,523, filed on Oct. 13, 2003. Priority to and the benefit of the present application is claimed based on US Pat. No. 6,239,523, the disclosure of which is incorporated herein by reference in its entirety. is incorporated by reference herein for all purposes. [Background technology]

[0002] A specialized dental laboratory typically creates dental prostheses based on patient-specific instructions provided by a dentist. Use computer-aided design (CAD) to design .

[0003] In a typical workflow, a dental laboratory receives information about a patient's oral condition from a dentist. To use the CAD / CAM system, a digital model of the patient's dentition is created. Segment the individual teeth and gingival areas in the digital model. However, it may be desirable to label the teeth and identify the precise tooth structure. Finding the dental and gingival boundaries can be difficult. Traditional manual segmentation The approach is slow, cumbersome and can be error-prone. It may be less intrusive, but it is possible to obtain a clearer picture of the teeth and other areas of interest in the digital model. This may result in inaccurate and / or missing boundaries. Summary of the Invention

[0004] A computer-implemented method for digitally segmenting teeth in a digital model The 3D model of the patient scan data of at least a portion of the patient's dentition is receiving a 3D digital model; and generating a panoramic image from the 3D digital model. Then, the first trained neural network is used to extract one or more regions of the panoramic image. and providing a labeled panoramic image. One or more regions of the 3D digital model are then mapped to one or more corresponding coarse digital regions in the 3D digital model. and providing a labeled 3D digital model of the surface of the device. Segment the labeled 3D digital model and provide the segmented 3D digital model This includes providing.

[0005] The system for digitally segmenting teeth in a digital model includes a processor. and a computer including instructions executable by a processor to perform the steps. and a readable storage medium, the steps including: Receiving a 3D digital model and generating a panoramic image from the 3D digital model and using the first trained neural network to generate a plurality of images of the panoramic image. The present invention provides a method for labeling a number of regions, providing a labeled panoramic image, and Map one or more regions of the image to one or more corresponding coarse regions in the 3D digital model. Mapping digital surface triangle labels to provide a labeled 3D digital model and segmenting the labeled 3D digital model. and providing a rule.

[0006] storing executable computer program instructions for segmenting the digital model; A non-transitory computer readable medium having computer program instructions for performing a dental implantation procedure on a patient. receiving a 3D digital model of at least a portion of the patient scan data; Generating a panoramic image from a 3D model and a first trained neural network The method further comprises labeling one or more regions of a panoramic image using the labeling method, and generating a labeled panoramic image. and converting one or more regions of the labeled panoramic image into a 3D digital model. The label is then mapped to one or more corresponding coarse digital surface triangle labels in the data Providing a labeled 3D digital model and segmenting the labeled 3D digital model and providing a segmented 3D digital model.

[0007] The computer-implemented method for generating a panoramic image includes imaging a patient of at least a portion of the patient's dentition. Receiving a 2D depth map of the scanned data and using a trained neural network and generating one or more digital tooth bounds corresponding to one or more digital teeth. determining a boundary region for one or more digital tooth bounds by a spline; Connecting spline regions and sampling one or more spline points on the spline and extracting one or more sampled spline points from one or more sampled spline points. determining a number of sampled digital surface points; and for each sampled digital surface point, and determining one or more associated digital surface points corresponding to the selected feature. [Brief description of the drawings]

[0008] [Figure 1]FIG. 2 shows a perspective view of a digital model of teeth and gums. [Diagram 2] FIG. 1 illustrates a diagram of an example of a convolutional neural network in accordance with some embodiments. [Diagram 3] FIG. 1 illustrates an example diagram of at least a portion of a YOLO network according to some embodiments. [Figure 4(a)] FIG. 1 illustrates an example diagram of at least a portion of a U-Net network in some embodiments. [Figure 4(b)] FIG. 1 illustrates an example diagram of at least a portion of a U-Net network in some embodiments. [Figure 5(a)] FIG. 1 is a diagram of an example of a pyramidal attention network. [Figure 5(b)] FIG. 2 is a diagram of an example of a pyramid attention feature. [Figure 5(c)] FIG. 13 is a diagram of an example of a global attention upsample feature. [Figure 6(a)] FIG. 1 is a diagram of an example of a ResNet neural network. [Figure 6(b)] FIG. 1 is a diagram of an example of a portion of a ResNet neural network. [Figure 7] FIG. 2 illustrates an example of a panoramic image with overlaid diagrams showing columns and rows in some embodiments. [Figure 8] 1 shows an example image of a 2D depth map. [Figure 9] 1 shows an image of an example of a labeled 2D depth map. [Figure 10(a)] FIG. 1 shows an image of a digital dental arch having one or more digital teeth with illustrated bounding regions. [Figure 10(b)] FIG. 1 shows an image of a digital dental arch with one or more digital teeth with illustrated bounding regions and digital bounding box centers. [Figure 10(c)] FIG. 2 illustrates digital bounding box centers. [Figure 10(d)]1 shows one or more digital tooth bounding area centers connected to each other by splines. [Figure 11] FIG. 1 illustrates an example of a digital model with sampling points on a spline. [Figure 12(a)] FIG. 1 illustrates an example of a digital model with sample digital surface points that correspond to sampling points on a continuous smoothing spline. [Figure 12(b)] FIG. 1 illustrates an example of a digital model with additional sample points. [Figure 12(c)] FIG. 2 illustrates an example of a digital model having multiple rays extending from a sample depth point. [Figure 13] FIG. 2 illustrates an example of a panoramic image with illustrated rows and columns. [Figure 14] FIG. 2 is a diagram showing an example of a panoramic image having depth information. [Figure 15] FIG. 1 shows an example of a panoramic image with labeled digital tooth bounding regions. [Figure 16(a)] FIG. 1 illustrates an example of a panoramic image that may include one or more digital bounding regions. [Figure 16(b)] FIG. 1 illustrates an example of a panoramic image that may include one or more digital bounding regions. [Figure 17] FIG. 1 illustrates an example of some embodiments of mapping to a 3D digital model. [Figure 18(a)] FIG. 1 illustrates an example of identifying an uncertainty region. [Figure 18(b)] FIG. 1 illustrates an example of identifying an uncertainty region. [Figure 19] FIG. 13 shows an example of determining the curvature on an edge connecting two triangles. [Figure 20(a)] FIG. 13 illustrates an example of determining the average curvature of an edge in some embodiments. [Figure 20(b)] 13A-13C show examples of segments after merging triangles in some embodiments. [Figure 20(c)]FIG. 1 illustrates an example of a segmented digital surface. [Figure 21] 1 illustrates an example of some embodiments of a digital surface mesh that includes multiple segments according to some embodiments. [Figure 22(a)] FIG. 1 illustrates an example of two-point segmentation in some embodiments. [Figure 22(b)] FIG. 1 illustrates an example of two-point segmentation in some embodiments. [Figure 22(c)] FIG. 1 illustrates an example of two-point segmentation in some embodiments. [Figure 22(d)] FIG. 1 illustrates an example of two-point segmentation in some embodiments. [Figure 22(e)] FIG. 1 illustrates an example of two-point segmentation in some embodiments. [Figure 22(f)] FIG. 1 illustrates an example of two-point segmentation in some embodiments. [Figure 22(g)] FIG. 1 illustrates an example of two-point segmentation in some embodiments. [Figure 23(a)] FIG. 2 illustrates an example of a portion of a boundary loop in some embodiments. [Figure 23(b)] FIG. 2 illustrates an example of a vertex diagram of a dual graph. [Figure 24] FIG. 1 shows an example of a fully segmented and labeled set of digital teeth and gingival regions. [Figure 25(a)] FIG. 1 illustrates a flowchart of an example of a computer-implemented method for digitally segmenting teeth in a digital model in some embodiments. [Figure 25(b)] FIG. 2 illustrates a flowchart of an example of a computer-implemented method for generating a panoramic image in some embodiments. [Figure 26] 1 illustrates a diagram of an exemplary processing system in accordance with some embodiments. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] For purposes of this description, certain aspects, advantages and novel features of the disclosed embodiments have been described herein. The methods, apparatus, and systems of the present disclosure are in no way limiting. Instead, this disclosure should not be construed as construing an exhaustive list of the various disclosed embodiments. All novel and non-obvious features and aspects, both alone and in various combinations with one another, are intended to be included in the present invention. The methods, apparatus, and systems are covered in any particular combination or subcombination. The present disclosure is not limited to any aspect or feature or combination thereof. It is not believed that any particular advantage or problems are solved. Neither is it necessary.

[0010] Some operations of the embodiments of the present disclosure may be presented in a particular sequential order for convenience of presentation. Although the specific ordering is described in the Unless otherwise required, it is to be understood that this description encompasses rearrangements. The operations described below may be rearranged or performed simultaneously in some cases. Moreover, for simplicity, the accompanying drawings are not intended to be limiting unless the disclosed methods are used in conjunction with other methods. In addition, the description may at times not illustrate the various ways in which the disclosed methods may be implemented. To explain the The actual operations corresponding to these terms may vary depending on the particular implementation. It is easily recognizable by traders.

[0011]

[0005] In some embodiments, the computer-implemented method further comprises: Receiving a 3D digital model of the survey data and creating a panoramic image from the 3D digital model and generating a first image of the panoramic image using the first trained neural network. labeling the plurality of regions and providing a labeled panoramic image; Map one or more areas of a panoramic image to one or more counterparts in a 3D digital model. The corresponding rough digital surface triangles are then mapped to the labelled 3D digital model. and segmenting the labeled 3D digital model. Alternatively, in some embodiments, providing a digital model. The computer-implemented method includes receiving a panoramic image and applying a first trained neural network to the panoramic image. The network is used to label one or more regions of a panoramic image and extract the labeled regions. Providing a panoramic image and labeling one or more regions of the panoramic image in 3D Map to one or more corresponding coarse digital surface triangle labels in the digital model. and providing a labeled 3D digital model. and providing a segmented 3D digital model. Cut.

[0012]

[0005] In some embodiments, the computer-implemented method further comprises: The method may include receiving a 3D digital model of the survey data ("Digital Model"). FIG. 1 illustrates a method for detecting the presence of a marker in accordance with the present invention, including, but not limited to, optical scanning, CT scanning, etc. By scanning a physical impression using any scanning technique known in the art. 1 shows an example of a digital model 100 that can be used to create a digital model 100, e.g. , may also be generated by an intraoral scan of the patient's dentition. No. 201801329 to Nikolskiy et al., which is incorporated by reference herein. 82(A1). Conventional scanners typically make physical impressions during scanning. / Capture the shape of the patient's dentition in 3D and digitize the shape into a 3D digital model The digital model 100 may be, for example, a physical impression / topology corresponding to the shape of the patient's dentition. In some embodiments, the polygon may include a plurality of interconnected polygons according to In some embodiments, the scanning process may include more than one digital triangle. For example, F Dental restoration design software, such as astDesign™ dental design software; It can generate STL, PLY, or CTM files that can be used together do.

[0013] Neural Networks In some embodiments, the computer-implemented method includes using one or more trained neural networks. Neural networks are a part of machine learning. Neural networks are computational models that typically contain nodes organized in layers. A layer of nodes may include, for example, an input layer, one or more hidden layers, and an output layer. Neural networks with more than one hidden layer - usually many more - can be used. The network is a deep neural network ("DNN") Information from one layer can be processed and provided to the next layer.

[0014] In some embodiments, the DNN may include at least one of the hidden layers of a deep neural network. At least one network uses convolutions instead of the usual matrix multiplications , in a convolutional neural network ("CNN") A convolutional layer applies a kernel function to a subset of the values ​​from the previous layer. The output value can be calculated by the following: We can train a CNN by adjusting the weights of the kernel function based on The same kernel function can be used to compute each value in a particular convolutional layer. One advantage of using CNNs is that they involve learning fewer weights during training. Another advantage of using CNN is that it can detect edge features, e.g. This can be done.

[0015] FIG. 2 illustrates an example of a CNN in some embodiments. The CNN has a first convolutional layer The first convolutional layer 20 may include one or more convolutional layers, such as 202. 2 applies a kernel (also called a filter), such as kernel 204, to an input image, such as image 203. 2. Apply across the input image, and optionally apply an activation function, to the first kernel output 2 The first convolutional layer 2 may generate one or more convolutional outputs, such as 08. Kernel 202 may include one or more feature channels. 2, and optionally, the application of the activation function to a first convolution, such as the convolution output 206. The kernel can then be scaled to produce an output of the input image 203 based on the stride length. and proceeds to the next set of pixels in The kernel can then be applied to the input image 20, resulting in a second kernel output. This can be done until all pixels in the CNN are applied. , generating a first convolved image 206, which may include one or more feature channels. The first convolved image 206 may, in some embodiments, be a first convolved image such as 207. In some cases, the activation function may include, for example, For example, it can be a RELU activation function. Other types of activation functions can also be used. do.

[0016] The CNN also includes one or more pooling layers, such as a first pooling layer 212. The first pooling layer may include a filter, such as pooling filter 214, as a first Any type of filter can be applied to the convolved image 206. For example, the filter can be a maximum filter (which returns the maximum value of the pixels to which the filter is applied), or a mean filter, which outputs the average value of the filtered pixels One or more pooling layers can downsample the size of the input matrix. For example, the first pooling layer 212 can be a first pulley. by applying a pooling filter 214 to provide a first pooled image 216. The first convolved image 206 can be reduced / downsampled. The ring image 216 may include one or more feature channels 217. Optionally, one or more additional convolutional layers (and activation functions) and pools For example, the CNN can apply a second convolutional layer 218, and Optionally, an activation function may be applied to include one or more feature channels 219. The second pooling layer 222 can output a second convolved image 220 that can be A pooling filter is applied to the second convolved image 220 to filter one or more feature channels. A second pooling image 224 may be generated, which may include the following: One or more convolutional layers (and activation functions) and one or more corresponding processors The output of the CNN can optionally include one or more fully connected layers. The fully connected layer may be part of the merging layer 230. The synthesis layer can provide an output prediction, such as output prediction 224. The force predictions 224 can include, for example, labels for the teeth and surrounding tissue.

[0017] CNNs can be structured and used in different ways. For example, You Only Loo An example of a You Only Look Once (YOLO) network is detailed in the You Only Look Once: Uniform Look Once (UNIF ied, Real-Time Object Detection, by Joseph Redmon, Santosh Divvala, Ross Girshic k, Ali Farhadi, submitted 8 Jun 2015, last revised 9 May 2016, v5 The entire contents of this document are incorporated herein by reference. For more details, see YOLO9000: Better, Faster, Stronger, by Joseph Redmon, Ali Farhad i, University of Washington, Allen Institute for AI, published 25 Dec 2016, arXi v, the entirety of which is incorporated herein by reference. For more information on an example network, see YOLOv3: An Incremental Improvement, by Joseph Redmon and Ali Farhadi, University of Washington, published 2018, also explained in ArXiv The entire contents of this document are incorporated by reference herein. The network, for example, receives a 2D digital model of the patient's dentition and calculates the circumference of each digital tooth. A digital bounding box is defined as the bounding area of ​​the digital tooth. A digital model can be output.

[0018] The YOLOv3 network (hereinafter referred to as the "YOLO network") as described above , multiple bounding boxes, and predicting class probabilities for each bounding box. In some embodiments, the network may include one or more convolutional networks that measure the ,YOLO network can divide the input image into S x S grid. Each of the data cells can predict B bounding boxes. A confidence score can be determined for the bounding box. indicates the model's confidence that the box contains the object, and the accuracy of the predicted box. The trust can be expressed as follows:

[0019]

number

[0020] In some embodiments, YOLO divides dimension clusters into bounded clusters. These can be used as anchor boxes to predict the anchor box. For example, YOLO uses four coordinates for the bounding box: x , t y , t w , t h Predict The cell can be aligned from the top left corner of the image (C x ,C y ) offset, The prior bounding box is of width p w and height p h If you have The predictions can correspond to: b x =σ(t x )+c x b y =σ(t y )+c y

[0021]

number

[0022]

number

[0023] In some embodiments, YOLO uses one or more convolutional layers to perform feature extraction. One or more of the convolutional layers may optionally include a residual operation. Figure 3 shows an example of YOLO feature extraction. The following is performed in each layer: Here is an example of the operations that can be performed:

[0024] [Table 1]

[0025] Layer 310 may be performed twice, layer 314 may be performed eight times, layer 318 may be performed eight times, Layer 322 can be performed four times, making the total number of convolutions for the entire network 53 convolutions. The avgpool can be global. Other configurations and Variations are also contemplated in the YOLO network. In some embodiments, the trained YOLO The LO network receives an image and provides bounding regions around each feature in the image. The features can be specified during training. YOLO training is done using the loss function The loss function can include minimizing the probability that the object is within a particular grid cell. When a particular predictor is a gradient, it only penalizes the classification error. If the bounding box coordinates are incorrect, the penalty is 100%. For example, if a particular predictor is the best of all predictors in a particular grid cell, This is the case when the IOU is high.

[0026] In some embodiments, the computer-implemented method comprises generating a semantic segmentation network. A semantic segmentation network can be used. It can be any neural network that provides a segmentation by segments. Some examples of semantic segmentation networks are: convNet, FPN, DeepLabV3, U-Net, and Pyramid Attenuation Examples of such networks include

[0027] In some embodiments, the computer-implemented method may, for example, include: It can be used as a segmentation network. U-Net is a biomedical image It is a convolutional neural network that can be used for segmentation. , U-Net: Convolutional Networks for Biomedical Image Segmentation, by Olaf Ronne berger, Philipp Fischer, and Thomas Brox, Computer Science Department and BIOSS Center for Biological Signalling Studies, University of Freiburg, Germany, arXiv , 18 May 2015, the entirety of which is incorporated herein by reference. A standard CNN typically consists of one or more convolutional / progressive layers, known as contraction layers. The U-Net architecture includes one or more convolution / pooling layers. A contraction layer can be combined with one or more convolution / upsampling layers. Therefore, the U-Net architecture can increase the resolution output. The method combines high-resolution features from the contraction path with the upsampled output. This can be achieved by using the U-Net architecture, which also uses context information. It may include multiple feature channels to provide information to higher resolution layers. In some configurations, the expansion path can be symmetrical to the contraction path, resulting in a U-shaped arch. Brings architecture.

[0028] FIG. 4(a) shows a cross-sectional view of a deflation path 402 and an expansion path 404 in some embodiments. The contraction path 402 includes a first convolutional layer 406, etc. , may include one or more convolution layers. Each convolution layer may include a first convolution 4 One or more convolutions can be performed, such as 10. In some embodiments, for example, a 3×3 kernel may be used. The loop can be used to perform one or more convolutions. Therefore, each convolutional layer can perform two convolutions.

[0029] In some embodiments, each convolution can generate a feature map, and each feature map can be In some embodiments, the shrinkage path may include one or more feature channels. Each convolution in a convolutional layer can maintain the same number of feature channels. The network is based on the image resolution and the number of feature channels of the input image, n. In the convolution layer 406, the first convolution 410 is the A first convolution can provide an n-channel feature map, such as feature map 411, The feature map 413 can provide an n-channel feature map such as the final feature map 412. Both the first feature map 411 and the last feature map 412 contain n channels. In some embodiments, each convolution may not be padded. Depending on the implementation, one or more convolutions can be padded. After each of the several convolutions, an activation function such as ReLu or other known methods in the art are used. Other activation functions known in the art can follow.

[0030] The U-Net CNN also performs pooling operations, such as the first pooling operation 408. In some embodiments, the pooling operation may include: The pooling operation may be, in some embodiments, This is performed on the final feature map generated after the last convolution in a given convolutional layer. For example, in FIG. 4(a), the first pooling operation 408 can be performed on the last feature The result can be fed to the next convolutional layer 416. The first pooling operation 408 can be any known technique in the art. The pooling operation may be any known pooling operation. For example, in some embodiments, the first pooling operation may be The pooling operation 408 can be a max pooling operation. The first pooling operation 408 can be a mean pooling operation. The pooling operation can be any value. For example, in some embodiments, the pooling operation is It can be a 2×2 pooling operation with a stride of

[0031] In some embodiments, each pooling operation is optionally followed by additional convolutional layers and For example, a first pooling operation 408 can be followed by a second pooling operation. This can be followed by a convolutional layer 416, followed by a third convolutional layer 418, and a fourth Convolutional layers 427 may follow, with a pooling operation between each convolutional layer. In some configurations, it may be possible to double the number of feature channels per convolutional layer in the contraction pathway 402. For example, the second convolutional layer 416 can increase the number of feature channels from the first convolutional layer 400. 6, and the third convolutional layer 418 increases the number of feature channels to The number of convolutional layers can be doubled to two for the first convolutional layer 416, and the fourth convolutional layer 427 can be The number of feature channels can be doubled in the third convolutional layer 418. , 2*n features are generated for each feature map provided by each convolution layer in the contraction path 402. For example, the first convolutional layer 406 may provide a It can receive an input image with two channels and perform one or more convolutions. After the second convolution, a feature map with 64 channels can be created. The convolution layer 416 has 128 channels after performing one or more convolutions. The third convolutional layer 418 can provide a feature map that is After performing the convolution, a feature map having 256 channels can be provided, The fourth convolutional layer 418 can provide a feature map with 512 channels. The dimensions of the image can be reduced with each convolution. For example, the input image is 572x57 The first convolution 410 has a resolution of 570×570. The second convolution can provide a feature map 411 having a 568×568 A final feature map 412 having resolution can be provided. Each pooling operation between convolutional layers can reduce the resolution by a factor of two. For example, The first pooling operation 408 produces a final feature map 4 with a resolution of 568×568. 12 can be reduced to a convolutional layer input 414 having a resolution of 284×284. .

[0032] The U-Net CNN performs the final pooling operation 422 in the contraction path 402 as the final contraction path folding. The U-Net CNN can be implemented using a convolutional layer 426. In some embodiments, one or more convolutions may be performed. The input to the convolution layer 426 has a resolution of 32x32 with 1024 feature channels. The output from the convolutional layer 426 is a 28× It can be a feature map with a resolution of 28.

[0033] The U-Net CNN performs upsampling of the feature maps in the dilation path 404. The dilation path 404 may include one or more layers, such as an upsampling layer 425. Each upsampling layer can include upsampling layers of feature channels. For example, upsampling 424 can reduce the number of layers by half from the convolutional layer 426. We halve the number of feature channels for these final feature maps to provide feature map 428. The upsampling can be, for example, a 2x2 deconvolution. Each upsampling layer concatenates the cropped feature maps from the contraction path. For example, the upsampling layer 425 can be a cropped image from the contraction path 424. for concatenating the extracted feature map 430 to the upsampled feature map 428. A concatenation operation 429 can be performed. Each upsampling is a multiplication of the previous upsampling. It is possible to double the resolution of the feature maps from the filtering layer, e.g., by upsampling 424 doubles the feature map resolution from the convolutional layer 426 to provide feature map 428. As an example, the final feature map from the convolutional layer 426 may be 28x28. In some cases, the feature map 428 may be 56x56 after upsampling 424. Each upsampling layer can also perform one or more convolutions. Each convolution can be followed by an activation function such as ReLU or any other function known in the art. Any other activation function known in the art can be followed in the upsampling layers. The convolution can be, for example, a 3x3 convolution. U-Net CNN is a 1x1 convolution. The convolution can be performed as the last convolution in the final upsampling layer. For example, the final upsampling layer 432 may, in some embodiments, be, for example, an output segment A final convolution 434 can be included to provide a convolution map 435. In some implementations, the output segmentation map 435 may be, for example, a 388×388 resolution. It may have a resolution of 100 MHz and may include two channels.

[0034] FIG. 4(b) shows another embodiment of the U-net neural network 4000. The Net neural network 4000 receives an input image, such as input image 4002, One or more convolutions can be performed in one or more convolutional layers. In some embodiments, the U-net neural network may generate a contraction path 4006, etc. The input image may include a contraction path and a dilation path, such as the dilation path 4021. The image may be of any resolution and may contain any number of channels. In this example, the input image 4002 has an input image resolution of 512×512 with n channels. ("N x 512 x 512"). In some embodiments, Each downward arrow in the figure, such as convolutional layer 4004, represents a Lea layer as known in the art. 4x4 fold with kyReLU activation function and regularization such as InstanceNorm In some embodiments, the convolutional layer can include a pool of In some embodiments, the deconvolution layer 4 in the expansion path 4021 may also include a ring. Each upward arrow, such as 032, represents a ReLU activation function and 4x4 deconvolution (stride 2x2, parity) with normalization such as InstanceNorm In some embodiments, the convolutional layer may include upsampling. A ring may also be included.

[0035] In some embodiments, each convolution layer in the contraction path 4006 reduces the resolution of its input image by half. For example, the first convolutional layer 4004 receives an input image 4002 and Take 128 pixels with a resolution of 128x256x256 (128 pixels with an image resolution of 256x256) The first feature map 4008 may have one or more channels. In some embodiments, the output feature map after each convolutional layer is input to another convolutional layer. For example, the first feature map 400 output from the first convolutional layer 4002 can be 8 can be input to the second convolutional layer 4005, which has a 245×128 A second feature map 4010 having a resolution of 1×128 may be provided. The feature map 4010 can be input to a third convolutional layer 4007, which It can output a third feature map 4012 having a resolution of 512x64x64. The third feature map 4012 can be input to a fourth convolutional layer 4009, The convolutional layer outputs a fourth feature map 4014 with a resolution of 1024×32×32. The fourth feature map 4014 can be input to the fifth convolutional layer 4011. The convolutional layer can generate a fifth feature map 401 having a resolution of 1024×16×16. The fifth feature map 4016 is input to the sixth convolutional layer 4013, which outputs the fifth feature map 4016. The convolutional layer can output a sixth feature map having a resolution of 1024×8×8. The sixth feature map 4018 can be output by the seventh convolutional layer 40 15, and the convolutional layer has a resolution of 1024 x 4 x 4. A feature map 4020 can be output.

[0036] In some embodiments, each deconvolution layer in the dilation path 4021 scales the resolution of its input image. In some embodiments, one or more deconvolution layers may be After convolution, the trimmed feature maps from the contraction paths can be concatenated. Depending on the model, the output feature maps after each deconvolution layer can be fed into another deconvolution layer. can.

[0037] For example, the first deconvolution layer 4032 receives the feature map 4020 and performs deconvolution. Then, the sixth feature map 4018 is concatenated (4030) to obtain a resolution of 2048×8×8. (2048 channels with an image resolution of 8x8) The second deconvolution layer 4033 can output a convolved feature map 4038. Receive the first deconvolution feature map 4038, perform deconvolution, and generate a fifth feature map The second deconvolved feature map is created by concatenating 4016 and has a resolution of 2048x16x16. The third deconvolution layer 4031 can output the second deconvolution feature 4036. Receive the feature map 4036, perform deconvolution, and concatenate the fourth feature map 4014. , and output a third deconvolved feature map 4034 having a resolution of 2048×32×32. The fourth deconvolution layer 4039 can be used to deconvolve the third deconvolution feature map 4034. , perform deconvolution, and concatenate the fifth feature map 4012, 4024×64×64 resolution. The fifth deconvolution layer 4037 receives the fourth deconvolution feature map 4042. 5, the fifth feature map 4010 can be concatenated, and deconvolution can be performed. and outputting a fifth deconvolved feature map 4040 having a resolution of 128×128. The sixth deconvolution layer 4035 receives the fifth deconvolution feature map 4040. Then, the fifth feature map 4008 can be concatenated, and the 256× It is possible to output a sixth deconvolution feature map 4048 having a resolution of 256×256. The seventh deconvolution layer 4044 receives the sixth deconvolution feature map 4048. The seventh pixel has a resolution of 1x512x512 and can perform convolution. The deconvolution feature map 4050 can be output.

[0038] In some embodiments, each deconvolution layer may be preceded by upsampling.

[0039] In some embodiments, training of the U-Net CNN involves pixel-by-pixel training across the feature maps. We can determine the energy function by softmax of Combine it with a loss function, where softmax is:

[0040]

number

[0041]

number

[0042] In some embodiments, the semantic segmentation network is a pyramidal ensemble. It is a Pyramid Attention Network (PAN) An example of PAN is Pyramid Attention Network for Semantic Segmentation by Han chao Li, Rengfei Xiong, Jie An and Lingxue Wang, arXiv:1805.10180v3, submitted N ov 25, 2018, the entirety of which is incorporated herein by reference. Other PAN architectures can be used. PANs increase the receptive field and In order to effectively recover the pixel localization details, the encoder and decoder are constructed using a feature pyramid. Feature Pyramid Attention ("FPA") features and global attention Includes Global Attention Upsample ("GAU") features In some embodiments, the GAU may, for example, select a discriminative multi-resolution feature representation. The decoder branch of the PAN may be part of the decoder branch for decoding the received signal.

[0043] FIG. 5(a) illustrates an example of a PAN in accordance with some embodiments. The PAN is implemented by an encoder 500. In some embodiments, the encoder may include an input image 5001, Any input image can be received. The encoder can include one or more stages, Each stage may, for example, be a layer that can perform a downsampling operation on the input data. During encoding, each stage generates a feature map of a different level. We can provide an array (tensor) of constituent features. Each stage corresponds to a feature layer. One or more output feature maps may be provided. In some embodiments, the FPA The feature can be between the encoder and the decoder. In some embodiments, the decoder is e.g. For example, it may contain 32 channels.

[0044] In some embodiments, the encoder may be any kind of ResNet architecture. An example of a ResNet architecture is ResNets, by Pablo Ruiz-Harvard University RSITY, August 2018, the entirety of which is incorporated herein by reference. In some embodiments, the encoder can be a ResNet-101 encoder. In some embodiments, different stages of the ResNet-101 encoder are For example, in ResNet-101, for example, C onv-1 5004 can output 64 channel feature maps, Res-2 5 The 008 can output 256 channel feature maps, while the Res-3 5010 can output 51 It can output 2-channel feature maps, while Res-4 5012 has 1024 channels It can output feature maps, and Res-5 5014 is a 2048 channel feature map. can be output.

[0045] In a preferred embodiment, the encoder can be a ResNet-34 encoder. In some implementations, the ResNet-34 encoder has a convolutional stage Conv-1 of 5004, and followed by Res-2 5008, Res-3 5010, and Res-4 5012 , and one or more stages such as Res-5 5014, In the ResNet-34 encoder, for example, Conv-1 500 4 can output 64 channel feature maps, while Res-2 5008 can output 64 channel feature maps. Res-3 5010 can output 128 channel feature map. Res-4 5012 can output 256 channel feature maps. and the Res-5 5014 can output a 512 channel feature map.

[0046] FIG. 6(a) shows an example of a ResNet-34 neural network. Input image 60 02 can be provided to the first ResNet convolutional layer Conv-1 6004. In some embodiments, the first ResNet convolutional layer has 64 channels and 2 This can be a 7x7 convolution with trides. In some embodiments, this is followed by This can be followed by a pooling step 6006 with a stride of 2. Some embodiments of the neural network may, for example, be a 3×3 fold network. Four adders can each perform two convolutions, bypassing the input every two convolutions. It can contain additional ResNet stages, or layers. The image resolution, as well as the number of channels (feature map dimensions) remain the same within each stage. can be done.

[0047] Figure 6(a) shows five stages: Conv-1 6004, Res-2 6010, Res- 3 6012, Res-4 6014, and Res-5 6016 Here is an example of the ResNet-34 neural network. Res-2 6010, Re Each block in the S-3 6012, Res-4 6014, and Res-5 6016 stages Each block represents a pair of 3×3 convolutions performed in series (i.e., the output of the first convolution is (This is the input for the second convolution in each box.) For example, the first block 6018 6020 and 6021 are a first Res-2 convolution and a second Res-2 convolution performed in series. In some embodiments, one or more ResNet stages may include a The first convolution in the first block of can increase its stride from 1 to 2. For example, the first Res-3 convolution 6024 in stage Res-3 6012 The first Res-4 convolution 6026 in stage 4 6014, and the second Res-5 convolution 6016 in stage The first Res-5 convolutions 6027 can each use a stride of 2, and the remaining Convolutions use a stride of 1. In some embodiments, the ResNet stage is bypassed. In bypass, the input of the first block is connected to the output of the first block. The sum is added to the input of the next block. For example, As shown, the input to the first block 6018 is a first input via a bypass 6028. This bypasses the block and is added to the output of the first block 6018. This sum is then This is provided as input to the next block 6040.

[0048] FIG. 6(b) shows an example of a bypass in more detail. A first block 6050 is a first block The input image 6052 may receive a three-dimensional input feature map 6052. 6054, such as a x3 convolution, and provides a first convolution output 6056. The first convolution output 6056 can receive a second convolution 6058, It can provide a block output 6060. The ResNet neural network is , via bypass 6062, the first block input feature map 6052 is The sum is then fed to the second block 6066 as the input feature matrix. This pattern can be provided as a step 6064 as illustrated in FIG. We can iterate through the entire ResNet stage like this:

[0049] Referring to Figure 6(a), the Conv-1 6004 stage is a 7x7, 64-channel, After performing tride 2 convolution, we can output an image size of 112x112. The Res-2 6010 stage consists of two 3x3, 64-channel convolutions in each block. It contains three blocks and can output an image size of 56x56. Res-3 6012 stages each perform two 3x3, 128 channel convolutions. It contains four blocks and can output an image size of 28 x 28. The s-4 6014 stage performs two 3x3, 256-channel convolutions in each block. It contains six blocks and can output an image size of 14x14. 5 6016 stages, each block performs two 3 × 3, 512 channel convolutions. It contains three blocks and can output an image size of 7x7. In some embodiments, the filtering stage 6029 may be implemented using average pooling, 1000-d fc, soft It can perform max and output image size 1x1.

[0050] As illustrated in FIG. 5(a), the output of the ResNet stage may be, in some embodiments, In addition, the output of a particular stage can also be input to the GAU. For example, the output of Res-2 5008 can be input to Res-3 5010. The output of the Res-3 5010 can also be input to the GAU-3 5016. It can be input to ES-4 5012 and also to GAU-2 5018. The output of Res-4 5012 is input to Res-5 5014, and the output of GAU-1 In some embodiments, the output characteristic of the last stage of the encoder may also be input to 5020. The feature map can be input to the FPA. For example, the output feature map of Res-5 5014 is The resulting data can be input into the FPA5022.

[0051] In some embodiments, the FPA uses a U-shaped structure to form three pyramids. In some embodiments, the FPA can combine features from different scales. We apply the context features of Kale to a fully convolutional network. ) ("FCN") based pixel prediction framework. FIG. 5B shows an example of an FPA according to some embodiments. The FPA 5030 may, for example, Receive feature maps 5034 from an encoder network, such as Resnet 5032 The resolution of the feature map may be, for example, 32×32 in some embodiments. In some embodiments, the FPA may include downsampling and subsequent For example, the FPA5030 can include downsampling. The signal processing branch 5038 may include an upsampling branch 5040. In some embodiments, the downsampling branch 5038 performs a 7×7 convolution. The first downsample convolution 5048 performs a 5×5 convolution. Then, a second downsample convolution 5050 and a 3×3 convolution are performed. 5052, which may include one or more convolutions, such as a third downsample convolution 5052. It can contain layers. Higher level feature maps have lower resolution and therefore larger clusters. In some embodiments, the pyramid structure is At each step, differently scaled information can be integrated, e.g. However, this allows for more precise incorporation of adjacent scales of context features. The output of the unsampled convolution is fed into another convolution layer with the same filter size. For example, the output of the third downsampled convolution 5052 is a 3x3 convolution. 5054, and the output of the second downsampled convolution 5050 is a 5×5 The output of the first downsampled convolution 5048 can be input to the convolution 5056. can be input to a 7x7 convolution 5058. The ring branches can include increasing feature map resolutions. For example, in the embodiment In some embodiments, the resolution is 4×4 in the first upsample branch stage 5042. and in the second upsample branch stage 5044 it can be 8×8. and in the third upsample branch stage 5046 it can be 32×32. Each upsampling branch may, for example, be , connections. The original features from the convolutional network can be, for example, a 1 × 1 convolution. After passing through a convolutional layer, such as convolutional layer 5072, which may perform convolution, For example, it may be multiplied pixel by pixel in the pixel by pixel multiplier 5070. One or more advantages of the structure include, for example, at least the ability to extract contextual information at different scales. We fuse the information from the two images to create pixel-level attention for high-level feature maps. and

[0052] In some embodiments, the FPA may include global pooling 5060, 1×1 convolution 50 62, and upsampling 5064. This may include a global pooling branch such as branch 5036.

[0053] The filters and resolution sizes provided are examples only. The image size can be any suitable value. The pulling branch may include more or fewer stages in some embodiments. In some embodiments, batch normalization is performed after every convolutional layer.

[0054] In some embodiments, the FPA may include a decoder, such as the decoder 5074 shown in FIG. As shown in the figure, in some embodiments, the decoder may provide input to Also, for example, GAU-1 5020, GAU-2 5018, and GAU-3 50 16. Therefore, the FPA 5022 transmits its output to the decoder 5074 and to the GAU-1 50 20. The GAU-1 5020 also supports Res-4 In some embodiments, the input may be from a ResNet layer such as 5012. ,GAU-2 5018 is, for example,,a ,corresponding ,ResNet, ,such ,as ,Res-3 5010. 5076. The decoder 5076 may receive inputs from the first decoder stage 5076 and from the second decoder stage 5078. In some embodiments, GAU-3 can be implemented using Res-2 50 as described above. 08 and from a second decoder stage 5078. In some embodiments, a third decoder stage 5080 may route the output of GAU-3 5016 to a second decoder stage 5082. 5078. In some embodiments, each decoder stage may sum its input The sum of forces can be carried out.

[0055] In some embodiments, each GAU may, for example, assist in selecting the category location specific details. Global context that can provide a global context for low-level features that can be FIG. 5(c) shows the GAU501. 5 shows an example of a GAU, such as 501.00. In some embodiments, the GAU may be implemented using low-level features 501. For one or more low-level features, e.g., 50102, convolution In some embodiments, the convolution layer 50102 may be, for example, For example, a 3 × 3 convolutional filter can reduce the channels of a feature map from a CNN. In some embodiments, high level features such as high level features 50104 may be used. Pool the global context from the signature in global pooling 50106 The global pooling 50106 can be used to, for example, generate feature maps 50108. In some embodiments, one or more feature maps may be provided. Chip 50108 may have, for example, a resolution of C×1×1, where C is the chip size. In some embodiments, the GAU is multiplied by the low-level features. From high-level features through convolution with batch normalization and ReLU nonlinearity that can be A global context can be generated. For example, in some embodiments, Layer 50110 provides a 1×1 convolution with batch normalization and ReLU nonlinearity. The convolution can be performed by multiplying the convolution layer 50102 with the convolution layer 50112. These low-level features may be multiplied together to produce one or more low-level features, such as weighted low-level features 50114. In some embodiments, the high level features are weighted. For example, in some embodiments, the high-level features 50104 to the weighted low-level features 50114 in the sum 50116. One or more advantages of such a GAU are, for example, the ability to scale feature vectors at different scales. This involves developing a map to provide guidance information to low-level feature maps using high-level features. In some embodiments, the PAN may provide a segmented image 5075. This can be done.

[0056] In some embodiments, ResNet uses, for example, a dilated convolutional method to extract feature maps. For example, in some embodiments, applies a dilation convolution rate of 2 to the res5b block. This is the case, for example, for ResN The output size from et can result in a feature map that is 1 / 16 of the input image. In the case of the ResNet-101 and ResNet-34 implementations, 7x7 convolutions The layer can be replaced, for example, with three 3×3 convolutional layers. Training can include poly learning rates as known in the art. For example, the initial rate can be set to (1-iter / max_iter) power Multiplying by Here, the power is 0.9 and the initial rate is 4e-3. For this example, the network was trained with a batch size of 16, momentum of 0.9, and weight decay of 0. Mini-batch stochastic gradient descent with 0001 (SG In some embodiments, the training can be done using, for example, The cross-entropy error at each pixel can be applied as a loss function. Between, for example, random left-right flips for all data sets and values ​​between 0.5 and 2 Random scaling can be used. In some embodiments, particularly the 3×3 kernel For the size of the convolution, mean pooling is preferable to max pooling. In some embodiments, the stride can be, for example, 16. One or more advantages of using N may include, for example, improved performance.

[0057] In some embodiments, the computer-implemented method further comprises: Implementing, including and / or carrying out one or more features of the In some embodiments, the computer-implemented method includes using one or more neural networks. A particular value can be used to train one or more neural networks. Although one or more features in are discussed, values ​​are provided as examples only. These are merely examples. Other suitable values ​​and variations may be used.

[0058] Panoramic Images In some embodiments, the computer-implemented method uses a trained neural network. Panoramic images can be generated from 3D digital models using a For example, it can be a 2D raster image containing one or more columns and one or more rows. In some embodiments, the computer-implemented method includes: Panoramic images can be advantageously used to, for example, capture more detail within a digital model. Panoramic images can be advantageously used to show more digital dentition from more sides, e.g. For example, a single image showing both the lingual and buccal gingival-crown margins of all teeth. FIG. 7 illustrates the overlapping of columns and rows, such as row 5102 and column 5104. 7 shows an example of a panoramic image with views of the same area. The number of columns and rows shown in FIG. 7 is for illustration purposes only. In some embodiments, the desired number of columns and rows can be specified in a configuration file or Add graphical user interface elements such as input fields for the values ​​of In some embodiments, the computer-implemented method may include, for example, The desired number of columns and rows can be loaded from a configuration file. The resolution of the panoramic image can be increased by increasing the number of columns, e.g. If the number of rows is set to, say, 320, then the performance A normal image can be 2048x320 (width x height). This could mean taking 2048 samples on the spline. Smaller dimensions can be processed faster, while larger dimensions can be processed better. One advantage is that it allows the user to choose between speed and spatial resolution. It can include being able to do.

[0059] In some embodiments, the neural network may be configured to recognize each digital tooth as, for example, a rectangular shape. One or more teeth that are labeled with labeled digital tooth bounding areas, such as the dentition boundary. One or more 2D images of at least a portion of a digital dental arch having a plurality of digital teeth. By providing a 2D depth map training dataset that can include depth maps It is possible to train the bounding area of ​​the digital tooth using other shapes. Some embodiments of the computer-implemented method optionally include:

[0005] In some embodiments, generating a 2D image from the digital model may include: The 2D image can be a 2D depth map. A 2D depth map has, within each pixel, It contains a 2D image that contains the distance from the orthographic camera to the object along a line that passes through the xcel. For example, the object may be, in some embodiments, a surface of a digital jaw model. In some embodiments, the input can be a 3D representation of the patient's dentition, e.g., jaw. The object, such as a digital model ("digital model"), and the camera orientation may be included. In some embodiments, the camera orientation can be determined based on the bite direction. The orientation is perpendicular to the occlusal plane, which can be adjusted using any technique known in the art. For example, one technique is AN AUTOMATIC IC AND ROBUST ALGORITHM OF REESTABLISHMENT OF DIGITAL DENTAL OCCLUSION, by Yu-Bi ng Chang, James J. Xia, Jaime Gateno, Zixiang Xiong, Fellow, IEEE, Xiaobo Zhou, and Stephen TC Wong in IEEE TRANSACTIONS ON MEDICAL IMAGING, VOL. 29, NO. 9, S EPTEMBER 2010, the entirety of which is incorporated herein by reference. Alternatively, in some embodiments, the occlusal orientation may be, for example, as described herein. As will be explained, the user uses an input device such as a mouse or a touch screen to The designation may be specified by manipulating a digital model on a display. The occlusal direction is, for example, as described in Nikolskiy et al., PROCESSING DIGITAL DENTAL IMPRESSION, USA Using the bite axis method described in patent application 16 / 451,968 No. 6,399,945, filed on Dec. 23, 2003, the entirety of which is incorporated herein by reference.

[0060] 2D depth maps are known in the art, including, for example, z-buffer or ray tracing methods. The image may be generated using any technique known in the art. For example, in some embodiments, The computer-implemented method may, for example, determine the depth of each pixel (j,k) to the maximum length and the pixel The color can be initialized to the background color. For each pixel in the projection of the polygon onto any digital surface, corresponding to pixel (j,k) Then we can determine the depth z of the polygon at (x,y) where z<pixel(j, k), then set the pixel depth to depth z. "z" is Although it is possible to point out the convention that the central visual axis of a camera is in the direction of the camera's z-axis, this is not necessarily the case. In some embodiments, the z-axis of the scene may not necessarily refer to the absolute z-axis of the scene. The method can also, for example, set the pixel color to something other than the background color. In some embodiments, the polygons can be, for example, digital triangles. Depending on the model, the depth in the map can be per pixel. 1 illustrates an example of a 2D depth map of a digital model in accordance with some embodiments.

[0061] To generate the training dataset, we first compute the depth of each 2D depth map in the training dataset. The digital teeth can be labeled by digital tooth bounding regions. An example of a 2D depth map 700 with labeled digital tooth bounding regions 702 is shown in FIG. Shown are labeled digital tooth bounding regions for each digital tooth in the 2D depth map. In some embodiments, the process may be performed manually by a user or technician, or automatically. In some embodiments, the computer-implemented method may include a 2D depth map. The neural network is fed a training set of 2D depth maps. It is possible to provide a network. 2D depth map trained neural network In some embodiments, the method further comprises receiving one or more 2D depth maps, and generating a 2D depth map. It is possible to automatically generate digital tooth bounding regions for each digital tooth in the In some embodiments, the computer-implemented method includes configuring the shape as a rectangle or a box. one or more digital tooth bounding regions each having a designated labeled digital tooth bounding region; The YOLO network can be trained using the D depth maps. The computer-implemented method includes: Each tooth bounding region is labeled and shaped to follow the overall shape of the tooth. One or more 2D depth maps are used to perform semantic segmentation. For example, in some embodiments, a computer-implemented method The method involves training a CNN, a U-Net CNN, or a PAN neural network. In some embodiments, the training data set may be, for example, 10,000 2D For example, in some embodiments, the depth map image may include any other suitable number of 2D The depth map images can also be used as a training data set.

[0062] After training, in some embodiments, the 2D depth map trained neural network: receiving one or more unlabeled 2D depth maps each having a digital dental arch; digital tooth bounding regions for each digital tooth within at least a portion of the digital dental arch; In some embodiments, the computer-implemented method may provide, for example, A trained neural network is used to generate digital tooth boundaries around each digital tooth. Each digital tooth bounding area can be roughly defined. It can provide a rough approximation of the position of each tooth when viewed from the direction. includes one or more digital tooth bounding regions around each digital tooth. An example of a digital model 800 with a digital dental arch with digital teeth is shown. Then, the bounding area of ​​the digital tooth is the bounding area of ​​the digital tooth as viewed from the occlusal direction 806. The bounding box is a digital bounding box 802 that bounds the object 804. Although box 802 is shown, each digital tooth bounding region may be The digital tooth may be of any suitable shape and / or size to define the boundaries of a particular digital tooth. It can be.

[0063] In some embodiments, the 2D depth map trained neural network may be 1 is a 2D depth map trained convolutional neural network according to an embodiment of the present invention. In other words, the 2D depth map trained CNN is a 2D depth map trained CNN as described above. A trained 2D depth map YOLO network. The 2D depth map may be received, and a 2D depth map may be generated. A digital tooth bounding area for each digital tooth may be provided. The data implementation method may, for example, in some embodiments, be based on a digital tooth bounding region. We can label all the bounded pixels with a unique label. The digital tooth bounding region provided by the depth map YOLO network is , for example, can be a digital tooth bounding box.

[0064] In some embodiments, the 2D depth map trained neural network In some embodiments, the present invention is a pre-trained semantic segmentation network. The 2D depth map trained semantic segmentation network is described above. In some embodiments, the 2D depth map trained U-Net can be used as is. The 2D depth map trained semantic segmentation network is described above. The PAN network is as shown in Fig. 1. The stationary network can receive a 2D depth map and compute a small part of the 2D depth map. providing a digital tooth bounding region for each digital tooth within at least a portion of the The computer-implemented method may, for example, in some embodiments, include generating a digital tooth bow. Label all pixels bounded by the binding region with a single label. A pre-trained 2D depth map semantic segmentation network The digital tooth bounding region provided by the Therefore, the trained digital tooth can be shaped to follow the shape of the corresponding digital tooth. The 2D depth map semantic segmentation network performs pixel-by-pixel segmentation. mentation / labeling can be provided.

[0065] The labels may, for example, distinguish each tooth from the others and / or distinguish each tooth from the surrounding gums and teeth. For example, the YOLO network is faster to train and apply. However, a semantic segmentation network can provide It cannot provide pixel-by-pixel labeling. Therefore, YOLO trained on 2D depth maps The network may be suitable for generating panoramic images in some embodiments. However, for example, in some embodiments, a 2D depth map trained semantic segmenter may be used. The mentation network can also be used to generate panoramic images. In some embodiments, each digital tooth is identified based on its size and location in the digital dental arch. can be identified.

[0066] In some embodiments, the computer-implemented method further comprises: In some embodiments, one or more of the The method may connect one or more of the bounding area centers to one another. In some embodiments, the computer-implemented method further comprises: The center of the digital tooth bounding area can be determined, for example, in the embodiment Depending on the geometry, it may be the geometric center of the digital tooth bounding area. FIG. 8B shows the digital tooth bounding region of the digital tooth 804. An example of the digital bounding box 802 is shown in FIG. The bounding box center 808 can be determined. The center 808 may be, for example, in some embodiments, a center of gravity of the digital bounding box 802. FIG. 10(c) shows the digital bounding box 802. 10(a) , such as the digital bounding box center 808 of An example of 800 digital bounding box centers is shown.

[0067] In some embodiments, the computer-implemented method further comprises: The digital tooth bounding area center is passed through the digital tooth bounding area center. In some embodiments, the computer-implemented method is a one-to-one loop that passes through the spline so that it forms a smooth convex hull. A plurality of digital tooth bounding area centers may be selected. In this case, the computer-implemented method derives a spline from the entire tooth bounding region center. In some embodiments, the spline is a smooth spline. In some embodiments, adjacent digital tooth bounding regions may be adjacent to each other. Bordering. This means that the digital tooth bounding area is bordered by nearby or adjacent teeth. This may be because the splat between the centers of the two digital tooth bounding regions The in can be determined using any technique known in the art. For example, The preferred spline between the centers of the two digital tooth bounding regions is a cubic Hermitian spline. A cubic Hermite spline may be used, for example, in some embodiments. The positions of the centers of adjacent digital tooth bounding regions and the distance between the centers of the two digital tooth bounding regions are shown in Fig. It is possible to include the first derivative at each of the bounding region centers. The computer implemented method includes: providing one or more splines connected together to form a spline; The computer-implemented method can be spliced ​​using any technique known in the art. For example, in some embodiments, the spline may be Determine the tangent at the center point, and then, any two digital tooth bounding area centers can be determined by constructing the spline as a Bézier curve between In some embodiments, the digital tooth bounding region center is located within one adjacent neighboring digital In the case where only the digital tooth bounding region center is present, the computer-implemented method uses a straight line. The center of the adjacent digital tooth bounding area is not included in the side of the center of the adjacent digital tooth bounding area. The spline can be made continuous.

[0068] FIG. 10(d) shows a first boundary connected to each other by a first spline 812. One or more bounding box centers 808 and 810, etc. 1 shows an example of a plurality of digital tooth bounding region centers. The figure also shows a second splice. A second digital bounding box center 810 is then added to the third digital bounding box center 814. The figure also shows how the first vertex, which is missing one neighbor, connects to the center of the vertex box 816. In the case of the bounding box center 808, the computer-implemented method calculates a first spline 812 can be extended at line 824. Similarly, In the case of the bounding box center 822 of the The line 826 can be extended at a line 828. So we connect the centers of the digital bounding boxes in the neighborhood to form a continuous spline 820. As mentioned above, the spline can be a continuous smoothing spline. Cut.

[0069] In some embodiments, the computer-implemented method further comprises: In some embodiments, the computer-implemented method includes: The number of points to sample can be determined based on the desired number of columns in the image. For example, in some embodiments, the number of sample points may be the same as the number of columns. In some embodiments, the computer-implemented method includes a user-configurable number In some embodiments, the number of samples may be It can be set and loaded from a configuration file, or the user can enter a value using an input device. The number of samples can be set by entering it into an input field in the GUI. A higher sampling rate will result in a higher resolution panoramic image. Correspondingly, lower sampling rates result in lower resolution but faster generated packets. As an example, the number of sampling points can be 2048. FIG. 11 shows a sample on a continuous smoothing spline 904 projected onto a digital model 906. An example of a ring point 902 is shown.

[0070] In some embodiments, the computer-implemented method further comprises: In some embodiments, the computer may determine the sampling digital surface points that correspond to the The computer-implemented method samples by projecting the sampling points onto a 3D digital model. The pulling digital surface points can be determined. FIG. 12(a) shows the 3D digital model. The sample data corresponding to the sampling points on the continuous smoothing spline 1004 for 1006 is 10 shows an example of a digital surface point 1002. In some embodiments, the computer-implemented method includes: At the sampling points on the spline, the spline is perpendicular to the spline, and the occlusion direction in the digital model is Determine a sample plane parallel to the spline and the digital The sampling points on the spline are calculated by determining their intersection with the surface. A ring digital surface point can be determined. For example, as illustrated in FIG. The computer-implemented method then performs a continuous smoothing step at the sampled digital surface points 1002. Determine a sample plane 1008 that is perpendicular to the spline 1004 and parallel to the occlusal direction 1010. It is possible.

[0071] In some embodiments, the computer-implemented method further comprises: In some embodiments, one or more digital surface points may be determined. , the computer implemented method includes: The sample is perpendicular to the continuous smoothing spline at the surface points and parallel to the occlusion direction in the digital model. A pull plane can be determined or a determined sample plane can be used.

[0072] In some embodiments, the computer-implemented method further comprises: Pulling digital surface points along depth rays to sample depth points Sampling digital surface The point depth can be moved downward along the occlusal direction. The method includes sampling digital surface points 1002 along sample point depth rays 1012. To sample depth points 1014, in some embodiments, sample points along the occlusal direction 1010 The sample point depth ray can be moved downward along the depth ray 1012. In some configurations, it can be parallel to the occlusal direction. The implementation method can move the sampled digital surface points within the sample plane. For example, the sampled digital surface point 1002 is moved within the sample plane 1008. In some embodiments, the sample point depth may be, for example, The sample point depth may be 15 mm along the occlusal direction. Other suitable sample point depths may be used. It is also possible.

[0073] In some embodiments, the computer-implemented method further comprises: Determine one or more rays and calculate one or more associated digital surface points. In some embodiments, the method includes computer-implemented The method involves directing a ray of light from a sample depth point in the sample plane to a digital image of the 3D digital model. One or more depth rays can be drawn by extending the sample point to the surface at different angles. In some embodiments, the computer-implemented method can determine a ray of light in the tilt angle range. In some embodiments, the tilt angle range is determined by the sample point depth. The inclination angle range can be -45 degrees to +45 degrees relative to the 35 degree beam. A user-configurable configuration file that can be loaded by the computer implementation. value and / or the user may use an input device, for example through the use of a GUI. In some embodiments, the computer-implemented method includes: Determining the number of rays from a sample depth point to be generated based on the number of desired rows in the image. In some embodiments, the computer-implemented method may include: The number can be set to the desired number of rows in the panoramic image. The computer implemented method determines the increment in tilt angle for one or more rays from a sample depth point. In some embodiments, the tilt angle increment can be determined as (total tilt angle range) / (desired For example, the number of desired columns may be set to 320 and the total inclination may be determined as If the bevel angle range is set to -45 degrees to +45 degrees, the increment of the bevel angle is (90 degrees ) / (320), which would be 0.28 degrees.

[0074] In some embodiments, the computer-implemented method further comprises forming a sample point depth ray; and determining one or more tilt angles on either side of the sample point depth ray as multiples of the tilt angle increment; One or more lines extending from the sample depth points to at least the digital surface of the 3D digital model. In this example, the computer-implemented method may determine one or more rays. The rays will be determined as multiples of 0.28 degrees on either side of the sample point depth ray. The beam or beams may, for example, be in the sample plane in some embodiments. The computer-implemented method includes: A digital surface point can be determined as a digital surface point that intersects a ray. The implementation is to perform one or more ray sweeps on either side of the sample point depth ray until the end of the range is reached. rays and their corresponding intersection digital surface points can be determined.

[0075] For example, as illustrated in FIG. 12(a), the computer-implemented method may include: A first positive inclination angle ray 1016 that is at a first positive inclination angle 1018 from the light ray 1012; and a first negative inclination angle ray 1012 that is at a first negative inclination angle from the sample point depth ray 1012. 020. The first positive tilt angle 1018 and the first negative tilt angle can be determined as: They are both inclination increments because they are the first inclination angles by which the ray is elongated. The second positive and negative tilt angle rays are incremented from the sample point depth ray 1012 by + / minus 2*(tilt angle increment), and the third positive and negative tilt angle rays are Pull point depth ray 1012 makes an angle of plus / minus 3*(tilt angle increment), etc. The computer-implemented method uses a first positive tilt angle ray 1016 and a digital model. 10. Determining a first digital surface point 1022 at the intersection of the rule 1006 with the digital surface. and a first negative inclination angle ray 1020 and a digital surface of the digital model 1006. A second digital surface point 1024 can be determined at the intersection of the first digital The surface point 1022 and the second digital surface point 1024 are sampled digital surface points 10 02. Only a single sample point and two rays are shown for illustration purposes. Although many more rays at different angles are possible. The number of sample points on the line can be, for example, 2100, and the number of rays can be, for example, , 320. For example, FIG. 12(b) shows a sample on spline 1054. The digital model 105 has additional sample points, such as ringed sample points 1052. 0. The sample point 1052 corresponds to a ray extending from the depth point corresponding to the sample point 1052. 1056. FIG. 12(c) shows a sample. 10 shows multiple rays, such as ray 1056 extending from a sample depth point for sample point 1052. .

[0076] In some embodiments, a central row of the panoramic image contains sampled digital surface points. In some embodiments, each sampled digital surface point and its associated digital The surface points can be columns in the panoramic image. In some embodiments, one of the central rows The side rows may contain digital surface points corresponding to rays with positive tilt angles, while the center row may contain digital surface points corresponding to rays with positive tilt angles. The rows on the right side may contain digital surface points that correspond to rays with negative tilt angles. Depending on the sampling method, the same slope is used to construct rays for all the digital surface points. In some embodiments, the angular increment of each sampled digital surface point may be The depths of the sample depth points are the same. In some embodiments, the center row is This corresponds to a ray that makes a zero inclination angle to the line.

[0077] FIG. 13 shows an example of a panoramic image, shown for illustrative purposes only. is a sampled digital surface point, such as sampled digital surface point 1103. The digital surface points in the column 1105 may include sampled digital surface points 1 103. The computer-implemented method includes: Light extending from the sample depth point of the digital surface point 1103 at different multiples of tilt angle increments From the lines, digital surface points in column 1105 can be determined. For example, the digital surface points in row 1104 are determined from rays with positive tilt angles. The digital surface points in row 1106 are determined from the negative tilt angle rays. For example, the first positive row 1110 corresponds to a sample point depth ray. do Slope angle = 1 * (slope increment) The digital surface point may include a first positive ray having a first The negative row 1112 is for the sample point depth ray Slope=-1*(Slope increment) The digital surface point may include a point intersecting a first negative ray having ... For this purpose, the tilt increment can be specified in a configuration file or by the user, for example using the GUI. The computer-implemented method may be specified through an editable input field by, for example: For example, in some embodiments, the number of rows may be determined from a set tilt angle increment.

[0078] Panoramic images are generated by sampling digital surface points with depth information and associated digital tables. In some embodiments, the computer-implemented method includes a digital display. The depth of a surface point is determined as the length of its corresponding ray from the digital surface point to the sample depth point. In some embodiments, the computer-implemented method includes sampling / associating The depth of a digital surface point is extended from the sampled / associated digital surface point to the sample depth point. The depth ray length can be determined as the sample point depth ray length. The computer-implemented method can store the depth of each pixel in the panoramic image. In some cases, the computer-implemented method may store the depth information as luminance. For example, in some embodiments, pixels with lighter colors are closer to the camera field of view on the digital display. The darker pixels represent areas of the digital surface that are further from the camera's field of view. FIG. 14 shows an example of a panoramic image with depth information. For example, the darker area 12 02 is further from the camera than a brighter area, such as brighter area 1204. Alternatively, In some embodiments, brighter colored pixels are located on the digital display that is farther from the camera field of view. The darker pixels represent the digital surface area that is closer to the camera. can be represented as:

[0079] The designations positive and negative tilt angles refer to the ray on one side of the sample point depth ray and the ray on the other side of the sample point depth ray. It should be noted that the terms are used interchangeably to distinguish them from the examples. For example, in some embodiments, the digital surface points in row 1104 are determined from rays with negative tilt angles. The digital surface points in row 1106 can be determined by the positive tilt ray. The surface points can be digital surface points determined from the surface.

[0080] At least one advantage of using a panoramic image is that the light rays are not reflected from, for example, an orthographic projection. Instead of being parallel as in the oblique view and diverging as in the oblique view, they point toward the root of each tooth. This can include improved visibility of the tooth-gingival border, as well as the ability to focus on the teeth and The accuracy of the boundaries of other regions, such as the gingiva and gums, can be improved.

[0081] Labeling Some embodiments of the computer-implemented method use a trained neural network. and labeling one or more regions of an input image to provide a labeled image. may include.

[0082] In some embodiments, the trained neural network may be any type of neural network. In some embodiments, the neural network may be, for example, In some embodiments, the neural network may be a CNN, as described above. The network is, for example, the YOLO neural network described above. In some embodiments, the neural network can be used to perform, for example, semantic segmentation. In some embodiments, the network may be a semantic The segmentation network may be, for example, a U-Net neural network as described above. In some embodiments, the network may be a semantic segmentation The application can be, for example, a PAN neural network as described above. .

[0083] In some embodiments, the neural network may be configured to recognize each digital tooth by, for example, labeling the tooth. One or more digital teeth, marked with a digital tooth bounding area and one or more images of at least a portion of the digital dentition. The method can be trained by providing a training data set that can be used to For this purpose, the training dataset is composed of digital teeth that are each bounded by a labeled digital tooth bounding region. The image may include one or more panoramic images, each of which is labeled with a region. The neural network trained on the dataset is a panoramic image trained neural network. It can become a real network.

[0084] In some embodiments, the labeled digital tooth bounding area may be, for example, rectangular. However, other shapes can be used to label the digital tooth bounding area. Each digital tooth in each image in the training dataset can be used as a digital The tooth bounding area is labeled. FIG. 15 shows the labeled digital tooth bounding area. 13 shows an example of a panoramic image 1300 having a digitally scanned area 1302. The labeled digital tooth bounding area for each tooth may be manually determined in some embodiments. , or may be labeled by an automated process. can contain 10,000 images, e.g., panoramic images. For example, In some embodiments, any other suitable number of panoramic images may be used as the training data set. Training of any neural network can be supervised or unsupervised. can.

[0085] In some embodiments, the YOLO network may also include a rectangle or box. Train by providing a shaped and labeled digital tooth bounding region. In some embodiments, U-Net or PAN neural networks can be used. Semantic segmentation networks such as Labeled digital tooth bounding regions are labeled to follow the shape of each digital tooth. It can be trained by providing

[0086] In some embodiments, the trained neural network may be, for example, In some embodiments, the trained CNN can be a panoramic image trained CNN. The neural network is, for example, a YOLO neural network trained on panoramic images as described above. In some embodiments, the neural network may be a trained neural network. The network is, for example, a semantic segmentation network trained on panoramic images. In some embodiments, the trained semantic segmentation The network is, for example, a U-Net network trained on panoramic images as described above. In some embodiments, the trained semantic neural network Segmentation can be performed, for example, using the panoramic image trained PAN neural network described above. Alternatively, in some embodiments, the neural network may be a trained neural network. The neural network can be a 2D depth map trained neural network. .

[0087] Although panoramic images are described as an example, in some embodiments the training data set may be Alternatively, each digital tooth may be labeled with a labeled digital tooth bounding region. The 2D depth map training dataset may include one or more 2D depth maps. The neural network trained using 2D depth map trained neural network For example, a training dataset can be one or more 2D depth maps. For example, it can be trained in the same way as described for panoramic images, except that This can be done.

[0088] In some embodiments, the computer-implemented method uses a trained neural network. It can label one or more regions of an input image using a labeling algorithm and provide a labeled image. For example, in some embodiments, a computer-implemented method may include receiving an input image and Or, multiple trained neural networks are used to extract information from one or more regions of the input image. In some embodiments, the training can include labeling the regions and providing labeled images. The pre-trained neural network is a panoramic image pre-trained neural network. In some embodiments, the computer-implemented method includes receiving a panoramic image and Or use multiple panoramic trained neural networks to extract the image from one of the panoramic images. Alternatively, multiple regions can be labeled to provide a labeled panoramic image.

[0089] Alternatively, in some embodiments, the computer-implemented method comprises receiving a 2D depth map; One or more 2D depth map trained neural networks are used to generate 2D depth maps. Label one or more regions of the map and provide a labeled 2D depth map. can be done.

[0090] In some embodiments, the trained neural network receives one or more images. It can take the digital tooth bounding region of each digital tooth in the image and generate it. The computer-implemented method may, for example, in some embodiments, include generating a digital tooth boundary. All pixels bounded by the mapping region can be labeled with a unique label. For example, in some embodiments, the panoramic image trained neural network may include one or more or multiple panoramic images, and Bounding regions can be generated automatically.

[0091] For example, in some embodiments, the panoramic image trained YOLO network The panoramic image may include a digital image for each tooth in at least a portion of the panoramic image. The computer-implemented method can provide a tooth bounding region. In some embodiments, all pixels bounded by the digital tooth bounding region are Cells can be labeled with unique labels. Panoramic image pre-trained YOLO net The digital tooth bounding area provided by the workpiece may be, for example, For example, the panoramic image trained YOLO network can be used as a bounding box. The network receives the panoramic image shown in FIG. 14 and calculates the digital bounding area. FIG. 1 may include one or more digital bounding regions, such as 1402. The labeled panoramic image can be output as shown in FIG. 6(a). The computer-implemented method may, for example, in some embodiments, include a method for determining the location of a tooth by a digital tooth bounding region. Label all the pixels bounded by the pixel with a unique label and label each pixel This can be achieved by neural network segmentation, or by This may also be referred to as segmentation.

[0092] In some embodiments, the neural network is a U-Net neural network or a PAN neural network. Panoramic image-trained semantic segmentation networks such as A panoramic image can be received, and digital images of at least a portion of the panoramic image can be displayed. For example, U-Net or Panoramic image pre-trained semantic segmentation networks such as PAN 14, receives a panoramic image as illustrated in FIG. 14, and calculates a digital tooth bounding region 140 4, it is possible to output a labeled panoramic image as shown in FIG. 16(b). The computer-implemented method may, for example, in some embodiments, include generating a digital tooth boundary. We label all pixels bounded by the filtering region with a unique label and This can provide labeling for neural network segmentation, It may also be referred to as rough segmentation.

[0093] In some embodiments, a 2D depth map trained neural network (e.g., CN N, YOLO, semantic segmentation, or any other kind of neural The network) can receive 2D depth map images and the panoramic image trained network As described above with respect to the global network, the digital image data within at least a portion of the panoramic image is A digital tooth bounding area for each tooth can be provided. The method may, for example, in some embodiments, include determining the boundary of the digital tooth bounding region. Every pixel in the image can be labeled with a unique label.

[0094] In some embodiments, the 2D label map with the labels is of the same dimensions as the 2D depth map. It can be something like that.

[0095]

[0008] In some embodiments, the computer-implemented method further comprises: Multiple regions are mapped to one or more corresponding rough digital surface polygons in the 3D digital model. This could include mapping the data to the label and providing a 3D digital model with the labels. In some embodiments, the computer-implemented method further comprises: applying a virtual ray to each pixel of the 2D label map. The light is then passed through a pixel, and the digital surface polygons that the light passes through are labeled with the same label as the pixel label. In some embodiments, the computer-implemented method further comprises labeling the virtual ray. It passes a polygon through any digital surface and identifies each device based on the label of the pixels it passes through. In some embodiments, the digital surface polygons can be labeled. The method converts all labeled regions from the 2D image, including any uncertain regions, into a 3D digital image. In some embodiments, the computer-implemented method may include projecting the mesh onto a 3D model. The method maps one or more unlabeled digital surface polygons to the nearest digital surface polygon. In some embodiments, the digital surface polysilicon can be labeled with the same label as the polysilicon. A gon can be, for example, a digital surface triangle.

[0096] In some embodiments, this mapping can be done, for example, by labeling panoramic images of digital teeth. The method may include projecting the tooth boundary region onto the 3D digital model. In this case, the computer-implemented method projects the digital tooth bounding box as a prism. The panoramic image is generated from a 3D digital model using light rays. The computer-implemented method uses the same rays to generate a digital tooth bounding box in 3D. In some embodiments, the computer The implementation method can determine the four corners of each digital tooth bounding box. For each corner, the computer-implemented method, in some embodiments, determines the rays that pass through the corner. In some embodiments, the computer-implemented method may include can connect each set of rays in the plane to their corresponding depth points. In some embodiments, the computer-implemented method includes: to be coarsely labeled as belonging to a single tooth, for example. In some embodiments, the computer-implemented method may include: Any digital surface polygon (which in some embodiments may be a digital surface triangle) In some embodiments, the computer-implemented The method can label each constructed prism with a different label. Thus, the computer-implemented method generates a digital surface that intersects every ray within the box. can be individually labeled.

[0097] FIG. 17 shows an example of the mapping in some embodiments. As illustrated in the figure, , the mapping may be, for example, a digital tooth bounding box 1502, A labeled panoramic image of the digital tooth boundary area, e.g., 150 mm, of the 3D digital model 0. The method includes: first computing a digital tooth bounding box 1502; The computer-implemented method may determine a corner of the first ray 1504. 1506 to the first digital tooth bounding box corner 1504, and / or or a ray extending through it. The method can determine second digital tooth bounding box corners 1508. The computer-implemented method further comprises: coupling a second ray 1510 to a second digital tooth bounding The box corners 1508 may be determined as rays extending to and / or through the box corners 1508. By this configuration, the first light beam 1506 and the second light beam 1510 can be projected onto a panoramic image. As explained in the section of this disclosure regarding image generation, they are in the same plane. In a Norma image, for example, the first ray 1506 and the second ray 1510 are in the same column. Therefore, the first ray 1506 and the second ray 1510 are located at a depth point 1512. The computer-implemented method generates a third data with a corresponding third ray 1516. The digital tooth bounding box corner 1513, and the corresponding fourth ray 1520 are The corresponding fourth digital tooth bounding box corner 1518 can be determined in a similar manner. The third light ray 1516 and the fourth light ray 1520 are in the same plane by construction, and Thus, they meet at their unique corresponding depth points. The computer-implemented method labels every digital surface point within a prism with, for example, the same label. In some embodiments, the computer-implemented method includes: Every ray within the bounding box can be individually labeled. In some cases, the computer-implemented method labels rays passing through the interior portion of the box. It is possible.

[0098] In some embodiments, the computer-implemented method determines one or more uncertainty regions. An uncertainty region is a region where the neural network can label one or more regions. If you do not provide enough information to classify the This can occur when generating information that is not possible with digital tooth bounding, for example. In some embodiments, the computer-implemented method includes: For an image or 2D depth map, e.g., the surface within the digital tooth bounding region All digital tables within a user-configurable uncertainty buffer distance from any boundary of a point Surface points can be determined as part of one or more uncertainty regions. YOLO Net In the case of a workpiece, for example, the digital tooth bounding area is The distance can be a box along the digital surface in some embodiments. In some embodiments, the uncertainty region buffer distance is, for example, 6 mm. Other suitable distances may be used, the uncertainty region buffer distance may be, for example: The settings may be stored and loaded from a configuration file, or may be configured, for example, by the user in some embodiments. The user uses an input device to create a graphical user interface (GUI). It is a user-configurable value that can be entered into the CAL User Interface ("GUI"). can.

[0099] An uncertain region is, for example, one that is not visible on the image given to the neural network. This can occur for any digital surface polygon (e.g., digital surface triangle). As discussed above, in some embodiments, the computer-implemented method may account for uncertainty regions. For example, in some embodiments, the computer-implemented method may include labeling the uncertainty regions. We can either leave them as they are or label them as areas of uncertainty. FIG. 8(a) shows an example of how a computer-implemented method may identify regions of uncertainty. The trained neural network then generates a digital tooth boundary region 1602 and other digital tooth boundary regions 1602. The computer-implemented method can, for example, determine the tusk boundary region by Confidence areas within the digital tooth boundary region 1602, such as confidence area 1604, that are visible The computer-implemented method may label the uncertainty based on distance 1605. Determining an uncertainty region, such as an uncertainty region 1607 based on the area 1606 and the distance 1603 For example, we can leave the uncertain region as unlabeled or label it as uncertain. 16(b) allows the user to designate areas as uncertain by This shows how to map an image onto a labeled 3D digital model. In the model, the computer-implemented method includes: 4, unlabeled (or labeled as uncertain) region 1616, and the second A labeled digital surface area 1618 can be determined. For example, a third labeled There may also be additional labeled areas, such as digital surface area 1620. In some embodiments, the uncertain regions are converted to labeled regions during segmentation. For example, in some embodiments, the computer-implemented method may include: Areas of uncertainty can be labeled during the commentary.

[0100] In some embodiments, the computer-implemented method comprises segmenting the labeled 3D digital model. In some embodiments, the 3D digital model can be segmented. The computer-implemented method includes: After generating labeled panoramic images / 2D label maps from the In some embodiments, the computer-implemented method comprises the steps of: Any kind of segmentation technique known in the field can be used to generate labeled 3D digital images. A segmented 3D digital model can be generated from the 3D model. Thus, the computer-implemented method may, for example, perform curvature-based segmentation. It is possible.

[0101] In some embodiments, the computer-implemented method includes performing one or more morphological erosion operations. One or more labeled regions may be applied to the In some configurations, a labeled gingival area may be included. The computer implementation method performs the morphological shrinkage operation on the 2D label map or the labeled 3D digital map. In some embodiments, the method may be performed on either the computer model. The implementation performs a morphological shrinkage operation on the 2D label map as follows: Can: A user-configurable radius (e.g., 5 pixels) and evaluate one or more nearby pixels If the pixels do not have the same label, then the computer-implemented method A cell and its surrounding pixels can be designated as unlabeled. Thus, the computer-implemented method performs this process on one or more pixels of the 2D label map. In some embodiments, the computer-implemented method includes: A morphological contraction operation is performed on the labeled 3D digital model as follows: You can: ) for a user-configurable radius ( One or more nearby polygons (e.g., digital triangles) within 5 mm If one or more nearby polygons do not have the same label, then The computer-implemented method treats a particular polygon and its surrounding polygons as unlabeled. In some embodiments, the computer-implemented method performs this process by Can be repeated for one or more polygons of a 3D digital model with a bell In some embodiments, the computer-implemented method further comprises the steps of: It can be applied to any labeled region, including

[0102] In some embodiments, one or more unlabeled digital surface polygons may remain. One or more unlabeled digital surface polygons may be added to the total number of digital surface polygons. This can occur when the number of unlabeled digital surface polygons is greater than the number of unlabeled digital surface polygons. The label was not determined for one or more occluded digital surface polys One or more unlabeled digital surface polygons can be generated by morphologones. This can be caused by G contraction.

[0103] Curvature-Based Segmentation

[00041] In some embodiments, the curvature based segmentation comprises:

[0013] In some embodiments, the computer-implemented method may include determining a curvature of the surface region. A digital model can be received and the curvature of the digital surface region can be determined. The data implementation method may use any technique to determine the curvature of the digital surface region. In some embodiments, the curvature determination may be performed automatically by a computer-implemented method.

[0104]

[00041] In some embodiments, the digital surface region comprises a triangle. The curvature of the triangle is determined by the side can be determined by taking the average of the curvatures of the triangle, or the average of the curvatures of the vertices of the triangle. do.

[0105] In some embodiments, the computer-implemented method calculates the curvature of the surface by taking the average of the curvatures of the edges. The curvature of the triangle can be determined by the following equation. 17 shows an example of determining the curvature at an edge 1786 that connects 0. The computer implemented method is shown as follows: The dihedral angle 179 formed at the side 1786 between its adjacent neighboring triangle 1788 2, the curvature at side 1786 can be determined. The second triangle 1786 is perpendicular to the side 1786 formed by the adjacent triangles 1790 and 1788. The angle formed between two adjacent triangles 1788 and 1790 in plane 1794 of FIG. For example, in some embodiments, the degree of A computer implementation can take sin(φ), where φ is the sum of two adjacent triangles. The dihedral angle 1792 between the angles 1790 and 1788. The curvature function can be repeated on every triangle edge.

[0106] Alternatively, in some embodiments, the computer-implemented method further comprises: For example, in some embodiments, the curvature of the triangle can be determined by taking The computer-implemented method optionally uses the connectivity information to reduce the search space. Then, we calculate the curvature at each vertex P by selecting a neighborhood (of size N) of vertices around P. The computer-implemented method can determine the quadric surface patch F(x,y,z)=0. The computer-implemented method can be used to fit a projection P of P onto a patch. 0, so that F(P0)=0. We can determine the curvature properties of F at P0 and assign the curvature properties to P.

[0107] In some embodiments, each data point (x,y,z) is perfectly located on the surface of F. Therefore, the computer-implemented method can, for example, 2 +by 2 +cz 2 +2e xy+2fyz+2gzx+2lx+2my+2nz+d=0 can be used. The computer implementation method is format A=D T From a 10x10 real symmetric eigenvalue problem in D, the patch surface The coefficients of (a,b,c,e,f,g,l,m,n,d) can be determined, where D i is an N x 10 design matrix, each row of which is

[0108]

number

[0109]

number

[0110]

number

[0111]

number

[0112]

number

[0113]

number

[0114] The computer-implemented method then derives matrices A and B from E, F, G, L, M, and N as:

[0115]

number

[0116]

number

[0117] The computer-implemented method applies the selected scalar function to the principal curvatures k1 and k2, The selected curvature function ("SCF") can be determined by For example, for the principal curvatures k1 and k2, the computer-implemented method may K) as K = k1 k2, or the mean curvature (H) as H = 1 / 2(k1 + k2). It can be determined.

[0118] Both methods of determining the curvature radius are, on average, 6 It can be any number of digital vertices, including 0, and is a user-selectable value. Choosing fewer points and a smaller radius can result in faster calculations. On the other hand, choosing a larger number of points and a larger radius allows for a more precise estimation of the curvature. The computer-implemented method can provide an estimate of the surface area of ​​a digital surface mesh, for example. This can be repeated for all vertices.

[0119] In some embodiments, the computer-implemented method further comprises: In some embodiments, the signal may be segmented into multiple digital segments. The computer-implemented method uses curvature-based segmentation to segment the digital dental impression surface into three parts. It is possible to segment in three dimensions (3D). This is the case, for example, in watershed segmentation. In some embodiments, segmentation may include This may be performed automatically by a computer-implemented method.

[0120] In some embodiments, the digital dental impression surface is connected at edges and vertices, The surface mesh may include one or more triangles. In accordance with the present invention, a computer-implemented method determines the curvature of every triangle in a digital surface mesh. The computer-implemented method may be used to calculate the mean curvature of the vertices of a particular triangle, as described above. Determine the curvature of each particular triangle by determining the average curvature of the sides of the particular triangle. A rate can be determined.

[0121] In one embodiment, the computer-implemented method comprises the steps of: A particular curvature is calculated by determining the curvature of each of the sides of the triangle and calculating the average of the side curvatures. FIG. 20(a) shows that a particular triangle 2402 is a first The curvature of the sides, including the first side 2405, the second side 2407, and the third side at 2409, An example of some embodiments of determining an average is shown. The first side 240 is determined based on the dihedral angle between the first triangle 2402 and the adjacent triangle 2408. The computer-implemented method can determine the curvature at a particular triangle 240. 2 and the adjacent triangle 2406 based on the dihedral angle as described in this disclosure. The curvature at the second edge 2407 can be determined by the computer-implemented method, The third side 2402 is determined based on the dihedral angle between a particular triangle 2402 and an adjacent triangle 2404. The curvature at 409 can then be determined. 405, the average curvature of the second side 2407, and the third side 2409 are determined, The curvature of the given triangle 2402 can be determined. For example, the curvature of a particular triangle 2402 may be stored in a lookup table. The computer-implemented method can use every triangle in the digital surface mesh to perform this The process can be repeated to determine the curvature at each triangle in the digital surface mesh. can.

[0122] In some embodiments, the computer-implemented method further comprises adding a user-selectable positive or negative sign to each It can be applied to the curvature of a triangle. For example, if the curvature is set to the most convex side In this case, any concave region is assigned a negative sign and any convex region is assigned a positive sign. If the curvature is set to the most concave edge, then any convex region will have a negative Concave / convex is a digital function of For the outward facing surface normal of a digital surface, For example, the computer-implemented method may assign positive values ​​to convex edges and negative values ​​to concave edges. For inward facing normals of digital surfaces, The method can, for example, assign positive values ​​to convex sides and negative values ​​to concave sides. In some embodiments, the segment boundaries correspond to the maximum curvature along the digital surface. .

[0123] After determining the curvature of each particular triangle, the computer-implemented method generates a 3D curvature-based segmentation. Based on the mentation, the triangles can be segmented. , watershed segmentation is used. For example, in some embodiments, The computer-implemented method determines the curvature for each triangle. The curvature for each triangle is The computer-implemented method may include storing the specific As triangles, one can start with the triangle with the smallest curvature. The method may, for example, look up a table to determine the curvature of triangles in the vicinity of the particular triangle being evaluated. In some embodiments, the computer-implemented method includes a lookup From the table, the curvature of nearby triangles can be determined. Any nearby triangles with a higher curvature are added to the segment to which the particular triangle being evaluated belongs. Any nearby triangles that have a curvature smaller than the curvature of a particular triangle can be added. is not added to a segment of a particular triangle. The computer-implemented method then Select the shape as the next particular triangle to evaluate, and repeat the process for every triangle. In some embodiments, a segment is assigned a sufficiently large number to be attributed to a label. (e.g., a majority or any other user-configurable value) of triangles, The computer-implemented method can then assign the segment to that label.

[0124] FIG. 20(a) shows some embodiments of watershed segmentation of triangles. As described herein, the computer-implemented method may include In one embodiment, the computer determines the curvature of all of the triangles in the surface mesh. The computer-implemented method stores the curvature of the triangle in a look-up table. Identify the triangle with the smallest curvature, e.g., specific triangle 2402. The computer-implemented method uses a look-up table to determine the triangle with the minimum curvature. The computer-implemented method may further include: In some embodiments, the computer-implemented method determines the curvature of the . From this, the curvature of the neighboring triangles can be determined. In this example, the neighboring triangle 2406 is If the curvature of the shape 2402 is larger than that of the shape 2403, then the neighboring triangles 2404 406 can be considered part of the same watershed as the particular triangle 2402 The computer-implemented method generates digital surface triangles 24 as illustrated in FIG. 02 with triangle 2406 into a single segment such as segment 2411 .

[0125] The computer-implemented method then, for example, calculates the curvature of the neighboring triangles 2404 relative to the curvature of the particular triangle. For example, the curvature of the neighboring triangle 2408 is the smallest curvature. If the curvature of 2402 is greater than the curvature of 2402, then triangle 2408 is The triangle 2402 is merged with the segment 2411 that contains it. Thus, after merging triangles 2408, segment 2412 is formed.

[0126] If the neighboring triangle has a lower curvature than the particular triangle 2402 in question, then 24. Then, the neighboring triangles are determined by the computer-implemented method to include the particular triangle 2402. For example, the neighboring triangle 2404 is lower than the triangle 2402. If the triangle has a small curvature, then 2404 finds the segment to which the particular triangle 2402 belongs. It will not be merged with Item 2412.

[0127] After processing the first particular triangle, the computer-implemented method further comprises: 2. Move to a new particular triangle, which may be a neighboring triangle. repeating the process of determining the segmentation with the new particular triangle being evaluated; The entire digital surface can be segmented. FIG. 20(c) shows, for example, 24 shows an example of a segmented digital surface mesh 2414 including a segment 2416.

[0128] After performing the triangle segmentation, the digital surface mesh is as shown in FIG. In some embodiments, the segmentation may include multiple segments. The number of participants may be increased by, optionally, merging two or more segments together. In some embodiments, the computer-implemented method may reduce The small segments are classified into four groups based on their mean curvature, mean size, area, perimeter, perimeter-to-area ratio, and Merging into a larger one based on geometric attributes, such as / or other geometric factors In some embodiments, the computer-implemented method may include, but is not limited to, a separate In some embodiments, the computer may not merge two segments that are assigned to the same label. The computer implementation methodology is to, after each merger, divide the merged segment into the same In some embodiments, the computer-implemented method includes: A merged segment is considered uncertain only if both previous segments were uncertain. The merging may be performed by a computer in some embodiments. This may be performed automatically by the method.

[0129] In some embodiments, the computer-implemented method further comprises: The computer-implemented method determines the merger priority of two neighboring segments. Ranking can be determined based on those attributes. Two segments are ranked based on those attributes. If the merger is possible based on the Data implementation methods prioritize based on geometric factors. For example, computer-implemented methods The method is: 1) the average curvature inside each segment and on their common boundary (on the boundary and on the segment) 2) the segments with smaller curvature difference from the interior of the vertex merge faster; and The ratio of the length of the common boundary to the minimum perimeter of the segments (the segment with the larger ratio Priorities can be determined based on which of the following (which of the two mergers will be the first to occur):

[0130] In some embodiments, the computer-implemented method stores the priority in a priority queue. The computer-implemented method may extract the highest priority from the queue and Merges the two segments and merges them into a newly formed segment in the queue. The computer-implemented method can update the priority between the two segments. This process can be repeated until no more units can be merged. do.

[0131] In some embodiments, for example, a segment that was attributed to the uncertainty region no longer exists. Smaller segments can be merged together until

[0132]

[0005] In some embodiments, the computer-implemented method comprises: After performing curvature-based segmentation, the labeled 3D digital model is A segmented 3D digital model can be provided.

[0133] Two-point segmentation In some embodiments, the computer-implemented method includes performing two-point segmentation. For example, in some embodiments, the computer-implemented method may include It receives the digital model, performs two-point segmentation, and creates a segmented 3D digital model. In some embodiments, the labeled 3D digital model may be provided. Labeling performed by one or more neural networks as described The labels on the labeled 3D digital model define local tooth regions. These are the approximate locations of each digital tooth in the labeled 3D digital model. In some embodiments, the two-point segmentation can help provide a more accurate boundary. One or more examples of the segmentation are described in SEMI-AUTOMATIC TOOTH SEGMENTATION by Nikolskiy et al., This can be found in U.S. Patent Application Publication No. 16 / 778,406. is incorporated herein by reference in its entirety.

[0134]

[0007] In some embodiments, the computer-implemented method comprises providing a labeled 3D digital representation of the patient's dentition. Receive the model, determine the center points of one or more labeled regions, and compute one or more Construct a best-fit parabola to connect the center points of the labeled regions of and determine whether the plane contains the center points. constructing a plane at each center point that is perpendicular to the best fit parabola at the center points; For each plane, determine the lingual and buccal points where the plane intersects the segment boundary and It is possible to construct first and second paths on either side of the plane to connect the buccal and nasal points. In some embodiments, the computer-implemented method includes constructing a plane at each center point. Instead, for each center point (such as the sample plane 1008 from FIG. 12(a)), The corresponding sample plane can be utilized or reused.

[0135] Figure 20(a) to Figure 20(g) show an example of two-point segmentation. As illustrated, the computer-implemented method may, for example, include: 2002. The computer-implemented method includes: For each labeled region, a center point, such as center point 2004, can be determined. , providing a center point as illustrated in FIG. The method is by best-fitting a smooth curve, e.g., a best-fit analytical curve, e.g., a parabola. As illustrated in FIG. 22(c), The computer-implemented method includes: A best-fit parabola, such as parabola 2010, can be constructed that connects the center points of the regions. To determine the best fit parabola, the computer-implemented method determines a least squares plane. For example, in the case of a center point, the computer-implemented method projects the center point onto a plane. For example, As illustrated in FIG. 22(d), a center point 8502 (shown as a black dot in the figure) ) are arranged in a least-squares plane. The computer-implemented method A first x-axis 8504 is generated in the plane, and a first y-axis 8506 is generated at 90 degrees to the x-axis in the plane. The computer-implemented method uses quadratic least squares regression, which is known in the art. In the equation y=ax 2 Determine the coefficients a, b, and c in +bx+c. For example, a parabola 8508 is determined by the computer-implemented method after determining the coefficients a, b, and c. The computer-implemented method may then, for example, find the intersection of the parabola 8508 and the center point 8502. The computer-implemented method determines the offset in steps for a user-selectable number of x-axis directions. For example, the computer-implemented method may generate a new x-axis with a corresponding y-axis 8512. Rotate the x-axis 8504 by rotating the x-axis around axis 8510 to determine the parabola 8514 In some embodiments, the number of x-axis directions may be user selectable and / or can be a default value. In some embodiments, the number in the x-axis direction is, for example, 100 In order to avoid very sharp parabolas, the computer-implemented method may be For example, if a is 150 meters -1 We can select the parabola with the smallest deviation, which is less than or equal to In some embodiments, the computer-implemented method optionally includes the step of: and can be located farther than the user-selectable and / or default maximum center point distance In some embodiments, the maximum center point distance is, for example, 5 mm. As illustrated in FIG. 22(c), the computer-implemented method can be used to find the best-fit parabolic Lines 2010 may connect the center points.

[0136] As illustrated in FIG. 22( e ), the computer-implemented method includes: A plane such as an isolation region 2020 at a center point is drawn such that the isolation region 2020 encompasses the center point 2004. 2004 , and can be constructed to be orthogonal to the best fit parabola 2010 at the center point 2004 . In some embodiments, instead of constructing a plane, the computer-implemented method may, for example, Utilizing a sample plane corresponding to each center point, such as sample plane 1008 from FIG. 12(a), It can be reused or repurposed.

[0137] As illustrated in FIG. 22( f ), for example, the computer-implemented method may include: For each plane, the plane 2030 is aligned along a first segment path 2036 and a second segment path 2037. A lingual point 2032 and a buccal point 2034 that intersect with the path 2072 can be determined. FIG. 22(g) shows a digital tooth with a center point 2051 through which a parabola 2059 passes. 2050. The computer implemented method calculates lingual points 2052 and buccal points 2062. A first path 2053 on a first side of the separation region 2057 and the separation region 20 57 and constructing a second path 2056 on the second side.

[0138] In some embodiments, the first segment path 2053 and the second segment path 20 56, for example, selects either the facial point 2062 or the lingual point 2052 as an initial digital representation. The surface points are used as the surface points, and the Dijkstra's shortest path algorithm, which is known in the art, is used. In some embodiments, the computer-implemented method may include, for example, By evaluating only the edges between the digital surface points on the first side 2054 of the isolated region 2057, A first segment path 2053 can be determined using a shortest path algorithm. In some embodiments, the computer-implemented method further comprises, for example, calculating the second surface points as digital surface points. Shortest path algorithm by evaluating only edges between the 2058 digital surface points on the side The second segment path 2056 may be determined using The computer-implemented method may, for example, accomplish the following:

[0139] 1. Set the initial digital surface points. In some embodiments, the initial digital surface points are set on the face side. Alternatively, in some embodiments, the initial digital surface point can be a lingual point 2052.

[0140] 2. Mark all digital surface points on one side of the separation region 2057 as unvisited. In some embodiments, one of the digital surface points is generated as an unvisited set of all unvisited digital surface points. The side can be the first side 2054. Alternatively, in some embodiments, one side can be This may be the second side 2058 .

[0141] 3. Specify tentative distance values ​​for every digital surface point on one side. Initial digital surface point The provisional distance values ​​for are set to 0, and the provisional distances for all other digital surface points on one side are The distance value can be, for example, infinity, or the highest possible, or less than the sum of the lengths of all the sides. is also specified to be a large value. Set the initial digital surface point as the current digital surface point.

[0142] 4. For the current Digital Surface point, all unvisited neighboring Digital Surface points on one side Considering the distances of the current digital surface points through the calculated virtual distances (e.g., The method further comprises determining the edge length between the digital surface point and a particular unvisited neighboring digital surface point. In some cases, the calculated virtual distance is calculated based on the curvature of the digital surface. The edge length between a face point and a particular unvisited nearby digital surface point can be determined. In some embodiments, the computer-implemented method may, for example, include determining the side length by using an angle-dependent factor For example, in some embodiments, the computer-implemented method may include: It can be determined as: f(e i )=|e i |exp(k sin α i ), or f(e i )=|e i |ex p(k α i ) Here, f(e i ) is the relationship between the current digital surface point and a particular unvisited neighboring digital surface point. Algorithm for edge e between i is the length of |e i | is the 3D distance between the two ends of the edge where k is a constant, such as 1.8 in some embodiments. The larger the constant k, the more likely it is that the path will pass through regions of higher curvature. The sign of the constant determines whether the concave or convex side is chosen by the algorithm. It is possible to specify what will happen, and i is side e i is the dihedral angle of the triangular surface at , sin α i For example, in some embodiments, α i Discovery is more important than calculation. is the sine of the angle at which the rotation speed can be increased. i and / or s in α i can be, for example, an angle-dependent factor. The boundary between the tooth and the gums is , can have a significantly higher curvature than the rest of the surface. i )=|e i |e xp(k sin α i ), the length of the side in the digital tooth-gum boundary area is The digital tooth-tooth image is thus significantly smaller than in other regions. The shortest path along the flesh line joint can be determined.

[0143] 5. Compare the newly calculated virtual distance with the current specified value and specify the smaller one. For example, if the current digital surface point A is labeled with a distance of 6, and you want to map it to a neighboring digital surface point A, If the edge connecting A to point B on the surface of the sphere has length 2, then the distance to B passing through A is becomes 6+2=8. If B was previously labeled at a distance greater than 8, If so, change it to 8; otherwise, keep the current value.

[0144] 6. After considering all of the unvisited neighbors of the current digital surface point, Mark the digital surface point as visited and remove it from the unvisited set. The digital surface points are never checked again.

[0145] 7. If the destination digital surface point is marked as visited (two specific digital when planning a route between surface points, or between digital surface points in an unvisited set The smallest hypothetical distance of, for example, infinity, or the highest possible, or all side lengths (When planning a complete traverse; initial digital (occurs when there is no connection between a surface point and the remaining unvisited digital surface points) Stop when the algorithm has finished.

[0146] Otherwise, select the unvisited digital surface point labeled with the smallest tentative distance, and Set this as the new "current digital surface point" and go back to step 4.

[0147] The destination digital surface point is the smallest hypothetical point among all "unvisited" digital surface points. (and therefore could be selected as the next "present") ), the algorithm can stop.

[0148] The computer-implemented method can then repeat the steps on the other side of the separation region 2057. (This may be performed simultaneously or sequentially.) Thus, the computer-implemented method 2053 along the first side 2054 and along the second side 2058. A second segment path 2056 can be determined along the first segment path 2 053 and the second segment path 2056 may be, for example, in some embodiments, digitally It can be the edge of a surface triangle.

[0149] In some embodiments, the first segment path 2053 is one of the sides 2066 of the triangle on the face side. 2064 on the lateral side and one side of the lingual triangle 2064. The second segment path 406 is the other side of the triangle on the facial side 2066 and the other side of the triangle on the lingual side. The second segment path 2053 is connected to the other side of the edge 2064. The segment path 2056, the facial triangle side 2066, and the lingual triangle side 2064 are In some embodiments, these may be combined to form a boundary loop. In some embodiments, the loop may have an inner loop region 2068 that is closed, e.g., enclosed. can be done.

[0150] In some embodiments, the computer-implemented method further comprises: The method may further include providing a segmented digital tooth from the segment path. As illustrated in FIG. 22( f ), the computer-implemented method includes: 36 and a second segment path 2072 to provide a segmented digital tooth 2070. This can include, for example, 6 and the digital in the boundary loop 2074 formed by the second segment path 2072. The digital surface of the digital tooth 2070 is provided by determining the triangular surface triangles. This may include:

[0151] FIG. 23(A) shows a digital tooth region 504 within a portion of the boundary loop 502 and the boundary loop 503. 5 shows a portion of a boundary loop 502 with a digital non-tooth region 506 outside the portion of the loop 502. The digital dental region 504 and the digital non-dental region 506 are, for example, 5, digital surface triangles 508 and 514, respectively. It may contain face triangles.

[0152]

[0005] In some embodiments, the computer-implemented method for determining a digital surface of a digital tooth comprises: The method may include generating a dual graph of the original digital surface triangles. The implementation method is such that the vertices of the dual graph correspond to the digital surface triangles of the original digital surface, Except where the edges of the pair graph are part of the boundary loop, the edges of the shared digital surface triangles A dual graph can be generated to correspond to the edges of the shared digital surface triangles. For example, as illustrated in FIG. 23(B), the computer-implemented method may include 5. Generate dual graph vertices 512 for digital surface triangles 514 within a portion of rule 500. The computer-implemented method may, for example, determine the edges of the shared digital surface triangles. The computer-implemented method may generate dual graph edges 516 for 518. , does not generate dual graph edges for the shared digital surface triangle edges 520. Why? , then edge 520 is part of bounded loop 502. The process can be repeated for every digital surface triangle in the digital model. Thus, in some embodiments, the computer-implemented method includes: In some embodiments, a dual graph can be generated for each digital surface. An edge in the rough can cross any edge in the original surface, but no edge in the dual graph can cross any edge in the boundary. In some embodiments, the dual graph is broken down into two parts, e.g., by bounded loops. It can contain separated interior and exterior dual graph domains. A graph domain is an interconnected dual graph domain enclosed by a bounding loop. The outer dual graph domain is the interconnected dual graph domain outside the bounding loop. A bounded loop may be a region outside the inner dual graph region in some embodiments. For example, as illustrated in Figure 23(B), , the dual graph is divided into inner dual graph regions 522 and 532 separated by bounded loops 502. It may include an exterior dual graph region 524 .

[0153]

[0008] In some embodiments, determining the digital tooth surface comprises performing a search on the dual graph. The computer-implemented method may include, for example, performing a step of: In some embodiments, a digital tooth surface can be determined based on the interconnections of: The search can be, for example, a breadth-first search as known in the art. The computer-implemented method may, for example, load the dual graph and the root of the starting vertex. In some embodiments, the root vertex of the starting dual graph can be a vertex of any dual graph. The computer-implemented method may select the current depth level before moving to the next depth level. Explore all nearby dual graph vertices at depth and label them as discovered. The computer-implemented method then proceeds to the next level of depth, e.g. The process can be repeated until there are no more undiscovered dual graph vertices. The computer-implemented method thus includes determining the interconnected vertices of the dual graph; The discovered dual graph can be determined. The breadth-first search technique can, for example, find one or more It is possible to apply the method to a digital tooth without having multiple tunnels. The computer-implemented method determines a conventional breadth-first search as follows: You can: Procedure Breadth_First_Search(Dual_Graph,start _vert) are as follows: Let Q be a queue Label start_vert as discovered Q.enqueue(start_vert) While Q is non-empty, do the following v:=Q.dequeue() All edges from v to y in Dual_Graph.adjacentEdges(v) For each edge, do the following: If y has not been labeled as discovered, then Label y as discovered y.parent:=v Q.enqueue(y)

[0154] In some embodiments, the search can be, for example, a depth-first search as known in the art. In a depth-first search, the computer-implemented method may, for example, In some embodiments, the starting vertex root is The vertices can be any dual graph vertices. The method can perform the traditional depth-first search on the dual graph as follows: The input is, say, a graph, Dual_Graph, G, and the vertices, vert, of G. The output is, for example, the set of vertices reachable from the vertices labeled as discovered. The vertex can be any vertex. The depth-first search technique can, for example, search for one or more vertices. In some embodiments, the present invention can be applied to digital teeth without the need for a console. The computer-implemented method can determine the conventional depth-first search as follows: R: The procedure Depth_First_Search(Dual_Graph,vert) is as follows: As shown below. Label a vert as discovered From the vertices in Dual_Graph.adjacentEdges(vert) to y For every directed edge in, do the following If vert y has not been labeled as discovered, then Depth_First_Search(Dual_Graph,y) recursively Call to

[0155] In some embodiments, the computer-implemented method includes, for example, performing a depth-first search on the dual graph. can be accomplished as follows: Procedure Depth_First_Search_Iterative(Dual_Gra ph,vert) are as follows: Let S be a stack S.push(vert) While S is non-empty, do the following vert = S.pop() If the vert has not been labeled as discovered, then Label a vert as discovered vert in Dual_graph.adjacentEdges(vert) For all edges from to y, do the following S.push(y)

[0156] In some embodiments, for example, if the discovered dual graph is within a bounded loop, and / or if the root vertex of the starting dual graph is inside the bounding loop, The computer-implemented method further comprises: Therefore, it will be determined that the image belongs to the digital tooth. The vertices of the dual graph are classified as belonging to the outer dual graph and therefore are related to the digital gingiva and other surroundings. In some embodiments, for example, the found digital teeth can be determined to be those of the surrounding digital teeth. If the dual graph is outside the bounding loop and / or if the starting dual graph is If the root vertex is outside the bounding loop, then the computer-implemented method The seen dual graph is the domain of the outer dual graph, and therefore the digital gingiva and other The computer-implemented method can determine that all of the digital teeth belong to the surrounding digital teeth. The other dual graph vertices belong to the domain of the inner dual graph and are therefore The computer-implemented method can determine that the vertices of the discovered dual graph For example, in some embodiments, the digital surface of the digital tooth and the surrounding teeth The digital surfaces that belong to the group and gingival regions can be determined. The final labels are In some embodiments, this may include, for example, the tooth number, e.g., tooth 1, tooth 2, tooth 3, etc., and the gums. In some embodiments, each tooth can be differentiated from other teeth and from the gingival area. 24 shows digital teeth 2202 and gingival area 2204 in the digital model 2200. A fully segmented and labeled set of digital teeth and gingival regions Here is an example.

[0157] In some embodiments, the received labeled 3D digital model is optionally It may also be a segmented 3D digital model. In some embodiments, the segments The 3D digital model was segmented using curvature-based segmentation. In some embodiments, the segmented digital model can be The segmentation may be performed using any technique known in the art.

[0158] In some embodiments, the curvature-based segmentation model may be, for example, can be refined by performing a two-point segmentation. One advantage of point segmentation, for example, can include improved accuracy.

[0159] One or more advantages of one or more features may be, for example, segmented, labeled The digital teeth are also labeled and segmented with non-dentition objects such as gingiva. The method may include providing a digital model having the area. One or more advantages of the present invention may include, for example, more accurate boundaries. One or more advantages of using images include, for example, the ability to view dentition features from both buccal and lingual sides; and preserving the gingival border while retaining the 2D depth map, etc. On any standard view, for example, the buccal and / or lingual surfaces are completely or are partially occluded, so the neural network has to see and segment them. Standard visual images, such as 2D depth maps, cannot capture the inter-dentition between the teeth and the gums, for example. The border of the teeth is not visible for all teeth and from all sides. This is because the teeth This can occur due to occlusion (hiding) of the field. Furthermore, the problem is more general: Whatever the viewing angle (perspective or orthographic camera) Some boundaries will not be visible. The advantage of numbers is, for example, that it is possible to distinguish the characteristics of the dentition from both the buccal and lingual sides, as well as the boundary with the gingiva. It may include holding.

[0160] FIG. 25(a) shows a computer-assisted method for digitally segmenting teeth in a digital model. 23 shows an example of a method for performing the method, in which, in 2302, a dental implant is performed on at least a portion of a patient's dentition. Receiving a 3D digital model of the patient scan data, at 2304, generating (or alternatively receiving) a panoramic image from the model; In 6, the first trained neural network is used to generate a pixel value of one or more of the panoramic images. The method labels multiple regions and provides labeled panoramic images. to map one or more areas of a labeled panoramic image to one or more areas in a 3D digital model. or multiple corresponding coarse digital surface triangle labels, and generate labeled 3D digital and, in 2310, segmenting the labeled 3D digital model. and providing a segmented 3D digital model. In some embodiments, the computer-implemented method may alternatively include, at 2304, A conventional 2D depth map image can be generated or received from the 3D model. An example of a typical 2D depth map image is shown in FIG.

[0161] The computer-implemented method of FIG. 25(a) may include one or more of the following features in a variety of In an embodiment, the following may be included alone or in combination: Alternatively, a first niche may be used to label multiple regions and provide a labeled panoramic image. 10. The method of claim 1, further comprising training a labeled neural network. The 3D digital model may contain one or more regions of uncertainty. Segmentation 3 D The digital model can include one or more individual dental regions, as well as non-dental regions. The first trained neural network is the first convolutional neural network The first convolutional neural network may include a U-Net neural network. The first convolutional neural network is the YOLO neural network. Segmenting a 3D digital model can include neural networks. , which may include geometrically segmenting the labeled 3D digital model. Geometric segmentation can include curvature-based segmentation. The objective segmentation can include two-point segmentation. That is, a second trained neural network is used to generate one or more digital determining one or more digital tooth bounding regions corresponding to the set of teeth; connecting one or more digital tooth bounding regions by splines; Sampling one or more spline points on the spline and number of sampled spline points to one or more sampled digital surface points determining one or more associated digital surface points corresponding to each sampled digital surface point; and determining a digital surface point for each digital tooth boundary. The method may further include determining a digital tooth bounding region center of the bounding region. and connecting one or more digital tooth bounding regions represents a digital tooth bounding region. The second trained neural network may include connecting the bounding region centers. The network can include a YOLO neural network. Neural networks can include U-net convolutional neural networks. Each sampled digital surface point and its corresponding associated digital surface point are Determining one or more associated digital surface points may be a column in each determining a sample depth point of the sampling digital surface point; and One or more respective rays extending in the sample plane at an oblique angle from the occlusal direction determining one or more digital surface points that intersect with the sample plane; and determining a direction perpendicular to the spline and the bite direction. The plurality of tilt angles can be within a tilt angle range, the tilt angle range being -45 degrees to +45 degrees. It is possible.

[0162] FIG. 25(b) illustrates an example of a computer-implemented method for generating a panoramic image. At 2350, a 2D depth map of the patient scan data of at least a portion of the patient's dentition is generated. and at 2352, using the trained neural network, or one or more digital tooth bounding regions corresponding to multiple digital teeth determining a region; and, in 2354, applying one or more digital Connecting the tooth bounding regions and, at 2356, one or more sampling a plurality of spline points; and, in 2358, sampling one or more Determine one or more sampled digital surface points from the sampled spline points. and at 2360, determining one or more of the sampled digital surface points corresponding to each sampled digital surface point. and determining a plurality of associated digital surface points.

[0163] The method in FIG. 25(b) may include one or more of the following optional features: The method may include, alone or in combination, determining a digital tooth bounding region center of the region; Connecting the plurality of digital tooth bounding regions may include connecting the plurality of digital tooth bounding regions to each other. The second trained neural net may include connecting the binning region centers. The work can include the YOLO neural network. The network can include a U-net convolutional neural network. The sampling digital surface points and their corresponding associated digital surface points are aligned in a sequence in the panoramic image. Determining one or more relevant digital surface points can be determining sample depth points of the ring digital surface points; and determining an occlusion direction from the sample depth points. Intersect with one or more respective rays extending in the sample plane at an oblique angle from the direction determining one or more digital surface points corresponding to the sample plane; and determining a direction perpendicular to the occlusion direction. The tilt angle can be within the tilt angle range, and the tilt angle range is -45 degrees to +45 degrees. can.

[0164] Some embodiments may include a processing system. The processing system may include a processor. and a computer program including instructions executable by a processor capable of performing steps and a data-readable storage medium, the steps including: receiving a 3D digital model of the patient scan data; and generating a panoramic image from the 3D digital model. generating an image and generating a panoramic image using the first trained neural network; and providing a labeled panoramic image of the one or more regions of the image. One or more areas of a panoramic image with a bell can be inserted into one or more areas of a 3D digital model. The corresponding rough digital surface triangles are then mapped to the labeled 3D digital model. and segmenting the labeled 3D digital model. and providing a D digital model. Alternatively, in some embodiments, the processor The instructions executable by the step include receiving a panoramic image and generating a first trained Using a neural network, we label one or more regions of a panoramic image. providing a labeled panoramic image; and displaying one or more of the labeled panoramic images. The area is then mapped to one or more corresponding rough digital surface triangle labels in the 3D digital model. and providing labeled 3D digital models of the Segmenting the digital model to provide a segmented 3D digital model; It can include.

[0165] One or more features described in this disclosure are part of a processing system; and and / or may be performed on a processing system.

[0166] FIG. 26 illustrates a processing system 14000 in some embodiments. 00 includes a processor 14030 and one or more steps described in this disclosure. A computer-readable storage medium having instructions executable by a processor to perform 14034.

[0167] In some embodiments, one or more steps may be performed by, for example, a user. In some embodiments, one or more features may be selected by a user, e.g., by a display. This can be accomplished using an input device while viewing the digital model at play. In some cases, the computer-implemented method further comprises: an input device for inputting a displayed For example, in some embodiments, The computer-implemented method rotates, scales, translates, and / or scales the digital model. can be manipulated in any manner as known in the art.

[00044] In some embodiments, one or more of the features are selected by a user using an input device. In some embodiments, one or more steps may be performed by, for example, a user It may be initiated using techniques known in the art, such as by selecting another button. In some embodiments, one or more features may be implemented automatically, without user intervention. This can be accomplished dynamically.

[0168]

[0005] In some embodiments, the computer-implemented method further comprises displaying the digital model on a display. from an input device, for example a mouse or a touch screen on a display. For example, the computer-implemented method may accept a start command. Upon receiving the start command, the computer-implemented method may include the steps of: Performing one or more steps using one or more features described in The computer-implemented method, upon receiving the manipulation command, may include: rotate, scale, translate, and / or change the image as known in the art. It may be operated differently in any manner.

[0169] One or more of the features disclosed herein may be implemented without manual or user intervention. The present invention may be implemented and / or accomplished automatically without the need for a computer. One or more of the disclosed features may be performed by a computer-implemented method. The features of the present invention, including but not limited to any method and system, For example, the functions performed by the The computing environment 14042 may include one or more computing devices. Various computing devices that may be incorporated into a computing system Devices (e.g. desktop computers, laptop computers, server computers) computers, tablet computers, gaming systems, mobile devices, programs The device may be any of the following: In some embodiments, the computing system is a cloud-based computing system. It may be a system.

[0170] For example, the computing environment 14042 may include one or more processing units 14030 and and memory 14032. The processing unit executes computer-executable instructions. The processing unit 14030 may be a central processing unit (CPU), a special-purpose A processor in an application-specific integrated circuit (ASIC), or any other type of processor. The multiple processing units 14030 may, for example, execute multiple computer-executable instructions in parallel. In a multiprocessing system, multiple processing units are used to increase processing power. For example, a typical computing environment might include , a central processing unit, and a graphics processing unit or co-processor. The register 14032 may be a volatile memory (e.g., a register, cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory The memory may be a single unit of memory (e.g., a memory), or some combination of both. In the form of computer executable instructions suitable for use in one or more of the methods described herein Store software that implements number innovations.

[0171] The computing system may have additional features. For example, in some embodiments, The computing environment may include storage 14034, one or more input devices 14 036, one or more output devices 14038, and one or more communication connections 1 4037. An interconnect mechanism such as a bus, controller, or network Interconnects the components of a computing environment. Typically, it is the operating system software. The software provides an operating environment for other software running within the computing environment. and coordinates the activities of the components of the computing environment.

[0172] The tangible storage 14034 may be removable or non-removable and may be a magnetic disk, magnetic tape, Magnetic or optical media such as discs or cassettes, CD-ROMs, DVDs, or information It can be used to store in a temporary manner and be accessed within a computing environment. Storage 14034 includes any other medium that can store data. The method further comprises storing instructions for software implementing one or more of the innovations to be implemented.

[0173] The input device may be, for example, a keyboard, a mouse, a pen, or a trackball. Touch input devices, voice input devices, scanning devices, any of a variety of sensors or another device that provides input to the computing environment, or a combination of both. For video coding, the input devices can be cameras, video cards, TV channels, etc. A video card or similar device that accepts video input in analog or digital format. or a CD-ROM or The output devices can be displays, printers, speakers, CD writers, etc. The input / output device may be a computer, a display, a printer, or another device that provides output from the computing environment.

[0174] A communications connection enables communication over a communications medium to another computing entity. Communication media may include computer-executable instructions, audio or video input or output, or or other data in a modulated data signal. A modulated data signal is a to set or change one or more of its characteristics in such a way as to encode information in By way of example, and not limitation, communication media may include electrical, optical, RF or other carrier media. Waves can be used.

[0175] Any of the methods of the present disclosure may include one or more computer-readable storage media 14034 (e.g., For example, one or more optical media disks, volatile memory components (DRAM or SR AM), or non-volatile memory components (such as flash memory or hard drives) The information is stored on a computer (e.g., a smartphone, a computing device, etc.) other mobile devices that contain hardware or programmable automation controllers embodied as computer-executable instructions executed on a computer (including any commercially available computer) (e.g., the computer-executable instructions may be implemented in one or more The term computer-readable storage medium includes signals and carriers. A computer-executable program for implementing the techniques of the present disclosure does not include a communication connection, such as a transmission. The instructions, and any data created or used during the performance of the embodiments of the present disclosure, The method may be stored on one or more computer-readable storage media 14034. The executable instructions may be, for example, a dedicated software application or a web browser. or other software applications (remote computing applications) Software applications accessed or downloaded through Such software may be part of, for example, a single local computer. on a computer (e.g., any suitable commercially available computer) or on one or more networks. Using a network computer in a network environment (e.g., the Internet, Area Network, Local Area Network, Client-Server Network (such as cloud computing networks), or other such networks (via

[0176] For clarity, only certain selected aspects of the software-based implementation are described. Other details that are well known in the art are omitted. It will be understood that the technique is not limited to any particular computer language or program. For example, the technology disclosed herein can be used in C++, Java, Perl, Python, Java Script, Adobe Flash, or any other suitable programming language Similarly, the techniques disclosed herein may be implemented by any particular computer-written software. The present invention is not limited to a specific computer or type of hardware. The specific details of the hardware are well known and need not be described in detail in this disclosure. .

[0177] Any functionality described herein may be implemented, at least in part, by software instead. It is also possible that the present invention may be implemented by one or more hardware logic components. It should be understood that, for example and without limitation, hardware logic may be used. An exemplary type of component is a Field Programmable Gate Array (FPGA). Field-programmable Gate Array, Program-specific Integrated Circuit (ASIC) specific Integrated Circuit), standard products for specific programs (ASSP: Program-spec ific Standard Product), System-on-a-chip (SOC) , Complex Programmable Logic Device (CPLD) etc.

[0178] Additionally, the present invention may be implemented in a variety of ways (e.g., to cause a computer to perform any of the methods of the present disclosure). Any software-based embodiment (including computer executable instructions) may be implemented using any suitable Uploaded or downloaded via communication means or accessed remotely Such suitable communication means may be, for example, the Internet, World Wide Web, Web, Intranets, Software Applications, Cables (Fiber Optic Cables) (including RF, microwave, and infrared communications); magnetic communications; electromagnetic communications (including RF, microwave, and infrared communications); electronic communications "includes any communication, telephone, or other such means of communication."

[0179] In view of the many possible embodiments to which the principles of the present disclosure may be applied, the illustrated embodiment is It is recognized that these are merely examples and should not be construed as limiting the scope of the present disclosure. sea ​​bream. [Explanation of symbols]

[0180] 100 Digital Models 202 First Convolutional Layer 203 Input images 204 Kernel 206 Convolution output, first convolution image 207 Feature Channel 208 First kernel output 212 First Pooling Layer 214 First Pooling Filter 216 First Pooling Image 217 Feature Channel 218 Second Convolutional Layer 219 Feature Channel 220 Second convolution image 222 Second Pooling Layer 224 Second Pooling Image 224 Output forecast 230 fully connected layer 302 layers 304 layers 306 layers 307 Residual 308 layers 310 layers 311 Residual 312 layers 314 layers 315 Residual 316 layers 318 layers 319 Residual 320 layers 322 layers 323 Residual 401 Input Image 402 Contraction Path 404 Expansion Path 406 First Convolutional Layer 408 First Pooling Operation 410 First Convolution 411 First feature map 412 Final feature map 413 Second Convolution 414 Convolutional Layer Input 416 Second Convolutional Layer 418 Third Convolutional Layer 422 Final pooling operation 424 Upsampling 425 upsampling layers 426 Convolutional Layer 427 Fourth convolutional layer, final contraction path convolutional layer 428 Feature Map 429 Concatenation 430 Feature Map 432 Final upsampling layer 434 Last Convolution 435 Output Segment Map 500 Digital Model 502 Boundary Loop 504 Digital Dental Area 506 Digital Non-dental Area 508 Digital Surface Triangles 512 Vertex 514 Digital Surface Triangles 516 sides 518 sides 520 sides 522 Interior Dual Graph Domains 524 Exterior Dual Graph Domains 700 2D Depth Maps 702 Labeled Digital Tooth Bounding Region 800 Digital Model 802 Digital Bounding Box 804 Digital Teeth 806 Occlusal direction 808 Digital bounding box center, first bounding box center 810 2nd bounding box center, 2nd digital bounding box center 812 First Spline 814 Second Spline 816 Third Digital Bounding Box Center 820 Continuous Spline 822 Last bounding box center 824 straight line 826 Last Spline 828 straight line 902 sampling points 904 Continuous Smoothing Spline 906 Digital Model 1002 sampling digital surface points 1004 Continuous Smoothing Spline 1006 3D Digital Models 1008 Sample Plane 1010 Occlusal direction 1012 depth rays, sample point depth rays 1014 sample depth points 1016 First positive inclination ray 1018 First positive tilt angle 1020 First negative tilt ray 1022 First Digital Surface Points 1024 second digital surface points 1050 Digital Model 1052 sample points 1054 Spline 1056 Ray of light 1102 Center row 1103 Sampling Digital Surface Points Line 1104 Column 1105 Line 1106 1110 1st Positive Row 1112 First negative row 1202 Darker Areas 1204 Brighter Areas 1300 Panorama Images 1302 Labeled digital tooth bounding area 1402 Digital Bounding Area 1404 Digital Tooth Bounding Region 1500 3D Digital Models 1502 Digital Tooth Bounding Box 1504 First Digital Tooth Bounding Box Corner 1506 First Ray 1508 Second Digital Tooth Bounding Box Corner 1510 Second Ray 1512 depth points 1513 Third Digital Tooth Bounding Box Corner 1516 The Third Ray 1518 Fourth digital tooth bounding box corner 1520 The Fourth Ray 1602 Digital Tooth Boundary Region 1603 distance 1604 Certainty Area 1605 distance 1606 Uncertain Area 1607 Uncertain Area 1614 First Labeled Digital Surface Area 1616 Unlabeled Region 1618 Second Labeled Digital Surface Area 1620 3rd Labeled Digital Surface Area 1786 sides 1788 triangle 1790 triangle 1792 dihedral angle 1794 The Third Plane 2000 labeled 3D digital models 2002 Labeled Regions 2004 center point 2010 Parabola 2012 Occlusal direction 2020 Separation Area 2030 plane 2032 Lingual point 2034 Buccal point 2036 First Segment Path 2050 Digital Teeth 2051 center point 2052 Lingual point 2053 1st route, 1st segment route 2054 First Side 2056 Second Path, Second Segment Path 2057 Separation area 2058 Second Side 2059 Parabola 2062 Cheek point, face point 2064 sides 2066 sides 2068 inner loop region 2070 Segmented Digital Teeth 2072 Second Segment Path 2074 Boundary Loop 2200 Digital Model 2202 Digital Teeth 2204 Gingival area 2402 Specific Triangle 2404 Adjacent triangles, nearby triangles 2405 First Side 2406 Adjacent Triangles, Nearby Triangles 2407 Second Side 2408 Adjacent triangles, nearby triangles 2409 Third Side 2010 Best Fit Parabola 2411 Segments 2412 Segments 2414 Segmented Digital Surface Mesh 2416 Segments 4000 U-net Neural Network 4002 input images 4004 convolutional layers, first convolutional layer 4005 Second convolution layer 4006 Contraction Path 4007 Third Convolutional Layer 4008 First feature map 4009 4th convolution layer 4010 Second feature map 4011 5th convolution layer 4012 Third feature map 4013 6th convolution layer 4014 4th feature map 4015 7th convolution layer 4016 5th feature map 4018 6th feature map 4020 7th feature map 4021 Expansion path 4031 3rd deconvolution layer 4032 First deconvolution layer 4033 Second deconvolution layer 4034 Third Deconvolution Feature Map 4035 6th deconvolution layer 4036 Second Deconvolution Feature Map 4037 5th deconvolution layer 4038 First deconvolution feature map 4039 4th deconvolution layer 4040 5th deconvolution feature map 4042 4th deconvolution feature map 4044 7th deconvolution layer 4048 6th deconvolution feature map 4050 7th deconvolution feature map 5001 input images 5002 encoder 5004 Conv-1 5008 Res-2 5010 Res-3 5012 Res-4 5014 Res-5 5016 GAU-3 5018 GAU-2 5020 GAU-1 5022 FPA 5030 FPA 5032 Resnet 5034 Feature Map 5036 Global Pooling Branch 5038 Downsampling Branch 5040 Upsampling Branch 5042 1st upsample branch stage 5044 Second Upsample Branch Stage 5046 3rd upsample branch stage 5048 First Downsample Convolution 5050 second downsample convolution 5052 third downsample convolution 5054 3x3 convolution 5056 5x5 convolution 5058 7x7 convolution 5060 Global Pooling 5062 1x1 convolution 5064 upsampling 5070 per-pixel multiplier 5072 Convolutional Layer 5074 Decoder 5075 Segmented Images 5076 First Decoder Stage 5078 Second Decoder Stage 5080 Third Decoder Stage 5102 rows Column 5104 6002 Input images 6004 First ResNet convolutional layer Conv-1 6006 Pooling Step 6010 Res-2 6012 Res-3 6014 Res-4 6016 Res-5 6018 1st Block 6020 First Res-2 Convolution 6022 Second Res-2 Convolution 6024 First Res-3 Convolution 6026 First Res-4 Convolution 6027 First Res-5 Convolution 6028 Bypass 6029 Pooling Stage 6040 Next Block 6050 1st Block 6052 1st block input feature map, input image 6054 First Convolution 6056 First convolution output 6058 Second Convolution 6060 1st block output 6062 Bypass 6064 Second block input feature map 6066 Second Block 8502 Center point 8504 1st x-axis 8506 1st y-axis 8508 Parabola 8510 x-axis 8512 y-axis 8514 Parabola 14000 Processing System 14030 Processor, Processing Unit 14032 Memory, Tangible Memory 14034 Computer-readable storage medium, storage, tangible storage 14036 Input Devices 14037 Communication Connection 14038 Output Device 14042 Computing Environment 50100 GAU 50101 Low-level features 50102 Convolution, Convolutional Layer 50104 High level features 50106 Global Pooling 50108 Feature Map 50110 Convolutional Layer 50112 Multiplier 50114 Weighted low-level features 50116 Total

Claims

1. A computer-implemented method for digitally segmenting teeth in a digital model. hand, receiving a 3D digital model of at least a portion of the patient's dentition from the patient scan data; 、 generating a panoramic image from the 3D digital model; and Using a first trained neural network, one or more of the panoramic images are and providing a labeled panoramic image. The labeled panoramic image includes one or more regions within the 3D digital model. or a corresponding coarse digital surface triangle label; and Providing a digital model, Segmenting the labeled 3D digital model; and The method includes:

2. labeling one or more regions of the panoramic image; and displaying the labeled panoramic image. training the first trained neural network to provide The method of claim 1 .

3. 2. The method of claim 1, wherein the labeled 3D digital model includes one or more regions of uncertainty. The method described.

4. The segmented 3D digital model may include one or more individual dental regions and non-dental regions. The method of claim 1 , comprising a fang region.

5. The first trained neural network is a first convolutional neural network. The method of claim 1 , comprising:

6. The first trained neural network is a semantic segmentation neural network. The method of claim 1 , further comprising a local network.

7. The first trained neural network comprises a YOLO neural network. The method according to claim 1 .

8. Segmenting the 3D digital model comprises segmenting the labeled 3D digital model. The method of claim 1 , further comprising: geometrically segmenting the pattern.

9. said geometrically segmenting comprising curvature-based segmentation. Item 9. The method according to item 8.

10. 10. The method of claim 8, wherein the geometric segmentation comprises two-point segmentation. Method of posting.

11. generating the panoramic image, A second trained neural network is used to generate a digital representation of one or more digital teeth. determining a corresponding digital tooth bounding region or regions; connecting said one or more digital tooth bounding regions by a spline; And, Sampling one or more spline points on the spline; One or more sampled spline points are extracted from the one or more sampled spline points. determining a matching digital surface point; determining one or more associated digital surface points corresponding to each sampled digital surface point; To do, The method of claim 1 , comprising:

12. Determining a digital tooth bounding region center for each digital tooth bounding region and connecting the one or more digital tooth bounding regions.

12. The method of claim 11, wherein the step of: connecting the digital tooth bounding area centers. Method of posting.

13. The second trained neural network comprises a YOLO neural network. The method according to claim 11 .

14. The second trained neural network is a semantic segmentation neural network. The method of claim 11 , further comprising a local network.

15. Each of said sampled digital surface points and its corresponding associated digital surface point is The method of claim 11 , including columns in a llama image.

16. Determining one or more relevant digital surface points determining a sample depth point for each sampled digital surface point; From the sample depth point, a sample plane is formed at one or more inclination angles from the occlusal direction. one or more digital surfaces intersected by one or more respective rays extending within the determining a point where the sample plane is perpendicular to the spline and the bite direction; To communicate, to decide, The method of claim 11 , comprising:

17. The one or more tilt angles are within a tilt angle range, the tilt angle range being between -45 degrees and +4 12. The method of claim 11, wherein the angle is 5 degrees.

18. 1. A system for digitally segmenting teeth in a digital model, comprising: A processor; a computer comprising instructions executable by said processor to perform steps A readable storage medium; wherein the steps include: receiving a 3D digital model of at least a portion of the patient's dentition from the patient scan data; 、 generating a panoramic image from the 3D digital model; and Using a first trained neural network, one or more of the panoramic images are and providing a labeled panoramic image. The labeled panoramic image includes one or more regions within the 3D digital model. or a corresponding coarse digital surface triangle label; and Providing a digital model, Segmenting the labeled 3D digital model; and Including, the system.

19. generating the panoramic image, A second trained neural network is used to generate a digital representation of one or more digital teeth. determining a corresponding digital tooth bounding region or regions; connecting said one or more digital tooth bounding regions by a spline; And, Sampling one or more spline points on the spline; One or more sampled spline points are extracted from the one or more sampled spline points. determining a matching digital surface point; determining one or more associated digital surface points corresponding to each sampled digital surface point; To do, The system of claim 18 , comprising:

20. 1. A computer-implemented method for generating a panoramic image, comprising: receiving a 2D depth map of patient scan data of at least a portion of the patient's dentition; A trained neural network is used to generate a digital representation of one or more sets of teeth. determining one or more digital tooth bounding regions corresponding to said tooth; connecting said one or more digital tooth bounding regions by a spline; And, Sampling one or more spline points on the spline; One or more sampled spline points are extracted from the one or more sampled spline points. determining a matching digital surface point; determining one or more associated digital surface points corresponding to each sampled digital surface point; To do, The method includes:

21. storing executable computer program instructions for segmenting the digital model; 10. A non-transitory computer readable medium for performing a method for a computer-readable apparatus, the computer program instructions comprising: receiving a 3D digital model of at least a portion of the patient's dentition from the patient scan data; 、 generating a panoramic image from the 3D digital model; and Using a first trained neural network, one or more of the panoramic images are and providing a labeled panoramic image. The labeled panoramic image includes one or more regions within the 3D digital model. or a corresponding coarse digital surface triangle label; and Providing a digital model, Segmenting the labeled 3D digital model; and A non-transitory computer readable medium comprising:

Images