Neural Network Margin Proposal

Abstract

A computer-implemented method / system / instructions for automatic margin line proposal includes receiving a 3D digital model of at least a portion of a jaw, the 3D digital model including digital preparation teeth, determining an internal representation of the 3D digital model using a first trained neural network, and determining margin line proposals from a base margin line and the internal representation of the 3D digital model using a second trained neural network.

Description

[Technical Field]

[0001] [Related applications] This application claims the benefits and priority of U.S. Utility Application No. 17 / 245,944, filed on 30 April 2021. The contents of this U.S. Utility Application, in whole, constitute part of this Specification by reference. [Background technology]

[0002] Specialized dental laboratories typically use computer-aided design (CAD) to design dental prostheses based on patient-specific instructions provided by dentists. For example, given a digital surface including the prosthesis, such as a preparation, it may be desirable to define margin lines.

[0003] Traditionally, a significant portion of a technician's work was typically dedicated to locating margin lines for preparations. In the conventional workflow, individual plaster casts for preparations were manually prepared by the technician. One of the main goals of this process was to generate clean, visible margins. Individual plaster casts with "visible" margins were scanned, and a digital surface was acquired. On such a digital surface, margin lines can generally be located with fewer clicks using curvature-based geometry tools.

[0004] In modern workflows, the plaster casting stage is omitted. Either a CT scan or an intraoral scan is performed instead. Both intraoral and CT scanners produce a complete digital jaw surface without individual preparation surfaces. For a complete jaw, a lot of work may be required. It is not possible to rely solely on curvature, and therefore, margin lines are usually drawn step by step. In subgingival cases, margin lines are missing or covered (partially or completely covered by gums and / or blood and saliva) and are formed by an experienced technician. Accurate margin detection in fully automated mode is not always possible due to the variety of shapes, the case of subgingival cases, and the requirements for accurate margin lines. Fully manual margin location / construction can be tedious and time-consuming for the physician or dental technician. [Overview of the Initiative] [Means for solving the problem]

[0005] A computer-aided method for automated margin line proposal includes receiving a 3D digital model of at least a portion of the jaw, the 3D digital model including digitally prepared teeth, using a first trained neural network to obtain an internal representation of the 3D digital model, and using a second trained neural network to obtain a margin line proposal from the base margin line and the internal representation of the 3D digital model. The first and second trained neural networks are trained using a training dataset that includes the uncropped digital surface of the jaw and the target margin line on the surface of the corresponding cropped digital surface. .

[0006] The automated margin line proposal system comprises a processor and a computer-readable storage medium containing instructions executable by the processor, the instructions comprising: receiving a 3D digital model of at least a portion of a jaw, the 3D digital model including digitally prepared teeth; and performing the steps of: obtaining an internal representation of the 3D digital model using a first trained neural network; and obtaining a margin line proposal from a base margin line and the internal representation of the 3D digital model using a second trained neural network. The first and second trained neural networks are trained using a training dataset that includes the uncropped digital surface of the jaw and the target margin line on the surface of the corresponding cropped digital surface. .

[0007] A non-temporary computer-readable medium stores executable computer program instructions for automatically proposing margin lines, the computer program instructions comprising: receiving a 3D digital model of at least a portion of a jaw, the 3D digital model including digitally prepared teeth; using a first trained neural network to obtain an internal representation of the 3D digital model; and using a second trained neural network to obtain margin line proposals from a base margin line and the internal representation of the 3D digital model. The first and second trained neural networks are trained using a training dataset that includes the uncropped digital surface of the jaw and the target margin line on the surface of the corresponding cropped digital surface. . [Brief explanation of the drawing]

[0008] [Figure 1] This is a 3D explanatory diagram of an example of a plaster cast, showing a perspective view. [Figure 2] This is a top perspective view of a 3D digital model of at least a portion of a digital jaw in several exemplary embodiments. [Figure 3] This is a perspective view of a 3D digital model of at least a portion of a digital jaw in several embodiments as an example. [Figure 4] These are perspective views of 3D digital point clouds in several exemplary embodiments. [Figure 5]Top perspective view of a 3D digital model of at least a portion of a digital jaw having a set occlusal direction, preparation die, and buccal direction in some embodiments as an example. [Figure 6] Diagram of a convolutional neural network in some embodiments as an example. [Figure 7] Top perspective view of an example of a 2D depth map of a digital model in some embodiments as an example. [Figure 8(a)] Diagram of a hierarchical neural network in some embodiments as an example. [Figure 8(b)] Diagram of a hierarchical neural network in some embodiments as an example. [Figure 9] Diagram of a deep neural network in some embodiments as an example. [Figure 10] Diagram of a computer-implemented method for automatic margin line proposal in some embodiments as an example. [Figure 11] Perspective view of an example of a 3D digital model showing a proposed margin line from a base margin line in some embodiments as an example. [Figure 12(a)] Perspective view of a 3D digital model having a preparation tooth and a proposed margin line in some embodiments as an example. [Figure 12(b)] Perspective view of a 3D digital model having a preparation tooth and a proposed margin line in some embodiments as an example. [Figure 13] Diagram of a computer-implemented method for automatic margin line proposal in some embodiments as an example. [Figure 14] Diagram of a system in some embodiments as an example.

Mode for Carrying Out the Invention

[0009] For the purposes of this description, certain aspects, advantages, and novel features of embodiments of the present disclosure are described herein. The disclosed methods, apparatus, and systems should not be construed as limiting. Rather, the present disclosure covers all novel and non-obvious features and aspects of the various disclosed embodiments, individually and in various combinations and partial combinations of them. These methods, apparatus, and systems are not limited to any particular aspect, feature, or combination thereof, and the disclosed embodiments do not require the existence of one or more particular advantages or the resolution of any problem.

[0010] The operation of some of the disclosed embodiments is described in a specific order for the sake of presentation, but it should be understood that this method of description includes reordering unless a specific order is required by the specific wording described below. For example, the operations described sequentially can, in some cases, be reordered or performed simultaneously. Furthermore, for simplicity, the accompanying diagrams may not show various ways in which the disclosed methods can be used in combination with other methods. In addition, the description may sometimes use terms such as “provide” or “achieve” to describe the disclosed methods. The actual operations corresponding to these terms may vary depending on the particular embodiment and will be readily apparent to those skilled in the art.

[0011] As used in this application and claims, the singular forms of terms ("a," "an," and "the") include the plural form unless the context clearly indicates otherwise. In addition, the term "include" means "to provide / comprise." Furthermore, the terms "combined" and "associated" generally mean to be combined or linked electrically, electromagnetically, and / or physically (e.g., mechanically or chemically), and do not exclude the existence of intermediate elements between combined or associated items unless specifically denied.

[0012] In some examples, values, procedures, or devices may be referred to as “minimum,” “best,” “smallest,” etc. It will be understood that such descriptions are intended to indicate that a choice can be made from many alternatives, and that such a choice does not need to be better, smaller, or otherwise preferable to the other choices.

[0013] In the following explanation, certain terms such as “up,” “down,” “upper side,” “lower side,” “horizontal,” “vertical,” “left,” and “right” may be used. These terms are used to make the explanation somewhat clearer when dealing with relative relationships, where applicable. However, these terms are not intended to imply absolute relationships, positions, and / or orientations. For example, with respect to an object, the “upper” surface may become the “lower” surface simply by inverting the object. Yet, the object is still the same object.

[0014] In the conventional workflow, individual plaster casts for preparation were manually fabricated by technicians. One of the main goals of this process was to generate clean, visible margins. Individual plaster casts with "obvious" margins were scanned, and a digital surface was obtained. Figure 1 illustrates, for example, an individual plaster cast 102 fabricated with a clearly visible margin line 104.

[0015] In modern workflows, the plaster casting stage is omitted. Either a CT scan or an intraoral scan is performed instead. Both intraoral and CT scanners generate only a complete digital jaw surface without individual preparation surfaces. Figure 2 shows an example of a 3D digital model of at least a portion of the digital jaw surface 1202. The digital jaw surface 1202 includes, for example, prepared teeth 1204, but margin lines are not established in the scanned model.

[0016] In some embodiments, a computer implementation can determine margin line proposals in a 3D digital model using one or more trained neural networks. In some embodiments, one or more trained neural networks can perform encoding and decoding. In some embodiments, at least one of the neural networks may be a hierarchical neural network ("HNN") that can be used, for example, to perform encoding.

[0017] Some embodiments may include receiving a 3D digital model of at least a portion of the jaw. The 3D digital model may include digitally prepared teeth. In some embodiments, the 3D digital model may be generated from a CT scanner. One example of a CT scan is described in Nikolskiy et al., U.S. Patent Application Publication No. 20180132982, which is incorporated herein by reference in its entirety. Other types of CT scanning systems known in the art may also generate 3D digital models. A computed tomography (CT) scanner can use X-rays to create detailed images of a physical impression. Multiple such images are then combined to form a 3D model of the patient's dental condition. A CT scanning system may include an X-ray source that emits an X-ray beam. The object being scanned may be placed between the source and an X-ray detector. The X-ray detector may be further connected to a processor, which is configured to receive information from the detector and convert this information into a digital image file. Those skilled in the art will recognize that the processor may include one or more computers capable of direct connection to the detector, wireless connection, connection via a network, or direct or indirect communication with the detector 148 by other means.

[0018] An example of a suitable scanning system includes the Nikon Model XTH255CT scanner, commercially available from Nikon Corporation. This exemplary scanning system includes a 225kV microfocus X-ray source with a 3μm focal size that provides high-performance image acquisition and volume processing. The processor may include a storage medium configured with instructions for managing the data collected by the scanning system. As described above, during the operation of the scanning system, the impression is placed between the X-ray source and the X-ray detector. As the impression is rotated in a given position between the X-ray source and the detector, a series of images of the impression are collected by the processor. An example of a single image. The image may be a radiograph, projection, or other form of digital image. In one embodiment, a series of images are collected as the impression is rotated in a given position between the X-ray source and the detector. In other embodiments, more or fewer images may be collected, as will be understood by those skilled in the art. Multiple images of the impression are generated by the processor of the scanning system and stored in a storage medium contained within the processor. In the scanning system, these multiple images can be used by software contained within the processor to perform additional operations. For example, in one embodiment, multiple images undergo tomographic reconstruction to generate a 3D virtual image from multiple 2D images generated by a scanning system. The 3D virtual image has the form of a volumetric image or volumetric density file generated from multiple radiographs via a reconstruction algorithm associated with the scanning system. The volumetric density file may contain one or more voxels. In one embodiment, the volumetric image is converted to a surface image using a surface imaging algorithm. In the illustrated embodiment, the volumetric image is converted to a surface image having a format suitable for use with dental restoration design software such as FastDesign® dental design software provided by Glidewell Laboratories, Inc. in Newport Beach, California, USA (e.g., .STL file format).

[0019] In some embodiments, 3D digital models can be generated from an optical scanner. For example, in some embodiments, 3D digital models can be generated by an intraoral scanner or other device. Digital jaw models can also be generated, for example, by an intraoral scan of the patient's dental condition. In some embodiments, each electronic image is obtained by a direct intraoral scan of the patient's teeth. This is usually done, for example, in a dental clinic or dental practice, and performed by a dentist or dental technician. In other embodiments, each electronic image is obtained indirectly by scanning an impression of the patient's teeth, scanning a physical model of the patient's teeth, or by other methods known to those skilled in the art. This is usually done, for example, in a dental laboratory, and performed by a dental technician. Thus, the methods described herein are suitable and applicable for use beside a patient's chair, in a dental laboratory, or in other environments.

[0020] Figure 3 shows an example of a digital jaw model 302. This digital jaw model can be generated by scanning a physical impression using any scanning technique known in the art, or by an intraoral scan of the patient's oral cavity (dental condition). The scanning techniques include, but are not limited to, optical scans and CT scans. Conventional scanners typically acquire the shape of the physical impression / patient's dental condition in three dimensions during the scan and digitize this shape into a three-dimensional digital model. The digital jaw model 302 may include, for example, multiple interconnected polygons with topologies corresponding to the shape of the physical impression / patient's dental condition of the jaw. In some embodiments, the polygons may include two or more digital triangles. In some embodiments, the scanning process can generate STL, PLY, or CTM files that are suitable for use with dental design software, such as FastDesign® dental design software provided by Glidewell Laboratories, Inc. in Newport Beach, California, USA.

[0021] In some embodiments, the 3D digital model can be a 3D digital point cloud. In the case of optical scanning, an optical scanner emits a light beam to scan and digitize a physical dental impression, such as any dental impression. Alternatively, the optical scanner can directly scan the patient's dental condition, such as in the case of an intraoral scanner. The data obtained from scanning the surface of a physical dental impression / dental condition can take the form of a collection of points, i.e., a point cloud, triangles, or a digital surface mesh. A 3D model can represent a physical dental impression or dental condition in digital form by using a collection of points in 3D space connected by various geometric entities, such as triangles. Scans can be stored, for example, locally or remotely, for use in the methods described herein. Scans can be saved as 3D scans, point clouds, or digital surface meshes for use in the methods described herein.

[0022] In the case of CT scans, the digital surface mesh and digital dental model may be created / determined by the Marching Cubes method or by other digital model generation methods and techniques known in the art, using the method described in Application No. 16 / 451,315 (U.S. Patent Application Publication No. 20200405455), “Processing CT Scan of Dental Impression,” which has been assigned to the assignee of this application. The aforementioned application is incorporated herein by reference in its entirety.

[0023] For example, point clouds can be automatically generated and / or adjusted (reduced) by a computer-aided implementation. The computer-aided implementation receives a volume density file generated by a CT scanner as input. The volume density file may contain voxels representing density values ​​at voxel locations within a volume density volume. The computer-aided implementation compares the ISO value of a selected density with the density of one or more voxels in the volume density file and generates digital surface points in the point cloud at the ISO value of the selected density. The ISO value of the density can be a selectable value that can be chosen by the user, and / or can be automatically determined in some embodiments. In some embodiments, if the ISO value of the selected density corresponds to the density of one or more voxels in the volume density file, the computer-aided implementation can generate digital surface points and place them in a virtual 3D space at locations in the point cloud corresponding to the locations of one or more voxels in the volume density file. In some embodiments, as discussed below, if the ISO value of the selected density lies between two voxel density values, the computer implementation can generate zero or more digital surface points and place them in a virtual 3D space at positions (which may be more) corresponding to positions (which may be more) between two voxel positions along a voxel edge. The computer implementation can optionally adjust the point cloud. The computer implementation can generate a digital surface mesh of either the point cloud or the adjusted point cloud.

[0024] Figure 4 shows an example of a generated point cloud 7000 visible on a display in several embodiments. The generated point cloud 7000 includes generated digital surface points such as digital surface points 7002 at any position of a selected density ISO value in the virtual 3D space.

[0025] In some embodiments, the 3D point cloud may include augmented information. This augmented information may include additional geometric data other than point coordinates, such as surface normal directions, arithmetic mean curvature values, and / or color from the generated digital surface mesh. The normal direction may be perpendicular to a plane, such as a digital surface, with respect to a given point. In some embodiments, the 3D point cloud may include a mesh representation, and the augmented information may be obtained from the generated digital surface mesh.

[0026] In some embodiments, the 3D digital model may include an occlusal direction, a buccal direction, and / or a preparation die region. In some embodiments, these features can provide, for example, a normalized orientation of the 3D digital model. For example, the occlusal direction may be the normal direction to the occlusal plane, the digital preparation die may be the region around the digitally prepared teeth, and the buccal direction may be the direction toward the cheek in the oral cavity. Figure 5 shows an example of a 3D digital model 500 of at least a portion of a patient's dentition, which may include, for example, a digital jaw 502 including digitally prepared teeth 504. The 3D digital model 500 may include an occlusal direction 506, a digital preparation die region 508, and a buccal direction 510.

[0027] In some embodiments, the 3D digital model may include an occlusal orientation. The occlusal orientation of the digital model can be determined using any technique known in the art. Alternatively, in some embodiments, the occlusal orientation can be specified, for example, by a user manipulating the digital model on a display using an input device such as a mouse or touchscreen, as described herein. In some embodiments, the occlusal orientation can be determined, for example, using the occlusal axis technique described in "Processing Digital Dental Impression" of Nikolskiy et al. U.S. Patent Application No. 16 / 451,968 (U.S. Patent Application Publication No. 20200405464). The entire U.S. Patent Application is incorporated herein by reference. In some embodiments, the occlusal orientation can be determined once for each 3D digital model. Alternatively, in some embodiments, the occlusal orientation can be determined automatically.

[0028] In some embodiments, the occlusal direction can be determined using a trained 3D convolutional neural network ("CNN") on a volume (voxel) representation. In some embodiments, the DNN can be a convolutional neural network ("CNN") which is a network that uses convolution instead of general matrix multiplication in at least one of the hidden layers of a deep neural network. The output values ​​of a convolutional layer can be computed by applying a kernel function to a subset of the values ​​of the preceding layers. A computer implementation can train a CNN by adjusting the weights of the kernel function based on training data. Each value in a particular convolutional layer can be computed using the same kernel function.

[0029] Figure 6 shows an example of a CNN in several embodiments. For illustrative purposes, a 2D CNN is shown. A 3D CNN may have a similar architecture but can use a 3D kernel (x, y, z axes) to provide a 3D output after each convolution. The CNN may include one or more convolutional layers, such as a first convolutional layer 202. The first convolutional layer 202 can apply a kernel (also called a filter), such as kernel 204, across an input image, such as an input image 203, and optionally apply an activation function to generate one or more convolutional outputs, such as a first kernel output 208. The first convolutional layer 202 may include one or more feature channels. By applying a kernel, such as kernel 204, and optionally an activation function, it can generate a first convolutional output, such as a convolutional output 206. The kernel can then proceed to the next set of pixels in the input image 203 based on the stride length, and apply kernel 204 and optionally an activation function to generate a second kernel output. The kernel can be applied to all pixels in the input image 203 in this manner. In this way, the CNN can generate a first convolved image 206 which may contain one or more feature channels. In some embodiments, the first convolved image 206 may contain one or more feature channels such as 207. In some cases, the activation function may be, for example, a RELU activation function. Other types of activation functions may also be used.

[0030] The CNN may also include one or more pooling layers, such as a first pooling layer 212. The first pooling layer may apply filters, such as a pooling filter 214, to the first convolved image 206. Any type of filter can be used. For example, the filter could be a maxim filter (which outputs the maximum value of the pixels to which the filter is applied) or an average filter (which outputs the average value of the pixels to which the filter is applied). One or more pooling layers may downsample the input matrix to reduce its size. For example, the first pooling layer 212 may reduce / downsample the first convolved image 206 by applying the first pooling filter 214 to provide the first pooled image 216. The first pooled image 216 may include one or more feature channels 217. The CNN may optionally apply one or more additional convolutional layers (and activation functions) and one or more pooling layers. For example, a CNN may apply a second convolutional layer 218 and optionally an activation function to output a second convolutional image 220 which may contain one or more feature channels 219. A second pooling layer 222 may apply a pooling filter to the second convolutional image 220 to generate a second pooled image 224 which may contain one or more feature channels. A CNN may include one or more convolutional layers (and activation functions) and one or more corresponding pooling layers. The output of the CNN may be sent to a fully connected layer which may optionally be part of one or more fully connected layers 230. One or more fully connected layers may provide output predictions such as output predictions 224. In some embodiments, the output predictions 224 may include, for example, labels for teeth and surrounding tissues.

[0031] In some embodiments, the trained occlusal direction 3D CNN can be trained using one or more 3D voxel representations, each representing a patient's dentition, along with optionally augmented data such as the surface normal of each voxel. The 3D CNN can perform 3D convolutions, which use a 3D kernel instead of a 2D kernel to process the 3D input. In some embodiments, the trained 3D CNN receives 3D voxel representations having voxel normals. In some embodiments, an N×N×N×3 floating-point tensor can be used. In some embodiments, N can be, for example, 100. Other suitable values ​​of N can be used. In some embodiments, the trained 3D CNN can include four levels of 3D convolution and two linear layers. In some embodiments, the training set of the 3D CNN can include one or more 3D voxel representations, each representing a patient's dentition. In some embodiments, each 3D voxel representation in the training set can include occlusal directions that have been manually marked by the user or by other techniques known in the art. In some embodiments, the training set may include tens of thousands of 3D voxel representations, each of which has a marked occlusal direction. In some embodiments, the training dataset may include 3D point cloud models, each with a marked occlusal direction.

[0032] In some embodiments, the 3D digital model may include the 3D center of the digital preparation die. In some embodiments, the 3D center of the digital preparation die may be manually set by the user. In some embodiments, the 3D center of the digital preparation die may be set using any technique known in the art.

[0033] In some embodiments, the 3D center of the digital preparation die can be determined automatically. For example, in some embodiments, the 3D center of the digital preparation die can be determined using a neural network in a 3D point cloud aligned by occlusion. In some embodiments, the trained neural network can provide the 3D coordinates of the center of the digital preparation bounding box. In some embodiments, the neural network can be any neural network capable of performing segmentation on the 3D point cloud. For example, in some embodiments, the neural network can be a PointNet++ neural network segmentation as described herein. In some embodiments, the digital preparation die can be determined by a sphere of a fixed radius around the 3D center of the digital preparation. In some embodiments, this fixed radius can be, for example, 0.8 cm for molars and premolars. For example, in some embodiments, other suitable values ​​for the fixed radius can be determined and used. In some embodiments, training the neural network may include using a sampled point cloud (without extension) of the digital jaw centered on the mass center of the jaw. In some embodiments, the digital jaw point cloud can be oriented so that the occlusal direction is perpendicular. In some embodiments, the training dataset can include a 3D digital model of a patient's dentition, such as a digital jaw, in which one or more points within the margin lines of the prepared teeth can be marked by the user using an input device or by any technique known in the art. In some embodiments, the neural network can utilize segmentation to return a bounding box containing the selected points. In some embodiments, the segmentation used can be, for example, PointNet++ segmentation. In some embodiments, the training set can be tens of thousands of points.

[0034] In some embodiments, the 3D center of the digital preparation die can be automatically determined based on a uniform depth map image of the jaw. In the training dataset, the die center position can be determined as the geometric center of the margin marked by the technician. In some embodiments, the final margin point from the completed state can be used. In some embodiments, the network can receive a depth map image of the jaw from an occlusal chart and return the position of the die center (X,Y) in the pixel coordinates of the image. For training, a dataset can be used that includes depth map images and the corresponding ground truth, i.e., floating-point X and Y values. In some embodiments, the training set can be tens of thousands of points.

[0035] In some embodiments, the 3D digital model may include a buccal orientation. In some embodiments, the buccal orientation is manually set by the user. In some embodiments, the buccal orientation can be determined using any technique known in the art. In some embodiments, the buccal orientation can be determined automatically. In some embodiments, the buccal orientation can be determined by providing a 2D depth map image of the 3D digital model mesh to a trained 2D CNN. In some embodiments, the trained 2D CNN processes the image representation. Some embodiments of the computer implementation may optionally include generating a 2D image from the 3D digital model. In some embodiments, the 2D image may be a 2D depth map. The 2D depth map may include a 2D image in which each pixel has the distance from the orthographic camera to the object along a line passing through that pixel. The object may, for example, in some embodiments, be the surface of a digital jaw model. In some embodiments, the input may include an object such as a 3D digital model of a patient's dentition ("digital model"), such as the jaw, and the orientation of the camera. In some embodiments, the orientation of the camera may be determined based on the occlusal orientation. The occlusal direction is the normal direction of the occlusal plane, and the occlusal plane of a digital model can be determined using any technique known in the art. Alternatively, in some embodiments, the occlusal direction can be specified, for example, by a user manipulating the digital model on a display using an input device such as a mouse or touchscreen, as described herein. In some embodiments, the occlusal direction can be determined, for example, using the occlusal axis technique described in "PROCESSING DIGITAL DENTAL IMPRESSION" of Nikolskiy et al. U.S. Patent Application No. 16 / 451,968 (U.S. Patent Application Publication No. 20200405464). The entirety of this U.S. Patent Application constitutes part of this specification by reference.

[0036] A 2D depth map can be generated using any technique known in the art, including, for example, a z-buffer or ray tracing. For example, in some embodiments, a computer implementation can initialize the depth of each pixel (j,k) to the maximum length and the pixel color to the background color. For each pixel in the projection of polygons onto a digital surface such as a 3D digital model, the computer implementation can determine the depth z of the polygon at (x,y) corresponding to pixel (j,k). If z < the depth of pixel (j,k), the pixel depth is set to depth z. "z" can refer to the rule that the central axis of the camera's field of view is in the direction of the camera's z-axis, and does not necessarily refer to the absolute z-axis of the scene. In some embodiments, a computer implementation can also set the pixel color to a color other than the background color. In some embodiments, the polygon can be, for example, a digital triangle. In some embodiments, the depth in the map can be pixel-wise. Figure 7 shows an example of a 2D depth map of a digital model in some embodiments.

[0037] In some embodiments, the 2D depth map image may include the Von Mises mean of 16 rotated versions of the 2D depth map. In some embodiments, the buccal direction may be determined after determining the occlusal direction and the 3D center of the digital preparation die. In some embodiments, the 2D depth map image may be an image of a portion of the digital jaw around the digital preparation die. In some embodiments, regression may be used to determine the buccal direction. In some embodiments, the 2D CNN may include, for example, GoogleNet Inception v3, which is known in the art. In some embodiments, the training dataset may include, for example, the buccal direction marked in a 3D point cloud model. In some embodiments, the training dataset may include tens of thousands to hundreds of thousands of images.

[0038] Some embodiments of the computer implementation may include using a first trained neural network to obtain an internal representation of a 3D digital model. In some embodiments, the first trained neural network may include an encoder neural network. In some embodiments, the first trained neural network may include a neural network for 3D point cloud analysis. In some embodiments, the first trained neural network may include a trained hierarchical neural network ("HNN"). In some embodiments, the HNN may include a PointNet++ neural network. In some embodiments, the HNN may be any message-passing neural network that processes geometric structures. In some embodiments, the geometric structure may include graphs, meshes, and / or point clouds.

[0039] In some embodiments, the computer implementation can use an HNN such as PointNet++ for encoding. PointNet++ is described in "PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space" (Charles R. Qi, Li Yi, Hao Su, Leonidas J. Guibas, Stanford University, June 2017). This entire document is incorporated herein by reference. Hierarchical neural networks can, for example, hierarchically process a set of points sampled in a metric space. An HNN such as PointNet++ or other HNNs can, in some embodiments, be implemented by finding local structures resulting from a metric. In some embodiments, an HNN such as PointNet++ or other HNNs can be implemented by first partitioning a set of points into two or more overlapping local regions based on a distance metric. This distance metric can be based on a basis space. In some embodiments, local features can be extracted. For example, in some embodiments, granular geometric structures can be found from small local neighborhoods. Small local neighborhood features can, in some embodiments, be grouped into larger units. In some embodiments, larger units can be processed to provide a higher level of features. In some embodiments, the process is repeated until all features of the entire set of points are obtained. Unlike volumetric CNNs that scan space using a fixed stride, local receptive fields in HNNs such as PointNet++ or other HNNs depend on both the input data and the metric. Also, in contrast to CNNs that scan a vector space that is independent of the data distribution, the sampling strategy in HNNs such as PointNet++ or other HNNs generates receptive fields in a data-dependent manner.

[0040] In some embodiments, an HNN such as PointNet++ or other HNNs can determine, for example, how to partition a set of points from an abstract set of points or local features using a local feature learner. In some embodiments, the local feature learner can be, for example, PointNet, or any other suitable feature learner known in the art. In some embodiments, the local feature learner can, for example, process a non-ordered set of points to perform semantic feature extraction. The local feature learner can abstract one or more sets of local points / features to a higher level representation. In some embodiments, the HNN can recursively apply the local feature learner. For example, in some embodiments, PointNet++ can recursively apply PointNet to nested partitions of an input set.

[0041] In some embodiments, an HNN can define a parcel of overlapping point sets by defining each parcel as a neighborhood ball in Euclidean space having parameters that may include, for example, the location and scale of the centroid. The centroid can be selected from the input set by, for example, sampling the farthest point known in the art. One advantage of using an HNN is that the local receptive field can depend on the input data and metrics, thus including, for example, efficiency and effectiveness. In some embodiments, the HNN can utilize neighborhoods at multiple scales, which can enable, for example, robustness and the acquisition of detail.

[0042] In some embodiments, the HNN can include hierarchical point set feature learning. For example, in some embodiments, the HNN can construct a hierarchical grouping of points and abstract local regions that grow progressively larger along the hierarchy. In some embodiments, the HNN can include multiple levels of set abstraction. In some embodiments, a set of points is processed and abstracted at each level to generate a new set with fewer elements. In some embodiments, the set abstraction level can include three layers: a sampling layer, a grouping layer, and a local feature learner layer. In some embodiments, the local feature learner layer can be, for example, PointNet. The set abstraction level can take an input of an N×(d+C) matrix consisting of N points having d-dimensional coordinates and C-dimensional point features, and can output an N'×(d+C') matrix of N' subsampled points having d-dimensional coordinates and new C'-dimensional feature vectors that can summarize the local context.

[0043] In some embodiments, the sampling layer can select or sample a set of points from the input points. In some embodiments, the HNN can define these selected / sampled points as the centroid of a local region. For example, input points to the sampling layer {x1, x2, ..., x n For}, use iterative farthest point sampling (FPS) to obtain a subset of points

number

number

[0044] The grouping layer can, for example, in some embodiments, find one or more sets of local regions by finding neighbors around each centroid. In some embodiments, the input to this layer can be a set of points of size N × (d + C) and the coordinates of a centroid having size N' × d. In some embodiments, the output of the grouping layer can include groups of point sets having size N' × K × (d + C). For example, in some embodiments, each group can correspond to a local region, where K can be the number of points in the neighborhood of the centroid point. In some embodiments, K can vary from group to group. However, the next layer, namely the PointNet layer, can, for example, convert a flexible number of points into a fixed-length local region feature vector. Neighbors can, in some embodiments, be defined by, for example, a metric distance. A ball query can, for example, in some embodiments, find all points within a radius to a query point. An upper limit can be set for K. In an alternative embodiment, K nearest neighbor (kNN) search can be used. kNN can find a fixed number of neighbors. On the other hand, the local neighborhoods of ball queries can guarantee a fixed domain scale, and therefore, for example in some embodiments, make one or more local domain features more generalizable across space. This may be preferable in some embodiments for, for example, semantic point labeling or other tasks requiring local pattern recognition.

[0045] In some embodiments, the local feature learner layer can encode local region patterns into feature vectors. For example, M⊆R n Let M be a set of points and d be the distance metric. Then X = (M, d) is the Euclidean space X n Assuming it is a discrete metric space inherited from, the local feature learner layer can find a function f that takes X as input and outputs semantic interest information about X. Function f can be a classification function that assigns labels to X, or a segmentation function that assigns point-specific labels to each member of M.

[0046] Some embodiments include, for example, x i ∈R d The set of non-ordered points {x1, x2, ..., x n Given}, PointNet can be used as a local feature learner layer that can define an aggregate function f:X→R that maps a set of points to a vector, as shown in the following equation.

number

[0047] In some embodiments, γ and h can be, for example, a multi-layer perceptron (MLP) network or other suitable alternative networks known in the art. The function f can be, for example, in some embodiments invariant under the rearrangement of input points and can approximate any continuous set function. The response of h in some embodiments can be interpreted as spatial coding of points. PointNet is described in "PointNet: Deep Learning on Point Sets for 3D Classification and Segmentation" by RQ Charles, H. Su, M. Kaichun, and LJ Guibas (2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 77-85). The entire document is incorporated herein by reference.

[0048] In some embodiments, the local feature learner layer can receive N' local regions of a point. The data size can be, for example, N' × K × (d + C). In some embodiments, each local region is abstracted in the output by, for example, its centroid and local features encoding the neighborhood of the centroid. The output data size can be, for example, N' × (d + C). In some embodiments, the coordinates of a point within a local region can be transformed into a local reference frame relative to the centroid point: for i = 1, 2, ..., K and j = 1, 2, ..., d

number

[0049] In some embodiments, the local feature learner can handle the non-uniform density of the input point set, for example, through a density adaptation layer. The density adaptation layer can learn to combine the features of regions scaled differently when the input sampling density changes. In some embodiments, the density adaptation hierarchical network is, for example, a PointNet++ network. The density adaptation layer can include, for example, multi-scale grouping ("MSG") or multi-resolution grouping ("MRG") in some embodiments.

[0050] In MSG, in some embodiments, multi-scale patterns can be obtained by applying grouping layers with different scales and then extracting the features of each scale. Extracting the features of each scale can be done, for example, by using PointNet in some embodiments. In some embodiments, for example, the features at different scales can be concatenated to provide multi-scale features. In some embodiments, the HNN can learn a combination of multi-scale features optimized by training. For example, random input dropout can be used, where the random input points are dropped input points with a probability of being randomized. For example, in some embodiments, as an example, a dropout rate of θ uniformly sampled from [0, p] when p is less than or equal to 1 can be used. As an example, p can be set to 0.95 in some cases so that an empty point set is not generated. For example, in some embodiments, other suitable values can be used.

[0051] In MRG, at level L i the features of one region can be, for example, the concatenation of two vectors. The first vector can be, for example, in some embodiments, the lower-level L i-1The first vector is obtained by summarizing the features in each sub-region from the first vector. This can be done using a set abstraction level. In some embodiments, the second vector can be features obtained by directly processing the raw points of the local region, for example, using a single PointNet. When the local region density is low, the first vector contains fewer points and includes sampling missings, so in some embodiments, the second vector can be weighted more heavily. When the local region density is high, the first vector can recursively provide finer details at a lower level due to inspection at a higher resolution, so in some embodiments, the first vector can be weighted more heavily.

[0052] In some embodiments, point features of a defined segmentation can be propagated. For example, in some embodiments, a hierarchical propagation strategy can be used. In some embodiments, feature propagation is N l From ×(d+C) points N l-1 This can include propagating point features to individual points, where N l-1 and N l (N l is N l-1 The following is the size of the point sets of input and output at the set abstraction level l. In some embodiments, feature propagation is N l-1 N in the coordinates of the individual points l This can be achieved through interpolation of the feature values ​​f of the individual points. In some embodiments, for example, an inverse distance-weighted average based on the k-nearest neighbor method can be used (where p=2 and k=3 in the following equation; other suitable values ​​can also be used). l-1The interpolated features in the model can, for example, be concatenated with skip-linked point features from a set abstraction level in some embodiments. In some embodiments, the concatenated features can be passed through a unit PointNet, for example, which can be analogous to a single convolution in a convolutional neural network. For example, in some embodiments, shared fully connected layers and ReLU layers can be applied to update the feature vector for each point. In some embodiments, the process can be repeated until the propagated features to the original set of points are obtained.

number

[0053] In some embodiments, a computer implementation can implement one or more neural networks as disclosed or known in the art. Any specific structure and values, as well as any other features, relating to one or more neural networks as disclosed herein are provided merely as examples, and any suitable variant or equivalent form can be used. In some embodiments, one or more neural network models can be implemented, for example, based on the Pytorch Geometry package.

[0054] Figures 8(a) and 8(b) show examples of HNNs in several embodiments. The HNN may include a hierarchical point set feature learner 802, the output of which can be used to perform segmentation 804 and / or classification 806. The hierarchical point set feature learner 802 uses points in 2D Euclidean space as an example, but can process input 3D images in 3D. As shown in the example in Figure 8(a), the HNN may receive an input image 808 having (N,d+C), perform a first sampling and grouping operation 810 to generate a first sampled and grouped image 812 having (N1,K,d+C). The HNN may then provide the first sampled and grouped image 812 to PointNet in 814, providing a first abstracted image 816 having (N1,d+C1). The first abstracted image 816 can undergo sampling and grouping 818 to provide a second sampled and grouped image 820 having (N2, K, d+C1). The second sampled and grouped image 820 is provided to the PointNet neural network 822, which can output a second abstracted image 824 having (N2, d+C2).

[0055] In some embodiments, the second abstracted image 824 can be segmented by HNN segmentation 804. In some embodiments, HNN segmentation 804 can take in the second abstracted image 824 and perform a first interpolation 830, the output of which can be concatenated with the first abstracted image 816 to provide a first interpolated image 832 having (N1,d+C2+C1). The first interpolated image 832 is provided to the unit PointNet in 834 to provide a first segmented image 836 having (N1,d+C3). The first segmented image 836 can be interpolated in 838, the output of which can be concatenated with the input image 808 to provide a second interpolated image 840 having (N1,d+C3+C). The second interpolated image 840 is provided to the unit PointNet in 842 to provide a segmented image 844 having (N,k). The segmented image 844 can, for example, provide a score for each point.

[0056] As shown in the example in Figure 8(b), the second abstracted image 824 can be classified by an HNN classification 806 in some embodiments. In some embodiments, the HNN classification can take in the second abstracted image 824 and provide it to a PointNet network (860), and its output 862 can be provided to one or more fully connected layers such as a connected layer 864, and its output can provide a class score 866.

[0057] Some embodiments of the computer implementation may include using a second neural network to derive margin line proposals from the base margin line and the internal representation of the 3D digital model.

[0058] In some embodiments, the base margin line can be pre-calculated once for each network type. In some embodiments, the network type can include molars and premolars. For example, in some embodiments, other suitable network types can be used. In some embodiments, the same base margin line can be used as the initial margin line for each scan. In some embodiments, the network type can include other types. In some embodiments, the base margin line is three-dimensional. In some embodiments, the base margin line can be determined based on the margin lines from the training dataset used to train the first and second neural networks. In some embodiments, the base margin line can be the pre-calculated arithmetic mean or average of the margin lines of the training dataset. In some embodiments, any type of arithmetic mean or average can be used.

[0059] In some embodiments, the margin line proposal can be a free-form margin line proposal. In some embodiments, the second trained neural network can include a decoder neural network. In some embodiments, the decoder neural network can perform guided decoding by concatenating its internal representation with specific point coordinates. In some embodiments, guided decoding can generate a closed surface as described in "A Papier-Mache Approach to Learning 3D Surface Generation" by T. Groueix, M. Fisher, VG Kim, BC Russell, and M. Aubry (2018 IEEE / CVF Conference on Computer Vision and Pattern Recognition, 2018, pp. 216-224). The entirety of this document is incorporated herein by reference.

[0060] In some embodiments, the decoder neural network may include a deep neural network ("DNN"). (See Figure 9.) Figure 9 is a high-level block diagram showing the structure of a deep neural network (DNN) 400 according to some embodiments of the present disclosure. The DNN 400 has multiple layers N i , N h,1 , N h,l-1 , N h,l , N o Includes the first layer N. i This is an input layer that can take in one or more dental condition scan datasets. The last layer N o This is the output layer. The deep neural network used in this disclosure can output probabilities and / or complete 3D margin line proposals. For example, the output may be a probability vector containing one or more probability values ​​for each feature or aspect of a dental model belonging to a particular category. In addition, the output may be margin line proposals.

[0061] Each layer N can contain multiple nodes that connect to each node in the next layer N+1. For example, layer N h,l-1 Each computing node in the layer h,l Connects to each computing node in the input layer N. i and output layer N o Layer N between h,1 , N h,l-1 , N h,l This is a hidden layer. In Figure 9, the nodes in the hidden layer indicated as "h" can be hidden variables. In some embodiments, the DNN400 may include multiple hidden layers, for example, 24, 30, 50, etc.

[0062] In some embodiments, DNN400 can be a deep feedforward network. DNN400 can also be a convolutional neural network, which is a network that uses convolution instead of general matrix multiplication in at least one of the hidden layers of the deep neural network. DNN400 can also be a generative neural network or a generative adversarial network. In some embodiments, the training module 120 manages the learning process of the deep neural network using a labeled training dataset. Labels are used to map features to probability values ​​of probability vectors. Alternatively, the training module 120 can also train a generative deep neural network in an unsupervised manner using an unstructured and unlabeled training dataset, which does not necessarily require a labeled training dataset.

[0063] In some embodiments, the DNN can be a multilayer perceptron ("MLP"). In some embodiments, the MLP may include four layers. In some embodiments, the MLP may be a fully connected MLP. In some embodiments, the MLP may utilize batch norm normalization.

[0064] Figure 10 shows a diagram of a computer implementation method for automatic margin line proposal in several embodiments as an example. In some embodiments, before starting margin line proposal for any 3D digital model, the computer implementation method can pre-calculate a base margin line 1003 in three dimensions (1001), where each point of the base margin line 1003 has a 3D coordinate such as coordinate 1005. The computer implementation method can receive a 3D digital model 1002 of at least a portion of the jaw. In some embodiments, the 3D digital model may have the form of a 3D point cloud. The 3D digital model may include, for example, prepared teeth 1004. The computer implementation method can use a first trained neural network 1006 to obtain an internal representation 1008 of the 3D digital model. In some embodiments, the first trained neural network 1006 may be a neural network such as an HNN that performs grouping and sampling 1007 and other operations on the 3D digital model, for example, in some embodiments. In some embodiments, a computer implementation can use a second trained neural network 1010 to determine a margin line proposal from a base margin line 1003 and an internal representation 1008 of a 3D digital model. In some embodiments, the second trained neural network can provide, for example, one or more three-dimensional displacement values ​​1012 of digital surface points of the base margin line 1003.

[0065] In some embodiments, a second trained neural network can determine the margin line displacement value in three dimensions from the base margin line. In some embodiments, the second trained neural network uses the bilateral chamfer distance as the loss function. In some embodiments, a computer implementation can provide a margin line proposal by moving one or more points on the base margin line by the displacement value. Figure 11 illustrates an example of one embodiment in which the base margin line 1102 of a 3D digital model 1100 is adjusted. In this example, one or more base margin line points, such as base margin line point 1104, can be displaced by the displacement value and direction 1106. Other base margin line points can also be adjusted similarly, for example, by their corresponding displacement values ​​and directions, to form a margin line proposal 1108.

[0066] Figure 12(a) shows one example of a proposed digital margin line 1204 for a digitally prepared tooth 1202 of a 3D digital model 1205. As can be seen in this figure, margin line proposals can be made even if the margin line is partially or completely covered by gums, blood, saliva, or other elements. Figure 12(b) shows another example of a proposed digital margin line 1206 for a digitally prepared tooth 1208 of a 3D digital model 1210. In some embodiments, the proposed margin line is displayed on the 3D digital model and can be manipulated by a user, such as a dental technician or physician, using an input device to make adjustments to the margin line proposal.

[0067] In some embodiments, a first neural network and a second neural network can be trained using the same training dataset. In some embodiments, the training dataset can include one or more training samples. In some embodiments, the training dataset can include 70,000 training samples. In some embodiments, each of the one or more training samples can include the occlusal direction, preparation die center, and buccal direction as normalized positioning and orientation for each sample. In some embodiments, the occlusal direction, preparation die center, and buccal direction can be set manually. In some embodiments, the training dataset can include an untrimmed digital surface of the jaw and a target margin line on the surface of the corresponding trimmed digital surface. In some embodiments, the target margin line can be prepared by a technician. In some embodiments, training can use regression. In some embodiments, training can include comparing margin line proposals to target margin lines using a loss function. In some embodiments, the loss function can be a chamfer loss function. In some embodiments, the chamfer loss function can include the following equation:

number

[0068] In some embodiments, training can be performed on a computing system that includes at least one graphics processing unit ("GPU"). In some embodiments, the GPUs may include, for example, two 2080-Ti Nvidia GPUs. Other suitable types, numbers, and equivalents of GPUs may be used.

[0069] In some embodiments, the computer implementation can be performed automatically. Some embodiments may further include displaying freeform margin lines on a 3D digital model. In some embodiments, the freeform margin lines can be adjusted by the user using an input device.

[0070] Figure 13 shows an example of a computer-aided method for automatic margin line proposal. This method may include, in 1302, receiving a 3D digital model of at least a portion of the jaw, the 3D digital model including digitally prepared teeth; in 1304, obtaining an internal representation of the 3D digital model using a first trained neural network; and in 1306, obtaining a margin line proposal from the base margin line and the internal representation of the 3D digital model using a second trained neural network.

[0071] Some embodiments include an automated margin line proposal processing system, i.e., a processor, and a computer-readable storage medium containing instructions executable by the processor, which includes receiving a 3D digital model of at least a portion of a jaw, the 3D digital model including digitally prepared teeth, and performing the steps of: obtaining an internal representation of the 3D digital model using a first trained neural network; and obtaining a margin line proposal from a base margin line and the internal representation of the 3D digital model using a second trained neural network.

[0072] In some embodiments, the computer implementation method, system, and / or non-temporary computer-readable medium may include one or more other features. For example, in some embodiments, the base margin line may include one or more digital points that define the margins of the digitally prepared teeth. In some embodiments, the 3D digital model may include a 3D point cloud. In some embodiments, the first trained neural network may include a trained hierarchical neural network ("HNN"). In some embodiments, the first trained neural network may include a neural network for 3D point cloud analysis. In some embodiments, the second trained neural network may include a decoder neural network. In some embodiments, the first and second trained neural networks are trained using a training dataset that includes an untrimmed digital surface of the jaw and a target margin line on the surface of the corresponding trimmed digital surface.

[0073] One or more advantages of one or more features in some embodiments may include, for example, that accurate margin lines are provided for prepared teeth even in subgingival cases where margin lines are missing or covered. One or more advantages of one or more features in some embodiments may include, for example, that margin lines are more accurate because the technician or any other user does not need to manually draw and / or guess the margin lines. One or more advantages of one or more features in some embodiments may include, for example, that it is faster to determine margin lines because the technician or any other user does not need to manually draw the margin lines for each 3D digital model. One or more advantages of one or more features in some embodiments may include, for example, that efficiency is increased due to the automation of determining margin lines. One or more advantages of one or more features in some embodiments may include, for example, that margin line suggestions are determined from a 3D digital model such as a point cloud, thereby providing more accurate margin line suggestions. One or more advantages of one or more features in some embodiments may include, for example, that the occlusal direction, the 3D center of the preparation die, and / or the buccal direction of the die are present / automatically determined, thereby providing input normalization and a more accurate determination of margin line suggestions. One or more advantages of one or more features in some embodiments may include, for example, that there is no need to draw margin lines step by step. One or more advantages of one or more features in some embodiments may include, for example, that a margin line suggestion is provided even in subgingival cases when the margin line is missing or covered (partially or completely covered by gums and / or blood and saliva), and the technician does not need to form the margin line. One or more advantages of one or more features in some embodiments may include, for example, that the margin is accurately detected in a fully automatic mode despite various shapes, subgingival cases, and the requirement for accurate margin lines.One or more advantages of one or more features in some embodiments may include, for example, that manual margin location / construction is avoided, and therefore time and effort are saved.

[0074] Figure 14 shows processing systems 14000 in several embodiments. The system 14000 may include a processor 14030 and a computer-readable storage medium 14034 having processor-executable instructions for performing one or more steps described in this disclosure.

[0075] In some embodiments, the computer implementation can, for example, display a digital model on a display and receive input from an input device such as a mouse or touchscreen on the display.

[0076] One or more of the features disclosed herein can be performed and / or achieved automatically, either manually or without user intervention. One or more of the features disclosed herein can be performed by computer implementation methods. The disclosed features can be implemented in a computing system, including, but not limited to, any methods and systems. For example, the computing environment 14042 used to perform these functions can be any of the various computing devices (e.g., desktop computers, laptop computers, server computers, tablet computers, gaming systems, mobile devices, programmable automation controllers, video cards, etc.) that can be incorporated into a computing system comprising one or more computing devices. In some embodiments, the computing system can be a cloud-based computing system.

[0077] For example, a computing environment 14042 may include one or more processing units 14030 and memory 14032. The processing units execute computer executable instructions. The processing units 14030 can be a central processing unit (CPU), a processor in an application-specific integrated circuit (ASIC), or any other type of processor. In some embodiments, one or more processing units 14030 can execute multiple computer executable instructions in parallel, for example. In a multiprocessing system, multiple processing units execute computer executable instructions to increase processing power. For example, a typical computing environment may include a graphics processing unit or co-processing unit in addition to a central processing unit. The tangible memory 14032 may be volatile memory (e.g., registers, cache, RAM), non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.), or any combination of these, accessible by the processing unit(s). The memory stores software that implements one or more innovations described herein in the form of computer-executable instructions suitable for execution by processing units (which may be more).

[0078] A computing system may have additional features. For example, in some embodiments, the computing environment includes a storage device 14034, one or more input devices 14036, one or more output devices 14038, and one or more communication connections 14037. An interconnection mechanism such as a bus, controller, or network interconnects the components of the computing environment. Typically, operating system software provides the operating environment for other software running in the computing environment and coordinates the activities of the components of the computing environment.

[0079] The tangible storage device 14034 may be removable or non-removable and may include magnetic or optical media such as magnetic disks, magnetic tapes or magnetic cassettes, CD-ROMs, DVDs, or any other media that can be used to store information in a non-temporary manner and can be accessed within a computing environment. The storage device 14034 stores software instructions for implementing one or more innovations described herein.

[0080] Input devices (which may be multiple) can include, for example, touch input devices such as keyboards, mice, pens, or trackballs; audio input devices; scanning devices; any of the various sensors; other devices that provide input to the computing environment; or a combination thereof. In the case of video encoding, input devices (which may be multiple) can include cameras, video cards, TV tuner cards, or similar devices that receive video input in analog or digital format, or CD-ROMs or CD-RWs that load video samples into the computing environment. Output devices (which may be multiple) can include displays, printers, speakers, CD writers, or other devices that provide output from the computing environment.

[0081] A communication connection (which may consist of multiple connections) enables communication to another computing entity via a communication medium. The communication medium carries information such as computer executable instructions, audio or video inputs or outputs, or other data in modulated data signals. A modulated data signal is a signal in which one or more of its properties are set or modified in a way that encodes information within the signal. The communication medium may, but is not limited to, be an electrical carrier, an optical carrier, an RF carrier, or other carrier.

[0082] Any of the disclosed methods can be stored in one or more computer-readable storage media 14034 (e.g., one or more optical media disks, volatile memory components (such as DRAM or SRAM), or non-volatile memory components (such as flash memory or hard drives)) and implemented as computer-executable instructions executed on a computer (e.g., a smartphone, other mobile devices including computing hardware, or any commercially available computer including a programmable automation controller) (for example, a computer-executable instruction causes one or more processors of a computer system to execute the above method). The term computer-readable storage media does not include communication connections such as signals and carriers. Any computer-executable instructions that implement the disclosed techniques, and any data created and used during the implementation of the disclosed embodiments, can be stored in one or more computer-readable storage media 14034. Computer-executable instructions can be, for example, part of a dedicated software application, a software application accessed or downloaded via a web browser, or other software applications (such as a remote computing application). Such software can run, for example, on a single local computer (e.g., any suitable commercial computer), or it can run in a network environment using one or more network computers (e.g., via the Internet, a wide area network, a local area network, a client-server network (such as a cloud computing network), or other such network).

[0083] For clarity, only certain selected embodiments of the software-based implementation are described. Other details well known in the art are omitted. For example, it should be understood that the disclosed art is not limited to any particular computer language or any particular computer program. For example, the disclosed art can be implemented by software written in C++, Java, Perl, Python, JavaScript, Adobe Flash, or any other suitable programming language. Similarly, the disclosed art is not limited to any particular computer or any particular type of hardware. Certain details of suitable computers and hardware are well known and do not need to be described in detail in this disclosure.

[0084] It should also be understood that any of the functions described herein can be performed at least partially by one or more hardware logic components instead of software. Examples of hardware logic components that can be used, but not limited to, include field-programmable gate arrays (FPGAs), integrated circuits for specific programs (ASICs), standard products for specific programs (ASSPs), system-on-a-chip (SOC) systems, and complex programmable logic devices (CPLDs).

[0085] Furthermore, any software-based embodiment (including, for example, computer executable instructions that cause a computer to execute any of the disclosed methods) can be uploaded, downloaded, or remotely accessed through suitable means of communication. Such suitable means of communication include, for example, the Internet, the World Wide Web, intranets, software applications, cables (including fiber optic cables), magnetic communications, electromagnetic communications (including RF communications, microwave communications, and infrared communications), electronic communications, or other such means of communication.

[0086] Given the many possible embodiments to which the principles of this disclosure can be applied, it should be recognized that the embodiments described are merely examples and should not be construed as limiting the disclosure.

Claims

1. Receiving a 3D digital model of at least a portion of the jaw, wherein the 3D digital model includes digitally prepared teeth, Using a first trained neural network, the internal representation of the 3D digital model is obtained, Using a second trained neural network, a margin line proposal is obtained from the base margin line and the internal representation of the 3D digital model. Includes, A computer-aided method for automatic margin line proposals, wherein the first and second trained neural networks are trained using a training dataset that includes an uncropped digital surface of the jaw and a target margin line on the surface of the corresponding cropped digital surface.

2. The method according to claim 1, wherein the base margin line includes one or more digital points that define the margin of the digitally prepared tooth.

3. The method according to claim 1, wherein the 3D digital model includes a 3D point cloud.

4. The method according to claim 1, wherein the first trained neural network includes a trained hierarchical neural network ("HNN").

5. The method according to claim 1, wherein the first trained neural network includes a neural network for 3D point cloud analysis.

6. The method according to claim 1, wherein the second trained neural network includes a decoder neural network.

7. Processor and A computer-readable storage medium, Receiving a 3D digital model of at least a portion of the jaw, wherein the 3D digital model includes digitally prepared teeth, Using a first trained neural network, the internal representation of the 3D digital model is obtained, Using a second trained neural network, a margin line proposal is obtained from the base margin line and the internal representation of the 3D digital model. A computer-readable storage medium containing instructions executable by the processor, which perform a step including the following: Equipped with, A system for automatically proposing margin lines, wherein the first and second trained neural networks are trained using a training dataset that includes the uncropped digital surface of the jaw and the target margin lines on the surface of the corresponding trimmed digital surface.

8. The system according to claim 7, wherein the base margin line includes one or more digital points that define the margin of the digitally prepared tooth.

9. The system according to claim 7, wherein the 3D digital model is a 3D point cloud.

10. The system according to claim 7, wherein the first trained neural network includes a trained hierarchical neural network ("HNN").

11. The system according to claim 7, wherein the first trained neural network includes a neural network for 3D point cloud analysis.

12. The system according to claim 7, wherein the second trained neural network includes a decoder neural network.

13. A non-temporary computer-readable medium for storing executable computer program instructions that automatically suggest margin lines, The aforementioned computer program instruction is Receiving a 3D digital model of at least a portion of the jaw, wherein the 3D digital model includes digitally prepared teeth, Using a first trained neural network, the internal representation of the 3D digital model is obtained, Using a second trained neural network, a margin line proposal is obtained from the base margin line and the internal representation of the 3D digital model. Includes, The first and second trained neural networks are trained using a training dataset that includes the uncropped digital surface of the jaw and the target margin lines on the surface of the corresponding cropped digital surface, in a non-temporal, computer-readable medium.

14. The medium according to claim 13, wherein the base margin line includes one or more digital points that define the margin of the digitally prepared tooth.

15. The medium according to claim 13, wherein the 3D digital model is a 3D point cloud.

16. The medium according to claim 13, wherein the first trained neural network comprises a trained hierarchical neural network ("HNN").

17. The medium according to claim 13, wherein the second trained neural network includes a decoder neural network.

Images