Neural Network Margin Proposal

Abstract

This invention provides a computer implementation method, system, and instructions for proposing an automated margin line. [Solution] The method includes receiving a 3D digital model of at least a portion of the jaw, the 3D digital model including digitally prepared teeth. The method also includes using a first trained neural network to obtain an internal representation of the 3D digital model, and using a second trained neural network to obtain a margin line proposal from the base margin line and the internal representation of the 3D digital model.

Description

[Technical Field]

[0001] [Related applications] This application is U.S. Utility Model Application No. 17 / 245,94, filed on April 30, 2021. We claim the benefits and priority of Article 4. The entirety of this U.S. utility application is cited. This shall form part of this specification. [Background technology]

[0002] Specialized dental laboratories typically use computer-aided design (CAD). Using this method, dental prostheses are designed based on patient-specific instructions provided by the dentist. For example, given a digital surface including a prosthesis such as a preparation. Therefore, it is sometimes desirable to define a margin line.

[0003] Traditionally, a significant portion of the technician's work involved typically determining the margin line position for preparation. It was assigned to a specific purpose. In the conventional workflow, individual plaster casts for preparation were used. However, it was made manually by technicians. One of the main goals of this process was to The goal was to create a visible margin in the line. Individual stones with an "easy-to-see" margin. The plaster cast is scanned and a digital surface is acquired. On such a digital surface Generally, using curvature-based geometry tools allows you to position margin lines with fewer clicks. The location can be identified.

[0004] In modern workflows, the plaster casting stage is omitted. CT scan or intraoral scan Either of the following will be performed instead. Both intraoral scanners and CT scanners are individual To generate a complete jaw digital surface without any preparation surfaces. This may require a lot of work. We cannot rely solely on curvature, and Therefore, margin lines are usually drawn one by one. In the case of subgingival edentulous Missing or covered (partially or completely covered by gums and / or blood and saliva) It is formed by experienced technicians. Accurate margin detection in fully automated mode is It is always possible due to various shapes, subgingival cases, and the requirement for accurate margin lines. This is not always the case. Completely manual margin positioning / construction is performed by a physician or dental technician. It can be boring and time-consuming. [Overview of the project] [Means for solving the problem]

[0005] The computer implementation method for automatic margin line proposal involves 3D digital analysis of at least a portion of the jaw. The process involves receiving a barrel model, and the 3D digital model undergoes digital preparation. Including teeth, and using the first trained neural network, the 3 To find the internal representation of the D digital model and the second trained neural network Using twerks, the base margin line and the internal representation of the 3D digital model are used. This could include requesting margin line proposals.

[0006] The automated margin line suggestion system consists of a processor and a computer-readable storage medium. This means receiving a 3D digital model of at least a part of the jaw, and the 3D digital The Tal model includes digitally prepared teeth and the first trained nuclei - Using a neural network to obtain an internal representation of the 3D digital model, and the second Using a trained neural network of 2 to obtain a base margin line and Obtaining a margin line proposal from the internal representation of the 3D digital model, including Steps to be executed, including computer-readable instructions executable by the processor The memory medium can be provided.

[0007] The non-temporary computer-readable medium stores executable computer Program instructions that automatically propose a margin line. The computer program instructions include receiving a 3D digital model of at least a part of the jaw, and the 3D digital model includes a digital prosthesis tooth. Using a first trained neural network to obtain an internal representation of the 3D digital model, and using a second trained Neural network to obtain a margin line proposal from the base margin line and the internal representation of the 3D digital model. It can include. Using to obtain an internal representation of the 3D digital model, and the second trained Neural network to obtain a margin line proposal from the base margin line and the internal representation of the 3D digital model. It can include.

Brief Description of Drawings

[0008] [Figure 1] It is a perspective view of a 3D explanatory diagram of an example of a plaster cast. [Figure 2] It is a top perspective view of a 3D digital model of at least a part of a digital jaw in some embodiments as an example. [Figure 3] It is a perspective view of a 3D digital model of at least a part of a digital jaw in some embodiments as an example. [Figure 4] It is a perspective view of a 3D digital point cloud in some embodiments as an example. [Figure 5]This is a top perspective view of a 3D digital model of at least a portion of a digital jaw having a set occlusal direction, preparation die, and buccal direction in some exemplary embodiments. [Figure 6] This is a diagram of a convolutional neural network in several embodiments as an example. [Figure 7] This is a top perspective view of an example of a 2D depth map of a digital model in several exemplary embodiments. [Figure 8(a)] This is a diagram of a hierarchical neural network in several exemplary embodiments. [Figure 8(b)] This is a diagram of a hierarchical neural network in several exemplary embodiments. [Figure 9] Here are some diagrams of deep neural networks in various embodiments as examples. [Figure 10] This is a diagram illustrating a computer-aided method for proposing an automated margin line in several exemplary embodiments. [Figure 11] This is a perspective view of an example of a 3D digital model showing the proposed margin lines from the base margin line in several embodiments as an example. [Figure 12(a)] This is a perspective view of a 3D digital model with prepared teeth and proposed margin lines in several exemplary embodiments. [Figure 12(b)] This is a perspective view of a 3D digital model having prepared teeth and proposed margin lines in several exemplary embodiments. [Figure 13] This is a diagram illustrating a computer implementation method for the proposed automated margin line in several exemplary embodiments. [Figure 14] This is a diagram of the system in several embodiments as an example. [Modes for carrying out the invention]

[0009] For the purposes of this description, certain aspects, advantages, and novel features of embodiments of the present disclosure may be described in this specification. As described in this document, the methods, apparatus, and systems disclosed are not limited in any way. It should not be interpreted. Instead, this disclosure provides all novel and These are non-obvious features and aspects, both individually and in various combinations and partial combinations with each other. These methods, apparatuses, and systems are characterized in any particular embodiment. The disclosed embodiments are not limited to either or any combination thereof, and the disclosed embodiments are 1 It is not necessary for there to be more specific advantages or for the problem to be solved. .

[0010] The operation of some of the disclosed embodiments is described in detail for the convenience of presentation. The explanation is given in the order listed below, but this method of explanation is specific to the particular wording described below. It should be understood that sorting is included unless order is required. For example, sequential The actions described can, in some cases, be rearranged and performed simultaneously. It is also possible to do so. Furthermore, for simplicity, the attached diagram shows the disclosed method combined with other methods. It may not show the various ways in which it can be used. In addition, the explanation is sometimes Use terms such as "provide" or "achieve" to describe the way the information is disclosed. There are also cases where the actual behavior corresponding to these terms varies depending on the specific embodiment. This is easily recognizable by those skilled in the art.

[0011] When used in this application and claims, the singular form of the term ("a", "an") is used. The pronouns "and" (and "the") include the plural form unless the context clearly indicates otherwise. Furthermore, the term "include" means "to provide / comprise". Furthermore, the terms "combined" and "associated" generally refer to electrical and electromagnetic relationships. This means to be bound or linked, both physically and / or mechanically (e.g., mechanically or chemically). And unless there is specific negative wording, the items being combined or associated with This does not exclude the existence of intermediate elements between them.

[0012] In some examples, values, procedures, or devices are referred to as "minimum," "best," "smallest," etc. Such descriptions may be referenced. Many alternatives can be chosen from. And such a choice is better, smaller, or worse than the other choices. It is understood that this is intended to show that there is no need for anything other than a preferred option. It is likely.

[0013] The following explanations use the terms "up," "down," "upper side," "lower side," "horizontal," "vertical," and "left." Certain terms such as "right" may be used. These terms are used where applicable. It is used to make explanations somewhat clearer when dealing with relative relationships. However, this These terms are not intended to mean absolute relationships, positions, and / or directions. For example, with respect to a certain object, the "upper" surface can be obtained simply by inverting the object. It could potentially become the "underside" surface. Nevertheless, the object is still the same object.

[0014] In the traditional workflow, each plaster cast for preparation is made by hand by a technician. It was created dynamically. One of the main goals of this process was to create clean and visible merges. The goal was to produce individual plaster casts with a "clear" margin. The image is then processed, and the digital surface is acquired. Figure 1 shows, for example, a clearly visible margin line. This shows an explanatory diagram of the individual plaster casts 102 that were made together with 104.

[0015] In modern workflows, the plaster casting stage is omitted. CT scan or intraoral scan Either of the following will be performed instead. Both intraoral scanners and CT scanners are individual It generates only a complete jaw digital surface without any preparation surfaces. Figure 2 shows the This shows an example of a 3D digital model of at least a portion of the digital jaw surface 1202. The digital jaw surface 1202 includes, for example, a preparation tooth 1204, but is scanned. A margin line cannot be established in this model.

[0016] In some embodiments, the computer implementation method involves one or more trained N Using a neural network, we determine margin line suggestions in 3D digital models. It can be determined. In some embodiments, one or more trained neural networks A network can perform encoding and decoding. In some embodiments, At least one of the network components is used, for example, to perform coding. Hierarchical neural networks ("HNN") that can do this and It is possible.

[0017] Some embodiments involve receiving a 3D digital model of at least a portion of the jaw. It can include. 3D digital models can include digitally prepared teeth. In some embodiments, the 3D digital model is generated from a CT scanner. This is possible. One example of a CT scan is described in Nikolskiy et al., U.S. Patent Application Publication No. 201801. It is described in Patent No. 32982. This U.S. Patent Application is cited in its entirety by This specification shall form part of the other types of CT scans known in this technology. Stems can also generate 3D digital models. Computed tomography (CT): Computed tomography scanners use X-rays to create detailed images of physical impressions. This can be done. Multiple such images are then combined to create a 3D model of the patient's dental condition. The model is formed. The CT scan system includes an X-ray source that emits an X-ray beam. This is possible. The object being scanned can be placed between the radiation source and the X-ray detector. The X-ray detector is further connected to a processor, which receives information from the detector. It is configured to receive and convert this information into a digital image file. The processor can connect to the detector directly, wirelessly, via a network, or otherwise One or more devices capable of direct or indirect communication with the detector 148 by the method described above. They will realize that they can be equipped with computers.

[0018] One example of a suitable scanning system is one commercially available from Nikon Corporation. This includes the Nikon Model XTH255CT scanner. This example scanning system is high-performance. A 225kV microphone with a 3μm focal size provides image acquisition and volume processing capabilities. Includes a lofocus X-ray source. The processor collects data from the scanning system. It may include a storage medium configured using instructions that manage the ski During the operation of the X-ray system, the impression is placed between the X-ray source and the X-ray detector. As the impression rotates in a predetermined position between the detector, a series of images of the impression are processed by the processor. This is how they are collected. An example of a single image. The image is a radiograph, projection, or other form of digital A 3D image can be obtained. In one embodiment, the impression is a predetermined image between the X-ray source and the detector. As the position is rotated, a series of images are collected. In other embodiments, as will be known to those skilled in the art... To understand this, more or fewer images can be collected. Multiple images are generated by the scanning system's processor and contained within the processor. They are stored in a storage medium. In a scanning system, these multiple images are stored in additional moving media. It can be used by software contained within the processor to perform the task. For example, in one embodiment, multiple images are generated by the scanning system. The 2D image undergoes tomographic reconstruction to generate a 3D virtual image. The image is obtained by reconstructing multiple radiographs via a reconstruction algorithm associated with the scanning system. The volumetric image or volumetric density generated from It has the form of a file. A volume density file can contain one or more voxels. In one embodiment, the volumetric image is converted into a surface image using a surface imaging algorithm. In the illustrated embodiment, the volumetric image is located in Newport Beach, California, USA. FastDesign® dental design software provided by Glidewell Laboratories. A format suitable for use with dental restoration design software such as .S. It is converted into a surface image (in TL file format).

[0019] In some embodiments, 3D digital models can be generated from an optical scanner. For example, in some embodiments, the 3D digital model is scanned using an intraoral scanner or other means. It can be generated by the device. A digital jaw model can also be generated, for example, of the patient's dental condition. It can be generated by scanning the oral cavity of the patient. In some embodiments, each electronic image The image is obtained by a direct intraoral scan of the patient's teeth. This is typically done, for example, in a dental clinic. It is performed in a hospital or dental clinic and by a dentist or dental technician. Other procedures Morphologically, each electronic image is a scan of the patient's tooth impression, a scan of the patient's tooth physical model. It is obtained by, or indirectly by other methods known to those skilled in the art. This is usually, for example, It is performed in a dental laboratory and by a dental technician. Therefore, it is not described herein. The method described is suitable for use beside the patient's chair, in a dental laboratory, or in other environments. Applicable.

[0020] Figure 3 shows one example of the digital jaw model 302. This digital jaw model is In this technology, scanning a physical impression using any known scanning technique Therefore, it can be generated, or it can be generated by an intraoral scan of the patient's oral cavity (dental condition). It is also possible. The above scanning techniques include, for example, optical scans, CT scans, etc. However, these are not the only limitations. Conventional scanners typically perform three steps during scanning. The physical impression / shape of the patient's dental condition is obtained, and this shape is used to create a 3D digital model. To digitize the process. The digital jaw model 302, for example, can be used to create physical impressions of the corresponding jaw / patients. It can include multiple interconnected polygons with topologies corresponding to the shape of the individual's dental condition. Yes. In some embodiments, a polygon can include two or more digital triangles. It is possible. In some embodiments, the scanning process is carried out, for example, in California, USA. FastDesign, provided by Glidewell Laboratories in Newport Beach. Suitable for use with dental design software such as gn(trademark) dental design software. It can generate STL, PLY, or CTM files.

[0021] In some embodiments, the 3D digital model can be converted into a 3D digital point cloud. In the case of optical scanning, the optical scanner emits a light beam, for example, any dental mark. Physical dental impressions of animals such as elephants can be scanned and digitized. Alternatively, optical scanning can be used. In the case of intraoral scanners, for example, it is possible to directly scan the patient's dental condition. Data obtained from scanning the surface of a physical dental impression / dental condition is a collection of points. , that is, it can take the form of a point cloud, triangle, or digital surface mesh. 3D The model is, for example, a point in 3D space connected by various geometric entities such as triangles. By using aggregates, physical dental impressions or dental conditions can be represented in digital format. This is possible. The scan can be used in the manner described herein, for example, It can be stored locally or remotely. Scanning is performed in the manner described herein. Save as a 3D scan, point cloud, or digital surface mesh for use in [location / application]. It is possible.

[0022] In the case of CT scans, the digital surface mesh and digital dental model are as provided for in this application. Application No. 16 / 451,315 (U.S. Patent Application Publication No. 20200405) assigned to the recipient. Use the method described in issue 455 of "PROCESSING CT SCAN OF DENTAL IMPESSION". And by the Marching Cubes method, or known in the art Other digital model generation methods and techniques can be used to create / determine the above application. The entire text shall constitute part of this specification by reference.

[0023] For example, point clouds are automatically generated and / or adjusted (reduced) by computer-aided methods. This can be done. The computer implementation method involves the volume density generated by the CT scanner. It receives a file as input. The volume density file contains voxels within the volume density volume. It may include voxels representing density values ​​at a given location. The computer implementation method is selected. The ISO value of the density is compared to the density of one or more voxels in the volume density file, and the point cloud is... Generate digital surface points at the selected ISO value of density. It can be a selectable value that the user can choose, and / or several implementation forms In this state, it can be automatically determined. In some embodiments, the ISO of the selected density If the value corresponds to the density of one or more voxels in the volume density file, the computer Depending on the implementation method, digital surface points are generated, and in a virtual 3D space, a volume density file is created. The position in the point cloud corresponding to the position of one or more voxels (multiple) within the voxels (multiple) It can be placed in (in some cases). In some embodiments, as will be discussed below If the ISO value of the selected density is between two voxel density values, then the computer... The method generates zero or more digital surface points, and in a virtual 3D space, The position corresponding to the position (or multiple position) between two voxel locations along the cell edge. It can be placed in (multiple locations). The computer implementation method allows for optional point placement. The group can be adjusted. The computer implementation method can be either the point cloud or the adjusted point cloud. It is possible to generate that digital surface mesh.

[0024] Figure 4 shows the generated point cloud 70 visible on a display in several embodiments. An example of 00 is shown. The generated point cloud 7000 is selected in the virtual 3D space. Generated digital surface points such as 7002 at any position of the ISO value of density Includes surface points.

[0025] In some embodiments, the 3D point cloud may include augmented information. For example, the surface normal direction, arithmetic mean curvature value, and / or from the generated digital surface mesh. It can include additional geometric data other than point coordinates, such as color. The normal direction is given The direction of a point can be perpendicular to a plane such as a digital surface. Several implementations In this state, 3D point clouds can include mesh representations, and augmented information is generated digitally. It can be obtained from the surface mesh.

[0026] In some embodiments, the 3D digital model is oriented in the occlusal direction, buccal direction, and / or pt It may include a reparation die region. In some embodiments, these features are For example, it can provide the normalized orientation of a 3D digital model. For example, bite The direction of alignment is the direction of the normal to the occlusal plane, and the digital preparation die is the digital pre Paration can be the area around the tooth, and the buccal direction is the direction towards the cheek in the oral cavity. It is possible. Figure 5 shows, for example, a digital preparation including tooth 504. 3D digital model of at least a portion of the patient's dental condition, including jaw 502. An example of 00 is shown. The 3D digital model 500 is in the occlusal direction 506, digital plate The paration die region 508 and the buccal direction 510 may be included.

[0027] In some embodiments, the 3D digital model can include the occlusal direction. The occlusal direction of the barrel model can be determined using any known technique in this technology. It is possible. Alternatively, in some embodiments, the occlusal direction is as described herein, for example. For example, a user can use an input device such as a mouse or touchscreen on the display. This can be specified by manipulating the digital model. In some embodiments The occlusal direction is, for example, as in Nikolskiy et al., U.S. Patent Application No. 16 / 451,968 (United States). (Patent Application Publication No. 20200405464) "PROCESSING DIGITAL DENTAL IMPRESSION This can be determined using the occlusion axis technique described in [the relevant document]. The entirety of the U.S. patent application shall constitute part of this specification by reference. In one embodiment, the occlusal direction can be determined once for each 3D digital model. In some embodiments, the occlusal direction can be determined automatically.

[0028] In some embodiments, the occlusal direction is controlled by a trained 3D convolutional neural network. Convolutional neural networks (CNNs) are used to represent volume (voxels). It can be obtained by using it. In some embodiments, DNN is a deep nucleus. In at least one of the hidden layers of the 3D network, convolution is performed instead of general matrix multiplication. Convolutional neural networks ("CNNs") are networks that use complex structures. This is possible. A convolutional layer applies a kernel function to a subset of the values ​​from the preceding layer. The output value can be calculated by the following method: The computer implementation method is training data We train a CNN by adjusting the weights of the kernel function based on the data. This can be done by using the same kernel function to calculate each value in a specific convolutional layer. It is possible.

[0029] Figure 6 shows an example of a CNN in several embodiments. For illustrative purposes, 2D A CNN is shown. A 3D CNN can have a similar architecture. However, it uses a 3D kernel (x, y, z axes) to provide a 3D output after each convolution. It is possible. A CNN includes one or more convolutional layers, such as the first convolutional layer 202. This can be done. The first convolutional layer 202 processes kernels across input images such as input image 203. Applying kernels such as 204 (also called filters) and optionally selecting activation functions By applying this, it is possible to generate one or more convolutional outputs, such as the first kernel output 208. The first convolutional layer 202 can contain one or more feature channels. By applying kernels such as RU204 and an arbitrarily selected activation function, convolution The kernel can generate a first convolved output such as the output 206. Then, based on the stride length, it proceeds to the next set of pixels in the input image 203. Then, kernel 204 and an arbitrarily selected activation function are applied to obtain the second kernel output. It can be generated. The kernel is applied to all pixels in the input image 203. And so, we can proceed in this way. In this way, CNNs can process one or more features A first convolved image 206 can be generated which may contain channels. The convolved image 206 is, in some embodiments, one or more features such as 207. It can include channels. In some cases, the activation function is, for example, RELU activation. This can be used as a characterization function. Other types of activation functions can also be used.

[0030] A CNN can also include one or more pooling layers, such as the first pooling layer 212. The first pooling layer is formed by first convolution of a filter such as a pooling filter 214. This can be applied to image 206. Any type of filter can be used. For example, a filter might output the maximum value of the pixels to which the filter is applied. (or) or average filter (outputs the average value of the pixels to which the filter is applied) This can be done. One or more pooling layers downsample the input matrix and adjust its size. This can reduce the amount of pooling. For example, the first pooling layer 212 is the first pooling fee By applying Ruta 214, the first convolved image 206 is reduced / downsunk. The first pooled image 216 can be provided by pulling it. The ringed image 216 can contain one or more feature channels 217. CNN This includes one or more optional additional convolutional layers (and activation functions) and one or more Puling layers. A second convolutional layer 218 and an arbitrary selection layer can be applied. For example, a CNN can have a second convolutional layer 218 and an arbitrary selection layer. A second feature channel 219 can be included by applying an activation function in a selective manner. The convolved image 220 can be output. The second pooling layer 222 is pooling A ring filter is applied to the second convolved image 220 to extract one or more feature channels. A second pooled image 224 can be generated, which can include the CNN. This consists of one or more convolutional layers (and activation functions) and one or more corresponding pooling layers. It can include. The output of the CNN can optionally include a portion of one or more fully connected layers 230. It can be sent to a fully connected layer that can perform output prediction 22 It is possible to provide output predictions of the 4th order. In some embodiments, the output prediction 224 is For example, it may include labels for the tooth and surrounding tissue.

[0031] In some embodiments, the trained occlusal direction 3D CNN is used for each patient One or more 3D voxel representations of the patient's dental condition, with optional selection of surface normals for each voxel. It can be trained using augmented data. 3D CNNs are 3D tatami mats. This 3D convolution can perform a 3D kernel instead of a 2D kernel. It is used to process 3D input. In some embodiments, trained 3D C The NN receives a 3D voxel representation having voxel normals. In some embodiments, An N×N×N×3 floating-point tensor can be used. In some embodiments, N can be, for example, 100. Other suitable values ​​for N can be used. In some embodiments, the trained 3D CNN performs four levels of 3D convolution. It can include a and can also include two linear layers. Some embodiments So, the training set for the 3D CNN consists of one or more sets, each representing a patient's dental condition. It may include a 3D voxel representation of the training set. In some embodiments, the training set Each 3D voxel representation in this context is manually created by the user or by other known methods in this technology. The technique may include occlusal directions marked by the method. In some embodiments, The training set can contain tens of thousands of 3D voxel representations, and each 3D voxel table The current has a marked occlusal direction. In some embodiments, training days Tasset uses 3D point cloud models and the marked occlusal direction in each 3D point cloud model. They can both be included.

[0032] In some embodiments, the 3D digital model is a digital preparation die. It may include a D center. In some embodiments, the digital preparation die The 3D center can be manually set by the user. In some embodiments, digital The 3D center of the preparation die is set using any technique known in this technology. It is possible.

[0033] In some embodiments, the 3D center of the digital preparation die is automatically determined. This is possible. For example, in some embodiments, three digital preparation dies The D center uses a neural network in a 3D point cloud aligned by occlusion. It can be obtained using. In some embodiments, trained neural networks The network provides the 3D coordinates of the center of the digital preparation bounding box. It can be provided. In some embodiments, the neural network is used to process 3D point clouds. In contrast, this refers to any neural network capable of performing segmentation. This is possible. For example, in some embodiments, the neural network is used in this disclosure. PointNet++ neural network segmentation as described above and It is possible. In some embodiments, the digital preparation die is digital The 3D center of the preparation can be determined by a sphere of a fixed radius around it. In some embodiments, this fixing radius is, for example, 0.8 cm for molars and premolars. This can be done. For example, in some embodiments, other suitable values ​​for the fixed radius can be found. It can be used in some embodiments to train a neural network. The sampling of the digital jaw centered on the jaw's mass center (extended) It may include using (without having) a digital jaw. In some embodiments, The point cloud can be oriented so that the occlusal direction is perpendicular. Several implementations In this state, the training dataset consists of one or more within the margin line of the prepared teeth. The above points can be made possible by the user using an input device or by the known technology Digital jaw that can include prepared teeth marked by any technique. This may include a 3D digital model of the point cloud of the patient's dental condition, etc. Several implementations In this state, the neural network uses segmentation to include selected points. It is possible to return a bounding box. In some embodiments, the used Segmentation can be done, for example, using PointNet++ segmentation. In some embodiments, the training set can number in the tens of thousands.

[0034] In some embodiments, the 3D center of the digital preparation die is at an equal depth of the jaw. It can be automatically determined based on the map image. The position of the die center is determined by the geometric center of the margin marked by the engineer. This is possible. In some embodiments, the final margin point from the completed state is used. It can be used. In some embodiments, the network can be used to measure jaw depth from occlusal signals. Receive a map image and return the position (X,Y) of the die center in the image's pixel coordinates. This is possible. For training, a depth map image and the corresponding ground truth, i.e., floating-point numbers. A dataset including X and Y values ​​can be used. In some embodiments, The number of training sets can reach tens of thousands.

[0035] In some embodiments, the 3D digital model may include a buccal orientation. In one embodiment, the buccal direction is manually set by the user. In this state, the buccal direction can be determined using any technique known in the art. In some embodiments, the buccal direction can be determined automatically. Morphologically, the buccal direction is trained using a 2D depth map image of a 3D digital model mesh. This can be obtained by providing it to a 2D CNN. Several embodiments The trained 2D CNN then processes the image representation. (Computer implementation method) Some embodiments of the law optionally allow the generation of 2D images from 3D digital models. This may include: In some embodiments, the 2D image may be used as a 2D depth map. This is possible. The 2D depth map has lines passing through each pixel. It can include a 2D image that includes the distance from the orthographic camera to the object. The object is, for example, In some embodiments, for example, it can be the surface of a digital jaw model. In the treatment method, the input is, for example, a 3D digital model of the patient's dental condition, such as the jaw ("digital"). This may include objects such as a "model" and the camera orientation. In some embodiments, The camera orientation can be determined based on the occlusal direction. The occlusal direction is the normal to the occlusal plane. The direction is such that the occlusal plane of the digital model can be any technique known in the art. It can be determined by... Alternatively, in some embodiments, the occlusal direction is as described herein. For example, when a user uses an input device such as a mouse or touchscreen... This can be specified by manipulating the digital model on the display. In one embodiment, the occlusal direction is, for example, as described in U.S. Patent Application No. 16 / 451 by Nikolskiy et al. "PROCESSING DIGITAL D" in Patent No. 968 (U.S. Patent Application Publication No. 20200405464) This can be determined using the occlusal axis technique described in "ENTAL IMPRESSION". The entire national patent application shall constitute part of this specification by reference.

[0036] 2D depth maps are already used in the technique, for example, with z-buffers or ray tracing. It can be generated using any technique of knowledge. For example, in some embodiments, The computer implementation method is, for example, to initialize the depth of each pixel (j,k) to the maximum length, The cell color can be reset to the background color. The computer implementation method is a 3D digital model. For each pixel in the projection of a polygon onto a digital surface such as a 3D model, the pixel (j,k The depth z of the polygon at (x,y) corresponding to ) can be found. z < pixels If the depth is (j,k), set the pixel depth to depth z. "z" is a turtle This can refer to the rule that the central axis of the field of view is in the direction of the camera's z-axis, and is not necessarily This does not refer to the absolute z-axis of the scene. In some embodiments, computer implementations For example, the pixel color can also be set to a color other than the background color. Several embodiments Then, the polygon can be, for example, a digital triangle. In some embodiments... In a map, depth can be measured per pixel. Figure 7 shows several examples. This shows an example of a 2D depth map of a digital model in an embodiment.

[0037] In some embodiments, the 2D depth map image is rotated 16 times. This version can include the average of Von Mises. In terms of morphology, the buccal direction is determined by the occlusal direction and the 3D center of the digital preparation die. It can be obtained afterwards. In some embodiments, the 2D depth map image is digital This can be an image of a portion of the digital jaw around the preparation dye. In some embodiments, regression can be used to determine the buccal direction. So, 2D CNNs are, for example, known in this technology, GoogleNet Inc. It can include pttion v3. In some embodiments, training data sets The points can include, for example, the buccal direction marked in a 3D point cloud model. In some embodiments, the training dataset includes tens of thousands to hundreds of thousands of images. It is possible.

[0038] Some embodiments of the computer implementation method involve a first trained neural This can include using a network to obtain an internal representation of a 3D digital model. In some embodiments, the first trained neural network is... This may include an INCORDA neural network. In some embodiments, the first The trained neural network is a neural network for 3D point cloud analysis. It may include a first trained neural The network includes a trained hierarchical neural network ("HNN"). It is possible to do so. In some embodiments, HNN is a PointNet++ neural network. It may include a network. In some embodiments, the HNN is a geometric structure This can be any message-passing neural network that performs the processing. In some embodiments, the geometric structure includes graphs, meshes, and / or point clouds. It is possible.

[0039] In some embodiments, the computer implementation method is an HNN such as PointNet++. It can be used for encoding. PointNet++ is "PointNet++: Deep Hiera "Charles R. Qi, Li Yi This is described in Hao Su, Leonidas J. Guibas, Stanford University, June 2017. This entire document shall constitute part of this specification by reference. A multi-level network, for example, uses a set of points sampled in a metric space. It can be processed hierarchically. PointNet++ or other HNNs such as HNNs are In some embodiments, this is carried out by determining the local structure resulting from the metric. This is possible. In some embodiments, HNNs such as PointNet++ or other HNNs are used. The neural network first partitions a set of points into two or more overlapping local regions based on the distance metric. This can be done by doing so. This distance metric is based on the basis space. This is possible. In some embodiments, local features can be extracted. For example, In some embodiments, granular geometric structures can be determined from small local neighborhoods. In some embodiments, small local neighborhood features are grouped into larger units. It is possible. In some embodiments, larger units provide a higher level of features. The process can be used to provide a set of points. In some embodiments, the process is used to provide a set of points. This process is repeated until all features of the whole are acquired. A fixed stride is used to scan the space. Unlike volumetric CNNs, PointNet++ or other HNNs use local volumetric CNNs. The receptive field depends on both the input data and the metric. It also does not depend on the data distribution. In contrast to CNNs that scan a vector space, PointNet++ or other HNNs Sampling strategies in HNNs such as those described above generate receptive fields in a data-dependent manner. ru.

[0040] In some embodiments, the HNN is PointNet++ or other HNNs, for example, How to use a local feature learner to learn from a set of points and an abstract set of points or local features It is possible to determine whether to partition. In some embodiments, the local feature learner is For example, PointNet, or any other suitable feature learner known in the technology It is possible. In some embodiments, the local feature learner is, for example, non-ordered By processing a set of points, semantic feature extraction can be performed. (Local feature learner) This allows us to abstract a set of one or more local points / features into a higher-level representation. In some embodiments, HNNs can recursively apply local feature learners. Yes, it is possible. For example, in some embodiments, PointNet++ can nest input sets. PointNet can be recursively applied to the resulting partition.

[0041] In some embodiments, the HNN includes, for example, the location and scale of the center of gravity. Each section is defined as a neighboring ball in Euclidean space with parameters that allow for this. By defining it, we can define the area of ​​the overlapping set of points. The centroid is, for example, In this technology, the input set can be selected by known furthest point sampling. One advantage of using HNNs is that the local receptive fields are dependent on the input data and metrics. Therefore, it is possible to include, for example, efficiency and effectiveness. Morphologically, HNNs can utilize neighbors at multiple scales. This is an example. For example, it can enable robustness and the acquisition of fine details.

[0042] In some embodiments, HNN can include hierarchical point set feature learning. N, for example, in some embodiments, constructs a hierarchical grouping of points and along the hierarchy This allows for the abstraction of local regions that gradually increase in size. In some embodiments, HNNs can include multiple levels of set abstraction. In some embodiments, point The set is processed and abstracted at each level to form a new set with fewer elements. A combination is generated. In some embodiments, the set abstraction level is in some embodiments This includes three layers: a sampling layer, a grouping layer, and a local feature learner layer. This is possible. In some embodiments, the local feature learner layer is, for example, PointNet. This can be done. The set abstraction level is, for example, d-dimensional in some embodiments. Input is an N×(d+C) matrix consisting of N points with coordinates and C-dimensional point features. It is possible to take d-dimensional coordinates and new C that can summarize the local context. N'×(d+C') of N' subsampled points having 'd-dimensional feature vectors' It can output matrices.

[0043] In some embodiments, the sampling layer selects or samples a set of points from the input points. HNN can be selected / sorted in some embodiments, for example. A sampled point can be defined as the centroid of a local region. For example, Input points to the layer {x1, x2, ..., x n Regarding}, iterative farthest point sampling ( Using FPS (farthest point sampling), a subset of points

number

number

[0044] The grouping layer, for example in some embodiments, finds neighboring points around each centroid. By doing so, one or more sets of local regions can be found. In some embodiments, The input to this layer is a set of points of size N × (d + C) and a set of centroids of size N' × d. This can be used as a target. In some embodiments, the output of the grouping layer is, for example, It can include a group of point sets having Iz N' × K × (d + C). For example, Iz In several embodiments, each group can correspond to a local region, and K is near the centroid. It can be the number of points within the circle. In some embodiments, K varies from group to group. This is possible. However, the next layer, namely the PointNet layer, can, for example, have a flexible number of layers. Points can be transformed into fixed-length local region feature vectors. Neighborhoods are defined in several embodiments. For example, it can be defined by metric distance. A ball query, for example, In some embodiments, all points within a radius to the query point can be found. An upper limit can be set. In an alternative embodiment, K nearest neighbors (kNN: K nearest neighbor) HBOR search can be used. kNN can find a fixed number of neighbors. On the other hand, the local neighborhood of a ball query can guarantee a fixed region scale, therefore For example, in some embodiments, one or more local regional features are more closely spaced. To make it generalizable. In some embodiments, this is done, for example, by semantic point labeling. This may be preferable for other tasks requiring pattern recognition or local pattern recognition.

[0045] In some embodiments, the local feature learner layer converts local region patterns into feature vectors. It can be encoded. For example, M⊆R n Let be a set of points, and let d be the distance metric. In this case, X=(M,d) is a metric in Euclidean space X n Discrete inherited from Assuming it is a metric space, the local feature learner layer takes X as input and analyzes X We can find a function f that outputs semantic interest information. Function f divides the label into X. A classification function that identifies the target, or a segmentation function that assigns a label to each member of M. It can be expressed as a function.

[0046] Some embodiments include, for example, x i ∈R d The set of non-ordered points {x1, x2, ... ,x n Given}, the set of points is mapped to a vector as shown in the following equation. PointNet is used as a local feature learner layer that can define the function f:X→R. It can be used.

number

[0047] In some embodiments, γ and h are, for example, multilayer perceptrons (MLPs). (layer perceptron) network, or other suitable alternative network known in the technology This can be used as a workpiece. The function f is, for example, in some embodiments, an arrangement of input points. It can be made invariant under substitution and can approximate any continuous set function. The response of h in this embodiment can be interpreted as spatial coding of a point. tNet is a project by RQ Charles, H. Su, M. Kaichun and LJ Guibas titled "PointNet: De ep Learning on Point Sets for 3D Classification and Segmentation” (2017 IEEE Co nference on Computer Vision and Pattern Recognition (CVPR), 2017, pp. 77-85) It is stated. This entire document shall constitute part of this specification by reference. do.

[0048] In some embodiments, the local feature learner layer receives N' local regions of a point. This is possible. The data size can be, for example, N' × K × (d + C). In several embodiments, each local region in the output is, for example, its centroid and the vicinity of the centroid It is abstracted by the local features to be encoded. The output data size is, for example, N' × ( It can be set to d+C). The coordinates of a point within a local region are, in some embodiments, the centroid. It is possible to transform into a local reference frame relative to a point: i=1,2,...,K and j Regarding =1,2,...,d

number

[0049] In some embodiments, the local feature learner processes input data points through, for example, a density adaptive layer. It can handle the uneven density of the sample. The density-adaptive layer can handle changes in the input sampling density. When doing so, learn to combine features from differently scaled regions. This is possible. In some embodiments, the density adaptive hierarchical network is, for example, Poin This is a tNet++ network. The density adaptive layer is, for example, in some embodiments, Multi-scale grouping ("MSG") or multi-resolution grouping It can include multi-resolution grouping ("MRG").

[0050] In MSG, in some embodiments, grouping layers with different scales are applied and then, by extracting the features of each scale, a multi-scale pattern can be obtained This can be done. Extracting the features of each scale can be done, for example, in some embodiments by using PointNet. In some embodiments it is possible, for example, to concatenate the features at different scales to provide multi-scale features In some embodiments, HNN can learn a combination of multi-scale features optimized by training For example, random input dropout, where the random input points are dropped points with a randomly perturbed probability, can be used For example, in some embodiments, as an example, a dropout rate of θ uniformly sampled from [0, p] where p is less than or equal to 1 can be used As an example, p can be set to 0.95 in some cases so that an empty point set is not generated. For example, in some embodiments, other suitable values can be used

[0051] In MRG, the features of one region at level L i can be, for example, the concatenation of two vectors The first vector can be obtained, for example, in some embodiments, by summarizing the features in each sub-region from the lower level L This can be done using the set abstraction level. The second vector can be, for example, in some embodiments, by directly processing the raw points of the local region using a single PointNet i-1 This can be used to obtain features by doing so. When the local region density is low, Vector 1 contains fewer points, and the first vector contains sampling missings, In some embodiments, the second vector can be weighted more heavily. Local region In cases of high density, the first vector is finer due to inspection at a higher resolution. Details can be provided recursively at a lower level, for example, several In this embodiment, the first vector can be weighted more heavily.

[0052] In some embodiments, the point features of the set segmentation can be propagated. For example, in some embodiments, a hierarchical propagation strategy can be used. In some embodiments, feature propagation is N l From ×(d+C) points N l-1 to individual points This can include propagating point features, where N l-1 and N l (N l is N l-1 Below Below is the size of the point set of inputs and outputs of the set abstraction level l. In the application morphology, feature propagation is N l-1 N in the coordinates of the individual points l Interpolation of the feature values ​​f of individual points This can be achieved through the following methods. In some embodiments, for example, the inverse distance based on the k-nearest neighbor method A weighted average can be used (where p=2 and k=3 in the following formula; other suitable values ​​can also be used). (Can be used). N l-1 The interpolated features in this case are, for example, several implementations. Morphologically, it can be linked to skip-linked point features from the collective abstraction level. In some embodiments, the connected features are, for example, connected through a unit PointNet. This can be done, and this is one convolution at a time in a convolutional neural network It can be made similar to an inclusion. For example, in some embodiments, the shared whole By applying coupled layers and ReLU layers, the feature vectors of each point can be updated. In that embodiment, the process is repeated until the propagated features to the original set of points are obtained. It can be returned.

number

[0053] In some embodiments, the computer implementation method is disclosed or is based on the technology. As is known, one or more neural networks can be implemented. Any particular structure relating to one or more neural networks as disclosed herein And the values ​​and any other features are provided merely as examples, and any suitable ones can be used. A modified or equivalent form can be used. In some embodiments, one or more N A neural network model, as an example, is based on the PyTorch Geometry package. It can be implemented.

[0054] Figures 8(a) and 8(b) show examples of HNN in several embodiments. The HNN may include a hierarchical point set feature learner 802, and its output is segmentation. It can be used to perform classification 804 and / or classification 806. Hierarchical point set features The symbol learner 802 uses a point in 2D Euclidean space as an example, but in 3D... It can process input 3D images. As shown in the example in Figure 8(a), HNN For example, an input image 808 having (N,d+C) is received, and the first sampling and grading are performed. Loop division operation 810 is performed to obtain a first sample having (N1, K, d+C) and The grouped image 812 can be generated. The HNN then processes 814. Then, the first sampled and grouped images 812 are provided to PointNet. And, a first abstracted image 816 having (N1,d+C1) can be provided. The first abstracted image 816 is subjected to sampling and grouping 818, ( A second sampled and grouped image 820 having N2, K, d+C1) It can be provided. The second sampled and grouped image 820 is Po Provided to the intNet neural network 822, the (N2,d+C2) is It can output an abstracted image 824 of 2.

[0055] In some embodiments, the second abstracted image 824 is HNN segmentation It can be segmented by n804. In some embodiments, HN N-segmentation 804 takes in the second abstracted image 824 and interpolates the first 830 can be performed, and its output is concatenated with the first abstracted image 816, (N A first interpolated image 832 having (1,d+C2+C1) can be provided. The interpolated image 832 of 1 is provided to the unit PointNet in 834, ( A first segment image 836 having N1,d+C3) can be provided. Segment image 836 can be interpolated in 838, and its output is the input image 8 When concatenated with 08, it provides a second interpolated image 840 having (N1,d+C3+C). This is possible. The second interpolated image 840 is provided to the unit PointNet. To provide a segmented image 844 having (842) and (N,k). This is possible. The segmented image 844 provides, for example, a score for each point. It is possible.

[0056] As shown in the example in Figure 8(b), the second abstracted image 824 is one of several embodiments. Then, it can be classified by the HNN classification 806. In some embodiments, HN N classification takes in the second abstracted image 824 and uses this in the PointNet network. It can be provided to the (860), and its output 862 is one or more such as coupling layer 864. It can be supplied to a fully connected layer, and its output can provide a class score of 866. ru.

[0057] Some embodiments of the computer implementation use a second neural network. Then, from the base margin line and the internal representation of the 3D digital model, the margin line This could include soliciting suggestions.

[0058] In some embodiments, the base margin line is drawn once per network type. It can be calculated in advance. In some embodiments, the network type is molar and It may include premolars. For example, in some embodiments, other suitable networks A type can be used. In some embodiments, the initial margin of each scan The same base margin line can be used as the line. In some embodiments, The network type can include other types. In some embodiments, - The margin line is three-dimensional. In some embodiments, the base margin line is The first neural network and the second neural network are trained. This can be determined based on the margin lines from the training dataset used. Yes, it is possible. In some embodiments, the base margin line is set on the training dataset. The margin lines can be the pre-calculated arithmetic mean or average of several. In the embodiment, any type of arithmetic mean or average can be used.

[0059] In some embodiments, the margin line proposal is a free-form margin line proposal. This is possible. In some embodiments, a second trained neural network The workpiece may include a decoder neural network. In some embodiments, The decoder neural network connects its internal representation to specific point coordinates for guidance. Decryption can be performed. In some embodiments, guided decoding is performed by T. Groueix "A Papier-Mache Approach" by M. Fisher, VG Kim, BC Russell, and M. Aubry to Learning 3D Surface Generation” (2018 IEEE / CVF Conference on Computer Visio Generate closed surfaces as described in n and Pattern Recognition, 2018, pp. 216-224). This is possible. The entirety of this document, by reference, constitutes part of this specification. Let's assume that.

[0060] In some embodiments, the decoder neural network is a deep neural network. It can include a network ("DNN": deep neural network). Here, see Figure 9. See also Figure 9, which shows deep neural networks according to some embodiments of the present disclosure. This is a high-level block diagram showing the structure of (DNN)400. DNN400 consists of multiple layers N i , N h,1 , N h,l-1 , N h,l , N o Includes the first layer N. i is one or more teeth This is an input layer that can take a state scan dataset. The last layer N o is the output layer The deep neural network used in this disclosure is based on probability and / or It can output complete 3D margin line proposals. For example, the output may be a certain type A probability vector containing one or more probability values ​​for each feature or aspect of a dental model belonging to a category. This is possible. In addition, the output can be a margin line proposal.

[0061] Each layer N may have multiple nodes that connect to each node in the next layer N+1. For example, layer N h,l-1 Each computing node in the layer h,l Each computing node in Continued. Input layer N i and output layer N o Layer N between h,1 , N h,l-1 , N h,l is the hidden layer In Figure 9, the node in the hidden layer indicated by "h" can be used as a hidden variable. Yes, it is possible. In some embodiments, the DNN400 has multiple hidden layers, for example, 24, 3 It can include hidden layers of 0, 50, etc.

[0062] In some embodiments, the DNN400 is a deep feedforward network This can be done. DNN400 is one of the hidden layers of a deep neural network. A network that uses convolution instead of general matrix multiplication in at least one of the following steps. It can also be a convolutional neural network. DNN400 is a generative neural network. It can also be a generative network or a generative adversarial network. In some embodiments, Training module 120 uses a training dataset with labels. This manages the learning process of the deep neural network. The labels are probability vectors. It is used to map features to the probability values ​​of Toll. Alternatively, it can be used in training modules. 120 uses an unstructured and unlabeled training dataset to Generative deep neural networks do not necessarily require bell-labeled training datasets. It is also possible to train a network using unsupervised methods.

[0063] In some embodiments, the DNN can be a multilayer perceptron ("MLP"). Yes, it is possible. In some embodiments, the MLP can include four layers. In some embodiments, the MLP may include a fully connected MLP. MLP utilizes BatchNorm normalization.

[0064] Figure 10 shows the computation of automated margin line proposals in several embodiments as an example. A diagram of the implementation method is shown. In some embodiments, any 3D digital model Before starting to propose a margin line, the computer-implemented method can pre-calculate the base margin line 1 003 in three dimensions (1001), and each point of the base margin line 1003 has 3D coordinates such as coordinates 1005, for example. The computer-implemented method can receive a 3D digital model 1002 of at least a part of the jaw. The 3D digital model can, in some embodiments, have the form of a 3D point cloud. The 3 D digital model can include, for example, a preparation tooth 1004. The computer implemented method can use a first trained neural network 1006 to obtain an internal representation 1008 of the 3D digital model. In some embodiments the first trained neural network 1006 can be, for example, in some embodiments, a neural network such as an HNN that performs grouping, sampling 1007 and other operations on the 3D digital model. In some embodiments, the computer-implemented method can use a second trained neural net work 1010 to determine a margin line proposal from the base margin line 1003 and the internal representation 1008 of the 3D digital model. In some embodiments the second trained neural network can provide, for example, one or more three-dimensional displacement values 1012 of the digital surface points of the base margin line 1003 and can.

[0065] In some embodiments, the second trained neural network can obtain a margin line displacement value in three dimensions from the base margin line. In some In some embodiments, the second trained neural network uses the BilateralChamferDistance as the loss function. In some embodiments, the computer-implemented method can provide a margin line proposal by moving one or more points of the base margin line by a displacement value. FIG. 11 shows an explanatory diagram of an example in some embodiments for adjusting the base margin line 1102 of the 3D digital model 1100. In this example, one or more base margin line points such as the base margin line point 1104 can be displaced by a displacement value and direction 1106. Other base margin line points can also be adjusted similarly by their corresponding displacement values and directions to form a margin line proposal 1108. In some embodiments, the computer-implemented method can provide a margin line proposal by moving one or more points of the base margin line by a displacement value. FIG. 11 shows an explanatory diagram of an example in some embodiments for adjusting the base margin line 1102 of the 3D digital model 1100. In this example, one or more base margin line points such as the base margin line point 1104 can be displaced by a displacement value and direction 1106. Other base margin line points can also be adjusted similarly by their corresponding displacement values and directions to form a margin line proposal 1108. In this example, one or more base margin line points such as the base margin line point 1104 can be displaced by a displacement value and direction 1106. Other base margin line points can also be adjusted similarly by their corresponding displacement values and directions to form a margin line proposal 1108. In this example, one or more base margin line points such as the base margin line point 1104 can be displaced by a displacement value and direction 1106.

[0066] FIG. 12(a) shows an example of a proposed digital margin line 1204 of the digital preparation tooth 1202 of the 3D digital model 1205. As can be seen in this figure, the margin line proposal can be made even when the margin line is partially or completely covered by the gingiva, blood, saliva, or other elements. FIG. 12(a) shows an example of a proposed digital margin line 1204 of the digital preparation tooth 1202 of the 3D digital model 1205. As can be seen in this figure, the margin line proposal can be made even when the margin line is partially or completely covered by the gingiva, blood, saliva, or other elements. FIG. 12(b) shows another example of a proposed digital margin line 1206 of the digital preparation tooth 1208 of the 3D digital model 1210. In some embodiments, the proposed margin line is displayed on the 3D digital model and can be manipulated by a user such as a dental technician or a doctor using an input device to make adjustments to the margin line proposal. In some embodiments, the proposed margin line is displayed on the 3D digital model and can be manipulated by a user such as a dental technician or a doctor using an input device to make adjustments to the margin line proposal. In some embodiments, the proposed margin line is displayed on the 3D digital model and can be manipulated by a user such as a dental technician or a doctor using an input device to make adjustments to the margin line proposal. In some embodiments, the proposed margin line is displayed on the 3D digital model and can be manipulated by a user such as a dental technician or a doctor using an input device to make adjustments to the margin line proposal. In some embodiments, the proposed margin line is displayed on the 3D digital model and can be manipulated by a user such as a dental technician or a doctor using an input device to make adjustments to the margin line proposal.

[0067] In some embodiments, a first neural network and a second neural network The models can be trained using the same training dataset. In some embodiments, the training dataset is one or more training samples It may include 7000 training datasets. It may contain 0 training samples. In some embodiments, one or more Each training sample has the normalized positioning and orientation of each sample, This may include the occlusal direction, the center of the preparation die, and the buccal direction. In the procedure, the occlusal direction, preparation die center, and buccal direction are set manually. This can be done. In some embodiments, the training dataset is a trimmed jaw Untrimmed digital surface and target mark on the surface of the corresponding trimmed digital surface It may include a gin line. In some embodiments, the target margin line is a technique A scientist can prepare this. In some embodiments, training uses regression. It is possible. In some embodiments, training is performed using a loss function, merging This may include comparing the line proposal to the target margin line. Morphologically, the loss function can be a chamfer loss function. In this embodiment, the chamfer loss function can include the following equation.

number

[0068] In some embodiments, training is performed using at least one graphics processing unit. Computing that can include a GPU (Graphics Processing Unit) can be executed on a system. In some embodiments, the GPU can include, for example, two 2080-Ti Nvidia GPUs. Other suitable GPU types, numbers, and equivalents can be used.

[0069] In some embodiments, the computer-implemented method can be executed automatically. Some embodiments can further include displaying a free-form margin line on a 3D digital model and, in some embodiments, the free-form margin line can be adjusted by the user using an input device.

[0070] FIG. 13 shows an example of a computer-implemented method for automatic margin line proposal. The method receives, at 1302, a 3D digital model of at least a portion of the jaw, where the 3D digital model includes digital preparation teeth, and at 1304 uses a first trained neural network to obtain an internal representation of the 3D digital model, and at 1306 uses a second trained neural network to obtain a margin line proposal from a base margin line and the internal representation of the 3D digital model.

[0071] Some embodiments include a processing system for automatic margin line proposal, i.e., a processor and a computer-readable storage medium that receives a 3D digital model of at least a portion of the jaw where the 3D digital model includes digital preparation teeth ​​​​​Then, using the first trained neural network, a 3D digital model To find the internal representation of and using a second trained neural network Based on the base margin line and the internal representation of the 3D digital model, margin line proposals are made. A processor executes a step which includes finding a ... Includes computer-readable storage media.

[0072] In some embodiments, a computer implementation method, system, and / or non-temporary computer A computer-readable medium may include one or more other features. For example, several implementations Morphologically, the base margin line defines the margin of the digitally prepared tooth. It may include one or more digital points. In some embodiments, a 3D digital model The data can include a 3D point cloud. In some embodiments, a first trained The neural network is a trained hierarchical neural network ("H This may include a first trained NN. The neural network may include a neural network for 3D point cloud analysis. In some embodiments, the second trained neural network is decoded It may include a duaneural network. In some embodiments, the first tray The trained neural network and the second trained neural network The work involves the uncropped digital surface of the jaw and the corresponding cropped digital surface. Using a training dataset that includes the target margin line on the surface of the surface, It will be done.

[0073] One or more advantages of one or more features in some embodiments are, for example, margin Even in cases where the tooth is missing or covered below the gum line, the precise measurement of the prepared tooth is possible. It may include that an engine line is provided. One or more advantages of the above features is, for example, that a technician or any other user can access the margin line. Since there is no need to draw and / or estimate the margin lines, they are more accurate. This may include one or more advantages of one or more features in some embodiments. For example, if a technician or another user manually draws the margin lines for each 3D digital model... Since there is no need to do so, it can be included that calculating the margin line is faster. One or more advantages of one or more features in some embodiments are, for example, margin This includes increased efficiency resulting from the automation of line selection. Yes, it is possible. One or more advantages of one or more features in some embodiments are, for example, Mar The proposed sine lines are determined from a 3D digital model such as a point cloud, and thereafter, This may include providing precise margin line suggestions. In some embodiments, One or more advantages of one or more features in the occlusal direction, preparation die The 3D center and / or the buccal direction of the die exist / are automatically determined, thereby input This can include providing normalization and a more accurate determination of margin line proposals. One or more advantages of one or more features in some embodiments are, for example, margin This may include the fact that it is not necessary to draw lines one by one. One of several embodiments. One or more of the above features are advantages, for example, that the margin line is missing or covered (tooth Even when partially or completely covered by the stalk and / or blood and saliva, the subgingival area In this case, a margin line proposal is provided, and the engineer does not need to form the margin line. This may include one or more advantages of one or more features in some embodiments. For example, it depends on the various shapes, the case of subgingival cases, and the requirements for accurate margin lines. However, it may also include the ability to accurately detect margins in fully automatic mode. One or more advantages of one or more features in that embodiment is, for example, manual margin This may include avoiding location / construction, and therefore saving time and effort. Cut.

[0074] Figure 14 shows processing system 14000 in several embodiments. M14000 includes a processor 14030 and implements one or more steps described in this disclosure. A computer-readable storage medium 1403 having instructions that can be executed by a processor. It can include 4.

[0075] In some embodiments, the computer implementation method, for example, displays a digital model. It can be displayed during gameplay, and input devices such as a mouse or touchscreen on the display... It can receive input from the device.

[0076] One or more of the features disclosed herein may be automatically activated without manual or user intervention. It can be performed and / or achieved. One or more of the features disclosed herein are It can be implemented by computer. The disclosed features can be implemented by any means and This includes, but is not limited to, systems and computing systems. This can be done in a computer. For example, the computer used to perform these functions The computing environment 14042 is a computer with one or more computing devices Various computing devices that can be incorporated into a running system (for example) Desktop computers, laptop computers, server computers, tablets Red computers, gaming systems, mobile devices, programmable automation It can be any of the following (e.g., a motion controller, video card, etc.). In several embodiments, the computing system is cloud-based computing It can be used as a rigging system.

[0077] For example, computing environment 14042 has one or more processing units 14030 and The processing unit may include memory 14032. The processing unit can execute computer instructions. This is executed. Processing unit 14030 is the central processing unit (CPU). ng unit), application-specific integrated circuit (ASIC) It can be a processor in (it), or any other type of processor. In one embodiment, one or more processing units 14030 are, for example, multiple computers Multiple executable instructions can be executed in parallel. In a multiprocessing system, multiple instructions can be executed in parallel. The processing unit increases processing power by executing computer executable instructions. For example, The typical computing environment includes a central processing unit in addition to a graphics processing unit. It may also include a processing unit or a co-processing unit. The tangible memory 14032 is a processing unit ( Volatile memory accessible by (e.g., registers, caches) (which may be multiple) RAM, non-volatile memory (e.g., ROM, EEPROM, flash memory, etc.) , or any combination of these two. Memory is as described herein. Software that implements one or more innovations is called a processing unit (or multiple units). It stores the information in the form of computer-executable instructions suitable for execution by a certain entity.

[0078] Computing systems can have additional features. For example, several In this embodiment, the computing environment includes a storage device 14034 and one or more input devices. Vice 14036, one or more output devices 14038, and one or more communication connections 140 37. Interconnection mechanisms such as buses, controllers, or networks are included. Interconnects the components of the computing environment. Typically, this is done using operating system software. The software provides the operating environment for other software running in the computing environment. It provides and coordinates the activities of the components of the computing environment.

[0079] The tangible storage device 14034 may be removable or non-removable. This also applies to magnetic disks, magnetic tapes or magnetic cassettes, CD-ROMs, DVDs, etc. It can be used to store information in a magnetic or optical medium, or in a non-temporary manner. This includes any other media that can be accessed within the computing environment. The device 14034 is a software that implements one or more innovations described herein. It memorizes the commands of the wearer.

[0080] Input devices (there may be multiple) include, for example, a keyboard, mouse, pen, or Trackballs and other touch input devices; voice input devices; scanning devices; various Any of the sensors; another device that provides input to the computing environment; or These can be combinations of them. In the case of video encoding, the input device (multiple) (In some cases) this may involve a camera, video card, TV tuner card, or video input. A similar device that receives analog or digital samples, or a video sample. It can be a CD-ROM or CD-RW that is read into the computing environment. Output devices (there may be multiple) include displays, printers, speakers, and CD players. It can be a device that provides output from a computing environment, or another device that provides output from a computing environment. .

[0081] A communication connection (which may consist of multiple connections) is established via a communication medium to connect to another computing entity. Enables communication with T. The communication medium is computer executable instructions, audio, etc. It carries information such as video input or output, or other data within modulated data signals. A modulated data signal has one or more of its characteristics that encode the information within the signal. It is a signal that has been set or modified in such a manner. For example, and not limited to, a communication medium is an electric signal. Air carriers, optical carriers, RF carriers, or other carriers can be used.

[0082] Any of the disclosed methods involves one or more computer-readable storage media 14034 (for example). , one or more optical media disks, volatile memory components (DRAM or SRAM, etc.) ), or non-volatile memory components (such as flash memory or hard drives) It is stored in a computer (for example, a smartphone, computing hardware) This includes other mobile devices, or programmed automation controllers. To be implemented as a computer executable instruction executed on a commercially available computer. (For example, a computer executable instruction can be one or more pieces of a computer system.) (To have the Rossesser perform the above method). The term computer-readable storage medium is used for signals and This does not include communication connections such as carrier waves. Computer executable instructions that implement the disclosed techniques In all cases, and in all data created and used during the implementation of the disclosed embodiments, It can be stored in one or more computer-readable storage media 14034. Executable instructions are, for example, dedicated software applications or web browsers. Software applications accessed or downloaded via the internet, or other Software applications (such as remote computing applications) It can be a part of. Such software, for example, a single local computer It can also be run on a machine (for example, any suitable commercially available computer), or one or more Using the above network computers (for example, the Internet, wide area network) Network, Local Area Network, Client-Server Network (CloudCo (e.g., computing networks, or via other such networks) It can also be executed within the work environment.

[0083] To clarify, only certain selected embodiments of the software-based implementations are included. It is explained. Other details that are well known in the technology are omitted. For example, The disclosed technology does not apply to any particular computer language or any particular computer program. It should be understood that this is not limited to M. For example, the technologies disclosed include C++, Java, Perl, Python, JavaScript, Adobe Flash, Alternatively, it may be implemented by software written in any other suitable programming language. This is possible. Similarly, the disclosed technology can be used on any particular computer and any particular type It is not limited to the hardware. Certain details of suitable computers and hardware are also relevant. The details are well known and do not need to be described in detail in this disclosure.

[0084] Any of the features described herein may be performed using one or more hardware instead of software. It is often possible to perform at least partially this through a logic component. It should be understood. Examples, not limitations, can be used as illustrations of types of hyphens. Hardware logic components include field-programmable gate arrays (FPGs). A: Field-programmable Gate Array), ASIC (Application-Specific Integrated Circuit), Special Standard products for fixed programs (ASSP), System-on-a-chip (SOC) IP) Systems, Complex Programmable Logic Devices (CPLDs) Includes grammable logic devices, etc.

[0085] Furthermore, software-based embodiments (e.g., any of the methods disclosed) Any of the computer executable instructions (including instructions that cause a computer to execute) are suitable for communication It can be uploaded, downloaded, or remotely accessed through various means. Suitable means of communication include, for example, the internet, the World Wide Web, and intranets. Networks, software applications, cables (including fiber optic cables), magnets Communications, electromagnetic communications (including RF communications, microwave communications, and infrared communications), electronic communications, or Including other such means of communication.

[0086] Considering the many possible embodiments to which the principles of this disclosure can be applied, the described practical The implementation methods described are merely examples and should not be interpreted as limiting the scope of this disclosure. It is.

Claims

1. Receiving a 3D digital model of at least a portion of the jaw, the 3D digital The model includes digitally prepared teeth, Using the first trained neural network, the 3D digital model To find the internal representation of the language, Using the second trained neural network, the base margin line And, to obtain margin line proposals from the internal representation of the 3D digital model, A computer-based method for automatically proposing margin lines, including the above.

2. The base margin line defines the margin of the digitally prepared tooth. The method according to claim 1, comprising one or more digital points.

3. The method according to claim 1, wherein the 3D digital model includes a 3D point cloud.

4. The first trained neural network is trained hierarchically The method according to claim 1, comprising a neural network ("HNN").

5. The first trained neural network is used for 3D point cloud analysis. The method according to claim 1, including a network.

6. The second trained neural network is a decoder neural network. The method according to claim 1, including a workpiece.

7. The first trained neural network and the second trained The neural network is configured to use the uncropped digital surface of the jaw and corresponding to Training including target margin lines on the surface of the trimmed digital surface. The method according to claim 1, which is trained using a dataset.

8. Processor and A computer-readable storage medium, Receiving a 3D digital model of at least a portion of the jaw, the 3D digital The model includes digitally prepared teeth, Using the first trained neural network, the 3D digital model To find Dell's internal representation, Using a second trained neural network, base margin And, to obtain margin line proposals from the internal representation of the 3D digital model. 、 A computer including instructions executable by the processor, which perform steps including the steps Readable memory medium and A system that automatically suggests margin lines, equipped with [specific feature / feature].

9. The base margin line defines the margin of the digitally prepared tooth. The system according to claim 8, comprising one or more digital points.

10. The system according to claim 8, wherein the 3D digital model is a 3D point cloud.

11. The first trained neural network is trained hierarchically The system according to claim 8, comprising a neural network ("HNN").

12. The first trained neural network is used for 3D point cloud analysis. The system according to claim 8, including a network.

13. The second trained neural network is a decoder neural network. The system according to claim 8, including a workpiece.

14. The first trained neural network and the second trained The neural network is configured to use the uncropped digital surface of the jaw and corresponding to Training including target margin lines on the surface of the trimmed digital surface. The system according to claim 8, which is trained using a dataset.

15. Non-executable computer program instructions that automatically suggest margin lines A temporary computer-readable medium, The aforementioned computer program instruction is Receiving a 3D digital model of at least a portion of the jaw, the 3D digital The model includes digitally prepared teeth, Using the first trained neural network, the 3D digital model To find Dell's internal representation, Using a second trained neural network, base margin And, to obtain margin line proposals from the internal representation of the 3D digital model. 、 Non-temporary computer-readable media, including [specific examples of such media].

16. The base margin line defines the margin of the digitally prepared tooth. The medium according to claim 15, comprising one or more digital points.

17. The medium according to claim 15, wherein the 3D digital model is a 3D point cloud.

18. The first trained neural network is trained hierarchically The medium according to claim 15, comprising a neural network ("HNN").

19. The second trained neural network is a decoder neural network. The medium according to claim 15, including the workpiece.

20. The first trained neural network and the second trained The neural network is configured to use the uncropped digital surface of the jaw and corresponding to Training including target margin lines on the surface of the trimmed digital surface. The medium according to claim 15, which is trained using a dataset.

Images