Automatic selection and locking of intraoral images

Abstract

A processing device receives an intraoral image of a first intraoral site and determines an identification of the first intraoral site. The processing device then locks the intraoral image and selects a portion of the intraoral image depicting a portion of the first intraoral site based at least in part on the identification of the first intraoral site. The processing device may then generate a model containing the first intraoral site based at least in part on the locked intraoral image, where a portion of the locked intraoral image is used for a first region of the model, and where the first region is used for a second region of the model. Data from one or more additional intraoral images that likewise depict the portion of the first intraoral site is not used for the first region of the model.

Description

[0001] This application is a divisional application based on patent application number 202010564054.6 filed on March 4, 2016 and entitled “Automatic Selection and Locking of Intraoral Images”. Technical Field

[0002] Embodiments of the present invention relate to the field of intraoral scanning, and in particular, to a system and method for improving the results of intraoral scanning. Background Art

[0003] In oral restorations designed to implant a dental prosthesis in the oral cavity, in many cases, the intraoral site where the prosthesis is to be implanted should be accurately measured and carefully studied so that, for example, a prosthesis such as a crown, denture, or bridge can be properly designed and sized to fit in the proper location. A good fit enables mechanical stresses to be properly transferred between the prosthesis and the jaw, and prevents gum infection and tooth decay through the interface between the prosthesis and the intraoral site. The intraoral site can be scanned to provide three-dimensional (3D) data of the intraoral site. However, if the area of ​​the prepared tooth containing the finish line lacks definition, the finish line may not be properly determined, and thus the boundaries of the restoration may not be properly designed. BRIEF DESCRIPTION OF THE DRAWINGS

[0004] The invention is illustrated by way of example and not by way of limitation in the figures of the accompanying drawings.

[0005] Figure 1 One embodiment of a system for performing intraoral scans and generating virtual three-dimensional models of intraoral locations is shown.

[0006] Figure 2 A flow chart showing a method for automatically locking an image group of an intraoral site according to an embodiment of the present invention.

[0007] Figure 3 A flow chart showing a method for locking multiple image groups of more than one intraoral location according to an embodiment of the present invention.

[0008] Figure 4 A flow chart of a method for stitching together a plurality of image groups of more than one intraoral location according to an embodiment of the present invention is shown.

[0009] Figure 5 A flow chart illustrating a method of detecting anomalies in a set of images of an intraoral site and replacing the anomalies with data from additional intraoral images, according to an embodiment of the present invention.

[0010] Figure 6 A flow chart illustrating a method of extending a model of an intra-oral location where an incomplete margin is detected, according to an embodiment of the present invention.

[0011] Fig. 7A Shown is a portion of an example dental arch during an intraoral scanning session after a first set of intraoral images of the prepared teeth have been generated.

[0012] Figure 7B shows the intraoral scanning session after a second set of intraoral images of teeth adjacent to the prepared teeth have been generated. Fig. 7A Example of dental arch.

[0013] Figure 7C Show Fig. 7A A first set of intraoral images is provided, wherein the first set of intraoral images includes an abnormality.

[0014] Fig.7D Shown from Figure 7C The model was created with the data from the first set of intraoral images and additional intraoral images.

[0015] Fig. 7E An intra-oral image set of the prepared tooth is shown, wherein the intra-oral image set fails to capture the entirety of the prepared tooth.

[0016] Figure 7F Shown from Fig. 7E The model was created with the data from the first set of intraoral images and additional intraoral images.

[0017] Figure 8 A block diagram of an example computing device is shown in accordance with an embodiment of the present invention. DETAILED DESCRIPTION

[0018] Described herein is a method and apparatus for improving the quality of scans, such as intraoral scans acquired of an intraoral location (e.g., a dental site) of a patient. During a scanning session, a user of a scanner (e.g., a dentist) may generate multiple different images (also referred to as scans) of an intraoral location, a model of an intraoral location, or other object. The images may be discrete images (e.g., point-and-shoot images) or frames from a video (e.g., a continuous scan). The physician may acquire a first set of intraoral images of the first tooth after preparing the first tooth for scanning. For example, if the first tooth is a prepared tooth (also referred to as a preparation), one or more operations may be performed before generating the first set of intraoral images to ensure that the quality of the first set of intraoral images is high. In one instance, the operations briefly expose the termination line of the prepared tooth to ensure that the termination line will be revealed in the first set of intraoral images.

[0019] After completing the first set of intraoral images, the physician may acquire additional sets of intraoral images of one or more adjacent teeth. The additional sets of intraoral images may include data of a portion of the first tooth that is the focus of the first set of intraoral images. In some cases, during creation of a 3D model using the intraoral images, the intraoral images from the additional sets are combined with (e.g., averaged with) the data for the first tooth from the first set of intraoral images, thereby reducing the quality of the first tooth in the 3D model.

[0020] In an embodiment, in order to prevent data from another group of intraoral images from reducing the quality of the first tooth in the 3D model, the first group of images is automatically locked after the first group of intraoral images is created. In addition, a portion of the first group of intraoral images depicting the first tooth can be dedicated to the generation of a 3D model of the first tooth. Thus, since the first group of intraoral images is locked, the additional group of intraoral images does not change the area of ​​the 3D model depicting the first tooth or add noise to it. In one embodiment, the identification of the first tooth is determined, and the first portion of the first group of intraoral images is selected based at least in part on the identification of the first tooth. Thus, when generating the 3D model of the first tooth, lower quality data from the additional group of intraoral images depicting the first tooth may not be applied. This can improve the image quality of the first tooth in a 3D model of an intraoral location (e.g., a portion of a jaw).

[0021] The embodiments described herein are discussed with reference to intraoral scanners, intraoral images, intraoral scanning sessions, and the like. However, it should be understood that the embodiments also apply to other types of scanners in addition to intraoral scanners. The embodiments can be applied to any type of scanner that obtains multiple images and stitches these images together to form a combined image or virtual model. For example, the embodiments can be applied to desktop scanners, computed tomography (CT) scanners, and the like. In addition, it should be understood that intraoral scanners or other scanners can be used to scan scanned objects other than intraoral locations within the oral cavity. For example, the embodiments can be applied to scans of physical models of intraoral locations or any other objects. Therefore, the embodiments describing intraoral images should be understood to be generally applicable to any type of image generated using a scanner, the embodiments describing intraoral scanning sessions should be understood to be applicable to scanning sessions of any type of object, and the embodiments describing intraoral scanners should be understood to be generally applicable to multiple types of scanners.

[0022] Figure 1 An embodiment of a system 100 for performing intraoral scanning and / or generating a virtual three-dimensional model of an intraoral location is shown. In one embodiment, the system 100 performs the following reference Figure 2-6 Describes more than one operation.

[0023] The system 100 includes a computing device 105 that can be coupled to the scanner 150 and / or the data storage 110. The computing device 105 can include a processing device, a memory, a secondary storage, one or more input devices (e.g., such as a keyboard, a mouse, an input tablet, etc.), one or more output devices (e.g., a display, a printer, etc.), and / or other hardware components. In some embodiments, the computing device 105 can be integrated into the scanner 150 to improve performance and / or mobility.

[0024] The computing device 105 can be connected to the data storage 110 directly or via a network. The network can be a local area network (LAN), a public wide area network (WAN) (e.g., the Internet), a private WAN (e.g., an intranet), or a combination thereof. Alternatively, the data storage 110 can be an internal data storage. Examples of network data storage include storage area networks (SANs), network attached storage (NASs), and storage services provided by cloud computing service providers. The data storage 110 can include a file system, a database, or other data storage devices.

[0025] In some embodiments, a scanner 150 for obtaining three-dimensional (3D) data of intraoral locations in a patient's mouth is operably connected to the computing device 105. The scanner 150 may include a probe (e.g., a handheld probe) for optically capturing a three-dimensional structure (e.g., using confocal focusing of an array of beams). An example of such a scanner 150 is manufactured by Align Technology, Inc. Intraoral digital scanners. Other examples of intraoral scanners include 3M TM True Definition Scanner and Apollo DI intraoral scanner and CEREC AC intraoral scanner manufactured by.

[0026] The scanner 150 can be used to perform an intraoral scan of a patient's oral cavity. An intraoral scanning application 108 running in the computing device 105 can communicate with the scanner 150 to implement the intraoral scan. The result of the intraoral scan can be one or more sets of intraoral images that have been discretely generated (e.g., by pressing a "generate image" button of the scanner for each image). Alternatively, the result of the intraoral scan can be one or more videos of the patient's oral cavity. The operator can start recording a video with the scanner 150 at a first position in the oral cavity, move the scanner 150 to a second position in the oral cavity while capturing the video, and then stop recording the video. The scanner 150 can transmit discrete intraoral images or intraoral videos (collectively referred to as intraoral image data sets 135A-135N) to the computing device 105. The computing device 105 can store the intraoral image data sets 135A-135N in the data storage unit 110. Alternatively, the scanner 150 can be connected to another system that stores the intraoral image data sets 135A-135N in the data storage unit 110. In such an embodiment, scanner 150 may not be connected to computing device 105 .

[0027] In one embodiment, the intraoral scanning application 108 includes an anomaly identification module 115, a marking module 118, a model generation module 125, an image locking module 128, an erasure module 132, and an expansion module 134. Optionally, one or more of the anomaly identification module 115, the marking module 118, the model generation module 125, the image locking module 128, the erasure module 132, and / or the expansion module 134 may be combined into a single module or separated into other modules.

[0028] According to an example, a user (e.g., a physician) can perform an intraoral scan on a patient. In doing so, the user can apply the scanner 150 to one or more intraoral locations of the patient. The scan can be divided into more than one section. As an example, a section can include a lower buccal area of ​​the patient, a lower lingual area of ​​the patient, an upper buccal area of ​​the patient, an upper lingual area of ​​the patient, one or more prepared teeth of the patient (e.g., a patient's tooth to which a dental device such as a crown or other dental prosthesis is to be applied), one or more teeth in contact with the prepared teeth (e.g., the teeth themselves are not affected by the dental device, but they are located next to one or more such teeth, or in contact with one or more such teeth when the mouth is closed), and / or a patient's bite (e.g., a scan conducted by guiding the scan toward the interface area of ​​the patient's upper and lower teeth while the patient's mouth is closed). Via such a scanner application, the scanner 150 can provide image data (also referred to as scan data) to the computing device 105. The image data can be provided in the form of intraoral image data sets 135A-135N, each of which can include 2D intraoral images and / or 3D intraoral images of specific teeth and / or regions in the intraoral position. In one embodiment, separate image data sets are created for the maxillary arch, the mandibular arch, the patient's bite, and each prepared tooth. Such images can be provided from the scanner to the computing device 105 in the form of one or more points (e.g., one or more pixels and / or groups of pixels). For example, the scanner 150 can provide such 3D images as one or more point clouds.

[0029] The manner in which the patient's oral cavity is scanned can depend on the procedure to be applied. For example, if an upper or lower denture is to be created, a complete scan of the edentulous dental arch of the upper or lower jaw can be performed. In contrast, if a bridge is to be created, only a portion of the entire dental arch can be scanned, the portion including the edentulous area, adjacent prepared teeth (e.g., abutting teeth), and the opposing dental arch and dentition. Thus, the dentist can enter the identification of the procedure to be performed into the intraoral scanning application 108. To do this, the dentist can select the procedure from an icon or from a plurality of preset options on a drop-down menu or the like via any other suitable graphical user interface. Alternatively, the identification of the procedure can be entered in any other suitable manner, for example, by means of a preset code, annotation, or any other suitable manner, where the intraoral scanning application 108 has been appropriately programmed to recognize the selection made by the user.

[0030] In a non-limiting example, dental procedures can be roughly divided into oral repair (restoration) and orthodontic procedures, and then further subdivided into the specific forms of these procedures. In addition, dental procedures can include identification and treatment of gum disease, sleep apnea, and intraoral diseases. The term "oral repair procedure" refers in particular to any process of design, manufacture or installation of a dental prosthesis at a dental site in the oral cavity (intraoral position), or its real or virtual model, or for designing and preparing for the intraoral position to accommodate such a prosthesis. For example, a prosthesis may include any restoration such as a crown, veneer, inlay, onlay, implant and bridge, and any other artificial partial or complete denture. The term "orthodontic procedure" refers in particular to any process of design, manufacture or installation of an orthodontic element at an intraoral position in the oral cavity, or its real or virtual model, or for designing and preparing for the intraoral position to accommodate such an orthodontic element. These elements may be appliances including but not limited to brackets and archwires, retainers, transparent calibrators, or functional appliances.

[0031] For a variety of oral restoration processes (e.g., creating crowns, bridges, veneers, etc.), the patient's existing teeth are ground into stumps. The ground teeth are referred to herein as prepared teeth, or simply as preparations. The prepared teeth have a termination line (also referred to as an edge line), which is the boundary between the natural (unground) portion of the prepared tooth and the prepared (ground) portion of the prepared tooth. The prepared teeth are usually created so that a crown or other prosthesis can be installed or placed on the prepared teeth. In many cases, the termination line of the prepared tooth is below the gum line. While the term "prepared body" generally refers to the stump of the prepared tooth, including the termination line and the shoulder of the tooth residue, the term "prepared body" herein also includes artificial stumps, pivots, cores, and posts, or other devices that can be implanted in the oral cavity to accommodate crowns or other prostheses. The embodiments described herein with reference to prepared teeth are also applicable to other types of preparations, such as the aforementioned artificial stumps, pivots, etc.

[0032] After creating the prepared tooth, the physician performs an operation to prepare the prepared tooth for scanning. Preparing the prepared tooth for scanning may include wiping blood, saliva, etc. on the prepared tooth, and / or separating the patient's gums from the prepared tooth to expose the termination line. In some instances, the physician will insert a thin wire around the prepared tooth between the prepared tooth and the patient's gums. The physician then removes the thin wire before generating an intraoral scan set of the prepared tooth. The soft tissue of the gums will then return to its natural position and in many cases shrink back onto the termination line after a short period of time. Thus, the physician scans the prepared prepared tooth using the scanner 150, and generates an intraoral image set (e.g., intraoral image data set 135A) of the prepared tooth before the soft tissue returns to its natural position.

[0033] After generating the intraoral image set of the prepared tooth, the physician can preview the intraoral image set (or the 3D model created therefrom) to determine whether the intraoral image set is of satisfactory quality. Then, if the quality is not satisfactory, the physician can rescan the prepared tooth (or a portion thereof), and if the quality is satisfactory, the physician can continue to generate additional sets of intraoral images (e.g., intraoral image sets 135B-135N) of adjacent teeth or other areas around the prepared tooth. For example, additional sets of intraoral images of these adjacent areas can be acquired to ensure that the dental prosthesis will fit in the patient's mouth. Additional sets of intraoral images can also capture portions of the prepared tooth after the gums have retracted back to the termination line and / or after blood and / or saliva have accumulated on the prepared tooth.

[0034] Thus, in one embodiment, after acquiring a first set of intraoral images (e.g., intraoral image dataset 135A) (e.g., of the prepared teeth), the image locking module 128 automatically locks the first set of intraoral images. The locked first set of intraoral images may be associated with the prepared teeth of the patient. In one embodiment, the image locking module 128 automatically locks the image dataset associated with the prepared teeth, but does not automatically lock other image datasets. Thus, the image locking module 128 may determine whether the new image dataset is associated with the prepared teeth, and if so, lock the image dataset.

[0035] The identification of the prepared teeth can be used by the image locking module 128 to automatically select the portion of the locked first set of intraoral images to be applied to the prepared teeth in the 3D model. Alternatively, the doctor can use a graphical user interface (GUI) to mark the portion of the locked set of intraoral images to be applied to the prepared teeth in the 3D model. In either case, the image locking module 128 can update the locked image data set so that only the portion of the locked image data set depicting the prepared teeth is locked, while the portion of the image data set depicting the gums, other teeth, etc. is not locked. In one embodiment, the image locking module 128 performs image processing to determine the outline of the prepared teeth and the termination line. All data representing the prepared teeth inside the termination line can be locked, while all data representing other intraoral features outside the termination line can be unlocked. In one embodiment, a buffer is applied, and all data within the termination line plus the buffer is locked. The buffer can be offset from the termination line by 1-3 mm, for example. Thus, the image locking module 128 can determine which data to retain in the locked image data set by an algorithm. Alternatively, the user can manually determine which data to retain in the locked image data set. For example, the user can outline the area he or she wants to retain through the graphical user interface. The locked image data set can then be unlocked by the user at any time.

[0036] During creation of the prepared teeth in the 3D model, data from the additional set of intra-oral images that may also include low-quality depictions of the prepared teeth may be ignored by the model generation module 125. Thus, the termination lines captured in the first image data set are not degraded by the other image data.

[0037] In another embodiment, additional intraoral image data sets may be generated for additional prepared teeth and / or other teeth, such as teeth adjacent to the scanned prepared teeth. The image locking module 128 may automatically (e.g., by an algorithm and / or without user input) lock some or all of the image data sets after the image data sets have been created and before additional intraoral images are acquired. The image locking module 128 may assign a separate layer or group identifier to each locked intraoral image data set 135A-135N. These layers may be used to refer to the entire image data set and may be used to display or hide image data sets and / or prioritize data from the image data sets for stitching these image data together.

[0038] In an example, a first image data set may be associated with a first tooth, and a second image data set may be associated with an adjacent second tooth. Data from the first image data set may overlap with data from the second image data set, and may deviate from data from the second image data set. In order to stitch the image data sets together, the differences between the overlapping regions of the intraoral sites depicted in the two image data sets should be corrected. One technique for correcting the differences is to average the data of the first image data set and the data of the second image data set for the overlapping regions. By using layers, weights may be assigned to the individual image data sets, and the average of the image data sets may be a weighted average. For example, if a user knows that data from a particular overlapping region of the first image data set is superior in quality to data from a particular specific overlapping region of the second image data set, the user may select a first image data set with a higher priority. Then, when averaging the differences between the image data sets, the model generation module 125 may give a heavier weight to the first image data set than to the second image data set.

[0039] The image locking module 128 can associate each intraoral image dataset 135A-135N with a specific tooth and / or can identify the intraoral location associated with each intraoral image dataset 135A-135N. In one embodiment, before generating the image dataset, the user indicates the tooth he or she scanned. Alternatively, the user can first acquire the image dataset and can then indicate the tooth imaged in the image dataset. In another embodiment, the intraoral scanning application 108 can instruct the user to scan a specific tooth, and the identification of the specific tooth can be associated with the image dataset. Thus, each locked intraoral image dataset 135A-135N can be associated with a specific tooth, which can be a prepared tooth or can be a non-prepared tooth.

[0040] When the scanning session is complete (e.g., all images of the intraoral site have been captured), the model generation module 125 can generate a virtual 3D model of the scanned intraoral site. To generate the virtual 3D model, the model generation module 125 can register (i.e., "stitch" together) the intraoral images generated from the intraoral scanning session. In one embodiment, performing image registration includes capturing 3D data for various points of the surface in multiple images (views from cameras) and registering the images by calculating transformations between the images. The images can then be integrated into a common reference frame by applying appropriate transformations to the points of each registered image.

[0041] In one embodiment, image registration is performed for each pair of adjacent or overlapping intraoral images (e.g., each consecutive frame of an intraoral video). An image registration algorithm is performed to register two adjacent intraoral images, which essentially involves the determination of a transformation that aligns one image with another image. Image registration may involve identifying a plurality of points in each image of the image pair (e.g., a point cloud), performing surface fitting on the points of each image, and using a local search around the points to match the points of the two adjacent images. For example, the model generation module 125 may match the points of one image with the nearest points interpolated on the surface of the other image, and iteratively minimize the distance between the matching points. The model generation module 125 may also find the best match of the curvature features at the points of one image with the curvature features at the points interpolated on the surface of the other image without iteration. The model generation module 125 may also find the best match of the rotated image point features at the points of one image with the rotated image point features at the points interpolated on the surface of the other image without iteration. For example, other techniques that may be used for image registration include techniques based on determining point-to-point correspondence using other features and based on minimization of point-to-surface distance. Other image registration techniques may also be used.

[0042] Many image registration algorithms perform fitting of surfaces to points in adjacent images, which can be done in many ways. Parametric surfaces such as Bezier and B-spline surfaces are the most common, although other surfaces can also be used. A single surface patch can be fitted to all points of an image, or alternatively, separate surface patches can be fitted to any number of subsets of the points of an image. Separate surface patches can be fitted to have a common boundary, or they can be fitted to overlap. A surface or surface patches can be fitted to interpolate multiple points by using a control point network with the same number of points as the grid of points being fitted, or a surface can approximate points by using a control point network that has a smaller number of control points than the grid of points being fitted. Image registration algorithms can also employ various matching techniques.

[0043] In one embodiment, the model generation module 125 can determine point matches between images, which can take the form of a two-dimensional (2D) curvature array. A local search for matching point features in corresponding surface patches of adjacent images can be performed by calculating features at points sampled in an area around a parametric similarity point. Once a corresponding set of points is determined between the surface patches of the two images, the determination of the transformation between the two sets of corresponding points in the two coordinate systems can be solved. In essence, the image registration algorithm can calculate a transformation between two adjacent images that will minimize the distance between a point on one surface and the nearest point to the above point found in an interpolated area on the surface of the other image used as a reference.

[0044] The model generation module 125 repeats the image registration for all adjacent image pairs of a series of intraoral images to obtain a transformation between each pair of images, thereby registering each image with the previous one. The model generation module 125 then integrates all images into a single virtual 3D model by applying appropriately determined transformations to each image. Each transformation can include rotations around one to three axes and translations in one to three planes.

[0045] In many cases, the data from one set of intraoral images does not perfectly correspond to the data from another set of intraoral images. For each intraoral image data set 135A-135N, the image locking module 128 can use the identification of the associated teeth to determine which parts of the image data set will be used exclusively for the creation of a specific area of ​​the 3D model (e.g., for the creation of the associated teeth in the 3D model). The image locking module 128 can analyze the image data in each intraoral image data set. For each image data set, the image locking module 128 can use the stored information about the associated teeth to determine from the analysis which parts or regions of the image data set represent the teeth, and which parts or regions of the image data set represent other intraoral objects such as gums or other teeth. The selection module 130 can then generate an outline of the tooth in the image data set. The generated outline can be used as a boundary. The data of the image data set within the outline can be used exclusively by the model generation module 125 to generate a specific associated tooth in the 3D model. Data from the image data set outside the outline may or may not be used to generate other features or objects in the 3D model. Additionally, data outside the contours may be combined with data from other image datasets to generate other features or objects in the 3D model.

[0046] In one embodiment, the operation of generating the tooth contour in the locked image dataset is performed using the image locking module (as described above). The image locking module 128 can then update the locked image dataset to lock the portion inside the contour of the image dataset and unlock the portion outside the contour of the image dataset.

[0047] The anomaly identification module 115 is responsible for identifying anomalies and / or other areas of interest (AOIs) from the intraoral scan data (e.g., intraoral images in an intraoral image dataset) and / or a virtual 3D model generated from the intraoral scan data. Such anomalies may include gaps (e.g., areas where scan data is missing), conflicting areas or defective scan data (e.g., areas where overlapping surfaces of multiple intraoral images cannot match), areas representing foreign matter (e.g., studs, bridges, etc.), unclear edge lines (e.g., edge lines of more than one prepared tooth), noise information, etc. The identified gaps may be gaps in the image surface. Examples of surface conflicts include bicuspid edges and / or other physiologically impossible tooth edges, occlusal line deviations, inclusion or absence of blood / saliva and / or foreign matter, differences in the depiction of edge lines, etc. In identifying anomalies, the anomaly identification module 115 may analyze only the patient image data (e.g., 3D image point clouds) and / or more than one virtual 3D model of the patient, and / or analyze relative to the reference data 138. The analysis may include direct analysis (e.g., pixel-based and / or other point-based analysis), application of machine learning, and / or application of image recognition. Such reference data 138 may include past data regarding the current patient (e.g., intraoral images and / or virtual 3D models), compiled patient data, and / or teaching patient data, some or all of which may be stored in the data store 110.

[0048] In order to identify abnormalities, the abnormality identification module 115 performs data processing to identify unexpected shapes, areas of low definition, areas of missing data, color deviations, etc. Different standards can be used to identify abnormalities of different categories. In one embodiment, areas of missing image data are used to identify abnormalities that may be blank. For example, voxels of areas not captured by the intraoral image can be identified. In one embodiment, the abnormality identification module inserts shapes for abnormalities based on geometric features around the abnormality and / or based on geometric features of the abnormality (if such features exist). Such geometric features can be determined by using edge detection, corner detection, blob detection, ridge detection, Hough transform, structural tensor and / or other image processing techniques.

[0049] The data about the current patient may include X-rays, 2D intraoral images, 3D intraoral images, 2D models, and / or virtual 3D models corresponding to the patient's visit during which the scan occurred. The data about the current patient may also include the patient's past X-rays, 2D intraoral images, 3D intraoral images, 2D models, and / or virtual 3D models (e.g., corresponding to the patient's past visits and / or the patient's dental records).

[0050] The patient data collected can include X-rays, 2D intraoral images, 3D intraoral images, 2D models, and / or virtual 3D models about a large number of patients. Such a large number of patients can include current patients, or can not include. The patient data collected can be anonymized and / or used according to regional medical confidentiality regulations (e.g., Health Insurance Portability and Accountability Act (HIPAA)). The patient data collected can include data corresponding to the scans of the types discussed herein and / or other data. Teaching patient data can include X-rays, 2D intraoral images, 3D intraoral images, 2D models, virtual 3D models, and / or medical illustrations (e.g., drawings and / or other images of medical illustrations) used in an educational environment. Teaching patient data can include volunteer data and / or cadaver data.

[0051] The anomaly identification module 115 may analyze patient scan data scanned during a later visit in the patient visit relative to additional patient scan data in the form of data from earlier in the patient visit (e.g., one or more early-visit 3D image point clouds and / or one or more early-visit virtual 3D models of the patient) (e.g., one or more later-visit 3D image point clouds and / or one or more later-visit virtual 3D models of the patient). The anomaly identification module 115 may additionally or alternatively analyze the patient scan data relative to reference data in the form of the patient's dental record data and / or the patient's data prior to the patient visit (e.g., one or more pre-visit 3D image point clouds and / or one or more pre-visit virtual 3D models of the patient). The anomaly identification module 115 may additionally or alternatively analyze the patient scan data relative to aggregated patient data and / or teaching patient data.

[0052] Identification of anomalies regarding missing and / or defective scan data may involve direct analysis by the anomaly identification module 115, such as determining that one or more pixels or other points are missing from the patient scan data of the patient and / or one or more virtual 3D models. Identification of anomalies regarding missing and / or defective scan data may additionally or alternatively involve employing the aggregated patient data and / or teaching patient data to determine that the patient scan data and / or virtual 3D model is incomplete (e.g., has a discontinuity) relative to the data represented by the aggregated patient data and / or teaching patient data.

[0053] The marking module 118 is responsible for determining how to present and / or call out the identified abnormality. The marking module 118 can provide an indication or indicator of the abnormality. The indication can be presented to the user (physician) in combination with or separately from a depiction of one or more teeth and / or gums of the patient (e.g., with one or more X-rays, 2D intraoral images, 3D intraoral images, 2D models, and / or virtual 3D models of the patient). The indication representation in combination with a depiction of the patient's teeth and / or gums can involve the indication being arranged so that the indication is associated with a corresponding portion of the teeth and / or gums.

[0054] The indication may be provided in the form of a logo, a mark, an outline, text, an image, and / or a sound (e.g., in the form of speech). Such an outline may be arranged to follow an existing tooth outline and / or gum outline (e.g., as a boundary) (e.g., via outline fitting). As an example, an outline corresponding to a blank may be arranged to follow an outline of missing data. Such an outline may be arranged relative to a missing tooth outline and / or gum outline (e.g., by outline extrapolation) to follow a projected path of the missing outline. As an example, an outline corresponding to missing tooth scan data may be arranged to follow a projected path of a missing tooth portion, or an outline corresponding to missing gums may be arranged to follow a projected path of a missing gum portion.

[0055] The data of the portion of the intraoral image dataset within the abnormality that forms the outline can be unlocked or removed from the locked intraoral image dataset. The abnormality can be identified by the user, and the user can then generate a new intraoral image that captures the abnormal area in the intraoral position. The portion of the new intraoral image corresponding to the inside of the outline of the abnormality is then used to replace the original data about the abnormality in the intraoral image dataset. The data can then be added to the locked image dataset. Thus, the abnormality can be automatically detected in a set of intraoral images, and additional intraoral images can be collected to overwrite the abnormality without affecting the remaining intraoral image datasets.

[0056] In one embodiment, after the abnormal identification module 115 identifies the abnormality, the abnormal identification module 115 can then determine whether there are any additional image data sets, which include data covering the area where the abnormality is identified. The abnormal identification module 115 can compare the data of the area from more than one additional image data sets with the locked image data set. Based on the comparison, the abnormal identification module 115 can determine that the data of the locked image data set covering the contour is replaced by the data from the additional image data set. Then, the part corresponding to the inside of the abnormal contour of other image data sets can be used to replace the original data about the abnormality of the intraoral image data set. This data can then be added to the locked image data set.

[0057] In one embodiment, for each additional image data set, different replacement options are presented to the user. Thus, for each additional image data set, the abnormal identification module 115 can utilize the image data of the outline that covers the abnormality from the additional image data set to replace the abnormality. Each replacement option can be presented to the user, and the user can then choose which replacement option to apply. Once the user's selection is received, the data from the additional image data set associated with the user selection can be used to replace the abnormality in the locked image data set.

[0058] The 3D model created by the model generation module 125 can be displayed to the user via the user interface of the intraoral scanning application. The 3D model can then be visually inspected by the user. The user can virtually manipulate the 3D model with respect to up to six degrees of freedom (i.e., translation and / or rotation with respect to one or more of three mutually orthogonal axes) via the user interface using appropriate user controls (hardware and / or virtual) to be able to view the 3D model from any desired direction. In addition to the anomaly identification module 115 algorithmically identifying anomalies for rescanning, the user can inspect (e.g., visually inspect) the generated 3D model of the intraoral location and determine that one or more areas of the 3D model are unacceptable.

[0059] Based on the inspection, the user may determine that a portion of the 3D model is inappropriate or undesirable and that the remainder of the 3D model is acceptable. The unacceptable portion of the 3D model may, for example, correspond to a portion of the actual tooth surface of the scanned intraoral location that is not sufficiently clearly defined in the 3D model. For example, in an initial 3D data collection step such as by scanning to generate a first 3D virtual model, the corresponding portion of the physical tooth surface may have been covered by foreign matter, such as saliva, blood, or debris. The corresponding portion of the physical tooth surface may also be obscured by other elements, such as a portion of the gums, cheeks, tongue, dental appliances, artifacts, etc. Alternatively, for example, in an initial 3D data collection step (e.g., via scanning) to obtain a generated first 3D virtual model, the unacceptable portion may be distorted or defective and may not properly correspond to the physical tooth surface (e.g., due to some defects in the actual scanning process).

[0060] Via the user interface, the user may mark or demarcate unacceptable portions of the 3D model. The erase module 132 may then delete or otherwise remove the marked portions (and associated portions of the locked image dataset and / or other image datasets used to create the unacceptable portions) from the 3D model. For example, the dental procedure of interest may be providing a dental prosthesis, and the deleted or removed portion of the 3D model may be a portion of a termination line of a tooth preparation that is present in the actual tooth surface but is not clearly presented in the 3D model (or in the intraoral image datasets 135A-135N used to create the 3D model).

[0061] The intraoral scanning application 108 can guide the user to generate one or more additional intraoral images of the dental part corresponding to the deleted or removed portion of the 3D model (and the corresponding intraoral image group). The user can then use the scanner 150 to generate one or more additional intraoral images, which at least partially cover the previously generated intraoral image. One or more additional intraoral images can be aligned with the 3D model (and / or with the intraoral image data set used to create the 3D model) to provide a composite of the 3D model and one or more additional intraoral images. In the composite, the pre-deleted / removed portion of the 3D model is at least partially replaced by the corresponding portion of the one or more additional intraoral images. However, the portion of the one or more additional images outside the deleted or removed portion of the 3D model is not applied to the composite or updated 3D model. In one embodiment, the portion of the new intraoral image corresponding to the erased portion of the locked image data is added to the locked image data set.

[0062] The expansion module 134 can perform similar operations as the abnormality identification module 115 and / or the erasure module 132. However, instead of identifying and correcting abnormal or unacceptable portions in the 3D model, the expansion module 134 can identify and / or connect portions of the 3D model at the edges of the 3D model. For example, the intraoral image dataset 135A may be missing a portion of a tooth, so that the tooth is cut off in the 3D model (e.g., a portion of the tooth is not shown in the 3D model). The expansion module 134 can algorithmically detect the edge of the 3D model where the teeth appear to be cut. Alternatively, the user can indicate the portion of the tooth that is not presented in the 3D model via a user interface. The user can mark or unmark the edge portion of the 3D model that makes the model incomplete.

[0063] The user can then use the scanner 150 to generate one or more additional intraoral images of the intraoral location (e.g., teeth) corresponding to the area of ​​the 3D model where data is missing. The one or more additional intraoral images can be registered to the 3D model. The expansion module 134 can then determine the portion of the one or more additional intraoral images that represents the area missing from the initial 3D model of the intraoral location (e.g., teeth). The portion of the one or more additional intraoral images can then be added to the 3D model to expand the 3D model for the intraoral location (e.g., teeth). In addition, portions of the one or more additional intraoral images can be attached to the locked image data set.

[0064] In one embodiment, the physician may have generated a complete or partial scan of one or more dental arches of the patient. Sometimes after the scan is completed, the patient may experience a change in dental health and may need to apply a bridge or other oral restoration in the location of a healthy tooth. In such a case, the dentist can take advantage of the previously completed scan. In particular, the physician can generate a prepared tooth, and then scan the prepared tooth to generate a locked intraoral image data set of the prepared tooth. The locked intraoral image data set can then be combined with the previously generated scan data to produce a new 3D model of the patient's dental arch. Most of the dental arch in the new 3D model will be based on data from the original scan, while the data for the prepared teeth will be based on the locked image data set.

[0065] Figure 2-6 A flow chart showing methods for processing a set of intraoral images and generating a virtual 3D model therefrom. These methods may be performed by processing logic comprising hardware (e.g., circuitry, dedicated logic, programmable logic, microcode, etc.), software (e.g., instructions running on a processing device), or a combination thereof. In one embodiment, the processing logic corresponds to Figure 1 A computing device 105 (e.g., corresponding to a computing device 105 executing an intraoral scanning application 108).

[0066] Figure 2A flow chart of a method 200 for automatically locking a set of images of an intraoral location is shown according to an embodiment of the present invention. At block 205 of method 200, an intraoral scanning session is started. During the intraoral scanning session, a dentist uses an intraoral scanner to create a set of intraoral images focused on a specific intraoral location (e.g., focused on a specific tooth). Processing logic can direct the dentist to the intraoral location (e.g., tooth) to be scanned, or the dentist can indicate the intraoral location to be scanned or has been scanned. Alternatively, processing logic can automatically (e.g., by algorithm) identify the intraoral location based on data from the set of intraoral images and / or based on one or more additional sets of intraoral images (e.g., focused on other intraoral locations). At block 210, processing logic receives a set of intraoral images of the intraoral location. At block 215, processing logic locks the intraoral image data set. This ensures that the portion of the intraoral image data set that depicts a specific area of ​​the intraoral location (e.g., depicts a specific prepared tooth, including its margin lines) will not be later modified or degraded by additional intraoral images.

[0067] refer to Fig. 7A , showing a portion of a dental arch 700 of an example during an intraoral scanning session. The dental arch includes two prepared teeth 708 and 710 and adjacent teeth 704, 706, 712 and the patient's gums 702. As shown, the prepared teeth 708, 710 have been ground into stumps to serve as abutments and to receive a bridge. The prepared tooth 708 includes a termination line 709, and the prepared tooth 710 includes a termination line 711. For this example, the termination lines 709, 711 are shown above the gum line to improve visibility. However, in multiple examples, the termination lines are below the gum line. In one example, a fine line may have been wrapped between the prepared tooth 708 and the surrounding gums and then removed to briefly expose the termination line 709 for scanning.

[0068] An intraoral image data set 713 including an intraoral image 714, an intraoral image 716, and an intraoral image 718 is shown. Each intraoral image 714-718 may have been generated using an intraoral scanner that is at a specific distance from the tooth surface to be imaged. At a specific distance, the intraoral images 714-718 have a specific scanning area and scanning depth. The shape and size of the scanning area generally depend on the scanner, and are represented by a rectangle in this article. Each image may have its own reference coordinate system and origin. Each intraoral image may be generated at a specific location (scanning station) using a scanner. The location and orientation of the scanning station may be selected so that the intraoral images together fully cover the entire target area. Preferably, the scanning station is selected so that, as shown in the figure, there is overlap between the intraoral images 714-718. Typically, when different scanners are used for the same target area, the selected scanning station will be different depending on the capture characteristics of the scanner used. Thus, a scanner capable of scanning a larger tooth area per scan (e.g., having a larger field of view) will use fewer scanning stations than a scanner capable of capturing 3D data of only relatively small tooth surfaces. Similarly, the number and arrangement of scanning stations for a scanner having a rectangular scanning grid (thereby providing a projected scanning area in the form of a corresponding rectangle) will typically differ from the number and arrangement of scanning stations for a scanner having a circular or triangular scanning grid (which will provide a projected scanning area in the form of a corresponding circle or triangle, respectively). The intraoral image dataset 713 is automatically locked and can be assigned as a first layer.

[0069] Reference Figure 2 , at block 220 of method 200, a portion of the first set of intraoral images is algorithmically selected using processing logic. The selected portion may correspond to the outline of a tooth or other feature of an intraoral location. The selected portion may be determined based on performing image analysis and applying object recognition techniques such as edge detection, edge matching, grayscale matching, gradient matching, bag of words models, etc. Reference data may be used to train processing logic to detect specific objects such as teeth. In one embodiment, known identifiers of teeth or intraoral locations are used to assist in the object detection process and select portions of the intraoral images.

[0070] For example, in Fig. 7A In the intraoral image dataset 713 of the example, an outline of the prepared tooth 708 can be generated. All parts of the intraoral images 714-718 of the intraoral image dataset 713 that are inside the outline can be fixed to prevent further changes. In one embodiment, the locked image dataset is updated so that the area inside the outline is locked in the image dataset, and the area outside the outline is unlocked.

[0071] At box 225 of method 200, processing logic receives one or more additional intraoral images depicting the intraoral location (e.g., depicting the teeth that are the focus of the locked intraoral image set). These additional intraoral images can be part of one or more additional intraoral image data sets, for example, for one or more additional teeth. At box 230, processing logic generates a virtual 3D model including the intraoral location. A selected portion of the locked intraoral image (e.g., a portion within a determined contour) is used to create a first area of ​​the model. For example, the selected portion can be used to create a specific prepared tooth in the 3D model. Areas of the 3D model are not created using data from the additional intraoral images.

[0072] Reference Figure 7B , showing during an intraoral scanning session after a second intraoral image dataset 721 has been generated for a tooth 706 adjacent to the prepared tooth 708 Fig. 7A The intraoral image dataset 721 includes intraoral images 722-728, which focus on the adjacent teeth 706. However, as shown, the area of ​​the intraoral image 726 in the second intraoral image dataset 721 also depicts the prepared teeth 708 and the termination line 709. However, because the first intraoral image dataset 713 has been locked, the data from the second intraoral image dataset 721 will not be used when creating the virtual representation of the prepared teeth 708 in the 3D model.

[0073] Figure 3 A flow chart of a method 300 for locking multiple image sets of more than one intraoral location is shown, according to an embodiment of the present invention. At block 302 of method 300, processing logic begins an intraoral scanning session. At block 304, processing logic receives a first set of intraoral images of a prepared tooth. At block 306, processing logic determines an identification of the prepared tooth. At block 308, processing logic locks the first set of intraoral images as a first layer.

[0074] At box 310, processing logic receives a second set of intra-oral images of another tooth adjacent to the prepared tooth. The adjacent tooth may be another prepared tooth, or may not be. At box 312, processing logic determines an identity of the adjacent tooth. At box 314, processing logic locks the second set of intra-oral images as a second layer.

[0075] At box 316, processing logic selects a portion of the first set of intra-oral images. The selection may be based at least in part on the identification of the prepared teeth. The selection of the portion may include forming an outline of the prepared teeth in the first set of intra-oral images, and selecting the portion within the outline. At box 318, processing logic selects a portion of the second set of intra-oral images. The selection may be based at least in part on the identification of adjacent teeth. The selection of the portion may include forming an outline of adjacent teeth in the second set of intra-oral images, and selecting the portion within the outline.

[0076] At block 320, processing logic generates a virtual 3D model of the intraoral site including the prepared tooth, adjacent teeth, and surrounding tissue.

[0077] Reference Figure 7A-7B , the intra-oral image dataset 713 may correspond to the first set of intra-oral images of the prepared teeth in method 300. Similarly, the intra-oral image dataset 721 may correspond to the second set of intra-oral images of the adjacent teeth in method 300. Therefore, those portions of the intra-oral image dataset 713 that depict the adjacent teeth 706 will not be used to recreate the adjacent teeth in the 3D model, and those portions of the intra-oral image dataset 721 that depict the prepared teeth 708 will not be used to recreate the prepared teeth in the 3D model.

[0078] Figure 4 A flow chart of a method 400 for stitching together multiple image groups of more than one intraoral location is shown according to an embodiment of the present invention. Processing logic may receive a first set of intraoral images (e.g., whose focus is the prepared teeth) and a second set of intraoral images (e.g., whose focus is adjacent teeth or a complete or partial dental arch). At box 405 of method 400, the processing logic identifies one or more differences in overlapping data between the first set of intraoral images and the second set of intraoral images. For example, the first set of intraoral images and the second set of intraoral images may depict the gums between the prepared teeth and the adjacent teeth, respectively. However, the descriptions of the gums in these intraoral image data sets may not be completely aligned. For example, blood and / or saliva may accumulate on the gums between the generation of the first intraoral image data set and the second intraoral image data set, or the positioning of the gums may be slightly different in the two intraoral data sets. In order to create a 3D model including the prepared teeth and the adjacent teeth, the data from the two intraoral image data sets should be merged and the data conflicts corrected.

[0079] At box 410, processing logic receives an indication of the priority of a first group of intraoral images and a second group of intraoral images. For example, it may be known that the first group of images is of higher quality or more important, so a higher priority may be assigned to the first group of images. At box 415, processing logic prioritizes the image data sets using the indication of the received priority. At box 420, processing logic applies a weighted average of the overlapping data between the first group of images and the second group of images to merge the overlapping data. The weights applied to the image data sets may be based on their priorities. For example, a first image data set assigned a higher priority may be assigned a weight of 70%, and a second group of intraoral images may be assigned a weight of 30%. Thus, when the data is averaged, the merged result will look more like a depiction from the first image data set and less like a depiction from the second image data set.

[0080] In one embodiment, the processing logic may render different versions of the 3D model, each version showing a different priority and / or a different weighting of the intraoral image data sets. The user may visually inspect the different renderings and select which rendering looks most accurate. The processing logic may then prioritize the image data sets accordingly and then apply an appropriate weighted average associated with the user selection to create the 3D model.

[0081] Figure 5 A flow chart of a method 500 for correcting anomalies in an intraoral image dataset and / or a 3D virtual model generated from such an intraoral image dataset is shown according to an embodiment of the present invention. The intraoral image dataset may be a collection of discrete images (e.g., acquired from a point-and-shoot mode) or multiple frames of an intraoral video (e.g., captured in a continuous scan or video mode). The intraoral image dataset may be targeted to a specific dental site (e.g., a tooth) of a patient and may be locked to retain the intraoral image dataset.

[0082] At box 505 of method 500, processing logic identifies anomalies within a locked intraoral image set and / or in a 3D model generated from a locked intraoral image set. Anomalies can be identified by performing image processing on an intraoral image data set and applying a set of criteria thereto. In one embodiment, processing logic determines whether a voxel or set of voxels in an intraoral image or 3D model meets more than one criterion. Different criteria can be used to identify different categories of anomalies. In one embodiment, missing image data is used to identify anomalies that may be blank. For example, voxels in an area that was not captured by an intraoral image can be identified.

[0083] In block 510, processing logic determines the boundaries of the anomaly, such as by generating an outline of the anomaly. In one embodiment, processing logic inserts a shape for the anomaly based on geometric features surrounding the anomaly and / or based on geometric features of the anomaly, if such features exist. For example, if the anomaly is a blank, the area surrounding the blank can be used to insert the surface shape of the blank. The shape of the anomaly can then be used to create the outline. All data outside the outline can remain locked and unchangeable, while data inside the outline can be replaced with data from a new intraoral image.

[0084] The processing logic can provide an indication of the abnormality via a user interface. The outline of the abnormality can be displayed in a manner that contrasts the abnormality with the surrounding image. For example, the tooth can be displayed in white, while the abnormality can be displayed in red, black, blue, green, or other colors. Additionally or alternatively, an indicator such as a flag can be used as an indication of the abnormality. The indicator can be away from the abnormality, but includes a pointer to the abnormality. The abnormality can be hidden or blocked in multiple views of the intraoral site. However, the indicator can be seen in all or most of such views.

[0085] refer to Figure 7C , a set of intraoral images 730 are shown, which include a prepared tooth 708 with a termination line 709, and an anomaly 732. As shown, the anomaly 732 has been contoured and has a specific shape (an ellipse in this example). The intraoral image set 730 includes intraoral images 714-718.

[0086] Back to Figure 5 At block 515, processing logic receives additional images of the intraoral location. The additional images may include images of the region of the 3D model or the initial set of intraoral images for which the anomaly was detected. At block 520, processing logic updates the virtual 3D model based on replacing data from the initial set of intraoral images within the boundary or outline with the additional data from the additional images of the intraoral location. Thus, the anomaly may be corrected without affecting the remainder of the virtual 3D model.

[0087] refer to Fig.7D , shows a virtual 3D model 740 created from the intraoral image set 730 and data from the additional intraoral image 709. The rendering of the prepared tooth 708 outside the outline of the anomaly 742 is not affected by the image data from the additional intraoral image 741. However, the portion of the prepared tooth 708 inside the outline of the anomaly 742 is rendered based on the data from the additional intraoral image 709.

[0088] Figure 6A flow chart of a method 600 for extending a model of an intraoral location in which an incomplete tooth or other object is detected is shown according to an embodiment of the present invention. At block 605 of method 600, processing logic determines that data for preparing a tooth (or other intraoral location) in a 3D model is incomplete. For example, processing logic may determine that an edge of the prepared tooth has been cut off. For example, the determination may be made based on comparing an expected contour of the prepared tooth or other tooth with a contour of the prepared tooth or other tooth in a calculated 3D model. If the calculated contour differs from the expected contour by more than a threshold amount, processing logic may determine that the prepared tooth or other tooth has been cut off in the model. In one embodiment, such a determination is made in response to a user indication that the prepared tooth or other tooth in the 3D model is incomplete. For example, a user may review the 3D model, determine that a portion of a tooth has been cut off, and manually enter an extension mode to add data for the cut off area. Alternatively, such a determination may be made algorithmically without first receiving user input (e.g., based on performing image processing).

[0089] refer to Fig. 7E , a set of intraoral images 750 includes intraoral images 714 and 752. The set of intraoral images 750 depicts a prepared tooth 708 having a termination line 709. However, an edge 754 of the prepared tooth 708 is cut away.

[0090] At box 610, processing logic receives an additional intra-oral image of the prepared tooth (or other teeth). At box 615, processing logic may identify the boundary of the edge of the prepared tooth. In one embodiment, this includes generating an outline of the edge of the prepared tooth at the boundary. In some instances, this may have been performed at box 605. The shape of the edge may be used to create the outline or boundary. All data inside the outline may remain locked and unchangeable.

[0091] At block 620, processing logic updates the model based on replacing data outside the boundary with additional data from the additional intraoral image. Processing logic determines which portion of the additional intraoral image of the prepared tooth depicts the portion of the prepared tooth that was excised in the initial set of intraoral images (e.g., outside the boundary of the edge of the excised data). The identified portion of the additional intraoral image can then be appended to the initial set of intraoral images and used to extend the prepared tooth (or other teeth) in the model.

[0092] refer to Figure 7F , showing the use of Fig. 7EA virtual 3D model 760 is created of the intraoral image set 750 with data from an additional intraoral image 762. The rendering of the prepared tooth 708 inside the outline of the prepared tooth 708 is not affected by the image data from the additional intraoral image 762. However, the portion of the intraoral image 762 showing the resected area outside the margin 754 is used to extend the prepared tooth 708 in the 3D model 760.

[0093] Figure 8 A diagram of a machine in the example form of a computing device 800 is shown, in which a set of instructions for causing the machine to perform any one or more of the methods discussed herein can be run. In alternative embodiments, the machine can be connected (e.g., networked) to other machines in a local area network (LAN), an intranet, an extranet, or the Internet. The machine can operate as a server or client in a client-server network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine can be a personal computer (PC), a tablet computer, a set-top box (STB), a personal digital assistant (PDA), a cellular phone, a network device, a server, a network router, a switch, or a bridge, or any machine capable of executing a set of instructions (sequential or otherwise) specifying the actions to be taken by the machine. In addition, although only a single machine is shown, the term "machine" should also be deemed to include any collection of machines (e.g., computers) that execute a set (or multiple sets) of instructions individually or collectively to perform any one or more of the methods discussed herein. The term "machine" should also refer to intraoral scanners and computing devices (e.g., Figure 1 An integrated all-in-one device that integrates a scanner 150 and a computing device 105).

[0094] The exemplary computing device 800 includes a processing device 802, a main memory 804 (e.g., read-only memory (ROM), flash memory, dynamic random access memory (DRAM) such as synchronous DRAM (SDRAM)), etc.), a static memory 806 (e.g., flash memory, static random access memory (SRAM), etc.), and a secondary memory (e.g., a data storage device 828), which communicate with each other via a bus 808.

[0095] Processing device 802 represents one or more general-purpose processors, such as a microprocessor, a central processing unit, etc. More specifically, processing device 802 can be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor that implements other instruction sets, or a processor set that implements a combination of instructions. Processing device 802 can also be one or more special-purpose processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), or a network processor, etc. Processing device 802 is configured to execute processing logic (instructions 826) for performing the operations and steps discussed herein.

[0096] The computing device 800 may also include a network interface device 822 for communicating with a network 864. The computing device 800 may also include a video display unit 810 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)), an alphanumeric input device 812 (e.g., a keyboard), a cursor control device 814 (e.g., a mouse), and a signal generating device 820 (e.g., a speaker).

[0097] The data storage device 828 may include a machine-readable storage medium (or more specifically, a non-transitory computer-readable storage medium) 824 on which is stored one or more sets of instructions 826, including any one or more methods or functions described herein. Non-transitory storage media refers to storage media other than carrier waves. The instructions 826 may also be stored completely or at least partially in the main memory 804 and / or in the processing device 802, and during its execution by the computer device 800, the main memory 804 and the processing device 802 also constitute computer-readable storage media.

[0098] The computer readable storage medium 824 may also be used to store an intraoral scanning application 850, which may correspond to Figure 1 . The computer-readable storage medium 824 may also store a software library comprising methods of the intraoral scanning application 850. Although the computer-readable storage medium 824 is shown as a single medium in the exemplary embodiment, the term "computer-readable storage medium" should be considered to include a single medium or multiple media (e.g., centralized or distributed databases, and / or associated caches and servers) that store one or more sets of instructions. The term "computer-readable storage medium" should also be considered to include any medium other than a carrier wave that is capable of storing or encoding a set of instructions that are executed by a machine and cause the machine to perform any one or more of the methods of the present invention. Therefore, the term "computer-readable storage medium" should be considered to include, but is not limited to, solid-state memories and optical and magnetic media.

[0099] It should be understood that the above description is intended to be illustrative rather than restrictive. After reading and understanding the above description, many other embodiments will be apparent. Although embodiments of the present invention have been described with reference to specific exemplary embodiments, it should be understood that the present invention is not limited to the described embodiments, but can be modified and changed within the spirit and scope of the appended claims. Therefore, the description and drawings should be regarded as illustrative rather than restrictive. Therefore, the scope of the present invention should be determined with reference to the full scope of the appended claims and the equivalents of these claims.

Claims

1. A non-transitory storage medium having instructions, which, when executed by a processing device, cause the processing device to perform operations comprising: receiving first intra-oral scan data of an intra-oral location, the first intra-oral scan data having been generated at a first time; receiving second intra-oral scan data of the intra-oral location, the second intra-oral scan data having been generated at a second time later than the first time; analyzing the second intraoral scan data of the intraoral location with respect to the first intraoral scan data of the intraoral location; identifying an area of ​​interest (AOI) based on the analysis; as well as Data of the AOI is removed from at least one of the first intraoral scan data, the second intraoral scan data, or a three-dimensional (3D) model of an intraoral site generated from the first intraoral scan data and the second intraoral scan data.

2. The non-transitory storage medium of claim 1 , further comprising additional instructions which, when executed by the processing device, cause the processing device to perform further operations, the further operations comprising: locking a first portion of the first intraoral scan data that does not include the AOI; as well as A second portion of the second intraoral scan data excluding the AOI is locked.

3. The non-transitory storage medium of claim 2, further comprising additional instructions which, when executed by the processing device, cause the processing device to perform further operations, the further operations comprising: receiving third intraoral scan data depicting the AOI; as well as A 3D model of the intraoral location is generated using the first portion of the intraoral location from the first intraoral scan data, the second portion of the intraoral location from the second intraoral scan data, and a depiction of the AOI from the third intraoral scan data, wherein the depiction of the AOI from the third intraoral scan data replaces data of the AOI from at least one of the first intraoral scan data or the second intraoral scan data.

4. The non-transitory storage medium according to claim 1, wherein: The AOI includes anomalies.

5. The non-transitory storage medium according to claim 4, wherein: Identifying the anomaly includes identifying a conflicting area between the first intra-oral scan data and the second intra-oral scan data.

6. The non-transitory storage medium according to claim 5, wherein: The conflict area includes at least one of the following areas: an area in which at least one of blood, saliva, or foreign matter is present in the first intra-oral scan data but is absent in the second intra-oral scan data; or At least one of blood, saliva, or foreign matter is present in the second intra-oral scan data and is absent in the first intra-oral scan data.

7. The non-transitory storage medium according to claim 5, wherein: The conflict region includes a difference in delineation of a margin line between the first intra-oral scan data and the second intra-oral scan data.

8. The non-transitory storage medium according to claim 1, wherein: Analyzing the second intra-oral scan data with respect to the first intra-oral scan data includes: a direct pixel-based analysis or a point-based analysis between the first intra-oral scan data and the second intra-oral scan data.

9. The non-transitory storage medium according to claim 1, wherein: The first intraoral scan data and the second intraoral scan data are generated from a single patient visit, wherein the second time is later in the single patient visit and the first time is earlier in the single patient visit.

10. A system comprising: The non-transitory storage medium according to any one of claims 1 to 9; as well as A processing device executes operations stored in the non-transitory storage medium.

11. A non-transitory storage medium having instructions which, when executed by a processing device, cause the processing device to perform operations comprising: receiving a first set of intraoral images of one or more intraoral locations; receiving a second set of intraoral images of the one or more intraoral sites; With respect to the first set of intraoral images of the one or more intraoral sites, analyzing the second set of intraoral images of the one or more intraoral sites; identifying, based on the analyzing, one or more differences in overlapping data between the first set of intra-oral images and the second set of intra-oral images; Determining a first merging result of the first group of intra-oral images and the second group of intra-oral images based on a first priority of the first group of intra-oral images and the second group of intra-oral images; determining a second merging result of the first group of intraoral images and the second group of intraoral images based on a second priority of the first group of intraoral images and the second group of intraoral images; receiving a selection of the first combined result; as well as A three-dimensional (3D) model of the one or more intraoral parts is generated using the first group of intraoral images, the second group of intraoral images, and a first priority between the first group of intraoral images and the second group of intraoral images, wherein the one or more differences in the overlapping data are resolved in the 3D model using the first priority.

12. The non-transitory storage medium according to claim 11, wherein: The first set of intra-oral images is generated at a first time, and the second set of intra-oral images is generated at a second time that is later than the first time.

13. The non-transitory storage medium according to claim 12, wherein: The first set of intra-oral images and the second set of intra-oral images are generated from a single patient visit, wherein the second time is later in the single patient visit and the first time is earlier in the single patient visit.

14. The non-transitory storage medium according to claim 11, wherein: The first priority level includes a first weight for the first set of intra-oral images and a second weight for the second set of intra-oral images; Determining the first merging result includes: applying a first weighted average of overlapping data between the first set of intra-oral images and the second set of intra-oral images using the first weight and the second weight to merge the overlapping data; The second priority level includes a third weight for the first set of intra-oral images and a fourth weight for the second set of intra-oral images; and Determining the second merging result includes applying a second weighted average of overlapping data between the first set of intra-oral images and the second set of intra-oral images using the third weight and the fourth weight to merge the overlapping data.

15. The non-transitory storage medium of claim 11, further comprising additional instructions which, when executed by the processing device, cause the processing device to perform further operations, the further operations comprising: A portion of the first set of intraoral images depicting a particular intraoral location of the one or more intraoral locations is targeted, wherein data of the second set of intraoral images also depicting the particular intraoral location is used in the generation of the 3D model.

16. The non-transitory storage medium according to claim 11, wherein: At least one of the first priority or the second priority is based at least in part on an identification of the one or more intra-port bits.

17. A system comprising: The non-transitory storage medium according to any one of claims 11 to 16; as well as A processing device executes operations stored in the non-transitory storage medium.

18. A non-transitory storage medium having instructions which, when executed by a processing device, cause the processing device to perform operations comprising: receiving first intraoral scan data including a first depiction of a dental site, the dental site including a prepared tooth; generating a virtual three-dimensional (3D) model of the dental part using the first intraoral scan data; processing at least one of the first intra-oral scan data or data from the virtual 3D model to identify a gum line of the prepared tooth, wherein at least a portion of the gum line is unclear; outputting instructions to generate second intra-oral scan data of the dental site that delineates the portion of the gum line that is unclear; receiving the second intra-oral scan data; and The virtual 3D model of the dental site is updated using the second intraoral scan data, wherein portions of the gum line that are unclear in the virtual 3D model are replaced using information from the second intraoral scan data.

19. The non-transitory storage medium of claim 18, the operations further comprising: A contour of the edge line is determined based on the processing.

20. The non-transitory storage medium according to claim 18, wherein: The processing is performed by a machine learning model.

21. The non-transitory storage medium of claim 18, the operations further comprising: Outputting the view of the virtual 3D model to a display; as well as An indication of an unclear portion of the margin line is provided in the view.

22. The non-transitory storage medium according to claim 18, wherein: The first intraoral scan data includes: a first group of intraoral images and a second group of intraoral images, and the operation further includes: determining a portion of one or more intra-oral images of the first set of intra-oral images to be locked; and Automatically locking determined portions of the one or more intraoral images, wherein the locked portions of the one or more intraoral images in the first set of intraoral images are used to depict a first area of ​​the virtual 3D model of the prepared teeth, and wherein portions of the second set of intraoral images that also depict the prepared teeth are not used for the first area of ​​the model.

23. The non-transitory storage medium of claim 18, the operations further comprising: receiving second intraoral scan data including a second depiction of the dental site; determining a first one of the first intra-oral scan data or the second intra-oral scan data for use in delineating a portion of the dental site; as well as locking at least a portion of the determined first one of the first intra-oral scan data or the second intra-oral scan data depicting a portion of the dental site; wherein the virtual 3D model is generated at least in part based on the first intraoral scan data and the second intraoral scan data, wherein a locked portion of the first one of the first intraoral scan data or the second intraoral scan data determined is used to depict a first area of ​​the virtual 3D model of a portion of the dental part, and wherein a portion of the second one of the first intraoral scan data or the second intraoral scan data also depicting a portion of the dental part is not used for the first area of ​​the virtual 3D model.

24. A system comprising: The non-transitory storage medium according to any one of claims 18 to 23; as well as A processing device executes operations stored in the non-transitory storage medium.

25. A method comprising: receiving a previously generated full intra-mouth scan of a patient's dental arch generated prior to generation of prepared teeth in the patient's dental arch; After the prepared teeth are generated, receiving an intraoral image data set of the prepared teeth; locking the intraoral image data set of the prepared tooth; as well as The locked intraoral image data set of the prepared teeth is combined with the previously generated full intraoral scan of the dental arch to generate a new three-dimensional (3D) model of the dental arch, wherein data for the prepared teeth in the 3D model are based on the intraoral image data set, and wherein data for hints of the dental arch are based on the previously generated full intraoral scan of the dental arch.

26. The method according to claim 25, further comprising: determining a portion of an image depicting the prepared tooth from the intraoral image data set of the prepared tooth; Therein, locking the intra-oral image dataset of the prepared tooth comprises: locking a certain portion of the image from the intra-oral image dataset of the prepared tooth.

27. The method of claim 25, further comprising: determining an identification of the prepared tooth represented in the intraoral image data set; as well as Based at least in part on the identification of the prepared tooth, a portion of the intraoral image dataset of the prepared tooth is selected for generating the 3D model.

28. The method of claim 25, further comprising: A boundary of the prepared teeth is determined from the intra-oral image dataset, wherein one or more portions of the previously generated full intra-oral scan of the dental arch inside the boundary are not applied to the 3D model.

29. The method according to claim 28, wherein: The boundary of the prepared tooth is the termination line of the prepared tooth.

30. The method of claim 29, wherein: The boundary of the prepared tooth is the offset of the termination line of the prepared tooth.

31. The method of claim 25, further comprising: Determining that the intraoral image data set of the prepared teeth has a higher priority than the previously generated full intraoral scan of the prepared teeth, wherein, in response to determining that the intraoral image data set of the prepared teeth has a higher priority, the intraoral image data set of the prepared teeth is automatically locked.

32. The method of claim 25, wherein: The intraoral image data set is combined with the previously generated full intraoral scan of the dental arch by applying weighted averaging to overlapping data between the intraoral image data set and the previously generated full intraoral scan of the dental arch for areas of the dental arch excluding the prepared teeth.

33. The method of claim 25, further comprising: identifying the prepared tooth; performing image processing on at least one of the 3D model or the intraoral image dataset to determine a termination line of the prepared tooth; as well as Based on the termination line, a contour on the generated 3D model is generated.

34. The method of claim 25, further comprising: performing image processing on at least one of the 3D model or the intraoral image dataset to determine a termination line of the prepared tooth; generating an offset of the termination line; as well as A portion of the intraoral image in the intraoral image data set that is within the offset of the termination line is determined, wherein only the portion of the intraoral image data set that is within the offset of the termination line is locked.

35. The method of claim 25, wherein: The previously generated full intraoral scan of the dental arch and the intraoral image dataset are generated in the same patient visit.

36. The method of claim 25, wherein: The previously generated full intraoral scan of the dental arch was generated in a first patient visit, and the intraoral image data set was generated in a second patient visit that is later than the first patient visit.

37. The method of claim 25, further comprising: determining that the intraoral image data set has a higher image quality than the previously generated full intraoral scan of the dental arch of the prepared teeth; as well as determining that data from the previously generated full intra-mouth scan of the dental arch degrades the quality of the 3D model of the prepared teeth; Therein, in response to determining that the intra-oral image data set of the prepared teeth has a higher image quality, the intra-oral image data set of the prepared teeth is automatically locked.

38. The method of claim 25, wherein: Receiving the previously generated full intra-mouth scan of the dental arch includes receiving a previously generated 3D model of the dental arch.

Images