Method and system for generating a model for use in a virtual extraction procedure of a target extraction object in a patient - Patents.com

Abstract

To provide a method that is executed on a computer to automatically generate a three-dimensional model for use in a virtual extraction procedure of a targeted extraction object in a patient.SOLUTION: The method is based on each of a surface scan of an anatomical region of the patient's oral cavity and a volumetric density scan of the anatomical region, the method including: cross-mounting in a common 3D coordinate system, labeled surface scan segments from a first dataset to labeled volumetric density scan segments from a second dataset; receiving an identifier of the targeted extraction object; identifying a 3D volumetric density model and a 3D surface model; and generating a third dataset comprising a 3D model of a socket, a 3D model of a portion of a dental crown, a 3D model of a portion of a target extracted tooth, and / or a 3D model of the extraction contour.SELECTED DRAWING: Figure 1

Description

[Background technology]

[0001] Digital software tools for anatomical treatment planning and / or prosthetic restoration design rely on accurate scan data of anatomical objects of interest, along with adjacent anatomical structures such as bones, tissues, nerve canals, organs, and other anatomical structures / objects / features, as well as artificial structures such as surgical implants, abutments, other fixation systems, artificial analogs of natural structures, and other naturally and artificially placed anatomical structures.

[0002] In the field of digital dentistry, scan data of anatomical structures are generally determined optically or radiologically. Optical scanners for taking 3D measurements directly from intraoral surface structures or from extraoral impressions of oral surface structures are widespread and economical. Surface data is usually represented by a surface mesh made of triangular elements and is routinely stored and exchanged between systems in STL or similar surface definition formats.

[0003] Radiological scanners such as digital volume tomographs (DVT) and computed tomographs (CT) use x-rays to generate volumetric data sets of anatomical structures. From these data sets, surface representations can also be determined by applying thresholding methods that analyze the intensity values ​​(measured in Hounsfield units) of individual scan elements (voxels) and determine whether they are above or below a certain threshold. Radiologically dense structures such as teeth (hereafter "volume dense" structures or objects) can be identified in this way, and the interfaces of identified volume dense scan structures can be modeled as triangulated surface data (e.g., triangular surface meshes), which can also be stored and exchanged between systems in STL or similar surface definition formats.

[0004] Calculations with MR scan data are more complex and it is preferable to apply contour analysis methods that work based on the gradients of adjacent scan elements, but even in such cases, volumetric structures or objects such as gingiva, gums and other tissues (mainly soft tissues) can be identified and their interfaces modeled as triangulated surface data for further processing. Summary of the Invention

[0005] The described analysis method can provide triangulated surface data for all modalities (imaging devices) of desired surfaces, e.g., visible surfaces, or boundary surfaces, e.g., inter-object surfaces or other non-visible surfaces between anatomical objects and / or structures in the scan data. In the context of the present invention, volumetric density scan data can include, e.g., all types of data described above, regardless of how it was acquired, but also other algorithmically and / or interactively determined structures, such as preparation lines, segmentation lines, anatomical landmarks, and desired contact points for use in virtual extraction procedures, virtual tooth restoration procedures, virtual prosthesis design and placement procedures, etc.

[0006] Optical surface scan data is typically preferred in the evaluation or calculation of virtual dental restorations because high accuracy can be achieved without exposing the patient to high doses of radiation.

[0007] In the context of the present disclosure, the term dental restoration part encompasses all kinds of objects that can be produced for dental esthetics and / or treatment of defects. For example, inlays, onlays, partial crowns, crowns, telescopic crowns, bridges, veneers, implant abutments, partial prostheses and prostheses. Also, in the context of the present disclosure, for the sake of simplicity, tooth replacement parts are encompassed by the term dental restoration part. The term virtual dental restoration (hereinafter more simply called dental restoration) is intended to include a suitable electronic representation of a dental restoration, i.e. a digital three-dimensional representation, preferably a surface representation, of the dental restoration part with sufficient accuracy. For example, the CAD / CAM data set of the corresponding virtual dental restoration can be transferred to a manufacturing machine to manufacture the dental restoration part.

[0008] When treating tooth defects, tooth preparation is usually performed, i.e. removal of caries, old fillings or missing tooth parts. What remains of each tooth is the remaining tooth substance, whose surface is divided into prepared (cavity) and non-prepared parts. The boundary line between the non-prepared and pre-prepared tooth surfaces is called the preparation line. In addition to the preparation line, which is only present in the case of pre-prepared teeth, each tooth also has at least one segmentation line that separates the tooth from extra-tooth structures such as gums and / or bone and / or adjacent teeth. As can be seen from these definitions, the segmentation line and / or the preparation line are the boundaries of the non-prepared tooth surfaces. In the context of the present disclosure, these boundary lines can be placed in the scan data mainly interactively, by using a graphical user interface, but algorithmic assistance based on the analysis of the surface curvature of the scan data is usually used in its determination.

[0009] Due to the complex structure of the scan data and the need to satisfy multiple functional-aesthetic criteria (adjustments to the opposing dentition taking into account jaw movements, adjustments to the adjacent teeth taking into account contact points, adjustments to the preparation line for optimal marginal fit, adherence to minimum material strengths for sufficient mechanical stability, consideration of the desired tooth shape in the anterior region, etc.), the process of designing a custom dental restoration, even with the assistance of interactive computer-aided design tools, still requires a human with knowledge and experience in dental restoration design and training to use such tools.

[0010] Adding to the complexity of customer prosthetic design, prosthetics are often designed to replace anatomical objects that have been removed from an existing oral condition. For example, a designer may be tasked with designing a temporary or permanent prosthesis to be placed on an abutment that is attached to an implant that is placed in the cavity of the extracted object. The designer can achieve greater accuracy by designing the prosthesis based on the patient's actual anatomical contours in the cavity where the extracted object was placed. Therefore, after the anatomical object is removed, the site often has to be optically rescanned before a surface model can be sent to the designer.

[0011] As dental care becomes more immediatized, new ways to shorten pre-planning are being sought, especially for procedures that can be completed in one or fewer visits than previously possible.

[0012] In situations where an anatomical object is targeted for extraction, methodologies are being explored to pre-plan the design of a prosthesis so that it can be immediately placed in place in the patient's mouth to replace the anatomical object being extracted. For example, it may be desirable to pre-plan the placement of an implant and pre-plan the design of a temporary abutment so that the implant can be placed and a custom temporary abutment placed immediately after the extraction (i.e., during the office visit). It may also be desirable to design and manufacture an abutment by additive or subtractive manufacturing (e.g., 3D printing or milling, respectively) and place it in the patient's mouth during the office visit to accommodate as immediate a situation as possible.

[0013] Ideally, a prosthesis should be designed to fit and conform to the contours of the fossa that is created when an anatomical object is removed. However, modern prosthesis design tools, such as CAD / CAM systems, only receive surface data (represented as a 3D triangular mesh, typically in STL format), and therefore miss hidden structures such as bones and other internal structures and contours. Thus, prosthesis design tools have no information about the fossa profile of the extracted anatomical object, but designers typically estimate the fossa profile of the extracted anatomical object when designing a prosthesis to fit in that place.

[0014] This can result in poor fit and misalignment when the prosthesis designed in this way is placed in the patient's mouth, which can lead to patient discomfort and other treatment-related problems. It is an object of the present invention to alleviate at least some of the drawbacks of known dental restoration or prosthesis design and placement procedures. In particular, it is an object of the present invention to provide a method and system that aids in improved pre-planning of dental restoration or prosthesis procedures using CAD / CAM tools.

[0015] This object is solved by a computer-implemented method for automatically generating a three-dimensional model for use in virtual removal of a target removal object of a patient according to independent claim 1 and by a system for automatically generating a three-dimensional model for use in virtual removal of a target removal object of a patient according to independent claim 14.

[0016] Dependent claims 2 to 13 represent embodiments of the invention.

[0017] In accordance with the present invention, a computer-implemented method automatically generates, by one or more computer processors, a three-dimensional model for use in the virtual extraction of a target extraction object of a patient, the target extraction object comprising an object in the patient's anatomical structure. The method is based on a surface scan of an anatomical region of the patient's oral cavity and a volume density scan of the same or a similar anatomical region, the anatomical region comprising at least the target extraction object. The method includes: receiving a first data set of labeled surface scan segments, each surface scan segment including a three-dimensional (3D) surface model of a corresponding object recognized and segmented from surface scan data of the surface scan, each surface scan segment having an associated label that identifies the surface scan segment; receiving a second data set of labeled volume density scan segments, each volume density scan segment including a three-dimensional (3D) volume density model of a corresponding object boundary surface recognized and segmented from volume density scan data of the volume density scan, each volume density scan segment having an associated label that identifies the volume density scan segment.

[0018] Recognition and segmentation of objects in the surface scan data of a surface scan and / or the volume density scan data of a volume density scan may be based on or assisted by automated data analysis, such as, for example, analysis by a suitably trained neural network capable of distinguishing anatomical objects and structures from one another in the surface scan data and / or volume density scan data.

[0019] In one embodiment, the associated label of each surface scan segment and / or each volume density scan segment identifies the respective surface scan segment or the respective volume density scan segment as corresponding to a tooth, a portion of a tooth, a prosthetic tooth, a portion of periodontium, a portion of bone, an implant, or other object or structure present in the patient's oral cavity and / or anatomical region.

[0020] In the present invention, the labeled surface scan segments from the first data set are cross-mounted with the labeled volume density scan segments from the second data set in a common 3D coordinate system. In one embodiment, the cross-mounting can be performed by using correspondingly labeled surface and volume density scan segments. Alternatively, or in addition, the cross-mounting can be performed by determining surface and volume density scan segments that are similar in shape and volume.

[0021] In one embodiment, the cross mount may include scaling, translation and rotation transformations applied to the surface scan segments and / or the volume density scan segments to achieve the best possible registration of the surface scan segments and the volume density scan segments in a common 3D coordinate system. In one embodiment, the cross mount parameters, such as scaling factors, translation and / or rotation transformation parameters for up to three spatial dimensions, may be stored and used to "overlap" the separately maintained labeled surface scan segments and labeled volume density scan segments or portions thereof, as needed in the following processing.

[0022] In one embodiment, cross-mounting includes generating a fifth data set of labeled segments, each segment including a combined three-dimensional (3D) model representing the anatomical object, the combined three-dimensional (3D) model including both the labeled surface scan segments and the labeled volume density scan segments. In some applications, portions of the labeled surface scan segments and / or the labeled volume density scan segments may be removed from the combined three-dimensional (3D) model representing the anatomical object.

[0023] For purposes of the following disclosure, when commonly represented features in the identified 3D volumetric density model and the identified 3D surface model are determined, this determination may be based on an operation of cross-mounting the labeled surface scan segments from the first data set and the labeled volumetric density scan segments from the second data set, thereby establishing an association between the labeled surface scan segments from the first data set and the labeled volumetric density scan segments from the second data set, and storing the association information in a temporary or non-transient storage component. Alternatively, determining commonly represented features in the identified 3D volumetric density model and the identified 3D surface model may be determined in a separate operation of establishing an association between corresponding labeled surface scan segments from the first data set and the volumetric density scan segments from the second data set, and optionally storing the association information in a temporary or non-transient storage component.

[0024] In the present invention, one or more computer processors receive the target extraction objects, including receiving a selection of target extraction objects that may be manually selected or a selection aided by a suitably trained neural network that can identify objects likely to be candidates for extraction, such as damaged or modified objects, anatomical location or orientation of objects, or unhealthy objects, for example, in the surface scan data and / or volume density scan data.

[0025] In certain embodiments, the target extraction object is a tooth or a portion of a tooth, such as a root.

[0026] In the present invention, both a 3D volume density model associated with the labels of the volume density scan segments and a 3D surface model associated with the labels of the surface scan segments corresponding to the identified target extraction object are identified.

[0027] The method of the present invention may also include a step of generating a third dataset comprising a model including a 3D model of the fovea that is equivalent to the identified 3D volume density model minus the portion of the identified 3D volume density model that is commonly represented in the identified 3D surface model.

[0028] For the purposes of this disclosure, it is assumed that the 3D model of the fossa substantially corresponds to the portion of the identified 3D volumetric density model.

[0029] In one embodiment, the 3D model of the cavity is generated by determining portions of the identified 3D volumetric density models that are not commonly represented in the identified 3D surface models, i.e., if the target extraction object is a tooth, the cavity model is derived by determining portions of the identified 3D volumetric density models that are not commonly represented in the identified 3D surface models, which typically includes at least a portion of the tooth root.

[0030] In another embodiment, the 3D model of the fovea is generated by determining portions of the identified 3D volume density model that are commonly represented in the identified 3D surface model and determining the difference between the determined commonly represented portions and the 3D volume density model by removing the determined commonly represented portions from the 3D volume density model.

[0031] In an embodiment, the determined portions of the identified 3D volumetric density model are stored in a computer readable data storage component, which may be a temporary or non-transient storage component, including a network storage component.

[0032] The method of the present invention may further comprise generating a third data set comprising at least a portion of the identified 3D volumetric density model commonly represented in the identified 3D surface model and / or a model comprising at least a portion of the identified 3D surface model commonly represented in the identified 3D volumetric density model. Thus, the third data set comprises a representation of at least a portion of a visible portion of the target extraction object, such as, for example, a dental crown or a prosthesis.

[0033] The method of the present invention may further comprise generating a third data set comprising at least a portion of the identified 3D volumetric density model excluding portions of the identified 3D volumetric density model commonly represented in the identified 3D surface model, such that the third data set comprises a representation of at least a portion of the unseen portions of the target extraction object, e.g., a tooth root, an implant, and / or an abutment.

[0034] The method of the present invention may further comprise generating a third dataset comprising models from at least a subset of the first dataset excluding portions of the identified 3D surface models that are commonly represented in the identified 3D volumetric density models.

[0035] In an embodiment, the method may include generating a fourth data set that is a copy of at least a subset of the first data set excluding portions of the identified 3D surface models that are commonly represented in the identified 3D volumetric density model, such that the third data set includes a representation of at least a portion of a visible portion of the patient's intraoral cavity excluding visible portions of the target extraction objects defined by the identified 3D surface models.

[0036] Thus, the third data set generated by the present invention includes a representation of at least a portion of the patient's anatomical region, with or without a representation of the target extraction object. This may aid in improving pre-planning of a dental restorative or prosthetic procedure using CAD / CAM tools such as those used by prosthetic designers, resulting in improved fit and alignment of the prosthesis in the patient's oral cavity. Furthermore, the information provided in the third data set is suitable for improving the design of the prosthesis, which may allow implant placement, placement of custom temporary or permanent abutments, and fitting of the prosthesis in one and the same consultation, which may allow the prosthesis to be placed immediately in place in place of the extracted anatomical object in the patient's oral cavity. In an embodiment, the method includes determining a boundary of the identified 3D surface model and generating a cut line from the determined boundary. This may include smoothing, straightening, or averaging the identified 3D surface model boundary. It may also include determining an intersection curve between the surface definition of the identified 3D surface model and the interface of the 3D volumetric density model. The cut line thus defined may then be projected onto the identified 3D volumetric density model and used to generate a sixth data set comprising the identified 3D surface model and / or the identified 3D volumetric density model on one side of the cut line including substantially all portions of the identified 3D volumetric density model commonly represented in the identified 3D surface model. Thus, the sixth data set may include a representation of at least a portion of the visible portion of the target removal object with an improved boundary substantially representing the demarcation line between the visible and invisible portions of the target removal object in the patient.

[0037] In an embodiment, the method further comprises augmenting any one of the first, second, third, fourth, fifth and / or sixth datasets, or the identified 3D surface model and / or the identified 3D volumetric density model, in the event of voids in the surface defined by the dataset or model, respectively, thereby filling such voids with the generated surface information. The generated surface information may be generated at least in part based on standard anatomical modeling, which may be determined by interpolating the boundaries of the voids and / or by a suitably trained neural network capable of generating corresponding cross sections of the anatomical objects and structures. Thus, the generated surface information may provide a steady continuous progression of the existing surface information of the dataset or model, respectively, and in the best case represents a true representation of the surfaces or interfaces of the patient's anatomical objects or structures in locations where the dataset or model is incomplete.

[0038] In an embodiment, the method includes storing any of the third, fourth, fifth, sixth and / or seventh data sets, and / or composite or cross-mounted versions thereof, in a computer readable data storage component, which may be a temporary or non-transitory storage component, including a networked storage component. The information may be saved in a data exchange format or a surface definition format such as STL, for import into a CAD / CAM system for use in the process of designing the prosthesis.

[0039] The present invention also relates to a system for automatically generating a three-dimensional model for use in the virtual removal of a target removal object of a patient, the system comprising one or more computer processing units configured, when executed, to configure the one or more computer processing units to load and execute computer readable instructions implementing the method of the present invention and the aforementioned embodiments.

[0040] In one embodiment of the system of the present invention, the system includes an image processing application and an electronic display device for displaying a graphical representation of at least one of the first, second, third, fourth, fifth, sixth and / or seventh data sets and / or composite or cross-mounted versions thereof or the identified 3D surface model and / or the identified 3D volumetric density model to a user for viewing and modification.

[0041] The image processing application is configured in conjunction with an electronic display device to automatically update the display of the graphical representations of the first, second, third, fourth, fifth, sixth and / or seventh data sets and / or their composite or cross-mounted versions or the identified 3D surface model and / or the identified 3D volumetric density model each time such data sets and / or models are updated and information is added or removed.

[0042] In addition to displaying the graphical representation of the first, second, third, fourth, fifth, sixth and / or seventh dataset and / or their composite or cross-mounted versions or the identified 3D surface model and / or the identified 3D volume density model, the image processing application also provides display controls responsive to user input for modifying the display of the graphical representation of the first, second, third, fourth, fifth, sixth and / or seventh dataset and / or their composite or cross-mounted versions or the identified 3D surface model and / or the identified 3D volume density model, thereby enabling, for example, modification of the viewing angle, rotation or pitch of the displayed information.

[0043] In addition to these display controls, the image processing application also provides a selection control which, when activated, allows a user to select at least one of the objects graphically represented on the display. After receiving a selection operation by a user of the image processing application using the selection control, the image processing application may highlight labeled surface scan image segments and / or volume density scan image segments in the graphical representation of the first, second, third, fourth, fifth, sixth and / or seventh data sets and / or composite or cross-mounted versions thereof or the identified 3D surface model and / or the identified 3D volume density model in response to the user's selection on the electronic display device.

[0044] The image processing application further provides a virtual removal control which, when activated, enables, for each graphically represented object selected by the user using the selection control, to remove the graphical representation of the target removal object from the graphical representation displayed on the electronic display device and to complement it with a graphical representation of a fovea model.

[0045] The present invention also relates to a computer-readable medium comprising computer-readable instructions that, when executed by one or more computer processing units, configure one or more computer processing units to implement at least one component of a system for automatically generating a three-dimensional model for use in virtual removal of a target removal object in a patient.

[0046] The present invention also relates to a computer program embodying computer readable instructions that, when executed by one or more computer processing units, configures the one or more computer processing units to implement at least one component of a system for automatically generating a three-dimensional model for use in virtual removal of a target removal target in a patient.

[0047] The present invention also relates to the use of the third, fourth, fifth, sixth and / or seventh data sets and / or composite or cross-mounted versions thereof in a virtual extraction procedure of a target extraction object in a patient.

[0048] The invention also relates to the use of said third, fourth, fifth, sixth and / or seventh data sets and / or composite or cross-mounted versions thereof in a prosthesis design CAD / CAM software tool to generate a prosthesis design anatomically representative of a patient's target extraction object, which may be performed prior to and / or during (preferably within a single visit) the patient visit during which the actual extraction procedure of the target extraction object is performed.

[0049] Such use may reduce the number of patient visits from many to as few as one by allowing accurate pre-planning of the extraction procedure and design and manufacture of anatomically correct prostheses and possibly abutments before or during the procedure. Upon removal of the target extraction object from the patient's anatomy, attachment of the abutments / prosthesis may occur in situ and immediately following removal of the target extraction object.

[0050] The present disclosure provides devices, systems, graphical user interfaces, computer-implemented tools, methods and processes for generating, displaying, exporting / storing and using anatomically accurate virtual 3D models of an anatomical cavity or fossa (hereinafter referred to as a "fossa") configured to fit an object (such as a tooth or implant) within the oral cavity of a mammal (such as a human, hereinafter referred to as a "patient").

[0051] In the context of the present disclosure, a virtual 3D model is a three-dimensional digital representation of a physical object (or fossa). Virtual 3D models are often represented as a point cloud or a mesh of triangles (or other polygons), as is commonly known and used in the field of computer graphics. The virtual 3D model of the fossa (or simply the "fossa model") implemented according to the described embodiments is a virtual 3D model that represents the anatomy of the actual fossa of a patient in which the relevant object is placed, and is determined and generated based on a surface scan of the patient's anatomical region (such as the oral cavity) and a volumetric density scan of the anatomical region, respectively.

[0052] In both surface and volume density scans, the anatomical region scanned includes at least an object placed in the actual cavity. In some use cases, the purpose behind generating the cavity model is to target an actual object placed in the patient's actual cavity for extraction (herein, a "target extraction object"). For example, the object may include a tooth to be extracted (herein, a "target extraction tooth") or an implant to be extracted (herein, a "target extraction implant").

[0053] Having access to and corresponding insights from a virtual socket model specific to the patient's situation can improve patient treatment planning, particularly diagnostic and treatment planning related to the patient's anatomy. For example, dental treatment planning can be improved, including treatment plans involving tooth extractions, implants, abutments and / or placement of prostheses such as artificial crowns. Furthermore, treatment workflow can be improved by reducing the number of steps and patient visits required to design an anatomically correct prosthesis, such as a temporary healing abutment or a permanent abutment.

[0054] Each socket model generated as described herein substantially matches the contours of the outer surface of an actual object that will be placed in the patient's anatomy. For example, a tooth socket model according to an embodiment of the invention is generated based on volumetric density scan data (e.g., from a CBCT scan) of an anatomical target area (including the target tooth) to match the contours of the root of the patient's actual tooth.

[0055] The present invention is further described in the following detailed description taken in conjunction with the drawings, in which like reference numerals refer to like elements unless otherwise noted.

[0056] It should be noted that, although the following detailed description describes particular embodiments of the invention in more detail, it is intended that features described in the context of only one embodiment can also be utilized in the context of or in combination with other embodiments of the invention, whether or not described in the detailed description (unless such combination of features produces meaningless results). The following detailed description is in no way intended to limit the invention to the specific embodiments and combinations of features thereof, and the invention is limited and defined exclusively by the appended claims. [Brief description of the drawings]

[0057] [Figure 1] 1 shows a flow diagram outlining a first aspect of a method according to an embodiment of the present disclosure. [Diagram 2] 1 illustrates an architecture diagram of a system according to an embodiment of the present disclosure. [Figure 3A] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3B] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3C] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3D] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3E] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3F] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3G] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3H] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3I] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3J] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3K] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3L] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3M] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 3N] 1 illustrates an exemplary embodiment of a method for generating a fovea model according to the present disclosure. [Figure 4] FIG. 4 is a flow diagram outlining a second aspect of a method according to an embodiment of the present disclosure. [Figure 5A] 1 illustrates a first exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 5B] 1 illustrates a first exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 5C] 1 illustrates a first exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 5D] 1 illustrates a first exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 6]4 depicts a flow diagram outlining a third aspect of a method according to an embodiment of the present disclosure. [Figure 7A] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 7B] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 7C] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 7D] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 7E] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 7F] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 7G] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 7H] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 7I] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 7J] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 8] 13 depicts a flow diagram outlining a fourth aspect of a method according to an embodiment of the present disclosure. [Figure 9A] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9B] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9C] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9D]1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9E] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9F] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9G] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9H] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9I] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9J] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9K] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9L] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9M] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9N] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9O] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9P] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9Q] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9R] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9S] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. [Figure 9T] 1 illustrates a second exemplary graphical user interface implementation of various aspects of the disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0058] One aspect of the present invention is the generation of a virtual three-dimensional cavity model that accurately models an anatomical cavity for placing an object within an anatomical region within a patient's oral cavity. Embodiments include a method for generating such a cavity model.

[0059] Figure 1 is a flow chart of a method 100 executed in a computer-implemented device, such as system 200 of Figure 2, for generating a 3D cavity model that accurately represents the anatomy of an actual cavity in a patient's anatomical region in which a relevant object, such as a tooth, is to be placed. The method 100 includes a step 101 of receiving a surface scan and a volume density scan (both stored in scan data 220 of the system 200) of an anatomical region of the patient's oral cavity (wherein the scanned anatomical region includes at least a target object, which in this example is in the form of at least a portion of the actual object that is to be placed in the patient's actual cavity).

[0060] Method 100 further includes step 102 of receiving a first data set of labeled surface scan segments and a second data set of labeled volume density scan segments (both stored in segment data 222) including individual segments corresponding to objects and / or features recognized by image recognition and segmentation processes in the surface scan and volume density scan of the target anatomical region. Steps 101 and 102 may occur outside of execution of application 215a, for example by remote service 212.

[0061] For example, the surface scan data 220 may be generated by a remote surface scanning service 240. The surface scanning service 240 may do any one or any combination of the following: collect surface scan data collected from a surface scanner (such as an optical scanner or camera), convert the surface scan data into a three-dimensional model accessible and readable by the processor 201 of the system 200, manipulate or convert the surface scan data as necessary into another format in preparation for display on the electronic display 209, store it in a computer readable file that can be received by the system 200, and store it in local memory 220 reserved for the scan data.

[0062] As another example of a remote service, the volume density scan data 220 may be generated by a remote volume density scanning service 240. The volume density scanning service 240 may do any one or any combination of the following: collect volume density scan data collected from a volume density scanner (using a volume density scanning device such as a cone beam computed tomography (CBCT) scanner), convert it into a three-dimensional model accessible and readable by the processor 201 of the system 200, manipulate or convert it into another format as necessary in preparation for display on the electronic display 209, store it in a computer readable file that can be received by the system 200, and store it in local memory 220 reserved for the scan data.

[0063] Another remote service 212 may include a segmentation service 244. The segmentation service receives scan data (surface (IOS) or volume density (CBCT or CT) scan data) and uses image processing, extraction and classification to segment the received scan image, including anatomical regions of interest, into various identified objects and associates classification tags with each of the segments. The segmentation service 244 may provide the segments in the form of individual virtual 3D segment models (also referred to herein simply as "segment models"), which are digital 3D representations of actual parts or features of the patient's anatomy. In one embodiment, the segmentation service 244 provides each segment as an individual digital 3D model, preferably (but not limited to) in STL format as a triangular mesh or point cloud.

[0064] Each of the remote services 212 returns data to the system 200 via a network adapter, where the data is stored in the appropriate bulk memory 206 (scan data 200 or segment data 222).

[0065] Either or both of steps 101 and 102 may be performed outside of the execution of application 215a, for example, by a separate process or application stored in local memory 204 and executed by local processor 201 of system 200. In other embodiments, either or both of steps 101 and 102 may alternatively or additionally be performed as an integrally executing process within application 215a.

[0066] Because the received surface and volume density scan segments are typically obtained using different modalities (and thus different, typically independent, scanning machines / devices), the scan data generated by each scanning modality is collected and stored according to a 3D coordinate system specific to the particular scanning machine / device that collected the data. In these situations, it is important to align the scan data from each scanner into a common 3D coordinate system so that similar objects from each scan are matched and appear to occupy the same space (as they should, since each represents the same object).

[0067] The (labeled) surface scan segments and the (labeled) volume density scan segments are cross-mounted to a common 3D coordinate system (step 103). As a result of the cross-mounting to the common 3D coordinate system, for each pair of surface scan segments and volume density scan segments that correspond to the same patient object in the scanned anatomical object region, there should be one or more respective points that "match" (i.e., the points of each segment of the segment pair substantially or exactly match) in the 3D space of the common 3D coordinate system. These matching points correspond to corresponding points on the actual anatomical object / feature of the patient. Matching points exist only for anatomical regions of the patient that could be imaged by a particular scanning modality.

[0068] Thus, because the surface scan data 220 includes only surface visible image data, and the volume scan data 220 includes both surface and subsurface image data, point matching can only occur for points of the volume density scan segments that correspond to surface visible points of the scanned object (as the surface scan segments do not include subsurface data points). Note that while the corresponding (or commonly represented) pair of points of each of the surface scan segments and the associated volume density scan segments should ideally match exactly in a common 3D coordinate system, due to differences in accuracy between scanning modalities and differences in the resolution and generation accuracy of the 3D scan segment models generated for each scanning modality, these points may only "substantially match", i.e., match within a margin of error.

[0069] Nevertheless, as a result of the cross-mounting, surface scan segments that correspond to (or are the core of) a scanned object should occupy approximately the same space in the 3D coordinate system as similar regions of the related object represented by the corresponding volume density scan segments.

[0070] At step 104, a selection is received indicating one or more target extraction objects.

[0071] The method further comprises generating 106, for each target extraction object, a 3D model of the fossa that fits an outer surface boundary of a portion of the target extraction object that is to be placed in the patient's actual fossa by identifying a 3D volume density model associated with a label of a volume density scan segment corresponding to the identified target extraction object, identifying a 3D surface model associated with a label of a surface scan segment corresponding to the identified target extraction object, and thereafter generating a 3D model of the fossa based on the corresponding (or commonly represented) identified surface scan segment models and the identified volume density scan segment models (respectively for each target extraction object).

[0072] In one embodiment, step 106 is performed by step 106a of determining commonly represented portions of the identified volumetric density models that do not occupy substantially the same 3D space as the identified surface models, and step 106b of setting the portions determined in step 106a as the 3D model of the fovea. In step 107, the generated fovea model is saved to a file and / or exported to a saved file for later use or distribution.

[0073] 2, the method 100 is executed on a computer implemented system 200 and includes computer readable instructions stored in a computer readable memory 202 that, when executed by a computer processor 201, perform the steps of the method. In an embodiment, the method is implemented as computer executable instructions stored in the memory 202 of the system 200, within a computer implemented application 215a (or "tool"), or accessible via a remote service via a device implemented network adapter 211 of the system 200.

[0074] The application 215a includes computer readable instructions that, when executed by a local or remote computer processor (not shown, but executing the service 112), perform the steps of the method 100. The application 215a may operate in association with the electronic display 209. In one embodiment, the application 215a includes a graphical user interface (GUI) that is displayed and presented on the electronic display 209.

[0075] The GUI can include a 3D model view pane for displaying a three-dimensional model of the patient's anatomical target area. The model displayed in the view pane is based on a surface scan and a volumetric density scan of the patient's target area. The GUI includes various user controls that allow a user to direct the application to perform various operations such as, but not limited to: selecting and loading a patient's scan records, selecting and manipulating display views, selecting content to display in the GUI, selecting target objects and / or target object identifiers that may be included in the patient's scan data, selecting and displaying identifiers, descriptions and images of implants, prostheses, materials, etc. in association with the patient's dental treatment plan.

[0076] One or more of such controls include an object selection tool that allows a user to select an object present in the patient scan data as a target extraction object. One or more of such controls include a cavity model generation tool that generates an independent 3D model of a cavity into which the target extraction object will be placed. Alternatively or in addition, one or more of such controls include a virtual tooth extraction tool that operates when activated by a user to extract a selected object from the region of interest.

[0077] Alternatively or in addition, one or more of such controls include an extraction tool that generates a dataset including a model of at least a portion of the exposed object (such as a dental crown) based on at least a portion of the identified 3D volumetric density model commonly represented in the identified 3D surface model and / or at least a portion of the identified 3D surface model commonly represented in the identified 3D volumetric density model.

[0078] Alternatively or in addition, one or more of such controls includes a separate extraction tool that generates a dataset including a model of at least a portion of the occluded object (such as a tooth root) based on at least a portion of the identified 3D volumetric density model excluding portions of the identified 3D volumetric density model that are commonly represented in the identified 3D surface model.

[0079] Alternatively or additionally, one or more of such controls include a tool that generates a dataset including a model from at least a subset of the surface scans, excluding portions of the identified 3D surface model that are commonly represented in the identified 3D volumetric density model.

[0080] 3A-3N show an exemplary embodiment of an aspect of the invention for generating a 3D cavity model from 3D surface scan segments (3D surface models of the corresponding object recognized and segmented from surface scan data of the surface scan) and volume density scan segments (3D volume density models of the corresponding object's interfaces recognized and segmented from volume density scan data of the volume density scan) associated with an object. In the illustrated embodiment, the object is a tooth and the generated cavity model is a tooth cavity that corresponds to part of the outline of the portion of the tooth below the gum line (i.e., the tooth root).

[0081] FIG. 3A shows an example of a 3D surface boundary model 1a generated based on a volume density scan. The individual structures of the model 1a correspond to the actual structures of the patient's oral situation. As shown, the 3D surface boundary model 1a embodies an anatomical representation of the patient's actual periodontium 2, bone 3, and teeth (labeled and shown as 11, 12, 13, 14, 15, 16, and 17 according to the Federation Dentair Internationale (FDI) notation (a tooth numbering system commonly used in the dental industry)), among other individual teeth (not labeled). In one embodiment, the surface boundary model 1a is generated from the 3D volumetric structures represented in the volume density scan, extracted by image recognition and segmentation techniques such as thresholding the intensity values ​​(measured in Hounsfield units) of the individual scan elements in the radiograph stack. In one embodiment, the surface boundary model 1a comprises a point cloud, a triangular or other polygonal mesh, or other 3D digital model.

[0082] FIG. 3B shows an example of a 3D surface model 1b generated based on a surface scan of the same anatomical region as FIG. 3A. The individual structures of model 1b correspond to actual structures in the patient's oral context. As shown, 3D surface model 1b embodies an anatomical representation of the patient's actual gum tissue 2, and teeth (labeled and shown as 11, 12, 13, 14, 15, 16 and 17 according to FDI notation). Because the bones in the patient's oral cavity are below the surface of the gums and teeth, typically no bone information is present in surface scan model 1b.

[0083] Because the surface scan and volume density scan are obtained by scanning the same intra-oral region of interest, both models 1a and 1b include respective model anatomical structures that represent the same particular anatomical structures of the patient, such as teeth 11, 12, 13, 14, 15, 16 and 17, and gums 2. Model 1b includes a point cloud, triangle or other polygonal mesh, or other 3D digital model generated from the 3D surface structures represented in the surface scan.

[0084] As noted in Figures 3A and 3B, both the volume density scan model 1a and the surface scan model 1b are surface models of the (generally same) anatomical region. The 3D surface models 1a and 1b do not include a representation of the internal anatomical structures. That is, neither model includes information about the fossa or other structures that lie beneath the visible outer surface of the object in the model. In the illustrated example where the object is a tooth, this means that the anatomical structure of the tooth socket in which the tooth sits cannot be ascertained, since the fossa is not visible in either the 3D volume density scan model (Figure 3A) or the 3D surface scan model (Figure 3B).

[0085] In both Model 1a of Figure 3A and Model 1b of Figure 3B, only the crown portion of the tooth is modeled, and neither scan model includes the anatomical structure of the root or cavity in which the tooth sits. While surface detail is useful when planning a patient's dental treatment, when planning surgical treatment and when designing a prosthesis for the patient, the lack of sub-surface information available in Models 1a, 1b relating to the patient's anatomical structure below the visible surfaces of the patient's oral cavity can hinder accurate planning and design.

[0086] To facilitate the generation of accurate 3D fovea models, in one embodiment, the respective volumetric density scan data and surface scan data from which the respective 3D surface models 1a, 1b are generated are submitted to a segmentation application. The segmentation application may be a remote service 244 or may be a local application (stored in the local memory 204 and executed by one or more processors 201). The segmentation processor processes each of the received surface and volumetric density scan data to automatically recognize individual recognized objects (via image recognition functionality) and extract and classify (via segmentation functionality) into labeled categories or classes.

[0087] For example, in an embodiment in which the objects are teeth within the patient's mouth, the segmentation processor receives each of the intraoral surface scan data and the CBCT or other volumetric density scan data and processes each of the scan data sets to recognize and label recognized objects as individually identified teeth, gums, bone, or other objects such as fillings, implants, etc. recognized in the received scan data. The segmentation processor marks the recognized objects with a corresponding object type label that is associated with the object type (or classification) of the recognized object.

[0088] For example, the segmentation processor may recognize objects in the 3D model or scan data that correspond to dental impressions 16 and assign the recognized object an object type label "16" (or any unique label that classifies the recognized object as a unique object type that corresponds to the patient's actual teeth 16). The segmentation processor recognizes and classifies (i.e., "labels") the recognized data segments in the scan data into a number of individual segments and corresponding to the various recognized object types.

[0089] Preferably, each segment comprises a 3D surface model representing an actual scanned object (e.g., a scanned tooth, a portion of a gum, a bone, an implant, etc.). In one embodiment, each segment corresponds to an individual object in the patient's oral cavity and is labeled as such with an associated label. Each segment comprises a free-standing 3D model represented as a three-dimensional triangular (or other polygonal) mesh.

[0090] 3C, the segmentation processor recognizes representations of individual teeth 11, 12, 13, 14, 15, 16, and 17 in the patient's volumetric density scan and segments each recognized representation of any of teeth 11, 12, 13, 14, 15, 16, and 17, gums 2, and bone 3 into corresponding independent segments 11a, 12a, 13a, 14a, 15a, 16a, 17a, 2a, and 3a, collectively forming a segmented volumetric density scan surface model 10a. Each segment is converted into an independent surface mesh, e.g., a 3D triangular mesh, and each segment can be independently selected (e.g., when presenting the segmented model in a graphical user interface (GUI) as described below).

[0091] Note that because segments 11a-17a, 2a, 3a are extracted from the volumetric scan data, each segment contains all available information from the volumetric density scan. This means that even objects (such as nerve canals) and parts of objects (such as tooth roots) that cannot be imaged in a surface scan because they are below the visible surface inside and outside the patient's mouth are modeled in the volumetric density scan segments. Each volumetric density scan segment contains the complete object information (based on what was imaged in the volumetric scan data), even if it is below the surface of the scanned anatomical region of the patient. Thus, each of teeth 11a-17a includes tooth root information that is clearly visible in the segmented model.

[0092] Similarly, with reference to Figure 3D, a segmentation processor may recognise representations of individual teeth 11, 12, 13, 14, 15, 16 and 17 in the patient's surface scan and segment each recognised representation of any of teeth 11, 12, 13, 14, 15, 16 and 17, as well as gums 2, into corresponding independently selectable segments 11b, 12b, 13b, 14b, 15b, 16b and 17b and 2b, which collectively form the segmented surface scan surface model 10b. Each segment is converted into an independent surface mesh, for example a 3D triangular mesh.

[0093] The segmented model 10a includes all segments of the scanned anatomical region of the patient, but with each segment being an independently selectable 3D model of the corresponding scanned object. Thus, each segment 11b-17b and 2b can be viewed alone, as shown, for example, in FIG. 3H (showing tooth segment 16b alone, corresponding to the patient's tooth 16). Tooth segment 16b includes only the portion of tooth 16 that is present in the surface scan. Thus, tooth segment 16b represents only the crown of tooth 16, since only the crown (the portion of tooth 16 that is above the gum line) is visible during the surface scan.

[0094] FIG. 3E shows the segmented volumetric density scan surface model 10a and the segmented surface scan surface model 10b cross-mounted in a common three-dimensional coordinate system. Independent imaging systems are typically used to capture each of the surface scan data and the volumetric density scan data. For example, an intraoral scanner (IOS) may be used to capture a surface scan of the patient's intraoral target area, and a CBCT scanner may be used to capture the patient's volumetric density scan. Both scans are important in providing important information and complement each other to provide a more complete picture of the patient's actual oral situation. Surface scanners (such as optical scanners) can capture very high resolution details of the visual topography of the patient's dentition, but can only capture surface details and not internal details.

[0095] In contrast, a volumetric density scan (such as a CT or CBCT scan) can capture the internal volume and density details of a patient's dentition, such as the dimensions and density of the jaws, complete teeth (including roots), and nerve pathways. Together, surface and volumetric density scans can form the basis of the dental treatment planning and prosthetic manufacturing process.

[0096] Because independent imaging systems capture image data relative to the specific 3D coordinate system of the imaging system that captures the scans, cross-mounting the scans into a single view pane with its own 3D coordinate system requires that both scans be aligned with each other. This process is often called scan matching or registration. Methods exist for aligning 3D meshes into a single 3D coordinate system.

[0097] In one embodiment, each of the surface scan data and volume density scan data is segmented into 3D triangular mesh segments corresponding to individual teeth and jaws, then key points of the corresponding tooth segments are determined from each of the surface scans and volume density scans of each tooth, and then the key points are aligned to a common 3D coordinate system. This process can be performed, for example, using CoDiagnostix™ dental implant planning software provided by Dental Wings, Inc. (a Straumann Group company).

[0098] It should be noted in FIG. 3E that the locations of the respective segments (11a, 12a, 13a, 14a, 15a, 16a, 17a, 2a, and 11b, 12b, 13b, 14b, 15b, 16b, and 17b, 2b) of each segmented model 10a and 10b correspond to the same actual anatomical structure (11, 12, 13, 14, 15, 16, 17, and 2) in the patient's mouth, respectively. As can be seen, it is very important that the segments corresponding to portions of the same actual anatomical structure obtained from each scan type (e.g., internal or surface) are placed in the same three-dimensional coordinate system. When they are properly cross-mounted, the segments representing the same actual anatomical structure will substantially coincide, as shown.

[0099] In one embodiment, a segmentation processor processes the scan data to identify and classify portions of the scan data into individual segments classified by anatomical structure type based on a set of labeled training data containing multiple instances of each of the anatomical structure types. In one embodiment, the segmentation processor is a convolutional neural network (CNN) trained on a large dataset of scan images obtained from many different people with different anatomical situations, including or lacking different teeth, gums, bones, and other natural and artificial anatomical structures (e.g., implants, prosthetics, etc.).

[0100] FIG. 3F illustrates the problem encountered when tooth scan segment 16b (see FIG. 3D) is removed from the surface scan. As shown, when the crown segment 16b is removed, a hole 16c is created in the model 10b where the crown 16b was originally. This is expected since the segmented surface scan 3D model 10b is generated based only on the surface scan data and does not include other subsurface information such as bone or root information. Thus, when the tooth segment 16b is selected and removed from the segmented surface scan model 10b (see FIG. 3D), there is no cavity information available and the surface scan model 10b only has the hole 16c where the tooth segment 16b was before it was removed.

[0101] FIG 3G shows the segmented surface scan 3D model after tooth segment 16b has been removed, and tooth segment 16a from the segmented volume density scan 3D model (FIG 3C) co-mounted in the same 3D coordinate system. FIG 3H shows tooth segment 16a separated and spaced apart from the other segments of volume density scan model 10b. Tooth segment 16a includes crown portion 16a. c and the tooth root portion 16a r The tooth root portion 16a of the tooth 16 is r is stem 16 at and three individual roots (lingual root 16a r_l, Middle buccal root 16a r_mbs (not visible in Fig. 3H), and distal buccal root 16a r_dbr ) The tooth 16 may have three roots, while other teeth may have only one root, or may have two roots or additional roots. For simplicity, the stem and individual roots of any tooth may be referred to collectively herein as the "roots." Crown 16a c Only the root portion 16a is visible above the gingival margin in the surface scan. r is not visible to the naked eye or to the intraoral scanner camera above the gum line.

[0102] Figure 3I shows an individual tooth segment 16b (derived from the segmented surface scan 3D model 10b of Figure 3G). As mentioned above, tooth segment 16B represents only the crown of tooth 16, since only the crown is visible to the surface scan camera (as the crown is above the gum line and visible to both the naked eye and the camera lens).

[0103] Note that because both the surface scan and the volume density scan image the same region of interest, both scans contain surface information related to the same actual corresponding anatomical structure (assuming each scan scanned the same region), which means that for a visible surface such as a tooth crown, both the surface scan and the volume density scan will contain surface information or surface boundary information, respectively, related to the tooth crown.

[0104] Surface scans using optical sensors tend to produce higher resolution images, resulting in surface 3D models with greater detail. Volume density scans typically do not provide the same level of accuracy as optical scans, or use modalities that are not medically safe to provide the same level of accuracy. For example, volume density scans produced using X-ray techniques, including CT or CBCT modalities, are based on X-ray radiation, and while they can provide highly accurate images using high doses of X-rays, they are not medically safe for patients. Therefore, CT and CBCT modalities used on patients are required to set the X-ray radiation dose very low, so that they are safer for the human body. The trade-off is that the images are less accurate. Thus, crown surface data from surface scans will generally contain more detail than crown surface data from volume density scans.

[0105] To generate a 3D model of the patient's oral situation after tooth extraction, the application performs a volume density scan of the root portion 16a of the tooth segment 16a. r Holds the crown part 16a c To do this, the application determines the gum line around the volume density scan tooth segment 16b based on points along the bottom edge of the surface scan tooth segment 16b.

[0106] FIG. 3J shows a cross-mount of a surface scan tooth segment 16a and a volume density scan tooth segment 16b (both displayed in the same 3D coordinate system). As shown, the gum line 16b gl is the set of points corresponding to the lower edge of the surface-scan tooth segment 16b.

[0107] Application: Gum line 16b gl Since we know the location of the crown portion 16a as all points on the same side of the gingival line (also called the cut line) where the surface scan tooth segment 16b is located within the application specific 3D coordinate system, c Calculate.

[0108] Briefly, the application removes all points of the volume density scan tooth segment 16a that coincide with or substantially fall within the same volume as the surface scan tooth segment 16b (i.e., crown) in the 3D coordinate system domain, i.e., corresponding portions of the volume density scan tooth segment and the surface scan tooth segment are commonly represented. r is obtained by subtracting 16b from 16a (and optionally removing any outlier points). c Root 16a after removal of r Since this is a surface model, only the outer points of the segment are present in the 3D model, and thus the crown portion 16a c When removed from segment 16a, interior 16a r_interior becomes empty. r The shape and morphology of the tooth root 16a, as defined from surface models of the individual segments obtained from the volume density scan, r is defined only by points on the exterior surface of

[0109] Thus, the inner contour of the root follows the outer contour of the root itself. r may be displayed together with the surface model 10a of FIG. 3F (with the crown segment 16a removed) to generate a surface 3D model 10d representing the patient's oral situation with the tooth 16 extracted. This is shown in FIG. 3L, FIG. 3M and FIG. 3N. FIG. 3L shows the 3D model 10d along a substantially horizontal plane, looking lingually to show the hole 16c and the fossa contour 16s (represented by 16br extracted from 16b). FIG. 3M shows the model 10d from another direction along the same horizontal plane as FIG. 3L, but from the rear of the model. From this perspective, the contour of the fossa is easy to see. FIG. 3N shows yet another view of the model 10d looking into the fossa where the tooth 16 has been virtually extracted. The contour of the fossa 16s is visible, showing where two of the three individual root apices were placed before the virtual extraction of the tooth 16.

[0110] The cavity 10S includes a stem 16Bt and the contours in which each of the individual tooth roots was placed prior to the virtual extraction. As shown, the cavity 16S includes a stem cavity portion 16 at and lingual root 16a r_l , Middle buccal root 16a r_mbs and distal buccal root 16a r_dbr The lingual root fossa 16sr_l and the mid-buccal root fossa 16s r_mbs and distal buccal root fossa 16s r_dbr The implant is fitted with three separate root cavities, each of which follows the contour of the root of the extracted tooth.

[0111] The tooth volume density scan segment model contains only the outer (boundary) surface of the tooth object and therefore does not contain any information about the interior of the tooth itself. That is, for segmentation, the segmentation processor generates a 3D mesh of the outer surface of the tooth without including any modeling of the tooth's interior. For closed objects such as teeth, the 3D mesh model is also a closed triangular mesh (the number of edges and triangular faces associated with any vertex is equal). Thus, the cut root segment 16a r (i.e., crown portion 16a c Since the interior of the segment 16) from which the root segment 16a has been removed is empty, r The inner surface of the root segment 16a r That is, the inner surface follows the same contour as the outer surface of the root segment 16a. r The same exterior wall of the wall is shown, but from the inside of the wall. c Removing the vertex results in an open mesh (i.e., there is at least one vertex in the mesh with a number of edges associated with that vertex that exceeds the number of triangular faces associated with that vertex). As used herein, a side face is a face with a number of adjacent faces (that share an edge) that is not equal to the number of edges of that face. The crown portion 16a of the volume density scan segment 16a c In the context of removing the remaining part of the volume density scan segment 16a, i.e., the cut root segment 16a rincludes a pair of edges along the margin line (where the crown meets the gum line) and is an open mesh. r Since there is no information inside the cut root segment 16a, the inner surface of the open mesh is r The outer surface is identical to that of the

[0112] A second aspect of the present disclosure relates to a virtual object extraction tool that incorporates the 3D fossa model generation methodology described above (e.g., in connection with FIGS. 3A-3N). Such a virtual object extraction tool can be implemented in a variety of anatomical treatment planning and design software tools.

[0113] For example, but not limited to, in various embodiments, a virtual object extraction tool is determined and generated in association with any and all of a virtual tooth extraction tool in a dental treatment planning software tool, an implant planning software tool, and / or a dental prosthesis design and manufacturing planning (CADCAM) software tool. For example, in a dental tooth extraction procedure, the novel apparatus, systems, graphical user interfaces, computer implemented tools, methods and processes presented herein allow for virtually performing a virtual extraction of a patient's tooth on an electronic display prior to the actual extraction of the tooth from the patient.

[0114] The virtual tooth extraction tool allows the dental professional to view a highly accurate model that accurately represents the contours of the actual cavity in which the object will be placed and that will remain exposed after the object is removed from the patient's oral cavity. The cavity model can be generated and displayed (along with the surrounding areas of the patient's oral cavity) after the virtual removal of the target extracted tooth. The cavity contour is accurately represented, and the cavity interior follows the contour of the outer surface of the patient's actual tooth (including the root) below the gum line (i.e., margin line).

[0115] The fossa model can be used to perform virtual object extraction, whereby a target extraction object (such as a tooth) can be extracted from a 3D model of the anatomical target area, and a virtual 3D fossa model can be added to the model (or made viewable with the model). Removing the object and displaying the 3D model with the virtual fossa visualized is in itself useful for actually extracting the object from the patient's anatomy. Additionally, it is useful in subsequent treatment planning, such as dental implant planning and prosthetic design planning, as described below.

[0116] 4, a virtual object extraction method 400 may include receiving 401 a surface scan and a volume density scan of an anatomical target region, and receiving 402 surface scan segments and volume density scan segments corresponding to objects recognized in the surface scan and volume density scan of the anatomical target region. Steps 401 and 402 may occur outside of the execution of an application or may be integral to the application, similar to the discussion of steps 101 and 102 in relation to the method of FIG.

[0117] The method 400 may further include step 403 of cross-mounting the surface scan segments and the volume density scan segments to a common 3D coordinate system. In step 404, the method includes displaying at least the cross-mounted segments associated with the target extraction objects and adjacent cross-mounted object segments on an electronic display. In step 405, the method includes receiving a selection of one or more target extraction objects. In step 406, the method includes identifying the surface scan segments and the volume density scan segments associated with the selected objects of interest. In step 407, the method includes generating, for each selected target extraction object, a 3D cavity model corresponding to the selected target extraction object.

[0118] Method 400 may further include step 408 of removing each of the identified surface and volume density scan segments corresponding to the selected target extraction object from the display and displaying the generated 3D cavity model on an electronic display. Method 400 may further include step 409 of saving and / or exporting a 3D surface model representative of the extracted tooth model including the generated 3D cavity model and associated segments of the anatomical region of interest, but excluding the segments corresponding to the selected target extraction object.

[0119] The functionality of any of steps 401, 401, 403, 405, 406 and 407 may be implemented as described in relation to the respective steps 101, 102, 103, 104, 105 and 106 of FIG. 1, or may be implemented by invoking a 3D model generation tool (215a of FIG. 2) which performs the functionality of the relevant method step of FIG. 1 and returns the generated 3D fovea model to the tool executing method 400.

[0120] The method 400 may be implemented as a computer-implemented virtual object removal application 215b running on a system such as the system 200 shown in Figure 2. Various aspects of the virtual object removal application 215b may be implemented as computer instructions stored in the local memory 204 of the system 200 and that, when executed by the processor(s) 201, may implement features and aspects of the method 400.

[0121] The computer instructions may include instructions to implement a graphical user interface (GUI), which may include a user-facing ("front-end") graphical user interface environment and a latent processing (or "back-end") GUI that performs functions to effectuate processing as dictated by user input received via the GUI environment controls. The front-end GUI includes input controls that may receive user controls, input and data via input devices 207 and load patient data into the environment. The front-end GUI further includes user control elements and output display areas, such as view panes, where user controls, various views of the scan data and model, and other information are displayed and output via output devices 208, 209 based on GUI control settings and user input to such controls.

[0122] 5A-5D illustrate an example embodiment of a graphical user interface (GUI) display environment 500 during various steps in a dental treatment planning workflow. The GUI display environment may be displayed on an electronic display 209 of a system 200 implementing a dental treatment planning application 215b. The GUI display environment 500 includes controls 501 (not shown separately) that enable a user to select and load patient scan data 220 onto the bulk memory 206 of the system 200 or onto an external memory device (not shown) accessible via the system 200 or accessible from a remote service 212 via a network adapter 211.

[0123] In the context of the above-described aspects of the invention, scan data 220 includes surface scan data and volume density scan data of the patient's anatomical region of interest. In some embodiments, both the surface scan and the volume density scan are acquired before the dental treatment planning application is launched, while in other embodiments, one or both of the surface scan data and the volume density scan data are acquired in conjunction with use of application 215b.

[0124] For example, the remote services 212 may include an optical scanner application in communication with an optical scanner that communicates optical scan data to a dental treatment planning application during or after completion of an intraoral scan of an area of ​​interest in the patient's oral context. Similarly, the remote services 212 may include a volumetric density scanner application in communication with a volumetric density scanner that communicates volumetric density scan data to application 215b during or after completion of an intraoral scan of an area of ​​interest in the patient's oral context.

[0125] The application 215a manages the display of graphical content in the GUI display environment 500, including monitoring user input for graphical controls received through user input devices such as a mouse, keyboard, joystick, voice recognition, etc. The user input may correspond to operations such as invoking various application functions specific to the substantive functionality of the application or GUI functions to change the layout or content of the functions displayed on the display. More specifically, the front-end GUI displays user input controls and monitors user input associated with the function controls. Upon receiving user input associated with a user input control, the GUI invokes the appropriate function corresponding to the particular user input control and the type and content of the user input. The GUI is also responsive to a back-end process that communicates with a back-end GUI, which in turn communicates with the front-end GUI to display information on the electronic display, remove the display, and / or modify the display.

[0126] Such user selection functionality may include, but is not limited to, displaying, removing, and changing the display of models, views, segments, and / or annotations, displaying, removing, and updating the appearance of various user controls and information displayed in the GUI display environment 500, and receiving and returning information to facilitate substantial functional features of the application 215b, including, but not limited to, substantial treatment evaluation, substantial treatment planning, virtual performance of a treatment or procedure (such as tooth extraction, implant placement, prosthetic design and placement, etc.).

[0127] 5A, the GUI display environment 500 includes global controls 501, such as file management, general display controls, and other controls common in GUI display environments. For example, the controls 501 may include a file selection control, a file save / export control, a view pane formatting control, etc. The GUI display environment 500 also includes controls 502 specific to patient data, such as arch and individual tooth model controls.

[0128] The GUI display environment 500 includes functional controls 503, including a tooth extraction control 512. Although the tooth extraction control 512 is generally shown as a single control, it may include multiple controls, such as a pop-up display panel guided dialog or other well-known GUI interaction techniques for displaying information, requesting information, or receiving user input. The GUI 500 also includes and displays at least one view pane 503 for displaying a 3D anatomical model (or a selected portion thereof) of a selected patient's oral situation, as obtained from the patient's scan data and selected via a selection control of the control 502.

[0129] The control 501 includes one or more controls (not shown) that allow a user to select the patient's surface and volume density scan data 220 from a local, bulk or remote memory to the bulk memory 206 of the system 200 when selected. In one embodiment, when the patient scan data 220 is initially loaded, the GUI display environment 500 may display one or more view panes 503 (only one shown) to present a visual overview of the patient's oral situation on the display. In FIG. 5A, the view pane 503 displays a 3D model 10A of the volume density scan. The environment 500 may include various additional views of the patient's oral situation based on the volume density scan data. For example, the environment 500 may include a panoramic view pane, an axial view pane, a cross-sectional view pane, and a tangential view pane (not shown).

[0130] An important purpose of the virtual tooth extraction tool accessible via control 512 is to virtually represent the situation in the patient's oral cavity following virtual extraction of one or more teeth or other objects selected for extraction. For example, in dental treatment application 215b, when a tooth is virtually extracted and displayed within GUI display environment 500, the resulting 3D model should include a representation of the tooth socket that will be visible to the naked eye upon extraction of the target extracted tooth.

[0131] Traditionally, dental treatment planning applications and dental prosthetic design software (such as dental CAD / CAM) software have been unable to provide accurate details of the socket of an extracted tooth because the object surface models generated from the scan data do not include subsurface information. In contrast to prior art dental treatment planning tools and / or dental prosthetic computer-aided design (CAD) and computer-aided manufacturing (CAM) tools, the virtual tooth extraction tool extracts information from both surface and volume density scans of the patient's dentition and utilizes information corresponding to the same target tooth from both scan types to automatically determine and provide a highly accurate, anatomically-based representation of the socket that will be left upon tooth extraction. Application 215b provides a virtual tooth extraction tool (activated via control 512) that automatically determines the contours of the socket of a selected tooth targeted for extraction and generates a virtual 3D model of the socket for display, storage and / or extraction.

[0132] As shown in Figure 5A, a user can select the tooth extraction control 512 to activate a virtual tooth extraction tool by moving a graphical cursor 520 via a mouse (not shown) over the control 512 and mouse clicking on the control 512. Figure 5B shows an embodiment of a pop-up dialog 513 that is displayed in the GUI environment 500 when the tooth extraction control 512 is activated. As shown, the dialog may include a tooth selection map that allows a user of the application 215b to select one or more individual teeth for virtual extraction.

[0133] In one embodiment, the user may select an individual tooth as a target extracted tooth by clicking on such tooth in the chart. The user may optionally select to save the socket model generated by the tool upon socket generation and / or to extract the extracted tooth model (including removing the target extracted tooth / tooth from the model in the view pane 503 and adding the socket model generated for such target extracted tooth / tooth) by selecting a respective selection click box or radio button or other such selection feature. Once the user has selected the target extracted tooth / tooth and the save / extract option, the user may click on the select button 517 to launch the extraction tool.

[0134] FIG. 5C shows a posterior view (a view of the maxillary arch from the back toward the front of the patient) of the 3D extracted tooth model 10D, which more simply shows a 3D socket model 16S corresponding to the socket from which the tooth 16 was extracted. The socket model 16S is displayed together with the 3D surface scan model from which the tooth 16 was extracted. FIG. 5D shows the same model 10D from a bottom view (looking up from below toward the maxilla). As shown, the tooth 16 is missing, but the interior 16c of the socket 16S is visible and follows the contour of the root of the extracted tooth 16.

[0135] A virtual tooth extraction is often a precursor to the placement of a dental implant. Accordingly, FIG. 6 is a flow chart of an exemplary method 600 of a third aspect of the present disclosure, namely, for generating a surface model with an extracted tooth model with an implant placed in a socket of a virtual extracted tooth. According to the method 600, in step 601, a dental implant planning application, such as application 215c executed in the system 200 of FIG. 2, obtains a virtual model of the patient's dentition based on both a volumetric density scan and a surface scan of a target area of ​​the patient's dentition.

[0136] In one embodiment, in step 602a, the dental implant planning application 215c places a virtual implant in the virtual model of the patient's dentition under control of user input received via the GUI control. In step 603a, the dental implant planning application 215c performs a virtual extraction of the tooth targeted for replacement by the implant when the user activates the extraction control, resulting in an extraction socket model and a replacement extraction model in which the target extracted tooth is removed from the virtual model of the patient's dentition. Since the virtual implant is already in the virtual model of the patient's dentition, the extraction model includes both the extraction socket model and the virtual implant model.

[0137] In an alternative embodiment, steps 602a and 603a are reversed. In this alternative embodiment, the tooth extraction is performed first in step 602b to generate an extraction model. In step 603b, a virtual implant is inserted into the extraction model. The virtual extraction model can then be saved to a file and / or exported (step 604). Optionally, a surface scan crown segment is exported in the same 3D coordinate system as the exported extraction model (step 605). Optionally, in step 606, the exported extraction model and crown segment file may be sent to a prosthesis design tool for import and design of the implant prosthesis.

[0138] Implant placement planning is often performed by pre-planning the selection and virtual placement of implants in a virtual model of the patient's dentition. Virtual implant planning and placement can be performed using dental implant planning tools, of which several already exist on the market. One example is the CoDiagnostix™ dental implant planning software offered by Dental Wings, Inc. (a Straumann Group company). Such dental implant planning software includes software-implemented tools that include a graphical user interface for importing and viewing the patient's internal and surface scans, selecting dental implants, and placing the selected dental implants into a model of the patient's dentition that has been generated based on the imported scans.

[0139] For example, FIG. 7A shows a GUI display environment 700 generated by an application (such as application 215c of FIG. 2) in which a patient's surface and volume density scans have been imported and loaded into memory 206 of system 200. FIG. 7A shows the patient's dental situation after a dental professional has selected and virtually placed an implant. Techniques for virtually placing an implant in a virtual model of a patient's dentition are already known in the art, for example, according to the use of CoDiagnostix™ dental implant planning software. In FIG. 7A, a virtual implant post 710 is placed and shown in various types of views in corresponding view panes of the graphical environment 700. In the illustrated example, the implant post 710 is shown virtually placed in a cross-sectional view (view pane 703d), an axial view (view pane 703c), a panoramic view (view pane 703b), a tangential view (view pane 703e), and a 3D view (view pane 703a).

[0140] As shown in the various views of panes 703a-703e, the placement of the implant is represented by the placement of an implant post or screw 710, which is the base portion of a full implant. The full implant includes the implant post 710, an abutment (not shown) that attaches to the implant post 710, and an abutment for a prosthesis or dental restoration (also not shown), which may be a crown, bridge, or denture.

[0141] In the initial planning stage, only the implant post 710 needs to be virtually placed. The implant planning software application provides a virtual implant placement guide 711, which does not correspond to a physical component, but is merely a visual indicator to assist the dental professional in placing the implant at the correct angle. In FIG. 7A, the virtual guide 711 is displayed in the 3D view pane 703a as a long cylindrical rod with a central axis coinciding with the central axis of the implant post 710 and a diameter corresponding to the diameter of the abutment attachment cavity in the implant post. Preferably, the cylinder of the virtual implant placement guide 711 extends above the occlusal surface of the tooth along its central axis, such that the length of the cylinder is significantly longer than the diameter of the cylinder. Preferably, the guide 711 is displayed in a color that contrasts with the colors used in the 3D model and other view panes, so that the application user can immediately see the guide relative to the content in each view pane.

[0142] The GUI display environment 700 includes a tooth extraction control 712 that, in an exemplary embodiment, is accessed by selecting a control from the view pane display controls 702 that corresponds to the portion of the patient's dentition where the implant under consideration will be placed. In the illustrated embodiment, the dental professional selects the lower arch control, right-clicks on it to bring up a context menu, and selects the tooth extraction control 712 from the context menu. Selecting the tooth extraction control can instruct the dental treatment planning application to automatically perform a virtual tooth extraction (using the principles described above). There are many ways to implement a control that launches the automatic virtual tooth extraction tool, but what is important is to provide one or more controls that allow the user to launch the tooth extraction workflow.

[0143] FIG. 7B illustrates a pop-up window 713 that is displayed in the GUI display environment 700 when a user selects the tooth extraction control 712. The pop-up window presents several options and user input controls for obtaining inputs required by the virtual tooth extraction tool, including a tooth selection control 718, a mode control 714. In the embodiment illustrated in FIG. 7B, the tooth selection control 718 displays a set of selectable tooth icons corresponding to the teeth in the selected portion of the patient's dentition selected by the user in FIG. 7A. The user (i.e., dental professional) can select the tooth corresponding to the tooth into which the virtual implant post 710 will be placed. In this example, the user selects tooth 35, which corresponds to the tooth on the left side of the lower arch into which the virtual implant post 710 will be placed. For the mode, the user selects "Mode: Alveolar Bone Extraction" 715 from the options drop-down menu 714, checks the checkbox control 716 to indicate that the extracted tooth should be saved as a separate file for future planning, and launches the virtual tooth extraction tool by clicking the extraction control 717.

[0144] 7C shows a 3D surface model of the patient's dentition in a view pane 703a. Upon completion, the virtual tooth extraction tool adds two 3D model files to the list of available model scans and 3D models in the control portion 702 of the GUI display environment 700. The user can select these files to display in pane 403a. One file is the extracted dentition model (indicated at 721 in the file list). The extracted tooth model was created by the virtual tooth extraction tool according to the techniques described in connection with FIGS. 1 and 4, and is a 3D surface model of the patient's dentition (including the previously placed implant post 310) with the selected tooth 35 removed from the model and a cavity generated and included in the model instead.

[0145] The second file is the Extracted Tooth Model in the file list (designated 722) which is a model of tooth 35 with the crown removed from the root.

[0146] Once these files have been created, the user can click on the Plan menu 723 of the GUI display environment 700 and select the Virtual Plan export control 724, as shown in FIG. 7D. In the next step, shown in FIG. 7E, the user can then select the format of the export file in the Format selection control 727 of the pop-up window 726. In this example, the user selects the STL format option 728 and clicks "Next" 729 to move to the next menu. In the next step, shown in FIG. 7F, the user selects the button control 732 to activate the option to export the selected model scan or segmentation without further processing. Clicking the Next button 733 causes the tooth extraction file 731 to be selected from the Export File Selection pop-up window 730, FIG. 7G. Clicking the Next button 732 causes the Implant Selection pop-up window 735, shown in FIG. 7H, to appear, showing the Scanbody Selection control 736. The Scanbody Selection control 736 includes a Scan Selection control 737 that allows the appropriate scanbody to be displayed and selected from a set of possible scanbody types. The user can select a scanbody from the menu of scanbodies to add the selected scanbody to the model for export. The user then clicks the Next button 738. In FIG. 7I, the user can select additional options such as selecting the 3D coordinate system in which to export the model and whether to export the exported objects as individual files that share the coordinate system. Clicking on the Export Plan line 741 launches the export function based on the selected options and parameters as selected by the user in the previous screen. The export file is saved to a known location in computer readable memory.

[0147] In FIG. 7J, the same process can be followed to export the surface scan crown segment of the virtual extracted tooth (and optionally the antagonist crown segment, i.e., the crown segment from the tooth in the opposing jaw that faces the target extracted tooth in the occlusal plane when the jaws are closed). It is important to ensure that the surface scan tooth segments are exported in the same coordinate system as the exported virtual extracted tooth model.

[0148] The exported file contains a 3D model of the patient's dentition with the virtual extraction of the tooth where the proposed implant will be placed. In place of the extracted tooth is a virtual socket where the virtual tooth once sat. The model includes the virtual implant post that the dental professional placed in the socket during the implant planning process.

[0149] The next step in the implant planning process is the design of a prosthesis or dental restoration, such as a temporary or permanent abutment with a crown, bridge or denture. Figure 8 is a flow diagram illustrating an exemplary process for designing an anatomically correct prosthesis, which is particularly useful in designing a temporary abutment for placement in the patient's mouth after implant placement while the tissue around the surgical site heals.

[0150] According to aspects of the present disclosure, designing a provisional abutment based on a virtual extracted tooth as described above allows for the design of a custom abutment (or other prosthesis) based on the patient's anatomy determined from the virtual extraction. That is, the provisional abutment (or other prosthesis) can be designed and manufactured even prior to the implant placement surgery, ready for immediate placement at the time of implant post placement (or shortly thereafter). Because the virtual extraction generates an anatomically correct socket specific to the patient's extracted tooth anatomy, there is no need to scan the patient's actual socket after actual extraction to obtain a socket surface profile.

[0151] This is a significant time saver, as bleeding at the surgical site prevents accurate surface scanning of the socket, and such scanning must usually be postponed until some initial healing has occurred.

[0152] The process described in FIG. 8 includes obtaining a virtual extracted tooth model based on both a volume density scan and a surface scan of the target area of ​​the patient's dentition (step 801), and obtaining a surface scan crown segment of the virtual extracted tooth (step 802). In an embodiment, this virtual extracted tooth model can be obtained according to the methods and systems described above. The obtained surface scan crown segment is imported into the model and used as the prosthetic crown (i.e., the part of the prosthetic that is visible above the gum line). In an embodiment, the design is done electronically in a CAD / CAM tool that provides a clone function to clone a virtual wax-up of the original crown.

[0153] The virtual extracted tooth model with the virtual implants placed in it is displayed in the GUI of the prosthesis design tool, allowing the user to view the model (step 804). The GUI provides user controls that allow the user to design the bottom of the prosthesis to follow the inner contour of the extraction socket in at least some portions (step 805).

[0154] The final design is a prosthesis design having an upper portion that substantially matches the surface scan crown segment and a lower portion with an outer surface that substantially matches the inner surface of the extraction socket obtained from the virtual extraction model. The prosthesis design may be exported to one or more electronic files in a format usable for subsequent manufacturing (step 806). A physical prosthesis is then manufactured based on the electronic files (step 807). In one embodiment, the electronic file includes 3D printing instructions for printing on a 3D printer or the electronic file is converted to 3D printing instructions. In another embodiment, the electronic file includes an STL file.

[0155] 9A-9R show a GUI display environment 900 for a prosthesis design application, such as a computer-aided design (CAD) or computer-aided manufacturing (CAM) tool. In one embodiment, the prosthesis design tool is an application 215d running on a system such as 200 previously described in FIG.

[0156] FIG. 9A is a GUI display environment 900 that is displayed on the electronic display 209 and has user input controls and display areas as described below. To design a prosthesis, the user clicks on control 901 (FIG. 9A) to start a new case, enters case information (e.g., in the text boxes associated with the new case: Case ID, Patient ID, and Dentist ID), selects a 3D model generated from the patient's scan data, and loads it into memory for use in the application 215b. In this example, a crown is designed. The user enters file names for the virtual extraction file (in this case as the lower tooth model input) and the extracted tooth file (as the lower wax-up), and loads the files into the system (e.g., by clicking the Save button (FIG. 9B)). The application displays the 3D model resulting from the selected virtual extraction file in the view pane 903 (FIG. 9C). If necessary, the application provides controls in the environment 900 for cleaning the scan (e.g., filling holes where the scan data is incomplete, smoothing scan lines, removing noise, etc.). The application may then provide tools to adjust the orientation of the model relative to the occlusal plane, if desired (FIG. 9D).

[0157] Before proceeding with the design, the user tags the tooth positions in the displayed model and indicates to the application 215 the positions of the teeth for which a prosthesis is to be designed and which teeth are adjacent to the teeth for which a prosthesis is to be designed (see FIG. 9E). Next, in FIG. 9F, the user selects the platform (implant manufacturer, implant type and connection) and the scanbody. These selections should match the implants and scanbodies selected in the dental implant planning application and generated based on the virtual extraction file.

[0158] Once the setup is complete, the user can proceed to design the prosthesis. In FIG. 6J, the surface scan crown segments from the 3D model have been exported as individual segments in a matching 3D coordinate system as the virtual extracted tooth model. Because the surface scan crown model was generated from an optical scan of the patient's original tooth (before the actual extraction of the real tooth), the surface scan crown segments can be used as a digital wax-up model without re-scanning the patient's mouth. Because the surface scan crown segments have been exported as individual segments in a matching 3D coordinate system as the virtual extracted tooth model, the surface scan crown segments of the virtual extracted tooth can be attached to the view pane along with the virtual extracted tooth model and directly used by the application 215d as the top of the prosthetic crown. By importing the surface scan crown segment file as a wax-up file (FIG. 9B), the designer can select the clone wax-up control (FIG. 9G) to instruct the prosthesis design application 215d to clone the wax-up model in the wax-up model file to use as the prosthetic crown surface. After the clone wax-up tool clones the wax-up model as a prosthesis, the user can make fine adjustments to the prosthesis shape via fitting, shaping, and sculpting controls available in the GUI display environment 900. The environment 900 also includes controls for rotating and changing the view of the model displayed in the view pane 903a so that the user can view the prosthesis 904 from any angle. For example, in FIG. 9H, the model 905 has been rotated so that the prosthesis 904 can be viewed buccally. This allows the designer to make adjustments to the prosthesis while viewing it in situ within the virtual extraction model 905.

[0159] After designing the visible surface of the crown based on the extraction surface scan crown segment obtained during the tooth extraction process, the user can design the bottom of the prosthesis. The application can provide the designer with controls to input restoration specifications such as material type, color (see Figure 9I), as well as what to output (e.g., output an STL file or output a purchase order, which can be communicated directly to a remote manufacturing facility).

[0160] In FIG. 9J, a virtual extraction model 905 is displayed in the view pane 903a. The case of FIG. 9A-FIG. 9T is a new case for designing a prosthesis for an implant, and the virtual extraction model 905 was exported (e.g., using the process described in connection with FIG. 7A-FIG. 7J) with the implant placed in the model, so the virtual extraction model 905 includes a virtual implant post. In this example, a provisional abutment is selected and automatically virtually connected to the implant post and displayed as shown. The user can then select the cement gap thickness (FIG. 9K), set the material thickness (FIG. 9L), and display the prosthesis 904 in the model 905, including the socket contour (or "emergence profile") (FIG. 9M). The user can then turn off the display of the model 905 to display only the prosthesis (FIG. 9N).

[0161] Figure 9O shows the model 905 turned on again for viewing (see the various model view on / off buttons in Figure 9O) and rotated to get a good side view of the fossa 910. With the anatomical transparency set low (to see the implant posts and abutments inside the fossa 910), it becomes apparent that the fossa contours can be used to design an anatomically correct prosthesis 904 that fits the patient's oral anatomy by guiding the design of the lower crown portion of the prosthesis to fit against the internal contours of the fossa 910. In this regard, the prosthesis design application provides controls for adjusting the shape and fit of the lower crown portion.

[0162] After the bottom of the crown prosthesis is shaped to fit the contours of the fossa 910, the designer can move on to specifying the shell of the prosthesis. The application can automatically calculate (FIG. 9P) the proximity information between the prosthesis surface and the teeth on either side of the prosthesis (when the prosthesis is virtually attached to the virtual implant post 906) and generate the shell surfaces (looking buccally in FIG. 9Q and occlusally down in FIG. 9R). FIG. 9S shows the final prosthesis design in the model with both the upper and lower jaws with the anatomical model set partially transparent to partially view the implant post 906, abutment and prosthetic crown. The user can easily check the placement and shape of the prosthesis by visualizing the model. FIG. 9T shows the same model and orientation as FIG. 9S with the anatomical structures fully opaque. The workflow goes through the usual steps before sending the crown for fabrication.

[0163] After the prosthesis design is complete, the design file can be exported and used for manufacturing. In some embodiments, the exported design file is sent to a manufacturing facility or a remote manufacturing service. In some embodiments, the exported prosthesis design file can be used to generate 3D printer instructions to send to a 3D printer that responds with instructions to 3D print the prosthesis.

[0164] The above-described aspects and embodiments of the present invention provide multiple advantages in virtual fossa visualization, anatomical treatment planning and anatomical prosthetic design. According to one advantage, anatomical treatment specialists and prosthetic designers can plan treatment and design anatomically accurate prosthetics based on an accurate three-dimensional model of the fossa, which is the site of treatment planning and prosthetic design, and which models the patient-specific fossa anatomy.

[0165] The model can be generated prior to the removal of the anatomical object from the patient, allowing accurate treatment planning and prosthetic design prior to or simultaneously (i.e., in parallel) with the actual surgical removal of the anatomical object from the patient. This means that the patient may only need to visit the treatment specialist once. During just one or two visits, the patient's anatomical target area including the anatomical object targeted for removal can be scanned. The scan data can be imported into digital treatment planning tools including a virtual fossa model generation tool and / or a virtual object removal tool, each of which generates a virtual fossa model that is included in a representation of the virtual anatomical model of the patient's anatomical target area (i.e., the area including the target removal object and adjacent anatomical objects or features).

[0166] The displayed virtual anatomical model includes a virtual socket model, allowing the treating professional to virtually place implants or other treatment bodies in the virtual socket model with precision, design and print 3D printable surgical treatment guides, and export the virtual anatomical model to the virtual socket model for use in another prosthetic design software tool to design an anatomically accurate prosthesis. If the treating professional has prosthetic fabrication equipment readily available, the prosthesis can be fabricated while the patient is in the office. If not, the prosthesis can be sent to a lab for fabrication, and the patient can return to the office once the anatomical area around the implant has healed sufficiently to allow the prosthesis to be placed over the implant.

Claims

1. 1. A computer-implemented method for automatically generating, by one or more computer processors, a three-dimensional model for use in virtual removal of a target removal object of a patient, the target removal object including an object in the patient's anatomical structure, the method being based on each of a surface scan of an anatomical region of the patient's oral cavity and a volume density scan of the anatomical region, the anatomical region including at least the target removal object, the method comprising: receiving a first data set of labeled surface scan segments, each surface scan segment including a three-dimensional (3D) surface model of a corresponding object recognized and segmented from surface scan data of the surface scan, each surface scan segment having an associated label that identifies the surface scan segment; receiving a second data set of labeled volume density scan segments, each volume density scan segment including a three-dimensional (3D) volume density model of a corresponding object boundary surface recognized and segmented from volume density scan data of the volume density scan, each volume density scan segment having an associated label that identifies the volume density scan segment; cross-mounting labeled surface scan segments obtained from the first data set with labeled volume density scan segments obtained from the second data set in a common three-dimensional coordinate system; receiving an identification of the target extraction object; identifying a 3D volume density model associated with a label of the volume density scan segment corresponding to the identified target extraction object; identifying a 3D surface model associated with a label of a surface scan segment corresponding to the identified target extraction object; a) a 3D model of a fovea equivalent to the identified 3D volume density model minus the portion of the identified 3D volume density model that is commonly represented in the identified 3D surface model; b) at least a portion of the identified 3D volumetric density model commonly represented with the identified 3D surface model, and / or the identified 3D surface model commonly represented with the identified 3D volumetric density model. At least a portion of c) at least a portion of the identified 3D volume density model, excluding portions of the identified 3D volume density model that are commonly represented in the identified 3D surface model; and / or d) removing from at least a subset of the first data set those portions of the identified 3D surface models that are commonly represented in the identified 3D volumetric density models; generating a third dataset comprising a model including: A method comprising:

2. The generating step includes: determining portions of the identified 3D volume density models that are not commonly represented in the identified 3D surface models; storing the determined portions of the identified 3D volumetric density model as a 3D model of the fossa in a computer readable data storage component; The method of claim 1 , comprising:

3. The generating step includes: determining portions of the identified 3D volume density models that are commonly represented in the identified 3D surface models; determining a difference between the determined commonly represented portion and the identified 3D volume density model by removing the determined commonly represented portion from the identified 3D volume density model; storing the determined difference as a 3D model of the fovea in a computer readable data storage component; The method of claim 1 , comprising:

4. The generating step includes: generating a fourth data set by removing from a copy of at least a subset of the first data set portions of the identified 3D surface models that are commonly represented in the identified 3D volumetric density models; storing the fourth data set in a computer readable data storage component; The method of claim 1 , comprising:

5. The method of claim 1 , wherein labeled surface scan segments obtained from the first data set are cross-mounted to corresponding labeled volume density scan segments obtained from the second data set.

6. 2. The method of claim 1 , wherein cross-mounting the first data set of labeled surface scan segments to the second data set of labeled volume density scan segments comprises generating a fifth data set of labeled segments, each segment including a combined three-dimensional (3D) model representing an anatomical object.

7. determining a boundary of the identified 3D surface model and generating a cutline from the boundary; projecting the cutline onto the identified 3D volumetric density model; The method of claim 1 , comprising:

8. 8. The method of claim 7, comprising generating a sixth data set comprising the identified 3D surface model and / or the identified 3D volume density model on a side of the cutline that includes substantially all portions of the identified 3D volume density model that are commonly represented in the identified 3D surface model.

9. 2. The method of claim 1, comprising generating a seventh data set from the first, second and / or third data sets, or the identified 3D surface model and / or the identified 3D volumetric density model, wherein voids in the surfaces defined by the data sets or models, respectively, are augmented by filling the voids with the generated surface information.

10. The method of claim 9 , comprising generating the generated surface information by interpolation of boundaries of the gap and / or based on standard anatomical modeling.

11. generating a seventh data set from the fourth data set, wherein gaps in the surface defined by the data set or model, respectively, are augmented by filling said gaps with generated surface information; and / or storing said fourth data set and / or a combined or cross-mounted version thereof in a computer readable data storage component. The method of claim 4 , comprising:

12. The method of claim 11 , comprising generating the generated surface information by interpolation of boundaries of the gap and / or based on standard anatomical modeling.

13. generating a seventh data set from the fifth data set, wherein gaps in the surface defined by said data set or model, respectively, are augmented by filling said gaps with generated surface information; and / or storing said fifth data set and / or a combined or cross-mounted version thereof in a computer readable data storage component. The method of claim 6, comprising:

14. The method of claim 13, comprising generating the generated surface information by interpolation of boundaries of the gap and / or based on standard anatomical modeling.

15. generating a seventh data set from the sixth data set, wherein gaps in the surface defined by the data set or model, respectively, are augmented by filling said gaps with generated surface information; and / or storing said sixth data set and / or a combined or cross-mounted version thereof in a computer readable data storage component. The method of claim 8 , comprising:

16. The method of claim 15, comprising generating the generated surface information by interpolation of the boundary of the gap and / or based on standard anatomical modelling.

17. The method of claim 1 , wherein the target extraction object is a tooth and the socket model comprises a three-dimensional model of the patient's tooth socket in which to place the target extraction object.

18. 2. The method of claim 1, wherein the associated label of each surface scan segment and / or each volume density scan segment identifies the surface scan segment or the volume density scan segment, respectively, as corresponding to a tooth, a portion of a tooth, an artificial tooth, a portion of periodontal tissue, a portion of bone, an implant, or other object or structure present in the patient's oral cavity and / or anatomical region.

19. The method of claim 1 , further comprising storing the third data set and / or any of the combined or cross-mounted versions thereof in a computer-readable data storage component.

20. 20. A system for automatically generating a three-dimensional model for use in the virtual removal of a target removal object of a patient, the system comprising one or more computer processing units configured to load and execute computer readable instructions that, when executed, configure the one or more computer processing units to perform a method according to any one of claims 1 to 19.

21. A computer readable medium comprising computer readable instructions which, when executed by one or more computer processing units, configure the one or more computer processing units to perform the method of any of claims 1 to 19.

22. A computer program embodying computer readable instructions which, when executed by one or more computer processing units, configures the one or more computer processing units to perform the method according to any of claims 1 to 19.

23. 20. Use of the third, fourth, fifth, sixth and / or seventh data set according to any of claims 1 to 19 and / or composite or cross-mount versions thereof in a prosthesis design CAD / CAM software tool for generating a prosthesis design anatomically representative of a patient's target extraction object in a virtual extraction procedure of the patient's target extraction object and / or before, during and / or during a single visit when an actual extraction procedure of the target extraction object is performed on the patient.

Images