Method for registering an imaging scan to a coordinate system and related system

Abstract

A method of relating an object to a coordinate system, the object being supported by a support element, the object and support element being contained within a housing, comprising engaging a registration element with the object or the support element, performing an optical surface scan of the object and the registration element using an optical scanner to form a three-dimensional surface image of the object, the three-dimensional surface image having a coordinate system associated therewith, the registration element being associated with the coordinate system, and correlating the object in the three-dimensional surface image with the registration element to register the object to the coordinate system. Related methods and systems are also provided.

Description

[Technical Field]

[0001] The present disclosure relates to imaging procedures, and more particularly to methods and systems for registering imaging scans to a coordinate system so that the imaging scans can be implemented with robotic guidance, such as a surgical robot. [Background technology]

[0002] Preoperative imaging, particularly in surgical spaces, and more specifically in dental surgical spaces, can involve several different processes. For example, one preoperative imaging process involves advance planning but also requires the patient to undergo two CT imaging scans. The first CT imaging scan is typically performed several weeks before the scheduled surgery, and the resulting patient imaging is used to plan the procedure (i.e., the surgical procedure plan is planned using the first CT imaging scan in conjunction with the first CT imaging scan). A second CT imaging scan is then performed on the day of the surgical procedure, with fiducial markers (or arrays or fiducial beads) in place on the patient during the second CT imaging scan. Images from the second CT imaging scan (including the fiducial markers / arrays within the images) are then matched / registered with the first CT imaging scan, for example, based on the alignment of imaged features (e.g., anatomical features) between the images from both scans. The surgical procedure preplanning associated with the images from the first CT imaging scan is then imported in association with the images from the second CT imaging scan. The image from the second CT imaging scan also includes fiducial markers (or array) therein. Thus, once the two CT imaging scans are registered such that the imaged features (e.g., anatomical features) are theoretically aligned between the two images, the patient's pre-planned surgical procedure and / or the location of the patient's imaged features (e.g., anatomical features) are also registered to the fiducial markers (or array) attached to the patient and thus transformed into patient coordinate space.

[0003] However, such procedures can be inaccurate. For example, if the patient's anatomy changes or shifts between the first and second CT imaging scans, it may be difficult to align the imaged features between the images, and / or the fiducial markers may ultimately be registered relative to an anatomical approximation in the actual patient coordinate space. Therefore, there is a need for a simplified procedure for creating an imaging-based plan for a surgical procedure that minimizes the number of required CT scans. There is also a need for such a procedure that minimizes the risk of changes to the patient's anatomy between the planning and surgical procedures. Additionally, there is a need for a procedure that reliably and consistently registers one or more images of the patient's anatomy (imaged features) and fiducial markers in the actual patient coordinate space so that the surgical plan can be more accurately executed. Summary of the Invention [Means for solving the problem]

[0004] These and other needs are met by aspects of the present disclosure, which provides, in one aspect, a method for relating an object to a coordinate system, wherein the object is supported by a support element, and the object and support element are contained within a housing. Such a method comprises engaging a registration element with the object or the support element; performing an optical surface scan of the object and the registration element using an optical scanner to form a three-dimensional surface image of the object, the three-dimensional surface image having a coordinate system associated therewith, the registration element being associated with the coordinate system; and correlating the object with the registration element in the three-dimensional surface image to register the object to the coordinate system.

[0005] Another aspect provides a method of controlling a robot with respect to an object in a coordinate system, the object being supported by a support element, the object and the support element being contained within a housing, the method comprising: engaging a registration element with the object or the support element; performing an optical surface scan of the object and the registration element using an optical scanner to form a three-dimensional surface image of the object, the three-dimensional surface image having a coordinate system associated therewith and the registration element being associated with the coordinate system; planning a procedure for the object; correlating the object with the registration element in the three-dimensional surface image to register the object to the coordinate system; and guiding the robot to perform the plan of the procedure for the object in relation to the registration of the object to the coordinate system.

[0006] In some aspects, prior to engaging the registration element with the object or support element, a radiological scan of the housing and the object and support element contained therein is performed to form a radiological image. The radiological image of the object is then image-matched with a three-dimensional surface image of the object to form a correlated image of the object in relation to the registration element. The object is then correlated with the registration element in the correlated image to register the object to a coordinate system.

[0007] Another aspect provides a system for relating an object to a coordinate system, the object being supported by a support element, the object and the support element being contained within a housing, the system comprising: a registration element adapted to engage the object or the support element; an optical scanner configured to perform an optical surface scan of the object and the registration element to form a three-dimensional surface image of the object, the registration element being associated with a coordinate system and the three-dimensional surface image having an associated coordinate system; and a controller having a processor and in communication with the optical scanner, the controller configured to correlate the object with the registration element in the three-dimensional surface image to register the object to the coordinate system.

[0008] Yet another aspect provides a system for guiding a robot relative to an object in a coordinate system, the object being supported by a support element, the object and the support element being contained within a housing. Such a system includes: a registration element adapted to engage the object or the support element; an optical scanner configured to perform an optical surface scan of the object and the registration element to form a three-dimensional surface image of the object, the registration element being associated with a coordinate system and the three-dimensional surface image having an associated coordinate system; and a controller having a processor and in communication with the optical scanner, the controller configured to enable formation of a treatment plan for the object, correlate the object with the registration element in the three-dimensional surface image to register the object to the coordinate system, and guide the robot to execute the treatment plan for the object in relation to the registration of the object to the coordinate system.

[0009] In some aspects, the computed tomography device is configured to perform a radiological scan of the housing and the object and support element contained therein to form a radiological image before the registration element is engaged with the object or support element, and the controller is configured to communicate with the computed tomography device, image-match the radiological image of the object with a three-dimensional surface image of the object to form a correlated image of the object in relation to the registration element, and correlate the registration element with the object in the correlated image to register the object to a coordinate system.

[0010] Accordingly, the present disclosure includes, but is not limited to, the following embodiments.

[0011] Exemplary Implementation 1: A method for relating an object to a coordinate system, the object being supported by a support element, the object and the support element being contained within a housing, the method comprising: engaging a registration element with the object or the support element; performing an optical surface scan of the object and the registration element using an optical scanner to form a three-dimensional surface image of the object, the three-dimensional surface image having a coordinate system associated therewith, the registration element being associated with the coordinate system; and correlating the object with the registration element in the three-dimensional surface image to register the object to the coordinate system.

[0012] Exemplary Implementation 2: A method of any above embodiment, or any combination of the above embodiments, comprising: performing a radiological scan of the housing and the object and support element contained therein to form a radiological image before engaging the registration element with the object or support element; image matching the radiological image of the object with a three-dimensional surface image of the object to form a correlated image of the object in relation to the registration element; and correlating the object with the registration element in the correlated image to register the object to a coordinate system.

[0013] Exemplary Implementation 3: The method of any preceding embodiment, or any combination of the preceding embodiments, wherein image matching the radiographic image with the three-dimensional surface image comprises image matching a three-dimensional structural representation of the object and support elements provided by a radiographic scan with a three-dimensional surface image of the object, support elements, and registration elements provided by an optical surface scan.

[0014] Exemplary Implementation 4: The method of any preceding embodiment, or any combination of the preceding embodiments, wherein the registration element is not radiopaque.

[0015] Exemplary Implementation 5: The method of any preceding embodiment, or any combination of the preceding embodiments, wherein the optical scanner is operably engaged with the distal end of a robotic tracking arm registered to a coordinate system.

[0016] Exemplary Implementation 6: The method of any preceding embodiment, or any combination of the preceding embodiments, wherein the registration element is arranged to have a known relationship with the coordinate system.

[0017] Exemplary Implementation 7: The method of any preceding embodiment, or any combination of the preceding embodiments, wherein engaging the registration element with the object or support element comprises: engaging the registration element with a reference frame; and engaging the reference frame with the object or support element.

[0018] Exemplary Implementation 8: A method of any preceding embodiment, or any combination of the preceding embodiments, comprising planning treatment for an object in relation to the three-dimensional surface image before correlating the object with registration elements within the three-dimensional surface image.

[0019] Exemplary Implementation 9: A method for controlling a robot relative to an object in a coordinate system, the object being supported by a support element, the object and the support element being contained in a housing, the method comprising: engaging a registration element with the object or the support element; performing an optical surface scan of the object and the registration element using an optical scanner to form a three-dimensional surface image of the object, the three-dimensional surface image having a coordinate system associated therewith, the registration element being associated with the coordinate system; planning a treatment for the object; correlating the object with the registration element in the three-dimensional surface image to register the object to the coordinate system, and guiding the robot to perform the treatment plan for the object in relation to the registration of the object to the coordinate system.

[0020] Exemplary Implementation 10: A method of any of the above embodiments, or any combination of the above embodiments, comprising: performing a radiological scan of the housing and the object and support element contained therein to form a radiological image before engaging the registration element with the object or support element; image matching the radiological image of the object with a three-dimensional surface image of the object to form a correlated image of the object in relation to the registration element; and correlating the object with the registration element in the correlated image to register the object to a coordinate system.

[0021] Exemplary Implementation 11: The method of any preceding embodiment, or any combination of the preceding embodiments, wherein image matching the radiographic image with the three-dimensional surface image comprises image matching a three-dimensional structural representation of the object and support elements provided by a radiographic scan with a three-dimensional surface image of the object, support elements, and registration elements provided by an optical surface scan.

[0022] Exemplary Implementation 12: A method of any of the above embodiments, or any combination of the above embodiments, wherein planning treatment for the object comprises planning treatment for the object in relation to its three-dimensional surface image, in relation to its radiological image, or in relation to its correlated image including the three-dimensional surface image and the radiological image.

[0023] Exemplary Implementation 13: The method of any preceding embodiment, or any combination of the preceding embodiments, wherein the registration element is not radiopaque.

[0024] Exemplary Implementation 14: A method of any of the above embodiments, or any combination of the above embodiments, wherein the optical scanner is operably engaged with the distal end of a robotic tracking arm registered to a coordinate system and configured to communicate with the robot.

[0025] Exemplary Implementation 15: The method of any preceding embodiment, or any combination of the preceding embodiments, wherein the registration element is arranged to have a known relationship with the coordinate system.

[0026] Exemplary Implementation 16: A method of any above embodiment, or any combination of the above embodiments, wherein engaging the registration element with the object or support element comprises: engaging the registration element with a reference frame; and engaging the reference frame with the object or support element.

[0027] Exemplary Implementation 17: A method of any of the above embodiments, or any combination of the above embodiments, wherein planning treatment for the object comprises planning treatment for the object before correlating the object with registration elements in its correlation image.

[0028] Exemplary Implementation 18: A system for relating an object to a coordinate system, the object being supported by a support element, the object and support element being contained within a housing, the system comprising: a registration element adapted to be engaged with the object or the support element; an optical scanner configured to perform an optical surface scan of the object and the registration element and form a three-dimensional surface image of the object, the registration element being associated with a coordinate system and the three-dimensional surface image having a coordinate system associated therewith; and a controller having a processor and in communication with the optical scanner, the controller configured to correlate the object with the registration element in the three-dimensional surface image to register the object to the coordinate system.

[0029] Exemplary Implementation 19: A system of any of the above embodiments, or any combination of the above embodiments, comprising a computed tomography device configured to perform a radiological scan of the housing and the object and support element contained therein to form a radiological image before the registration element is engaged with the object or support element, and a controller configured to communicate with the computed tomography device, image-match the radiological image of the object with a three-dimensional surface image of the object to form a correlated image of the object in relation to the registration element, and correlate the registration element with the object in the correlated image to register the object to a coordinate system.

[0030] Exemplary Implementation 20: A system of any of the above embodiments, or any combination of the above embodiments, wherein the controller is configured to image-match a three-dimensional structural representation of the object and support elements provided by a radiological scan with a three-dimensional surface image of the object, support elements, and registration elements provided by an optical surface scan.

[0031] Exemplary Implementation 21: A system of any of the above embodiments, or any combination of the above embodiments, wherein the registration element is not radiopaque.

[0032] Exemplary Implementation 22: A system of any of the above embodiments, or any combination of the above embodiments, wherein the optical scanner is operably engaged with the distal end of a robotic tracking arm registered to a coordinate system and configured to communicate with a controller.

[0033] Exemplary Implementation 23: The system of any of the above embodiments, or any combination of the above embodiments, wherein the registration element is arranged to have a known relationship with the coordinate system.

[0034] Exemplary Implementation 24: A system of any of the above embodiments, or any combination of the above embodiments, comprising a reference frame with an engaged registration element, the reference frame being engaged with the object or support element.

[0035] Exemplary Implementation 25: A system of any of the above embodiments, or any combination of the above embodiments, wherein the controller is configured to enable the formation of a treatment plan for the object in relation to the three-dimensional surface image before the object is correlated with registration elements within that three-dimensional surface image.

[0036] Exemplary Implementation 26: A system of any of the above embodiments, or any combination of the above embodiments, wherein the controller is configured to track the position of the robotic tracking arm and an optical scanner operably engaged with its distal end relative to a coordinate system.

[0037] Exemplary Implementation 27: A system for guiding a robot relative to an object, the object being supported by a support element, the object and the support element being contained within a housing, the method comprising: a registration element adapted to engage with the object or the support element; an optical scanner configured to perform an optical surface scan of the object and the registration element and form a three-dimensional surface image of the object, the registration element being associated with a coordinate system and the three-dimensional surface image having an associated coordinate system; and a controller having a processor and in communication with the optical scanner, the controller configured to enable the formation of a treatment plan for the object, correlate the object with the registration element in the three-dimensional surface image to register the object to the coordinate system, and guide the robot to execute the treatment plan for the object in relation to the registration of the object to the coordinate system.

[0038] Exemplary Implementation 28: A system of any of the above embodiments, or any combination of the above embodiments, comprising a computed tomography device configured to perform a radiological scan of the housing and the object and support element contained therein to form a radiological image before the registration element is engaged with the object or support element, and a controller configured to communicate with the computed tomography device, image-match the radiological image of the object with a three-dimensional surface image of the object to form a correlated image of the object in relation to the registration element, and correlate the registration element with the object in the correlated image to register the object to a coordinate system.

[0039] Exemplary Implementation 29: A system of any of the above embodiments, or any combination of the above embodiments, wherein the controller is configured to image-match a three-dimensional structural representation of the object and support elements provided by a radiological scan with a three-dimensional surface image of the object, support elements, and registration elements provided by an optical surface scan.

[0040] Exemplary Implementation 30: A system of any of the above embodiments, or any combination of the above embodiments, wherein the controller is configured to enable the formation of a treatment plan for the object in association with the three-dimensional surface image, in association with the radiological image, or in association with the correlated image including the three-dimensional surface image and the radiological image.

[0041] Exemplary Implementation 31: A system of any of the above embodiments, or any combination of the above embodiments, wherein the registration element is not radiopaque.

[0042] Exemplary Implementation 32: A system of any of the above embodiments, or any combination of the above embodiments, wherein the optical scanner is operably engaged with the distal end of a robotic tracking arm registered to a coordinate system and configured to communicate with the controller and the robot.

[0043] Exemplary Implementation 33: The system of any of the above embodiments, or any combination of the above embodiments, wherein the registration element is arranged to have a known relationship with the coordinate system.

[0044] Exemplary Implementation 34: The system of any preceding embodiment, or any combination of the preceding embodiments, comprising a reference frame with an engaged registration element, the reference frame being engaged with the object or support element.

[0045] Exemplary Implementation 35: A system of any of the above embodiments, or any combination of the above embodiments, wherein the controller is configured to enable the formation of a treatment plan for the object in relation to the three-dimensional surface image before the object is correlated with the registration elements in that correlation image.

[0046] Exemplary Implementation 36: A system of any of the above embodiments, or any combination of the above embodiments, wherein the controller is configured to track the position of the robotic tracking arm and an optical scanner operably engaged with its distal end relative to a coordinate system.

[0047] These and other example implementations, features, aspects, and advantages of the present disclosure will become apparent from the following detailed description read in conjunction with the accompanying drawings, briefly described below. The present disclosure includes any combination of two, three, four, or more features or elements as set forth in this disclosure or recited in one or more of the claims, regardless of whether such features or elements are expressly combined or otherwise recited in the description of a particular embodiment or in the claims herein. This disclosure is intended to be read collectively such that all separable features or elements of the disclosure, in any of its aspects and embodiments, are deemed to be intended to be combinable, unless the context of the disclosure clearly dictates otherwise.

[0048] Thus, methods according to aspects of the present disclosure provide these and other advantages, as further detailed herein. Importantly, these advantages include reducing or eliminating the need for a second radiological scan as part of the preoperative process for a robotic surgical procedure. That is, by performing an optical three-dimensional surface scan of a feature of interest (e.g., an intraoral scan) and including a fiducial marker array (e.g., mounted on a splint, which is engaged to the patient) in the intraoral scan, the imaged fiducial marker array in the scan is used to register the optical three-dimensional surface scan to the location of the patient and / or feature (e.g., an anatomical feature). Once the optical three-dimensional surface scan is registered to a coordinate system associated with the patient-engaged fiducial marker array, the imaged feature from the intraoral scan (e.g., a three-dimensional surface image of the anatomical structure) can be registered to the imaged feature from the original / first CT scan (e.g., a radiological image of the anatomical structure). The treatment plan, and other information associated therewith, created from the original / first CT scan can then be applied with reference to the patient and / or the patient's anatomical features (e.g., within a coordinate system associated with the fiducial marker array).

[0049] In further aspects, an optical three-dimensional surface scan (e.g., an intraoral scan) may obviate the need for both the first and second CT scans. For example, in some cases, a surgeon may not need a CT scan (e.g., radiographic imaging of anatomical features) to accomplish the intended procedure. This may be applicable, for example, to robotic procedures beyond dental implants and may extend to, for example, tooth preparation procedures, which involve drilling or polishing a tooth surface (e.g., to remove decayed areas of the tooth) and preparing the remaining part of the tooth to receive a crown (e.g., an artificial tooth) thereon. In such instances, the intraoral scan alone may be sufficient for treatment planning purposes, and fiducial markers included in the intraoral scan can be used to register the intraoral scan to the patient and / or patient features (e.g., within a coordinate system associated with the fiducial marker array) and to implement robotic guidance.

[0050] Having thus described the disclosure in general terms, reference is now made to the accompanying drawings, which are not necessarily drawn to scale. [Brief explanation of the drawings]

[0051] [Figure 1] FIG. 1 is a schematic diagram of an optical scanner configured to perform an optical surface scan of an object, the object contained within a housing that includes a patient's mouth or maxillofacial structure, in a system for relating the object to a coordinate system and controlling a robot relative to the object within the coordinate system, according to one aspect of the present disclosure. [Figure 1A] FIG. 1 is a schematic diagram of an optical scanner configured to perform an optical surface scan of an object, the object contained within a housing that includes a patient's mouth or maxillofacial structure, in a system for relating the object to a coordinate system and controlling a robot relative to the object within the coordinate system, according to one aspect of the present disclosure. [Figure 2]FIG. 1 is a schematic diagram illustrating an object supported by a support element, according to one aspect of the present disclosure, where the object includes a tooth, the support element includes a corresponding jaw and / or gum, and the registration element is adapted to be engaged with the object or the support element. [Figure 3] FIG. 1 is a schematic diagram of an optical scanner configured to perform an optical surface scan of an object, the object may be contained within a housing that includes a patient's mouth or maxillofacial structure, according to one aspect of the present disclosure. [Figure 4] FIG. 1 is a schematic diagram of an optical scanner configured to perform an optical surface scan of an object, the object may be contained within a housing that includes a patient's mouth or maxillofacial structure, according to one aspect of the present disclosure. [Figure 5] FIG. 1 is a schematic diagram of a computed tomography device configured to perform a radiation scan of a housing and objects and support elements contained therein to form a radiation image of the housing and objects and support elements contained therein, according to one aspect of the present disclosure. [Figure 6] FIG. 1 is a diagram illustrating a schematic diagram of a radiographic image of an object being image-matched with a three-dimensional surface image of the object, according to one aspect of the present disclosure. [Figure 7] FIG. 1 is a diagram that schematically illustrates a method for relating an object to a coordinate system, according to one aspect of the present disclosure. [Figure 8] FIG. 1 illustrates a schematic diagram of a method for controlling a robot relative to an object in a coordinate system, according to an aspect of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0052] The present disclosure will now be described more fully hereinafter with reference to exemplary embodiments thereof. These exemplary embodiments are described so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art. Indeed, this disclosure may be embodied in many different forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided so that this disclosure will satisfy applicable legal requirements. The various aspects of the present disclosure discussed above, as well as many other aspects of the present disclosure, are described in further detail herein.

[0053] An embodiment of the present disclosure involves a system 100 for relating an object 200 to a coordinate system 300 and controlling a robot 800 relative to the object 200 within the coordinate system 300, as shown, for example, in FIGS. 1-4 , where the object 200 is supported by a support element 220, and the object 200 and support element 220 are contained within a housing 240. In one example, the object 200 can include a tooth, the support element 220 can include a corresponding jaw and / or gums, and the housing 240 can include a patient's mouth or maxillofacial structure. In a particular embodiment, such a system 100 includes a registration element 170 adapted to engage the object 200 or the support element 220 (see, e.g., FIG. 2 ). An optical scanner 400 is configured to perform an optical surface scan of the object 200 (see, e.g., FIGS. 3 and 4 ) and the registration element 170 to form a three-dimensional surface image of the object 200 (see, e.g., FIG. 6 , right image). The optical scanner 400 may be handheld and configured to be manually operated by an operator (see, e.g., FIG. 1 ), or may be otherwise operably engaged with the distal end 520 of the robotic tracking arm 500 (see, e.g., FIG. 1A ). The three-dimensional surface image has a coordinate system 300 associated therewith, and the registration element 170 is associated with the coordinate system 300. The registration element 170 (see, e.g., FIG. 2 ), adapted to be engaged with the object 200 or the support element 220, may comprise, for example, one or more fiducial markers 175 arranged to define or be in a known relationship with the coordinate system 300. More specifically, in some examples, the one or more fiducial markers 175 may be arranged such that at least the location and orientation of the one or more fiducial markers 175 is known or readily determinable relative to or registered to the coordinate system 300. The controller 190 has a processor and is configured to communicate with the optical scanner 400, and in some examples, with the robotic tracking arm 500.The controller 190 is further configured to correlate the object 200 with the registration element 170 within the three-dimensional surface image to register the object 200 to the coordinate system 300. That is, because the registration element 170 has one or more fiducial markers 175 that define a known position / orientation or reference or registration for itself in the coordinate system 300, an optical surface scan of the object 200 / support element 220 including the registration element 170 allows the object 200 to be spatially correlated with the registration element 170 based on the three-dimensional surface image. When the spatial correlation of the object 200 is known with respect to the registration element 170 and with respect to the coordinate system 300, the object 200 is thus registered in the coordinate space 300 with respect to the registration element 170.

[0054] In some examples, system 100 may further include a computed tomography device 700 (see, e.g., FIG. 5 ) arranged to perform a radiological scan of housing 240 and object 200 and support element 220 housed therein to form a radiological image of object 200 or support element 220 (see, e.g., FIG. 6 , left image) before registration element 170 is engaged with object 200 or support element 220. In such examples, controller 190 is arranged to communicate with computed tomography device 700 to receive imaging data therefrom representing the radiological scan / radiological image. Upon receiving the imaging data, controller 190 is further arranged to image-match the radiological image of object 200 with a three-dimensional surface image of object 200 to form a correlation image of object 200 in relation to registration element 170 (see, e.g., FIG. 6 ). Once the radiological image and the three-dimensional surface image are correlated, the controller 190 is configured to correlate the registration element 170 with the object 200 in the correlated image to register the object 200 to the coordinate system 300.

[0055] Controller 190 is configured to enable the formation of a treatment plan for object 200 in relation to the three-dimensional surface image before object 200 is correlated within the three-dimensional surface image with registration element 170. When optical scanner 400 is engaged with robotic tracking arm 500, controller 190 may be further configured to track the position of robotic tracking arm 500 and optical scanner 400 operatively engaged with its distal end 520 relative to coordinate system 300. In either example, once the object 200 is correlated with the registration element 170 as determined relative to the coordinate system 300 associated with the registration element 170 within the three-dimensional surface image from the optical surface scan, the treatment plan determined and created for the three-dimensional surface scan can subsequently be used to guide a robot 800, such as a surgical robot, having a surgical instrument 820 engaged therewith (and operably engaged with a robotic tracking arm 500 in communication with the registration element 170 attached to the patient) to execute the treatment plan with the surgical instrument 820 on the object 200 in relation to the registration of the object 200 to the coordinate system 300.

[0056] In one aspect of the present disclosure, a method of relating object 200 to coordinate system 300 involves an arrangement in which object 200 is supported by support element 220, and object 200 and support element 220 are housed within housing 240. In one example, such an arrangement may include a tooth as the object, a corresponding jaw and / or gums as the support element, and an oral or maxillofacial structure as the housing. Accordingly, the method may include engaging registration element 170 with the object or support element ( FIG. 7 , block 900). In some examples, registration element 170 is provided within or otherwise in a known relationship to coordinate system 300, as disclosed herein. An optical surface scan of the object and registration element is then performed using the optical scanner 400 within the housing (in some examples, the optical scanner 400 may be handheld and manually operated, or the optical scanner 400 may be operatively engaged with the distal end 520 of the robotic tracked arm 500) to form a three-dimensional surface image of the object in relation to the registration element 170 ( FIG. 7 , block 920). The optical surface scan by the optical scanner 400 digitizes an image of the object and registration element 170 in the form of a three-dimensional surface image, which is then correlated with the registration element ( FIG. 7 , block 940) so that the object is subsequently registered in a coordinate system (e.g., transforming the image from image coordinate space (a coordinate space internal to the image) to patient coordinate space (a coordinate space relative to the patient or the registration element 170)).

[0057] Another aspect of the present disclosure is directed to a method of controlling a robot 800 relative to an object 200 in a coordinate system 300, such method involving an arrangement having the object 200 supported by a support element 220, with both the object 200 and the support element 220 contained within a housing 240. In one example, such an arrangement may include teeth as the object, corresponding jaws and / or gums as the support elements, and a mouth or maxillofacial structure as the housing. Thus, the method may include engaging the registration element 170 with the object 200 or the support element 220 (FIG. 8, block 1000), and then performing an optical surface scan of the object and the registration element using an optical scanner 400 (in some examples, the optical scanner 400 may be handheld and manually operated, or the optical scanner 400 may be operably engaged with the distal end 520 of a robotic tracking arm 500) to form a three-dimensional surface image of the object, the three-dimensional surface image having a coordinate system 300 associated therewith, and the registration element 170 being associated with the coordinate system 300 (FIG. 8, block 1020). A treatment plan for the object is formed from the three-dimensional surface image ( FIG. 8 , block 1040), for example, before correlating the object with the registration elements in the three-dimensional surface image ( FIG. 8 , block 1060) to register the object to a coordinate system (e.g., transform the image from image coordinate space (a coordinate space internal to the image) to patient coordinate space (a coordinate space relative to the patient or registration elements 170)). Robot 800, having surgical instrument 820 engaged therewith, can then be directed to execute the treatment plan with surgical instrument 820 on object 200 in conjunction with the registration of object 200 to coordinate system 300 ( FIG. 8 , block 1080).

[0058] In some embodiments, particularly with respect to the two method embodiments disclosed above, a radiological scan of the housing and the object and support element contained therein may be performed to form a radiological image of the housing, object, and support element before the registration element is engaged with the object or support element (FIG. 7, block 960; FIG. 8, block 1100). The radiological image of the object is then image-matched with a three-dimensional surface image of the object to form a correlation image of the object in relation to the registration element (FIG. 7, block 970; FIG. 8, block 1120). The object is then correlated with the registration element in the correlation image (e.g., transforming the correlation image from image coordinate space (a coordinate space internal to the correlation image) to patient coordinate space (a coordinate space relative to the patient or the registration element 170)) to register the object to a coordinate system (FIG. 7, block 980; FIG. 8, block 1140).

[0059] In some embodiments, image matching the radiological image with the three-dimensional surface image may involve image matching the three-dimensional structural representation of the object and supporting elements provided by the radiological scan with the three-dimensional surface image of the object, supporting elements, and registration elements provided by the optical surface scan. In this way, a treatment plan may then be formed based on and taking into account both the aesthetic aspects of the treatment as seen in the three-dimensional surface image of the optical surface scan (e.g., an intraoral scan showing, for example, the gums and soft tissue surrounding the teeth) and particular anatomical structures of interest as seen in the three-dimensional structural representation of the radiological scan (e.g., a computed tomography (CT) scan showing, for example, nerves, tooth roots, jawbone, and / or related structures).

[0060] In some examples, because the registration element is associated with an optical surface scan, the registration element need not be radiopaque, but may be radiopaque if necessary or desirable. In another example, the registration element 170 may include, for example, one or more fiducial markers, fiducial beads, etc., engaged with the reference frame (e.g., a splint) before the reference frame is engaged with the object or support element. Engagement of the reference frame (e.g., a splint) with the object (e.g., a tooth) or support element (e.g., a gum or jaw) may be achieved, for example, by a suitable adhesive (e.g., an epoxy) provided therebetween. In another example, for example, in the case of an edentulous patient, an implant anchor may be implanted in the jawbone, and the reference frame (e.g., a splint) may be securely attached thereto by a removable fastener.

[0061] In yet another example, the registration element may be positioned / configured to have a known relationship to the coordinate system. For example, an optical surface scanning device (e.g., an intraoral scanning device, intraoral scanner, or other suitable optical or optical-based three-dimensional surface scanner / digitizer) may be operably engaged with the distal end of a robotic tracking arm. The robotic tracking arm may then be tracked by a controller in communication therewith, and the known or tracked position of the robotic tracking arm may include its distal end. In such an example, because the intraoral scanner is engaged with the distal end of the robotic tracking arm, the position of the imaging portion of the intraoral scanner may also be known by the controller. The known position(s) of the robotic tracking arm and the intraoral scanner engaged with the distal end of the robotic tracking arm allow the controller to associate a coordinate system with the position(s) of the robotic tracking arm and the intraoral scanner. Furthermore, the intra-oral scanner may be constructed and arranged such that the digitized image of the registration element (and / or, for example, one or more fiducial markers engaged therewith) includes or indicates a ranging relationship between the intra-oral scanner and the imaged object (e.g., the distance between the imaging element of the intra-oral scanner and a point on the imaged surface), where the relationship of the registration element to a coordinate system is known to the controller when imaged by the intra-oral scanner engaged with the distal end of the robotic tracking arm.

[0062] In another example, the relationship of the registration element to the coordinate system may be known or determinable in a different manner, in addition to or instead of the relationship between the intraoral scanner and the registration element. For example, an emitter-detector arrangement may be provided that communicates between the registration element and the intraoral scanner and / or robotic tracking arm. In another example, a transceiver-transmitter arrangement, a transceiver-reflector arrangement, a transmitter-receiver arrangement, or a sensor arrangement may be implemented as needed, such that the registration element is in communication with the intraoral scanner / robotic tracking arm, is in a known position relative thereto, and is therefore in a known relationship with respect to the coordinate system.

[0063] Once the optical surface scan is completed, a treatment for the object can be planned in relation to the three-dimensional surface image of the object, either before or after the object is correlated with the registration elements in the three-dimensional surface image. In some specific embodiments, the treatment for the object is planned in relation to the three-dimensional surface image of the object before the object is correlated with the registration elements in the three-dimensional surface image to register the object and the planned treatment to a coordinate system. In examples where a robotic surgical procedure is planned, the robot (or a robotic arm supporting a surgical tool for performing the surgical procedure) can then be guided by a controller to execute the treatment plan for the object using the surgical tool in relation to the registration of the object to the coordinate system. Such treatment planning and robotic guidance based on optical surface scans can be advantageous, for example, when the planned treatment is for the object itself (e.g., a tooth) or otherwise involves surface features related to or associated with the object. In one example, such a treatment may involve polishing the object (tooth) in preparation for receiving a dental crown.

[0064] In examples in which a radiological scan (e.g., a computed tomography (CT) scan) is also performed before the optical surface scan, the radiological image may be correlated with the three-dimensional surface image, e.g., using an image matching procedure implemented by a controller or other suitable computing device upon receiving both the radiological and optical surface scan data. That is, in some embodiments, the three-dimensional structural representation of the object and support elements provided by the radiological scan may be image matched (e.g., based on anatomical features common to both images) with the three-dimensional surface image of the object, support elements, and registration elements provided by the optical surface scan to form a correlation image. Once the correlation image is formed, the object can then be correlated with the registration elements in the correlation image to register the object to a coordinate system. Depending on the nature of the treatment (e.g., whether the treatment is performed on subsurface anatomical features / structures, on surface features, or both), the treatment for the object may be planned in conjunction with its three-dimensional surface image from the optical surface scan, from its radiological image from the radiological scan, or from its correlation image including the three-dimensional surface image and the radiological image. In this way, treatment for the object may also be planned before the object is correlated with the registration elements in its correlation image.

[0065] In this manner, the disclosed methods and systems may advantageously reduce or eliminate the need for a second radiological scan (e.g., a CT scan) as part of the preoperative process for a robotic surgical procedure, thereby reducing patient X-ray exposure. By performing an optical three-dimensional surface scan of a feature of interest (e.g., an intraoral scan) and including a fiducial marker array (e.g., mounted on a splint, with the splint engaged to the patient) in the intraoral scan, the imaged fiducial marker array in the scan is used to register the optical three-dimensional surface scan to the location of the patient and / or feature (e.g., an anatomical feature). Once the optical three-dimensional surface scan is registered to a coordinate system associated with the patient-engaged fiducial marker array, the imaged features from the intraoral scan (e.g., a three-dimensional surface image of an anatomical structure) can be registered to the imaged features from the original / first CT scan (e.g., a radiological image of an anatomical structure). The treatment plan, and other information associated therewith, created from the original / first CT scan can then be applied with reference to the patient and / or the patient's anatomical features (e.g., within a coordinate system associated with the fiducial marker array) based on its relationship to the coordinate system provided by the intraoral scan.

[0066] In further aspects, an optical three-dimensional surface scan (e.g., an intraoral scan) may eliminate the need for both the first and second CT scans. For example, in some cases, a surgeon may not need a CT scan (e.g., radiographic imaging of anatomical features) to accomplish the intended procedure. This may be applicable, for example, to robotic procedures beyond dental implants and may extend to tooth preparation procedures, for example, involving drilling or polishing a tooth surface (e.g., to remove decayed areas of the tooth) and preparing the remaining part of the tooth to receive a crown (e.g., an artificial tooth) thereon. In such instances, the intraoral scan alone may be sufficient for treatment planning purposes, and fiducial markers included in the intraoral scan can be used to register the intraoral scan to the patient and / or patient features (e.g., within a coordinate system associated with the fiducial marker array).

[0067] For example, implementation of optical surface scanning using an intraoral scanner allows the mouth (or teeth therein) and fiducial markers to be scanned and imaged during surgery while the patient is unconscious, whereas the patient must often remain awake and conscious during a CT scan with fiducial marker(s) in place. Images from an intraoral scan (with fiducial marker(s) in place) can then be correlated with images from a previous CT scan (without the fiducial marker(s) in place) by matching anatomical shapes or features between the images. Because the images captured by the intraoral scanner are already related to the fiducial marker(s) and thus integrated and registered to patient coordinate space through interaction with the fiducial marker(s), the intraoral scan / images (and CT scans / images, if implemented) containing the object and fiducial marker(s) can thus be easily correlated with a mechanically tracked (robotic) arm registered to the patient coordinate space reference and thus easily integrated into an overall robotic system to guide the robot during a procedure.

[0068] Many variations and other embodiments of the disclosure set forth herein will come to mind to one skilled in the art having the benefit of the teachings presented in the foregoing descriptions and the associated drawings. It is to be understood, therefore, that the disclosure is not limited to the particular embodiments disclosed, and that variations and other embodiments are intended to be included within the scope of the appended claims. Although specific terms are employed herein, they are used in a generic and descriptive sense only and not for purposes of limitation.

[0069] In this specification, terms such as "first," "second," etc. are used to describe various steps or calculations, but it should be understood that these steps or calculations are not limited by these terms. These terms are used only to distinguish one operation or calculation from another. For example, a first calculation may be referred to as a second calculation, and similarly, a second step may be referred to as a first step, without departing from the scope of this disclosure. As used herein, the terms "and / or" and " / " include any and all combinations of one or more of the associated listed items.

[0070] As used herein, the singular forms "a," "an," and "the" are intended to include the plural forms unless the context clearly dictates otherwise. It will be further understood that as used herein, the terms "comprises," "comprising," "includes," and / or "including" specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Accordingly, the terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting.

Claims

1. 1. A method of operating a medical device for relating an object to a coordinate system and planning a robotic procedure on the object, the method comprising: an optical scanner; and a controller including a processor and in communication with the optical scanner, the object including a tooth supported by a jaw of a patient, the object and the jaw being contained within the patient's head; an optical scanner performing an optical surface scan of the object and a registration element comprising one or more fiducial markers engaged with the object or the jaw to form a three-dimensional surface image of the object, the three-dimensional surface image having a coordinate system associated with the three-dimensional surface image such that the registration element is associated with the coordinate system, the optical scanner being hand-held and configured to be manually operated or operably engaged with a distal end of a robotic tracked arm; determining a distance between the registration element and the optical scanner from the three-dimensional surface image according to a ranging relationship between the optical scanner and the object, such that the controller determines a relationship between the registration element and the coordinate system; a controller correlating the object with the registration elements within the three-dimensional surface image to determine a relationship between the object and the registration elements, whereby the object is registered to the coordinate system via the determined relationship with the registration elements; a controller forming a plan of a robotic procedure for the object in relation to the three-dimensional surface image; A method comprising:

2. a computed tomography device configured to communicate with the controller performing a radiological scan of the patient's head and the object and jaw contained therein to form a radiological image before the registration element is engaged with the object or the jaw; a controller image matching the radiographic image of the object with the three-dimensional surface image of the object to form a correlated image of the object in relation to the registration element; the controller correlating the object with the registration element in the correlation image according to a relationship between the object and the registration element such that the object is registered to the coordinate system via the determined relationship with the registration element; The method of claim 1 , comprising:

3. 3. The method of claim 2, wherein the controller image-matching the radiological image with the three-dimensional surface image comprises the controller image-matching a three-dimensional structural representation of the object and jaw provided by the radiological scan with a three-dimensional surface image of the object, jaw, and registration elements provided by the optical surface scan.

4. The method of claim 1 , wherein the optical scanner is operatively engaged with a distal end of a robotic tracking arm registered to a coordinate system.

5. 1. A system comprising: a medical device for relating an object to a coordinate system and planning a robotic procedure on the object, the object including a tooth supported by a jaw of a patient, the object and the jaw being contained within the patient's head; an optical scanner of the medical device arranged and configured to perform an optical surface scan of the object and a registration element comprising one or more fiducial markers engaged with the object or the jaws to form a three-dimensional surface image of the object, the three-dimensional surface image having a coordinate system associated with it such that the registration element is associated with the coordinate system, the optical scanner being hand-held and arranged to be manually operated or operably engaged with the distal end of a robotic tracking arm; a controller of a medical device having a processor and in communication with the optical scanner, the controller being configured to: determine a distance between the registration element and the optical scanner from the three-dimensional surface image according to a ranging relationship between the optical scanner and the object to determine a relationship between the registration element and a coordinate system; correlate the object with the registration element in the three-dimensional surface image to determine a relationship between the object and the registration element, the object thereby being registered to the coordinate system via the determined relationship with the registration element; and form a plan of a robotic procedure for the object in association with the three-dimensional surface image prior to correlating the object with the registration element in the three-dimensional surface image; A system comprising:

6. 6. The system of claim 5, further comprising a computed tomography device configured to perform a radiological scan of the patient's head and the object and jaw contained therein to form a radiological image before the registration element is engaged with the object or jaw, and a controller of the medical device configured to communicate with the computed tomography device, image-match the radiological image of the object with a three-dimensional surface image of the object to form a correlation image of the object in relation to the registration element, and correlate the object with the registration element in the correlation image according to the relationship between the object and the registration element so that the object is registered to the coordinate system via the determined relationship with the registration element.

7. The system of claim 6, wherein the controller of the medical device is configured to image-match a three-dimensional structural representation of the object and jaw provided by a radiological scan with a three-dimensional surface image of the object, jaw, and registration elements provided by an optical surface scan.

8. 6. The system of claim 5, wherein the optical scanner of the medical device is operably engaged with a distal end of the robotic tracking arm registered to the coordinate system and configured to communicate with the controller of the medical device.

9. The system of claim 5 , comprising a reference frame having engaged registration elements, the reference frame being engaged with the object or the jaw.

10. 10. The system of claim 8, wherein the controller of the medical device is configured to track the position of the robotic tracking arm and the optical scanner operatively engaged with its distal end relative to a coordinate system.

11. 1. A system for guiding a robot relative to an object, the object including a tooth supported by a jaw of a patient, the object and the jaw being contained within the patient's head; 11. A system comprising a medical device for relating an object to a coordinate system and planning a robotic procedure on the object according to any one of claims 5 to 10, wherein a controller of the medical device is in communication with an optical scanner of the medical device and the robot, the controller of the medical device being arranged to guide the robot to perform the plan of the robotic procedure on the object in relation to the registration of the object to the coordinate system. A system comprising:

12. 8. The system of claim 6 or 7, wherein the controller of the medical device is configured to form a plan of a robotic procedure for the object in relation to the radiological image or a correlated image comprising a three-dimensional surface image and a radiological image.

13. The system of claim 11 , wherein the optical scanner of the medical device is operably engaged with a distal end of a robotic tracking arm registered to a coordinate system and configured to communicate with the robot.

Images