Method for registering imaging scans using a coordinate system and an associated system

Abstract

A method for associating an object with a coordinate system, the object being supported by a support element, and the object and the support element being contained in a housing, the method comprising engaging a registration element with the object or the support element, optically surface scanning the object and the registration element using an optical scanner to form a three-dimensional surface image of the object, the three-dimensional surface image having a coordinate system associated therewith, wherein the registration element is associated with the coordinate system; and associating the object with the registration element in the three-dimensional surface image so that the object is registered with the coordinate system. Associated methods and systems are also provided.

Description

Background Art Technical Field

[0001] The present disclosure relates to imaging procedures, and more particularly, to methods and systems for registering imaging scans using a coordinate system to enable imaging scans to be implemented in the guidance of a robot, such as a surgical robot.

[0002] Related technical notes

[0003] Preoperative imaging, particularly in the surgical space, and more specifically in the dental surgical space, may involve several different processes. For example, one preoperative imaging process involves pre-planning, but also requires the patient to undergo two CT imaging scans. The first CT imaging scan is typically performed a few weeks before the planned surgery, and the resulting patient imaging is used to plan the procedure (i.e., the planning of the surgical procedure is associated with the first CT imaging scan and uses the first CT imaging scan). A second CT imaging scan is then performed on the day of surgery, and fiducial markers (or arrays or fiducial beads) are placed on the patient during the second CT imaging scan. Then, for example, based on the alignment of imaging features (e.g., anatomical features) between the images from the two imaging scans, the image from the second CT imaging scan (including the fiducial markers / arrays in the image) is matched / registered with the image from the first CT imaging scan. The pre-planning of the surgical procedure associated with the image from the first CT imaging scan is then imported into a file associated with the image from the second CT imaging scan. The image from the second CT imaging scan also includes the fiducial markers (or arrays) therein. Thus, once the imaging features (e.g., anatomical features) are theoretically aligned between the two images such that the two CT imaging scans are registered, the patient's pre-planned surgical procedure and / or the location of the patient's imaging features (e.g., anatomical features) will also be registered with the fiducial markers (or arrays) attached to the patient and thereby converted to patient coordinate space.

[0004] However, such procedures may be subject to inaccuracies. For example, if the patient's anatomy changes or shifts between a first CT imaging scan and a second CT imaging scan, it may be difficult to align imaging features between images and / or fiducial markers may ultimately be registered relative to anatomical approximations in the actual patient coordinate space. Therefore, there is a need for a simplified procedure for creating image-based plans for surgical procedures to simplify the number of CT scans required. In addition, there is a need for a procedure that minimizes the risk of changes in the patient's anatomy between the planning stage and the surgical procedure stage. In addition, there is a need for a procedure that reliably and consistently records (multiple) images and fiducial markers of the patient's anatomy (imaging features) in the actual patient coordinate space so that the surgical plan can be executed more accurately. Summary of the invention

[0005] Various aspects of the present invention meet the above and other needs. In one aspect, the present invention provides a method of associating an object with a coordinate system, the object being supported by a support element, the object and the support element being contained in a housing. Such a method includes engaging a registration element with the object or the support element; optically surface scanning the object and the registration element using an optical scanner to form a three-dimensional surface image of the object, the three-dimensional surface image having the coordinate system associated therewith, wherein the registration element is associated with the coordinate system; and associating the object with the registration element in the three-dimensional surface image so that the object is registered with the coordinate system.

[0006] Another aspect provides a method of controlling a robot relative to an object in a coordinate system, the object being supported by a support element, the object and the support element being contained in a housing. Such a method comprises engaging a registration element with the object or the support element; optically surface scanning the object and the registration element using an optical scanner to form a three-dimensional surface image of the object, the three-dimensional surface image having the coordinate system associated therewith, wherein the registration element is associated with the coordinate system; planning a program for the object; associating the object with the registration element in the three-dimensional surface image so that the object is registered with the coordinate system; and directing the robot to execute the plan of the program for the object, the plan being associated with the registration of the object with the coordinate system.

[0007] In some aspects, prior to engaging the registration element with the object or the support element, the housing and the object and the support element contained therein are radiographically scanned to form a radiographic image. The radiographic image of the object is then image matched with the three-dimensional surface image of the object to form a correlated image of the object relative to the registration element. The object and the registration element are then correlated in the correlated image so that the object is registered with the coordinate system.

[0008] Another aspect provides a system for associating an object with a coordinate system, wherein the object is supported by a support element, the object and the support element being contained in a housing. Such a system includes a registration element adapted to engage with the object or the support element, an optical scanner arranged to perform an optical surface scan of the object and the registration element to form a three-dimensional surface image of the object, the registration element being associated with the coordinate system, and the three-dimensional surface image having the coordinate system associated therewith, and a controller having a processor and communicating with the optical scanner, wherein the controller is arranged to associate the object with the registration element in the three-dimensional surface image so that the object is registered with the coordinate system.

[0009] In yet another aspect, a system for guiding a robot relative to an object is provided, the object being supported by a support element, the object and the support element being contained in a housing. Such a method comprises a registration element adapted to engage with the object or the support element, an optical scanner arranged to perform an optical surface scan of the object and the registration element to form a three-dimensional surface image of the object, the registration element being associated with the coordinate system, and the three-dimensional surface image having the coordinate system associated therewith, and a controller having a processor and in communication with the optical scanner, the controller being arranged to allow formation of a plan of a program for the object to relate the object to the registration element in the three-dimensional surface image, so that the object is registered with the coordinate system, and to guide the robot to execute the plan of the program for the object, the plan being associated with the registration of the object with the coordinate system.

[0010] In some aspects, the computed tomography device is arranged to perform a radiographic scan of the shell and the object and the support element contained therein to form a radiographic image before engaging the alignment element with the object or the support element, wherein the controller is arranged to communicate with the computed tomography device to perform image matching on the radiographic image of the object with the three-dimensional surface image of the object so as to form a correlated image of the object relative to the alignment element, and to correlate the object with the alignment element in the correlated image so as to align the object with the coordinate system.

[0011] The present disclosure therefore includes but is not limited to the following embodiments:

[0012] Example implementation 1: A method for associating an object with a coordinate system, the object being supported by a support element, the object and the support element being contained in a shell, the method comprising engaging a registration element with the object or the support element, performing an optical surface scan of the object and the registration element using an optical scanner to form a three-dimensional surface image of the object, the three-dimensional surface image having a coordinate system associated therewith, wherein the registration element is associated with the coordinate system; and associating the object with the registration element in the three-dimensional surface image so that the object is registered with the coordinate system.

[0013] Example implementation 2: The method as described in any one of the preceding embodiments or any combination of the preceding embodiments, comprising, before engaging the alignment element with the object or the support element, performing a radiographic scan on the shell and the object and the support element contained in the shell to form a radiographic image; performing image matching on the radiographic image of the object with the three-dimensional surface image of the object to form a correlated image of the object relative to the alignment element; and correlating the object with the alignment element in the correlated image so that the object is aligned with the coordinate system.

[0014] Example implementation 3: A method as described in any one of the preceding embodiments or any combination of the preceding embodiments, wherein image matching of the radiographic image with the three-dimensional surface image includes image matching of the three-dimensional structural representation of the object and the supporting element provided by the radiographic scan with the three-dimensional surface image of the object, the supporting element, and the registration element provided by the optical surface scan.

[0015] Example embodiment 4: The method as recited in any one or any combination of the preceding embodiments, wherein the registration element is radiopaque.

[0016] Example embodiment 5: The method as in any one or any combination of the preceding embodiments, wherein the optical scanner is operably engaged with a distal end of a robotic tracking arm, the robotic tracking arm being registered with the coordinate system.

[0017] Example embodiment 6: The method as in any one or any combination of the preceding embodiments, wherein the registration element is arranged in a known relationship with the coordinate system.

[0018] Example embodiment 7: A method as described in any one of the preceding embodiments or any combination of the preceding embodiments, wherein engaging the alignment element with the object or the support element includes engaging the alignment element with a reference frame; and engaging the reference frame with the object or the support element.

[0019] Example Implementation 8: The method as described in any one of the preceding embodiments or any combination of the preceding embodiments includes planning a procedure for the object relative to the three-dimensional surface image of the object before associating the object with the alignment element in the three-dimensional surface image of the object.

[0020] Example implementation 9: A method for controlling a robot relative to an object in a coordinate system, the object being supported by a support element, the object and the support element being contained in a shell, the method comprising engaging a registration element with the object or the support element; performing an optical surface scan of the object and the registration element using an optical scanner to form a three-dimensional surface image of the object, the three-dimensional surface image having the coordinate system associated therewith, wherein the registration element is associated with the coordinate system; planning a program for the object; associating the object with the registration element in the three-dimensional surface image so that the object is registered with the coordinate system; and guiding the robot to execute the plan of the program for the object, the plan being associated with the registration of the object and the coordinate system.

[0021] Example embodiment 10: The method as described in any one of the preceding embodiments or any combination of the preceding embodiments, comprising, before engaging the alignment element with the object or the support element, performing a radiographic scan on the shell and the object and the support element contained in the shell to form a radiographic image; performing image matching on the radiographic image of the object with the three-dimensional surface image of the object to form a correlated image of the object relative to the alignment element; and correlating the object with the alignment element in the correlated image so that the object is aligned with the coordinate system.

[0022] Example embodiment 11: A method as described in any one of the preceding embodiments or any combination of the preceding embodiments, wherein image matching of the radiographic image with the three-dimensional surface image includes image matching of the three-dimensional structural representation of the object and the supporting element provided by the radiographic scan with the three-dimensional surface image of the object, the supporting element, and the registration element provided by the optical surface scan.

[0023] Example embodiment 12: A method as described in any one of the preceding embodiments or any combination of the preceding embodiments, wherein planning the procedure for the object includes planning the procedure for the object relative to the three-dimensional surface image of the object, the radiographic image of the object, or the related image of the object including the three-dimensional surface image and the radiographic image.

[0024] Example Embodiment 13: The method as recited in any preceding embodiment or any combination of the preceding embodiments, wherein the registration element is radiopaque.

[0025] Example Implementation 14: The method as described in any one or any combination of the preceding embodiments, wherein the optical scanner is operably engaged with a distal end of a robotic tracking arm, the robotic tracking arm is aligned with the coordinate system and is arranged to communicate with the robot.

[0026] Example Implementation 15: The method as in any one or any combination of the preceding embodiments, wherein the registration element is arranged in a known relationship with the coordinate system.

[0027] Example Implementation 16: The method as described in any one of the preceding embodiments or any combination of the preceding embodiments, wherein engaging the alignment element with the object or the support element includes engaging the alignment element with a reference frame; and engaging the reference frame with the object or the support element.

[0028] Example Implementation 17: A method as described in any one of the preceding embodiments or any combination of the preceding embodiments, wherein planning the procedure for the object includes planning the procedure for the object before associating the object with the alignment element in the associated image of the object.

[0029] Example embodiment 18: A system for associating an object with a coordinate system, the object being supported by a supporting element, the object and the supporting element being contained in a shell, the system comprising a registration element adapted to engage with the object or the supporting element; an optical scanner arranged to perform an optical surface scan of the object and the registration element to form a three-dimensional surface image of the object, the registration element being associated with the coordinate system, and the three-dimensional surface image having the coordinate system associated therewith; and a controller having a processor and communicating with the optical scanner, the controller being arranged to associate the object with the registration element in the three-dimensional surface image so that the object is registered with the coordinate system.

[0030] Example embodiment 19: The system as described in any one of the preceding embodiments or any combination of the preceding embodiments, comprises a computed tomography device, which is arranged to perform a radiographic scan of the shell and the object and the supporting element contained therein to form a radiographic image before engaging the alignment element with the object or the supporting element, wherein the controller is arranged to communicate with the computed tomography device to perform image matching of the radiographic image of the object with the three-dimensional surface image of the object so as to form a correlated image of the object relative to the alignment element, and to correlate the object with the alignment element in the correlated image so as to align the object with the coordinate system.

[0031] Example embodiment 20: A system as described in any one of the preceding embodiments or any combination of the preceding embodiments, wherein the controller is arranged to perform image matching of the three-dimensional structural representation of the object and the supporting element provided by the radiographic scan with the three-dimensional surface image of the object, the supporting element, and the registration element provided by the optical surface scan.

[0032] Example Embodiment 21: The system of any preceding embodiment or any combination of the preceding embodiments, wherein the registration element is radiopaque.

[0033] Example Embodiment 22: The system of any one or any combination of the preceding embodiments, wherein the optical scanner is operably engaged with a distal end of a robotic tracking arm, the robotic tracking arm being registered with the coordinate system and arranged to communicate with the controller.

[0034] Example Embodiment 23: The system as in any one or any combination of the preceding embodiments, wherein the registration element is arranged in a known relationship with the coordinate system.

[0035] Example Embodiment 24: The system of any one or any combination of the preceding embodiments, comprising a reference frame having the registration element coupled thereto, the reference frame coupled to the object or the support element.

[0036] Example embodiment 25: A system as described in any one of the preceding embodiments or any combination of the preceding embodiments, wherein the controller is arranged to allow a plan for the procedure for the object to be formed relative to the three-dimensional surface image of the object before the object is associated with the alignment element in the three-dimensional surface image of the object.

[0037] Example Implementation 26: The system of any one or any combination of the preceding embodiments, wherein the controller is arranged to track the position of the robotic tracking arm and the optical scanner operably engaged with the distal end of the robotic tracking arm relative to the coordinate system.

[0038] Example embodiment 27: A system for guiding a robot relative to an object, the object being supported by a support element, the object and the support element being contained in a housing, the method comprising a registration element adapted to engage with the object or the support element; an optical scanner arranged to perform an optical surface scan of the object and the registration element to form a three-dimensional surface image of the object, the registration element being associated with the coordinate system, and the three-dimensional surface image having the coordinate system associated therewith; and a controller having a processor and communicating with the optical scanner, the controller being arranged to allow formation of a plan for a program for the object to associate the object with the registration element in the three-dimensional surface image so that the object is registered with the coordinate system, and to guide the robot to execute the plan for the program for the object, the plan being associated with the registration of the object with the coordinate system.

[0039] Example embodiment 28: The method as described in any one of the preceding embodiments or any combination of the preceding embodiments, includes a computed tomography device, which is arranged to perform a radiographic scan of the shell and the object and the support element contained in the shell to form a radiographic image before engaging the alignment element with the object or the support element, wherein the controller is arranged to communicate with the computed tomography device to perform image matching on the radiographic image of the object with the three-dimensional surface image of the object so as to form a correlated image of the object relative to the alignment element, and to correlate the object with the alignment element in the correlated image so as to align the object with the coordinate system.

[0040] Example embodiment 29: A method as described in any one of the preceding embodiments or any combination of the preceding embodiments, wherein the controller is arranged to perform image matching on the three-dimensional structural representation of the object and the supporting element provided by the radiographic scan with the three-dimensional surface image of the object, the supporting element, and the registration element provided by the optical surface scan.

[0041] Example embodiment 30: A method as described in any one of the preceding embodiments or any combination of the preceding embodiments, wherein the controller is arranged to allow the plan for the procedure for the object to be formed relative to the three-dimensional surface image of the object, the radiographic image of the object, or the related image of the object including the three-dimensional surface image and the radiographic image.

[0042] Example Embodiment 31: The system of any preceding embodiment or any combination of the preceding embodiments, wherein the registration element is radiopaque.

[0043] Example embodiment 32: A system as described in any one or any combination of the preceding embodiments, wherein the optical scanner is operably engaged with a distal end of a robotic tracking arm, the robotic tracking arm is aligned with the coordinate system and is arranged to communicate with the controller and the robot.

[0044] Example Embodiment 33: The system as in any one or any combination of the preceding embodiments, wherein the registration element is arranged in a known relationship with the coordinate system.

[0045] Example Embodiment 34: The system of any one or any combination of the preceding embodiments, comprising a reference frame having the registration element coupled thereto, the reference frame coupled to the object or the support element.

[0046] Example embodiment 35: A system as described in any one of the preceding embodiments or any combination of the preceding embodiments, wherein the controller is arranged to allow a plan for the procedure for the object to be formed relative to the three-dimensional surface image of the object before the object is associated with the alignment element in the associated image of the object.

[0047] Example embodiment 36: A system as described in any one of the preceding embodiments or any combination of the preceding embodiments, wherein the controller is arranged to track the position of the robotic tracking arm and the optical scanner operably engaged with the distal end of the robotic tracking arm relative to the coordinate system.

[0048] These and other example embodiments, features, aspects and advantages of the present disclosure will be apparent from reading the following detailed description together with the accompanying drawings briefly described below. The present disclosure includes any combination of two, three, four or more features or elements set forth in the present disclosure or recited in any one or more of the claims, regardless of whether such features or elements are explicitly combined or otherwise recited in the specific embodiment description or claims herein. The present disclosure is intended to be read as a whole, so that any separable features or elements of any aspect and embodiment of the present disclosure in the aspects and embodiments of the present disclosure should be considered as combinable as expected, unless the context of the present disclosure clearly indicates otherwise.

[0049] Thus, as further described in detail herein, methods according to aspects of the present disclosure provide these and other advantages. Importantly, these advantages include reducing or eliminating the need for a second radiographic scan as part of a pre-procedure for robotic surgery. That is, by performing an optical three-dimensional surface scan (e.g., an intraoral scan) of a feature of interest, and including a fiducial marker array in the intraoral scan (e.g., mounted on a splint, where the splint is engaged with a patient), the imaged fiducial marker array in the scan is used to align the optical three-dimensional surface scan with the location of the feature (e.g., an anatomical feature) and / or the patient. Once the coordinate system associated with the optical three-dimensional surface scan and the fiducial marker array engaged with the patient is aligned, the imaged features from the intraoral scan (e.g., a three-dimensional surface image of the anatomical structure) can be aligned with the imaged features from the original / first CT scan (e.g., a radiographic image of the anatomical structure). The plan for the procedure developed based on the original / first CT scan and other information associated therewith can then be applied with reference to the patient and / or the location of the patient's anatomical features (e.g., within a coordinate system associated with the fiducial marker array).

[0050] In a further aspect, an optical three-dimensional surface scan (e.g., an intraoral scan) may eliminate the need for both a first CT scan and a second CT scan. For example, in some cases, a surgeon may not need CT (e.g., radiographic imaging of anatomical features) to complete the intended surgery. This may apply, for example, to robotic procedures other than dental implants, and may also extend to, for example, tooth preparation procedures involving drilling or grinding a tooth surface (e.g., removing a decayed area of ​​a tooth) and preparing the remainder of the tooth to receive a crown (e.g., a denture) thereon. In such instances, an intraoral scan alone may be sufficient for the purpose of the planned procedure, and an array of fiducial markers included in the intraoral scan may be used to align the intraoral scan with the position of the patient and / or patient features (e.g., within a coordinate system associated with the array of fiducial markers), as well as to enable robotic guidance. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Having thus described the present disclosure in general terms, reference will now be made to the accompanying drawings, which are not necessarily drawn to scale, and in which:

[0052] Figure 1 Schematically illustrating an optical scanner according to one aspect of the present disclosure, the optical scanner being arranged to perform an optical surface scan of an object, wherein the object may be contained within a housing comprising an oral or maxillofacial structure of a patient, the housing being located within a system for associating the object to a coordinate system and for controlling a robot relative to the object in the coordinate system;

[0053] Figure 1A Schematically illustrating an optical scanner according to one aspect of the present disclosure, the optical scanner being arranged to perform an optical surface scan of an object, wherein the object may be contained within a housing comprising an oral or maxillofacial structure of a patient, the housing being located within a system for associating the object to a coordinate system and for controlling a robot relative to the object in the coordinate system;

[0054] Figure 2 Schematically illustrating an object supported by a supporting element according to one aspect of the present disclosure, wherein the object comprises teeth, the supporting element comprises corresponding jaws and / or gums, and wherein a registration element is adapted to engage with the object or the supporting element;

[0055] Figure 3 and Figure 4 Schematically illustrating an optical scanner according to one aspect of the present disclosure, the optical scanner being arranged to perform an optical surface scan of an object, wherein the object may be accommodated within a housing comprising an oral or maxillofacial structure of a patient;

[0056] Figure 5 Schematically illustrates a computed tomography apparatus according to one aspect of the present disclosure, the computed tomography apparatus being arranged to radiographically scan a housing and an object and a support element accommodated therein to form a radiographic image thereof;

[0057] Figure 6 schematically illustrates a radiographic image of an object being image matched with a three-dimensional surface image of the object according to one aspect of the present disclosure;

[0058] Figure 7 A method of associating an object to a coordinate system according to one aspect of the present disclosure is schematically shown; and

[0059] Figure 8 A method for controlling a robot relative to an object in a coordinate system according to one aspect of the present disclosure is schematically illustrated. DETAILED DESCRIPTION

[0060] The present disclosure will now be described more fully below with reference to embodiments of the present disclosure. These exemplary embodiments are described so that the present disclosure will be thorough and complete, and will fully convey the scope of the present disclosure to those skilled in the art. In fact, the present disclosure can be embodied in many different forms and should not be construed as being limited to the embodiments set forth herein; rather, these embodiments are provided so that the present disclosure will satisfy applicable legal requirements. The various aspects of the present disclosure mentioned above and many other aspects of the present disclosure are further described in detail herein.

[0061] For example Figure 1-Figure 4 As shown, various aspects of the present disclosure are directed to a system 100 for associating an object 200 to a coordinate system 300 and for controlling a robot 800 relative to the object 200 in the coordinate system 300, wherein the object 200 is supported by a support element 220, and wherein the object 200 and the support element 220 are contained within a housing 240. In one example, the object 200 may include teeth, the support element 220 may include a corresponding jaw and / or gums, and the housing 240 may include an oral or maxillofacial structure of the patient. In a particular aspect, such a system 100 includes a registration element 170 (e.g., see FIG. 1 ) adapted to engage with the object 200 or the support element 220. Figure 2 ). The optical scanner 400 is arranged to scan the object 200 (see, for example, Figure 3 and Figure 4 ) and the registration element 170 perform optical surface scanning and form a three-dimensional surface image of the object 200 (see, for example, Figure 6 , right image). The optical scanner 400 is arranged to be handheld and manually manipulated by an operator (see, for example, Figure 1 ), or may otherwise be operably engaged with the distal end 520 of the robotic tracking arm 500 (see, e.g., Figure 1A ). The three-dimensional surface image has a coordinate system 300 associated therewith, and the registration element 170 is associated with the coordinate system 300. Suitable for being associated with the object 200 or the support element 220 (see, for example, Figure 2) may include, for example, one or more fiducial markers 175 arranged to define the coordinate system 300 or arranged in a known relationship with the coordinate system 300. More specifically, in some instances, the one or more fiducial markers 175 are arranged so that at least the position and orientation of the one or more fiducial markers 175 are known, or readily determinable relative to the coordinate system 300, or registered with the coordinate system 300. The controller 190 has a processor and is arranged to communicate with the optical scanner 400 and, in some instances, the robotic tracking arm 500. The controller 190 is further arranged to correlate the object 200 with the registration element 170 in the three-dimensional surface image so as to register the object 200 to the coordinate system 300. That is, since the registration element 170 has one or more fiducial markers 175 that themselves define a known position / orientation or reference or registration in the coordinate system 300, an optical surface scan of the object 200 / support element 220 including the registration element 170 allows the object 200 to be spatially correlated with the registration element 170 based on a three-dimensional surface image. When the spatial correlation of the object 200 with respect to the registration element 170 is known with respect to the coordinate system 300, the object 200 thus becomes registered with respect to the registration element 170 in the coordinate space 300.

[0062] In some examples, the system 100 may further include a computed tomography device 700 (see, for example, Figure 5 ), the computer tomography device 700 is arranged to perform a radiographic scan of the housing 240 and the object 200 and the support element 220 contained therein before the registration element 170 is engaged with the object 200 or the support element 220 to form a radiographic image thereof (see, for example, Figure 6 , left image). In such an example, the controller 190 is arranged to communicate with the computed tomography device 700 to receive imaging data representing a radiographic scan / radiographic image therefrom. Upon receiving the imaging data, the controller 190 is also arranged to compare the radiographic image of the object 200 with a three-dimensional surface image of the object 200 (e.g., see Figure 6 ) to perform image matching so as to form a correlated image of the object 200 relative to the registration element 170. Once the radiographic image and the three-dimensional surface image are correlated, the controller 190 is arranged to correlate the object 200 with the registration element 170 in the correlated image so as to register the object 200 with the coordinate system 300.

[0063] The controller 190 is arranged to allow planning of a procedure for the object 200 with respect to its three-dimensional surface image before the object 200 is associated with the registration element 170 in its three-dimensional surface image. If the optical scanner 400 is engaged with the robotic tracking arm 500, the controller 190 may further be arranged to track the position of the robotic tracking arm 500 and the optical scanner 400 operably engaged with the distal end 520 of the robotic tracking arm 500 relative to the coordinate system 300. In either instance, once the object 200 is associated with the registration element 170 (as determined with respect to the coordinate system 300 associated with the registration element 170 in the three-dimensional surface image from the optical surface scan), the plan of the procedure determined and developed with respect to the three-dimensional surface scan can then be used to guide a robot 800 (such as a surgical robot) having a surgical instrument 820 engaged therewith (and operably engaged with a robotic tracking arm 500 in communication with the registration element 170 attached to the patient) to perform the plan of the procedure on the object 200 via the surgical instrument 820 and associated with the registration of the object 200 with the coordinate system 300.

[0064] In one aspect of the present disclosure, a method of associating an object 200 to a coordinate system 300 involves an arrangement in which the object 200 is supported by a support element 220, and in which the object 200 and the support element 220 are contained within a housing 240. In one example, such an arrangement may include teeth as the object, corresponding jaws and / or gums as the support element, and oral or maxillofacial structures as the housing. Thus, the method may include engaging the registration element 170 with the object or the support element ( Figure 7 , block 900). In some examples, the registration element 170 is disposed in the coordinate system 300 or otherwise exists in a known relationship in or with the coordinate system 300, as disclosed herein. The object and the registration element within the housing are then optically surface scanned using an optical scanner 400 (in some examples, the optical scanner 400 can be handheld and manually manipulated, or the optical scanner 400 can be operably engaged with the distal end 520 of the robotic tracking arm 500) to form a three-dimensional surface image of the object relative to the registration element 170 ( Figure 7 , block 920). The optical surface scan performed by the optical scanner 400 digitizes the image of the object and the registration element 170 in the form of a three-dimensional surface image, and the three-dimensional surface image of the object is then correlated with the registration element so that the object is then registered with the coordinate system (e.g., the image is converted from image coordinate space (a coordinate space internal to the image) to patient coordinate space (a coordinate space relative to the patient or the registration element 170)) ( Figure 7 , box 940).

[0065] Another aspect of the present disclosure relates to a method of controlling a robot 800 relative to an object 200 in a coordinate system 300, wherein the method relates to an arrangement having an object 200 supported by a support element 220, and both the object 200 and the support element 220 are contained within a housing 240. In one example, such an arrangement may include teeth as the object, corresponding jaws and / or gums as the support element, and oral or maxillofacial structures as the housing. Thus, the method may include engaging a registration element 170 with the object 200 or the support element 220 ( Figure 8 , block 1000), and then optically surface scanning the object and the registration element using an optical scanner 400 (in some examples, the optical scanner 400 can be handheld and manually manipulated, or the optical scanner 400 can be operably engaged with the distal end 520 of the robotic tracking arm 500) to form a three-dimensional surface image of the object, wherein the three-dimensional surface image has a coordinate system 300 associated therewith, and wherein the registration element 170 is associated with the coordinate system 300 ( Figure 8 , block 1020). For example, before associating the object with the registration element in the three-dimensional surface image, a plan for a procedure with respect to the object is formed from the three-dimensional surface image ( Figure 8 , block 1040) to register the object with the coordinate system (e.g., transform the image from image coordinate space (a coordinate space internal to the image) to patient coordinate space (a coordinate space relative to the patient or the registration element 170)) ( Figure 8 , block 1060). The robot 800 with the surgical instrument 820 engaged therewith may then be directed to perform the planned procedure on the object 200 with the surgical instrument 820 and associated with the registration of the object 200 with the coordinate system 300 ( Figure 8 , box 1080).

[0066] In some aspects, particularly with respect to the two method aspects disclosed above, the housing and the object and support element contained therein may be radiographically scanned to form a radiographic image of the housing, the object, and the support element prior to the engagement of the registration element with the object or the support element. Figure 7 , box 960, Figure 8 , box 1100). Then, the radiographic image of the object is image matched with the three-dimensional surface image of the object to form a correlated image of the object relative to the registration element ( Figure 7 , box 970, Figure 8 , block 1120). The object is then associated with the registration element in the associated image to register the object with the coordinate system (e.g., the associated image is transformed from image coordinate space (a coordinate space internal to the associated image) to patient coordinate space (a coordinate space relative to the patient or the registration element 170)) ( Figure 7 , box 980, Figure 8, box 1140).

[0067] In some aspects, image matching the radiographic image to the three-dimensional surface image can involve image matching the three-dimensional structural representation of the object and support elements provided by the radiographic scan to the three-dimensional surface image of the object, support elements, and registration elements provided by the optical surface scan. In this manner, a plan for the procedure can then be formed based on and taking into account the aesthetics of the procedure seen in the three-dimensional surface image of the optical surface scan (e.g., an intraoral scan showing, for example, gums and soft tissue surrounding the teeth) and the specific anatomical structures seen in the three-dimensional structural representation of the radiographic scan (e.g., a computed tomography (CT) scan showing, for example, nerves, tooth roots, jawbone, and / or related structures).

[0068] In some instances, since the registration elements are associated with optical surface scanning, the registration elements need not be radiopaque, but if necessary or desired, the registration elements can be radiopaque. In other instances, the registration elements 170 may include, for example, one or more fiducial markers, fiducial beads, etc. that engage with the reference frame (e.g., a splint) before the reference frame is engaged with the object or support element. Engagement with the object (e.g., a tooth) or support element (e.g., gum or jaw) and the reference frame (e.g., a splint) can be achieved by, for example, a suitable adhesive (e.g., epoxy resin) disposed therebetween. In other instances, for example, in the case of an edentulous patient, an implant anchor can be implanted in the jawbone and the reference frame (e.g., a splint) can be securely attached thereto by a removable fastener.

[0069] In other examples, the registration element may be arranged / configured to be in a known relationship with the coordinate system. For example, an optical surface scanning device (e.g., an intraoral scanning device, an intraoral scanner, or any other suitable light-based or optical-based three-dimensional surface scanner / digitizer) may be operably engaged with the distal end of the robotic tracking arm. In turn, the robotic tracking arm may be tracked by a controller communicating therewith, and the known or tracked position of the robotic tracking arm may include its distal end. Since the intraoral scanner is engaged with the distal end of the robotic tracking arm in such examples, the controller may also know the position of the imaging portion of the intraoral scanner. The (multiple) known positions of the robotic tracking arm and the intraoral scanner engaged with the distal end of the robotic tracking arm allow the controller to associate the coordinate system with the (multiple) positions of the robotic tracking arm and the intraoral scanner. In addition, the intraoral scanner may be configured and arranged so that the digitized image of the registration element (and / or, for example, one or more fiducial markers engaged therewith) includes or indicates a ranging relationship between the intraoral scanner and the imaging object (e.g., the distance between the imager of the intraoral scanner and a point on the imaging surface). Using such a range finding relationship, the controller knows the relationship of the registration elements to the coordinate system when imaging with an intra-oral scanner engaged with the distal end of the robotic tracking arm.

[0070] In other examples, the relationship of the registration element to the coordinate system may be known or determined in a different manner in addition to or in lieu of the relationship between the intraoral scanner and the registration element. For example, an emitter-detector arrangement may be provided in communication between the registration element and the intraoral scanner and / or the robotic tracking arm. In other examples, a transceiver-transceiver arrangement, a transceiver-reflector arrangement, an emitter-receiver arrangement, or a sensor arrangement (as the case may be) may be implemented such that the registration element is in communication with the intraoral scanner / robotic tracking arm and is in a known position relative to the intraoral scanner / robotic tracking arm, and is therefore arranged in a known relationship relative to the coordinate system.

[0071] Once the optical surface scan is completed, a procedure for the object may be planned relative to the three-dimensional surface image of the object before or after the object is associated with the registration element in its three-dimensional surface image. In some specific aspects, the procedure for the object is planned relative to the three-dimensional surface image of the object before the object is associated with the registration element in its three-dimensional surface image so as to align the object and the planned procedure with the coordinate system. In an instance of planning a robotic surgical procedure, a controller may guide a robot (or a robotic arm supporting a surgical tool to perform a surgical procedure) to perform a surgical plan associated with the registration of the object in the coordinate system using the surgical tool for the object. Such program planning and robotic guidance based on optical surface scanning may be advantageous, for example, in an instance of a planned procedure for the object itself (e.g., a tooth), or in an instance involving surface features associated or associated with the object in other cases. In one instance, such a procedure may involve grinding an object (a tooth) in preparation for receiving a crown.

[0072] In the case where a radiographic scan (e.g., a computed tomography (CT) scan) is also performed prior to the optical surface scan, the radiographic image can be correlated with the three-dimensional surface image, for example, using an image matching program implemented by a controller or other suitable computer device to receive data from the radiographic scan and the optical surface scan. That is, in some aspects, the three-dimensional structural representation of the object and support elements provided by the radiographic scan can be image matched with the three-dimensional surface image of the object, support elements, and registration elements provided by the optical surface scan (e.g., based on anatomical features common to both images) to form a correlated image. Once the correlated image is formed, the object and the registration element can be correlated in the correlated image to align the object with the coordinate system. Depending on the nature of the procedure (e.g., whether the procedure is performed on subsurface anatomical features / structures, or on surface features, or both), the procedure for the object can be planned relative to the three-dimensional surface image from the optical surface scan, the radiographic image from the radiographic scan, or from a correlated image including the three-dimensional surface image and the radiographic image. In this way, a procedure for an object can also be planned before the object is associated with the registration element in its associated image.

[0073] Thus, the disclosed methods and systems reduce or eliminate the need for a second radiographic scan (e.g., a CT scan) as part of the preoperative process for robotic surgery, and it may be advantageous to reduce the patient's x-ray exposure by removing the second radiographic scan. By performing an optical three-dimensional surface scan (e.g., an intraoral scan) of a feature of interest, and including a fiducial marker array in the intraoral scan (e.g., mounted on a splint, where the splint is engaged with the patient), the imaged fiducial marker array in the scan is used to register the optical three-dimensional surface scan with the location of the feature (e.g., an anatomical feature) and / or the patient. Once the optical three-dimensional surface scan and the coordinate system associated with the fiducial marker array engaged with the patient are registered, the imaged features from the intraoral scan (e.g., a three-dimensional surface image of the anatomical structure) can be registered with the imaged features from the original / first CT scan (e.g., a radiographic image of the anatomical structure). The plan of the procedure developed based on the original / first CT scan and other information associated therewith can then be applied with reference to the location of the patient and / or the patient's anatomical features (e.g., within the coordinate system associated with the fiducial marker array) based on the relationship to the coordinate system provided by the intraoral scan.

[0074] In a further aspect, an optical three-dimensional surface scan (e.g., an intraoral scan) may eliminate the need for both a first CT scan and a second CT scan. For example, in some cases, a surgeon may not need a CT scan (e.g., radiographic imaging of anatomical features) to complete an intended procedure. This may apply, for example, to robotic procedures other than dental implants, and may also extend to, for example, tooth preparation procedures involving drilling or grinding a tooth surface (e.g., removing a decayed area of ​​a tooth) and preparing the remainder of the tooth to receive a crown (e.g., a denture) thereon. In such instances, an intraoral scan alone may be sufficient for the purpose of the planned procedure, and an array of fiducial markers included in the intraoral scan may be used to register the intraoral scan with the position of the patient and / or patient features (e.g., within a coordinate system associated with the array of fiducial markers).

[0075] For example, implementing optical surface scanning using an intraoral scanner allows the oral cavity (or teeth within the oral cavity) and fiducial markers to be scanned and imaged during surgery while the patient is unconscious, whereas the patient must typically be awake and conscious during a CT scan with fiducial marker(s). The images from the intraoral scan (with the fiducial marker(s) in place) can then be correlated with the images from the previous CT scan (without the fiducial marker(s) in place) by matching anatomical geometry or features between the images. Since the imaging captured by the intraoral scanner is already associated with the fiducial marker(s) and is therefore integrated into and registered with the patient coordinate space via interaction with the fiducial marker(s), the intraoral scan / image (and CT scan / image, if implemented) including the object and fiducial marker(s) is easily associated with a mechanically tracked (robotic) arm referenced to the patient coordinate space and can therefore be easily integrated into an overall robotic system to guide the robot in a procedure.

[0076] Thanks to the teachings presented in the foregoing description and the associated drawings, those skilled in the art to which the present disclosure relates will appreciate the many modifications and other aspects of the present disclosure set forth herein. Therefore, it should be understood that the present disclosure is not limited to the specific aspects disclosed, and modifications and other aspects are intended to be included within the scope of the appended claims. Although specific terms are employed herein, these terms are used only in a general and descriptive sense and not for limiting purposes.

[0077] It should be understood that, although the terms first, second, etc. can be used herein to describe various steps or calculations, these steps or calculations should not be limited to these terms. These terms are only used to distinguish one operation or calculation from another operation or calculation. For example, without departing from the scope of the present disclosure, the first calculation can be referred to as the second calculation, and similarly, the second step can be referred to as the first step. As used herein, the term "and / or" and the symbol " / " include any and all combinations of one or more items in the associated listed items.

[0078] As used herein, the singular forms "a", "an", and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that when used herein, the terms "comprises", "comprising", "includes", and / or "including" specify the presence of stated features, integers, steps, operations, elements, and / or components, but do not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. Therefore, the terminology used herein is for the purpose of describing specific embodiments only and is not intended to be limiting.

Claims

1. A method of associating an object with a coordinate system, the object being supported by a patient's jaw, the object and the jaw being received in the patient's head, the method comprising: engaging a registration element with the subject or the jaw, the registration element comprising one or more fiducial markers; optically surface scanning the object and the registration element using an optical scanner to form a three-dimensional surface image of the object, the three-dimensional surface image having the coordinate system associated therewith such that the registration element is associated with the coordinate system; determining a distance between the registration element and the optical scanner based on a ranging relationship between the optical scanner and the object to determine a relationship between the registration element and the coordinate system; correlating the object with the registration element in the three-dimensional surface image using a controller having a processor and in communication with the optical scanner to determine a relationship between the object and the registration element, thereby registering the object with the coordinate system via the determined relationship with the registration element; as well as Prior to correlating the object with the registration element in the three-dimensional surface image, a procedure for the object is planned with the controller relative to the three-dimensional surface image of the object, the procedure being a robotic procedure performed by a robot.

2. The method of claim 1, comprising: Prior to engaging the registration element with the object or the jaw, performing a radiographic scan of the patient's head and the object and the jaw received in the patient's head to form a radiographic image; performing image matching of the radiographic image of the object with the three-dimensional surface image of the object to form a correlated image of the object relative to the registration element; as well as The object is correlated with the registration element in the correlation image according to the relationship between the object and the registration element, such that the object is registered with the coordinate system via the determined relationship with the registration element.

3. The method according to claim 2, characterized in that Image matching the radiographic image with the three-dimensional surface image includes image matching the three-dimensional structural representation of the object and the jaw provided by the radiographic scan with the three-dimensional surface image of the object, the jaw and the registration element provided by the optical surface scan.

4. The method according to claim 1, characterized in that The optical scanner is operably engaged with a distal end of a robotic tracking arm, the robotic tracking arm being registered with the coordinate system.

5. The method according to claim 1, characterized in that Engaging the registration element with the subject or the jaw comprises: engaging the registration element with a reference frame; and The reference frame is engaged with the subject or the jaw.

6. A system for associating an object with a coordinate system, the object being supported by a patient's jaw, the object and the jaw being received in the patient's head, the system comprising: a registration element adapted to engage with the subject or the jaw, the registration element comprising one or more fiducial markers; an optical scanner arranged to perform an optical surface scan of the object and the registration element to form a three-dimensional surface image of the object, the three-dimensional surface image having the coordinate system associated therewith such that the registration element is associated with the coordinate system; as well as a controller having a processor and in communication with the optical scanner, the controller being arranged to determine a distance between the registration element and the optical scanner based on a ranging relationship between the optical scanner and the object to determine a relationship between the registration element and the coordinate system, and to correlate the object with the registration element in the three-dimensional surface image to determine the relationship between the object and the registration element, thereby registering the object with the coordinate system via the determined relationship with the registration element, and allowing a plan of a program for the object to be formed relative to the three-dimensional surface image of the object before correlating the object with the registration element in the three-dimensional surface image, the program being a robotic program performed by a robot.

7. The system as described in claim 6 includes a computed tomography device, which is arranged to perform a radiographic scan on the patient's head and the object and the jaw accommodated in the patient's head before engaging the alignment element with the object or the jaw to form a radiographic image, wherein the controller is arranged to communicate with the computed tomography device to perform image matching on the radiographic image of the object with the three-dimensional surface image of the object so as to form a correlated image of the object relative to the alignment element, and to correlate the object with the alignment element in the correlated image according to the relationship between the object and the alignment element, so that the object is aligned with the coordinate system via the determined relationship with the alignment element.

8. The system according to claim 7, characterized in that The controller is arranged to image match the three-dimensional structural representation of the object and the jaw provided by the radiographic scan with the three-dimensional surface image of the object, the jaw and the registration element provided by the optical surface scan.

9. The system according to claim 6, characterized in that The optical scanner is operably engaged with a distal end of a robotic tracking arm that is registered with the coordinate system and arranged to communicate with the controller.

10. The system of claim 6, comprising a reference frame having the registration element engaged therewith, the reference frame being engaged with the subject or the jaw.

11. The system according to claim 9, characterized in that The controller is arranged to track the position of the robotic tracking arm and the optical scanner operably engaged with the distal end of the robotic tracking arm relative to the coordinate system.

12. A system for guiding a robot relative to an object, the object being supported by a patient's jaw, the object and the jaw being received in the patient's head, the system comprising: A system for associating an object with a coordinate system according to any one of claims 6-9 and 10-11, wherein the controller is in communication with the optical scanner and the robot, and wherein the controller is arranged to direct the robot to execute the plan of the program for the object, the plan being related to the registration of the object and the coordinate system, the program being a robot program performed by the robot.

13. The system of claim 12, wherein: The controller is arranged to allow forming the plan of the robotic procedure for the object with respect to a radiographic image of the object or a correlated image of the object comprising the three-dimensional surface image and the radiographic image.

14. The system of claim 12, wherein: The optical scanner is operably engaged with a distal end of a robotic tracking arm that is registered with the coordinate system and arranged to communicate with the robot.

Images