Surgical navigation system

Abstract

To provide a surgical navigation method using a computer system.SOLUTION: A surgical navigation method includes the steps of: (a) fixing a camera 11 to a surgical site on a patient such that the camera does not move with respect to the surgical site; (b) restricting a body portion of the patient including the surgical site to an operating table supporting the patient such that the body portion does not move with respect to the operation table; and (c) detecting a position of a medical instrument having a tracker array by using the camera.SELECTED DRAWING: Figure 2A

Description

Background Art

[0001] In the field of image-guided surgery or surgical navigation, a camera in the operating room (often called a "localization camera") tracks the position of surgical tools in three-dimensional space. This position information is transferred from the localization camera to a computer. The computer monitor displays multi-planar three-dimensional radiographic images of the patient's anatomical structures related to the surgical procedure that are uploaded to the computer. The anatomical structures of the patient located in the operating room are registered to the radiographic image data using a probe or registration array tracked by the localization camera. In image-guided surgery, registration is the process of transforming a three-dimensional radiographic image dataset (image space) to correlate with the three-dimensional coordinates of the corresponding patient's anatomical structures (surgical space). After registration of the relevant anatomical structures, a navigation system 100, such as shown in FIG. 1, can present the position of the tracked surgical tool relative to the anatomical structures of the radiographic image displayed on the display 102. The surgical tool or instrument to be tracked has its own tracking array 103 with IR marker elements, whereby the system can detect and track the surgical instrument via the localization camera 101. To make this process accurate, the three-dimensional spatial relationship between the localization camera and the patient's anatomical structures needs to be known and maintained. If the localization camera moves, or if the patient's anatomical structures move during surgery, the accuracy of the navigation system drops. To compensate for this problem, a tracked tool known as a dynamic reference frame 104 is fixed in relation to the patient's anatomical structures. When the patient's anatomical structures and the localization camera move relative to each other, their three-dimensional relationship is recalculated by the navigation system computer and the registration solution is updated. Generally, the dynamic reference frame is fixed to a clamp fixed to the patient's anatomical structures near the surgical site 105.

[0002] The above means can play a role in maintaining the accuracy of image-guided surgery, but it has drawbacks. To track the dynamic reference frame, the localization camera measures the spatial coordinates of the markers on the reference frame 104 along the x-axis, y-axis, and z-axis. The accuracy of these measurements varies depending on the viewing angle between the camera and the markers. For example, it is common practice to move the camera to maintain a line of sight for identifying the position of the dynamic reference frame so that objects such as microscopes, various instruments, and additional operating room personnel entering the surgical field can enter. Similarly, the patient is often repositioned during surgery, such as when the operating table is rotated or lifted. In such cases, when the camera moves relative to the markers on the dynamic reference frame, navigation errors are inadvertently induced.

[0003] Instruments and techniques are still needed to minimize or eliminate the relative movement between the camera and the dynamic reference frame during image-guided surgery.

Summary of the Invention

[0004] One embodiment of the present invention is a surgical navigation method using a computer system. This method includes (a) fixing a camera relative to the surgical site of a patient so that the camera does not move relative to the surgical site, (b) restraining the body part of the patient including the surgical site relative to the operating table supporting the patient so that the body part does not move relative to the operating table, and (c) detecting the position of a medical instrument having a tracker array by the camera.

[0005] Other embodiments include placing a registration array over the surgical site and generating a 3D radiation image registered to the camera using the registration array. The registration array is removed from the patient, and then the camera tracks the position of the surgical instrument during the surgical procedure.

[0006] An additional embodiment of the present invention is an operating table including pelvic support pads inclined on at least two opposing inner and lower sides. The pelvic support pads have at least three foam layers having different densities, the patient boundary layer has the lowest density among the at least three layers, and the bottom layer is the layer with the highest density. A fixing sheet is used to fix the patient to the pelvic support pads.

[0007] In the following drawings and detailed description, additional embodiments are described.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2A

Figure 2B

Figure 3A

Figure 3B

Figure 4

Figure 5A

Figure 5B

Figure 6A

Figure 6B

Figure 6C

Figure 6D

Figure 7

Figure 8

Figure 9A

Figure 9B

Figure 10

Figure 11

Figure 12

Figure 13A

Figure 13B

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Best Mode for Carrying Out the Invention

[0009] An example of the components used in the navigation system 1 of the present invention is presented in FIG. 2A. These components include a system station or a mobile cart 2. In the station 2, a display 5 and user interface components such as a conventional keyboard 6 or a mouse 7 are installed. FIG. 2 also shows an optical sensor 10 (for example, a camera 11) and a path system 25 removably installed in the station 2, both of which will be described in detail later.

[0010] The navigation system includes a computer system 220 shown as a system diagram in FIG. 2B. The computer system 220 may include a memory 222, a processor 224, the display 5 described above, a network interface 228, and an input device 230. The memory 222 may store an application 236 and / or CT data 234. When executed by the processor 224, the application 236 may cause the display 5 to present a user interface 238. The processor 224 may be a general-purpose processor, a dedicated graphics processing unit (GPU) configured to perform specific graphic processing tasks while leaving the general-purpose processor available for other tasks, and / or any number of such processors or combinations thereof. The keyboard 6 and the mouse 7 serve the function of data input devices. Alternatively, the display 5 may be touch-sensitive and / or voice-activated, and the display 5 may be capable of serving the functions of both input and output devices. The network interface 228 may be configured to connect to a network, either wired or wireless. Through the network interface 228, the computing system 220 may receive data from related devices such as, for example, the camera 11 and the path system 25, and send commands to them. Also, the computing system 220 may receive patient image data, such as computerized tomography (CT) image data, from a server such as, for example, a hospital server, an Internet server, or other similar servers, for use during the planning and execution of a surgery. The application 236 may be one or more software programs stored in the memory 222 and executed by the processor 224. The software programs perform computer-related functions described below, including the generation of navigation images and various user interfaces.

[0011] Many aspects of the present invention relate to software operations of medical images, while one embodiment of the present invention relates to improvements to the operating room (OR) table. As will become apparent later, one important feature of certain embodiments of the present invention is to fix the anatomical structure of a patient undergoing surgery so as not to move relative to the OR table. FIGS. 3A and 3B present an improved OR table that provides a stabilization surface useful for fixing the patient so as not to move during a surgical procedure. FIG. 3A shows the upper part of a rail-type OR table 70 (such as an Allen Advanced or Jackson radiolucent OR table) having a series of patient support surfaces or pads, such as leg support pads 71, pelvic support pads 75, and head support pads 73. The support pads may, in some cases, also be referred to as "table adapters". FIG. 3B shows a cross-section of the pelvic or hip structure of a patient in the prone position located on the pelvic support pad 75, and also shows the structure of the pelvic support pad 75. This embodiment of the pelvic support pad 75 includes at least three foam layers, each layer having a different density or stiffness. The top layer (or "patient boundary layer") 76 is the layer with the lowest stiffness, and the bottom layer 78 is the layer with the highest stiffness. The bottom layer 78 is fixed to a very rigid backing layer 79 (for example, by an adhesive), and the backing layer 79 is attached to a support bracket 74 connected to the rails of the OR table.

[0012] In one example embodiment, the top layer 76 has a stiffness or hardness of 30 to 40 on the A-scale Shore hardness, the middle layer 77 has a hardness of 50 to 70, the bottom layer 78 has a hardness of 70 to 90, and the backing layer 79 has an even higher hardness. Generally, the layer thickness varies from 0.5 centimeters to 10 centimeters, preferably 1 to 5 centimeters. In a particular embodiment, the layers increase in thickness from the top layer 76 towards the bottom layer 78. In a preferred embodiment, the layers are formed of viscoelastic polyurethane foam.

[0013] Figures 4 and 5 show a patient in a prone position on the OR table 70. At this position, the pelvic support pads 75 that are inclined towards two opposing inner and lower sides serve to support the patient's pelvis or hip structure, and it is understood that the patient's position is stabilized by the patient's weight. Figure 4 also presents how a series of patient fixation sheets 85 extend over the patient and are fixed to the OR table on both sides of the patient to further immobilize the patient's pelvis and thoracic region. Generally, the fixation sheet is a breathable stretchable fabric sheet or band with a width of 4 inches to 18 inches so as to disperse the force applied to the patient's body and not suppress blood circulation. In one example, the fixation sheet may be an iFIX Fleece strip sold by Interventional Systems USA located in Stanford, Connecticut. However, as an alternative, the fixation sheet may be a co-flexible stretch bandage, a custom-designed belt with sufficient padding material, or substantially any means for securely and safely fixing the relevant patient's anatomical structure to the OR table without interfering with the surgical process.

[0014] FIG. 5A enables visualization of one method of the present invention that includes fixing an optical sensor (e.g., camera 11 in FIG. 5A) so that camera 11 does not move relative to surgical field 90 using a patient fixed to an OR table. In the embodiment of FIG. 5A, this relative fixation is achieved by fixing the patient to the OR table as described above and then fixing camera 11 to the OR table by any conventional means. FIG. 5A shows a bed mounting bracket 13 that grips the rails of the OR table and a plurality of positioning arm segments 14A - 14C that connect mounting bracket 13 to camera 11. Locking sleeve 16A may be loosened so that positioning arm segment 14B can rotate relative to arm segment 14B, and then locking sleeve 16A is tightened to lock the relative position of the arm segments. Also, arm segment 14B may extend telescopically from arm segment 14A, and locking sleeve 16A also fixes this telescoping relationship. Similarly, locking hinge 15 allows positioning arm segment 14C to rotate relative to 14B and then be locked in the desired relative position. Locking sleeve 16B allows camera 11 to rotate relative to arm segment 14C and then be locked in a fixed position. By using this mechanical structure, camera 11 is positioned such that it does not interfere with the surgical procedure and maintains any tracking array (e.g., tracking array 40 in FIG. 5A, whose function will be described in detail later) used during the surgical procedure within its field of view. In the illustrated embodiment, camera 11 may be a monocular position - specific camera sold by Intellijoint Surgical of Kitchener, Canada.

[0015] FIG. 5A shows the relative fixation between the surgical site and the camera by fixing the patient and the camera to the OR table, although this relative fixation may be ensured by other means. For example, if the OR table is securely fixed to the floor of the operating room, the camera can likewise be fixed to the floor of the OR. If the patient is immobile relative to the OR table, the camera and the surgical site are fixed relative to each other. Also, in an additional embodiment shown in FIG. 5B, the camera 11 may be fixed to the patient's skeletal structure, an example of which may be a camera 11 attached to a rod 20 or a similar structure pinned to the patient's ilium or other pelvic bone, or directly fastened to the patient's vertebrae. In this latter method, in the case of a surgical site with relatively little movement relative to the pelvis (e.g., the lumbar spine), the fixation of the patient to the OR table may not be as important. In a preferred embodiment, the camera 11 is equipped with an accelerometer and / or a gravitometer. These sensors may serve to detect any inadvertent movement of the camera 11 (e.g., a surgical staff member accidentally hitting the camera). If these sensors detect movement of the camera, the system may generate a warning regarding the movement of the camera that may have changed the relative fixation between the camera and the surgical site.

[0016] As described above, the anatomical structure of a patient located within the operating room is registered to the radiographic image data using a probe or registration array that can be tracked by the camera. FIG. 4 shows an intraoperative x-ray imaging system 200 (e.g., the O-arm Imaging System sold by Medtronic, Inc., Minneapolis, Minnesota) arranged to create a radiographic image of the patient's anatomical structure at the surgical site. Above the surgical site, a registration array 60 is arranged. A more detailed description of the registration array 60 will be provided later. Here, it is only necessary to note that the registration process can transform the three-dimensional radiographic image data to correlate with the three-dimensional coordinates of the corresponding patient's anatomical structure located on the OR table using a camera 11 that is fixed relative to the patient's anatomical structure and detects the registration array 60.

[0017] Generally, a surgical navigation system uses some kind of surgical instrument alignment guide. The alignment guide is often a cylindrical tube or sleeve through which surgical instruments, such as awls, driver / screw assemblies, etc., are directed. Looking at FIG. 5A, another aspect of the present invention is a system and method (and related software) for placing an alignment guide over a surgical field as a preparation for using the alignment guide with a surgical instrument. As presented in FIG. 5A, the instrument guide 37 is attached to a targeting platform 26 that is part of the entire path system 25. A robotic arm 27 extends from the targeting platform 26 and is coupled to the instrument guide 37. Using the robotic arm 27, the targeting platform 26 can expand and contract the instrument guide 37 by a short distance (a few centimeters) and also rotate it. The targeting platform 26 is fixed to the OR table by a series of adjustable positioning arms similar to those that support the camera 11 as described above. An instrument guide tracking array 40 (sometimes referred to as the "first marker array") is fixed to the instrument guide 37, enabling the camera 11 to track the position and orientation of the instrument guide 37. In the illustrated embodiment, the targeting platform 26 may be the Micromate® personal robotic assistance system sold by iSYS Medizintechnik, located in Kirchbichl, Austria (commonly known as Interventional Systems).

[0018] In order to properly position the instrument guide 37, it is necessary to align the instrument guide with the intended path of the surgical instrument or implant component (e.g., pedicle screw). Since the targeting platform 26 can only make relatively small adjustments to the instrument guide 37, the surgical staff must manually position the targeting platform 26 such that the instrument guide 37 is placed relatively close (within a few centimeters) to the intended path. Thereafter, the approximate position of this targeting platform can be fixed in place by locking hinges and locking sleeves on an adjustable positioning arm that supports the targeting platform. FIG. 5A also shows a control unit 35 that can be used to wirelessly control the various functions of the targeting platform 26 described herein.

[0019] Figures 6A-6D show a novel user interface for assisting the surgeon in achieving an initial gross positioning of the targeting platform 26 and the instrument guide 37. As presented in Figure 6A, the instrument guide tracking array 40 is positioned within the field of view of the camera 11. A general algorithm for controlling this user interface is presented in Figure 19. Step 301 of Figure 19 takes into account the input of a surgical plan by software that enables the surgeon to overlay the path of the implant component on a radiographic image. In step 302, the computer system converts the image coordinates of the planned path into camera coordinates. Thereby, in step 303, the computer system can display a disk or "path target" representing each path of the surgical plan. Figure 6B shows the display of this path target 47 representing the position and orientation of one of the planned implant paths (e.g., the first implant component in the surgical plan). Similarly, a 3D artifact 46 representing the position and orientation of the instrument guide detected by the camera is also shown on the display. In the illustrated embodiment, the path target 47 is shown as an annulus and has an inner annulus diameter 48. The 3D artifact 46 is cylindrical in shape with a diameter close to the inner annulus diameter 48 (and thus is also referred to as the "cylinder 46"). The software tracks the movement of the instrument guide tracking array 40 in three dimensions via data from the camera 11. In response to the camera data, the software updates the image of the cylinder 46 on the display as the instrument guide moves within the field of view of the camera. This is represented by step 304 of Figure 19 where the system receives the position and orientation of the instrument guide and converts this information into camera coordinates. In step 306, the updated position of the cylinder 46 is shown on the display. If the cylinder 46 is not in position alignment with the path target 47, the system receives the next position of the instrument guide.

[0020] As shown in FIG. 6A, the instrument guide is positioned between the user and the display. The software displays and updates the position of the cylinder 46 such that the movement of the instrument guide 37 results in the same movement from the user's perspective with respect to the cylinder 46 on the display. For example, movement of the instrument guide in the left direction as seen from the user's perspective is shown on the display as movement of the cylinder 46 in the left direction. Similarly, movement of the instrument guide towards the user's direction is shown on the display as downward movement of the cylinder 46 on the display. Similarly, rotation of the instrument guide 37 is represented as rotation of the cylinder 46 in the same direction on the display.

[0021] As described above, the robotic arm of the targeting platform 26 has a limited range of mechanical “reach”, i.e., a distance within which the targeting platform itself can control the positioning of the instrument guide. As shown in FIG. 6C, when the system detects that the distance of the instrument guide from the planned path is within the range of the mechanical reach of the targeting platform, the software shows on the display an indicator to that effect (e.g., the “within reach” indicator shown in FIG. 6C). After the instrument guide and the robotic arm have been moved by the user within the range of the mechanical reach of the targeting platform, the computer takes over control and provides the targeting platform and the robotic arm with instructions that enable the robotic arm to automatically position-align the instrument guide with the planned implant path. This can be essentially achieved in accordance with the same steps 305 - 306 of FIG. 19. When the instrument guide is position-aligned with the planned path, the condition of step 306 of FIG. 19 becomes affirmative. In FIG. 6D, the cylinder 46 is shown at the center of the path target 47 and an indicator such as “position-aligned” is displayed.

[0022] Other embodiments of the present invention are systems and methods for generating a more accurate path estimate of a surgical instrument during a surgical procedure. FIG. 7 shows an instrument guide 37 and its path array 40 that are positionally aligned over a surgical site. Also shown is a user inserting a surgical instrument 55 (e.g., an awl) through the instrument guide 37. A surgical instrument tracking array 50 (also sometimes referred to as a "second tracking array" or "second marker array") is secured to the surgical instrument 55. As will be appreciated, based on the detection of the camera 11 with respect to the tracking array of the medical instrument, the navigation system software determines the orientation of the tracking array (and thus the orientation of the medical instrument), and then can display to the system an estimated path of the medical instrument superimposed on a radiographic image of the patient's anatomical structure at the surgical site (see, e.g., FIGS. 9A and 9B).

[0023] Calculation of the instrument path based on the IR marker elements 52 of the instrument tracking array 50 may be acceptable for some applications, but in many cases it is desirable to enable the most accurate instrument path. FIG. 8 shows a novel approach to estimating the instrument path, which uses the IR marker elements of the instrument tracking array 50 together with the IR marker elements 42 of the instrument guide tracking array 40. In the example of FIG. 8, the camera detects eight IR marker elements and essentially treats the two tracking arrays at the time of image capture as an eight-marker element array of an instantaneous single rigid body. Even if the instrument tracking array 50 changes its position relative to the guide tracking array 40 (e.g., as the tip of the surgical instrument moves through the instrument guide and into the patient's body), the system software calculates an updated path based on the changes in the "eight-marker element array" detected by the camera 11. In other words, the system calculates a path based on the positions of the IR marker elements of both the instrument tracking array 50 and the guide tracker array 40 in a first time frame, and then calculates an updated path based on the subsequent positions of the IR marker elements of the instrument tracking array 50 and the guide tracker array 40 in a second time frame.

[0024] As will be appreciated, the greater the number of IR marker arrays and the greater the geographical separation between IR marker elements, the more suitable it is for providing more accurate path estimation. There can be numerous variations of this technique. As an example, FIG. 8 can show that the IR marker elements 52 of the instrument tracking array 50 have a different spatial arrangement from the IR marker elements 42 of the guide tracking array 40. Thereby, the system can distinguish between the two tracking arrays and, in some cases, when a particular situation of the instrument and / or surgical procedure is given and it is likely to generate a more accurate path estimation, it is possible to give a greater weight to the IR marker elements of one tracking array than the other in path calculation.

[0025] An additional aspect of the present invention is a system and method for indicating an acceptable positional alignment of a surgical instrument. Specifically, as schematically presented in FIG. 10, the system identifies the positions of both the instrument tip 56 and the instrument rear 57. In one example, this may be done based only on the instrument tracking array 50. Since the system knows the dimensions of the instrument and how the tracking array 50 is fixed to the instrument, by determining the position of the tracking array 50, the system can calculate the positions of the instrument tip and the rear. In other examples, the lower end of the instrument guide 37 is determined (using a similar method based on the position of the guide tracking array 40), and the position of the lower end of the instrument guide is used for estimating the position of the instrument tip. As a further additional alternative, the method described in FIG. 8 can be used for determining the tip and the rear of the instrument.

[0026] Once the positions of the instrument tip and the rear portion are calculated, an estimated instrument path 58 can be calculated based on the positions of the tip and the rear portion. As presented in FIG. 10, thereafter, a comparison between the planned surgical path 45 and the estimated instrument path 58 can be made. An offset distance "d" between the planned surgical path 45 and, for example, the estimated instrument path 58 at the instrument tip 56 can be calculated to determine the variation of the instrument path from the planned path. Of course, this offset distance "d" can be calculated at the rear portion or at any position along the instrument. If the offset distance "d" exceeds a set threshold, the system presents a warning of position inconsistency on the display (see, for example, FIG. 9B). Similarly, the warning of position inconsistency may also include presenting an image of the surgical instrument in red on the display. Similarly, the image of the surgical instrument can be presented in green on the display if the offset distance is less than the threshold. In a particular exemplary embodiment, this threshold is about 2 mm or less.

[0027] FIG. 21 is a flowchart of one method for determining and showing an acceptable positional alignment between a medical instrument and a planned surgical path. In steps 320 and 321, the system determines the positions of the tip and the rear portion of the instrument guide and the surgical instrument (such as an owl) based on each tracker array and the known dimensional relationships between the instrument guide and the surgical instrument and each tracker array. In step 322, the tip and the rear portion of the instrument guide are used as two points for generating an extension line. Since the instrument guide is oriented to follow the planned surgical path, this extension line also lies along the planned surgical path. In step 323, the system finds the closest point on this straight line to the tip (and / or rear portion) of the surgical instrument. Step 324 calculates the distance between the owl tip and the closest point on the straight line. In step 325, if this distance is less than a given threshold, in step 327, the system displays the surgical instrument on the radiographic image using the closest point of the tip to the straight line. If this distance exceeds the threshold, step 326 displays a red status for the owl and the instrument guide.

[0028] As described above with reference to FIG. 4, certain embodiments of the present invention include using a registration array 60 when generating a radiation image. FIG. 11 shows one embodiment of the registration array 60 in more detail. This example of the registration array 60 is formed by a marker frame (or marker support structure) 61 extending outward from a plate portion 63. The marker frame 61 supports IR marker elements 62, and the row of openings in the plate portion 63 forms a "reference" 64, which will be described in more detail later.

[0029] FIG. 14 provides an overall overview of the software functions performed by one embodiment of the present invention. At start 250, the system displays a setup state at step 251. Generally, the next step (252) displays an instruction to acquire a radiation image. The system receives the radiation image at step 253 and automatically registers the image to the camera at step 254. Specific aspects of the registration process are described in more detail elsewhere in this specification. After the image is acquired and registered, it is displayed at step 255. Step 256 enables the surgical practitioner to activate a surgical plan and display such a plan. For example, the position of pedicle screws in the patient's anatomical structure can be determined and displayed on the radiation image as part of the planning process. Step 257 displays a guidance view (an example of which is described above) to assist in aligning the instrument guide along an appropriate path over the surgical site. Step 258 displays the instrument in relation to the radiation image as soon as the instrument enters the camera's field of view (e.g., FIG. 9A).

[0030] Before generating the radiographic image, the registration array 60 is positioned over or adjacent to the surgical site. FIG. 4 shows the registration array 60 fixed to the patient at the surgical site by tape, although other fixation means may be used if the registration array 60 can be easily removed after completion of the radiographic imaging. In many imaging / registration techniques, it is important that the registration array does not move relative to the surgical site while the radiographic image is being generated. Thus, another aspect of the invention is a method for monitoring the registration array with respect to movement during the imaging process. An example of this process is presented in the flowchart of FIG. 20. In step 310, the computer system begins monitoring, by camera 11, for movement of the registration array 60 (or specifically the IR marker elements 62 on the registration array 60). In step 311, the starting position is set equal to the current position most recently detected by camera 11. A dwell counter is set to 0. In the loop of steps 312-315, the computer system continuously (e.g., at the frame rate of the system) determines the current position of the registration array and then determines the difference (displacement or distance) between the starting position and the current position at that time. As used herein, "frame rate" means the rate at which the system can calculate the position of the tracker array and update the position of each instrument on the display. As long as the difference in distance in step 314 does not exceed a distance threshold, in step 315 the dwell counter is incremented and the loop continues. This distance threshold can vary, but in one example the distance threshold is 1 mm.

[0031] If the distance exceeds the distance threshold, step 316 determines whether the dwell counter exceeds the counter threshold at step 316. If this condition is not met, the starting position is reset to the current position at that time. If this condition is met, the system displays an array operation error indicator. The counter threshold represents a given amount of time while the distance threshold has not been exceeded (expressed in terms of the number of loops in which the dwell counter is incremented). Basically, it is the time during which the registration array has not moved to a recognizable extent, i.e., to an extent that would result in an unacceptable error in the registration process. Of course, rather than counting software loops, the counter threshold could simply be a specified amount of time, and the dwell counter could be the elapsed time. In fact, the algorithm of FIG. 20 does not trigger an error or warning if there are a series of minute changes in the position of the registration array, for example, if the registration array is constantly moving within the camera's field of view while the OR personnel are fixing the registration array over the surgical site. However, if the registration array moves after the remaining stationary period, this movement is likely to be accidental (e.g., during the radiation imaging process), and an error notification is generated. In one embodiment, the threshold time in step 316 is set to 3 seconds, but in other embodiments, the threshold time could be as little as 1 second.

[0032] FIG. 12 presents an example of an error message that can be generated when the system detects movement of the registration array during the generation of a radiation image. Similar error messages can be generated if the camera fails to detect the array within its field of view or if the system fails to detect the plate of the registration array within the radiation image.

[0033] A further aspect of the present invention relates to the structure of the registration array 60 shown in FIG. 11. The plate portion 63 is generally rectangular and can be seen to have a series of "reference points" 64 formed in the plate portion. In FIG. 11, the reference points are openings or acute-angled depressions formed in the plate, but may also be protrusions, i.e., ridges or spheres attached to the plate portion. FIG. 11 shows 12 reference points, but there may be fewer or more reference points 64, and many embodiments have at least 6 reference points. The marker support structure 61 extends upwardly at or near one of the short sides of the rectangular plate portion 63 (e.g., within 20% of the full length of the plate). The marker support structure 63 is in a plane that is substantially perpendicular (e.g., within the range of perpendicular to + / - 30°) to the plane containing the plate portion 63. A preferred embodiment of the marker support structure includes at least 4 spherical IR marker elements 62.

[0034] FIG. 13A presents a registration array 60 used in conjunction with the capture of a 3D radiation image having an image volume 95. In this embodiment, the length of the plate portion 63 must generally match the length of the image volume 95, and the length of the plate portion 63 is 75% or more of the length of the image volume. In many embodiments, the plate portion 63 has a length of 150 mm to 300 mm and a width of 50 mm to 150 mm.

[0035] As shown in FIG. 13A, when the image volume 95 of the radiographic image includes the plate portion 63, the reference 64 can be used to assist in registering the radiographic image with respect to the 3D space visible by the camera 11. However, before registering the image, the reference needs to be identified within the 3D image data by the software running on the computer system. As presented in the flowchart of FIG. 15, at step 260, the image volume is received, and then at step 261, the volume is cropped to include the posterior portion (e.g., half) of the volume where the registration plate is expected to be found. By cropping this volume (which may be referred to as the "first" image volume in some cases), the amount of data processed when searching for the plate is reduced, thus reducing the processing time. At step 262, an initial threshold is set for the image features that are likely to distinguish the image elements related to the plate portion 63 from other image elements (e.g., the bone structure shown in FIG. 13A). In one embodiment, the image feature for calibrating the threshold is grayscale, i.e., the image features related to the plate are likely to have different grayscale values from other image elements. Steps 263 and 264 determine whether the plate can be identified from other portions of the cropped image volume. If not, the threshold is increased (e.g., the grayscale value is increased), and other attempts are made to find the plate. Therefore, the process for finding the registration plate includes the step of repeatedly increasing the image threshold.

[0036] An example of an algorithm for finding the plate at step 263 is shown in FIG. 16. Step 273 of FIG. 16 starts with the first cropped volume, and at step 274, connected component analysis is performed to identify all the objects within the volume. Next, at step 275, the geometric features of each object, including size, shape, and volume, are determined. Finally, at step 276, the object that best matches the geometric features of the plate is selected as the result.

[0037] Return to FIG. 15 and step 264. If a plate is found, step 266 crops a "second" image volume (e.g., approximately 120% of the plate volume) associated with the identified plate as preparation for searching for a reference within the plate. In step 267, the program begins searching for the reference using, as an example, the algorithm shown in FIG. 17. The second image volume in step 280 undergoes dilation, erosion, and masking in steps 281, 282, and 283, which are initial processing steps. As presented in FIG. 13B, the initial version of the second image volume (showing the reference opening) is dilated to produce an image showing a mostly filled reference opening. Next, erosion erases any visual traces of the reference opening. Thereafter, the initial image is used as a mask for the image processed by erosion to create a final representation that constitutes a mirror image of the reference opening, i.e., a series of small cylindrical shapes replacing the reference opening. Next, in FIG. 17, step 284 uses connected component analysis to find all objects in this latest reference image (e.g., here the cylindrical shapes represent the reference). Step 285 calculates the geometric shapes of all objects found in step 284, and step 286 removes (rejects outliers) objects that do not match the expected cylindrical size and the shape of the reference using geometric features such as elongation and volume. In step 287, the inter-point distance (IDP) is calculated for the other objects, and the positions of these objects are compared to the known grid spacing of the reference in the original plate to calculate the IDP error for each object. In step 288, all objects that do not correspond to the grid spacing (i.e., have an unacceptable IDP error) are removed as outliers.

[0038] Finally, in step 289, random sample consensus (RANSAC) analysis is used to identify and remove all objects that are not in the same plane as most of the references (i.e., the estimated plane of the plate).

[0039] FIG. 18 presents an example of RANSAC analysis. In step 291, a set of points representing the objects existing after the IDP error analysis is established. In step 292, three points are randomly selected to define a plane, and then step 293 calculates the distance from all points to the plane. Step 294 removes the points that are not on or near the plane. In step 295, if the number of remaining points is below a given minimum value (i.e., the expected number of criteria), the process is restarted and a new plane is defined. If the number of remaining points is greater than the minimum value, in step 296, the remaining points are estimated to correspond to the criteria. Returning to FIG. 15, in step 269, the position of the criteria is classified so as to correspond to the reference model within the marker array coordinates (i.e., the coordinates determined by a camera that has the marker elements extending on the registration plate in its field of view). In other words, since the system knows the positional relationship between the criteria and the IR marker elements on the registration array 60, the position of the criteria in the coordinate system of the camera can be generated. Finally, in step 270 of FIG. 15, a reference registration for converting points from camera coordinates to image coordinates is calculated.

[0040] The term "about" generally means a numerical value that approximates and a numerical value with small variations that do not have a significant impact on the implementation of the disclosed embodiments. When numerical limitations are used, unless otherwise indicated by the context, "about" means that the numerical value can vary by + / −5%, + / −10%, or in certain embodiments + / −15%, and in some cases + / −20%. Similarly, "substantially" generally means at least 85% to 99% of the feature modified by this term. For example, "substantially all" means 85% or more, 90% or more, or 95% or more, etc.

[0041] Although preferred embodiments of the present disclosure have been described, the described embodiments are merely exemplary, and it should be understood that the scope of the present invention should be defined only by the appended claims that recognize all equivalents within the full scope with numerous variations and modifications that would occur to those skilled in the art upon reading this specification.

Claims

1. 1. A surgical navigation system for indicating alignment of a surgical instrument, the surgical navigation system comprising: (a) a computer and a display; (b) data representing a 3D radiological image of a surgical site on a patient; and (c) an instrument guide having a first marker array having a plurality of IR marker elements; (d) the surgical instrument having a second marker array having a plurality of IR marker elements; and (e) an optical sensor configured to detect markers on the first marker array and the second marker array; and Equipped with (f) the computer includes a processor and memory storing software that, when executed, causes the computer to: (i) receiving optical sensor data relating to positions of the first marker array and the second marker array and generating on the display a graphical representation of a path on the 3D radiological image based on the optical sensor data; (ii) calculating a path based on positions of IR marker elements of both the first marker array and the second marker array during a first time frame; (iii) calculating an updated path based on subsequent positions of IR marker elements of both the first marker array and the second marker array during a second time frame; This is a surgical navigation system.

2. 2. The surgical navigation system of claim 1, wherein when the instrument guide is positioned between a user and the display, (i) movement of the instrument guide to the left from the user's perspective is shown on the display as movement of the three-dimensional artifact to the left, and (ii) movement of the instrument guide towards the user is shown on the display as movement of the three-dimensional artifact downward.

3. The surgical navigation system of claim 1 , further comprising a robotic arm having a mechanical reach controllable by the computer.

4. The software, when executed, causes the computer to: (i) indicating on the display a path landmark representing a position and an orientation of the planned implant path; (ii) presenting on the display a three-dimensional artifact representative of the position and orientation of the instrument guide; (iii) tracking three-dimensional movement of the first marker array via data from the optical sensor; (iv) updating the three-dimensional image on the display in response to the optical sensor data as the instrument guide moves within the field of view of the optical sensor; (v) indicating the three-dimensional artifact as being aligned with the path target when the instrument guide is aligned with the intended implant path. The surgical navigation system of claim 3 .

5. 5. The surgical navigation system of claim 4, wherein the computer provides instructions to the robotic arm as the robotic arm is moved by the user within a mechanical reach of the path target, enabling the robotic arm to align the instrument guide with the planned implant path.

6. The surgical navigation system of claim 5 , wherein the computer shows an indicator on the display when the user moves the robotic arm within the mechanical reach of the path target.

7. The surgical navigation system of claim 6 , wherein the path target is annular in shape having an inner diameter, and the three-dimensional artifact is cylindrical in shape having a diameter approximating the inner diameter of the annulus.

8. The surgical navigation system of claim 6 , wherein a rotation of the instrument guide is represented on the display by a corresponding rotation of the cylindrical three-dimensional artifact.

9. 1. A system including a computer readable storage device for storing instructions and / or data, the instructions and / or data, when executed on a computer system, causing the computer system to: (a) receiving a 3D radiological image of a surgical site including a registration array, the registration array including (i) a plate having at least six fiducials formed thereon, and (ii) a marker support structure associated with a plane including the plate for positioning a marker relative to the plane of the plate; (b) identifying coordinates of fiducials within the 3D radiological image; (c) receiving an image of the registration array from an optical sensor; and (d) registering fiducials of the registration array detected by the optical sensor to coordinates of fiducials in the 3D image; Configure the system to:

10. The criteria are: (i) cropping a first image volume of the 3D image in which the registration plate is expected to be located; (ii) detecting the registration plate in the first image volume and cropping a second image volume that includes the registration plate; (iii) detecting fiducials within the second image volume and classifying the fiducials; The system of claim 9 , wherein the detection is performed by a method comprising:

11. The system of claim 10 , wherein detecting the registration plate comprises iteratively increasing an image threshold.

12. The system of claim 11 , wherein the image threshold is a grayscale.

13. The step of detecting the registration plate includes: (i) performing a connected component analysis on the second image volume; (ii) determining geometric features in the second image volume; and (iii) identifying an object that matches a geometric feature of the registration plate; The system of claim 9 , comprising:

14. The step of detecting the reference comprises: (i) dilating the second image volume to form a dilated image; (ii) eroding the dilated image to form an eroded image that closes the fiducial opening; (iii) masking the eroded image with an image from the second image volume to generate the reference image; The system of claim 10 , comprising:

15. The step of detecting the reference comprises: (i) performing connected component analysis on the reference image; (ii) computing a set of geometric features from the reference image; (iii) removing outliers from the set of geometric features; (iv) calculating an inter-point distance (IPD) error for each object; (v) removing outliers using the IPD errors; (iv) removing outliers using RANSAC; The system of claim 14 , comprising:

Images