Surgical Navigation System

Abstract

To provide a surgical navigation method using a computer system.SOLUTION: A method includes steps of: (a) fixing a camera 11 to a surgical site of a patient so as to prevent the camera from moving with respect to the surgical site; (b) restricting a body portion of the patient including the surgical site with respect to a surgical bed supporting the patient so as to prevent the body portion from moving with respect to the surgical bed; and (c) detecting a position of a medical device including a tracker array with the camera.SELECTED DRAWING: Figure 2A

Description

[Background technology]

[0001] In the field of image-guided surgery or surgical navigation, a camera (often referred to as a "localization camera") in an operating room tracks the position of a surgical tool in three-dimensional space. This position information is transferred from the localization camera to a computer. The computer monitor displays a multiplanar three-dimensional radiographic image of the patient's anatomical structures relevant to the surgical procedure, which has been uploaded to the computer. The patient's anatomical structures located in the operating room are registered to the radiographic image data using a probe or registration array tracked by the localization camera. In image-guided surgery, registration is the process of transforming a three-dimensional radiographic image dataset (image space) to correlate with the three-dimensional coordinates of the corresponding patient's anatomical structures (operative space). After registering the relevant anatomical structures, a navigation system 100, such as that shown in FIG. 1, can present the position of the tracked surgical tool relative to the anatomical structures in the radiographic image displayed on a display 102. The tracked surgical tool or instrument has its own tracking array 103 with IR marker elements, allowing the system to detect and track the surgical instrument via the localization camera 101. For this process to be accurate, the three-dimensional spatial relationship between the localization camera and the patient's anatomy must be known and maintained. If the localization camera moves, or if the patient's anatomy moves during surgery, the accuracy of the navigation system will decrease. To compensate for this problem, a tracked tool known as a dynamic reference frame 104 is fixed in relationship to the patient's anatomy. As the patient's anatomy and the localization camera move relative to each other, their three-dimensional relationship is recalculated by the navigation system computer, and the registration solution is updated. Typically, the dynamic reference frame is fixed to a clamp that is fixed to the patient's anatomy near the surgical site 105.

[0002] While the above measures can help maintain the accuracy of image-guided surgery, they have drawbacks. To track the dynamic reference frame, the localization camera measures the spatial coordinates of markers on the reference frame 104 in the x-, y-, and z-axes. The accuracy of these measurements varies depending on the viewing angle between the camera and the markers. It is common practice to move the camera to maintain a line of sight for locating the dynamic reference frame to accommodate objects such as a microscope, various instruments, and additional operating room personnel entering the surgical field. Similarly, the patient is often repositioned during surgery, such as when the operating table is rotated or raised or lowered. In such cases, movement of the camera relative to the markers on the dynamic reference frame can unintentionally induce navigation errors.

[0003] There remains a need for instruments and techniques that minimize or eliminate relative motion between the camera and the dynamic reference frame during image-guided surgery. Summary of the Invention

[0004] One embodiment of the present invention is a method for surgical navigation using a computer system, the method including: (a) fixing a camera relative to a surgical site on a patient to prevent the camera from moving relative to the surgical site, (b) restraining a body part of the patient including the surgical site to an operating table supporting the patient to prevent the body part from moving relative to the operating table, and (c) detecting, with the camera, a position of a medical instrument having a tracker array.

[0005] Another embodiment includes placing a registration array over the surgical site and using the registration array to generate 3D radiographic images that are registered to a camera. The registration array is removed from the patient, and the camera then tracks the position of the surgical instrument during the surgical procedure.

[0006] An additional embodiment of the present invention is an OR table including at least two opposing inwardly and downwardly sloping pelvic support pads. The pelvic support pads have at least three foam layers with different densities, with the patient boundary layer being the lowest density of the at least three layers and the bottom layer being the highest density layer. A fixation sheet is used to fix the patient to the pelvic support pads.

[0007] Additional embodiments are described in the figures and detailed description that follow. [Brief explanation of the drawings]

[0008] [Figure 1] 1 shows a prior art surgical navigation system. [Figure 2A] 1 illustrates a workstation including hardware for the surgical navigation system of the present invention. [Figure 2B] FIG. 1 shows a diagram of a computer system that executes the software necessary to operate the surgical navigation system. [Figure 3A] 1 illustrates a patient support frame for an OR table according to the present invention. [Figure 3B] A cross-sectional view of a patient's buttocks resting on a pelvic support pad on an OR table is shown. [Figure 4] 1 shows a surgical navigation system used during the creation of radiological images of a surgical site. [Figure 5A] 1 shows an instrument guide positioned over a surgical site. [Figure 5B] 1 shows how the camera is attached to the patient's skeletal structure. [Figure 6A] 1 illustrates one position alignment user interface method for a surgical navigation system. [Figure 6B] 13 illustrates the position alignment user interface method when the instrument is not aligned. [Figure 6C] 13 illustrates the position alignment user interface method with the instrument in reach. [Figure 6D] 13 illustrates the position alignment user interface method with the instrument position aligned. [Figure 7] 1 illustrates a surgical instrument being used with an instrument guide. [Figure 8] 1 illustrates conceptually how multiple marker arrays can be used to determine alignment. [Figure 9A] 1 shows an image of proper alignment generated by the surgical navigation system. [Figure 9B] 1 shows an image of a registration error generated by a surgical navigation system. [Figure 10] 1 conceptually illustrates alignment of a surgical instrument with a surgical guide at a surgical site. [Figure 11] 1 shows one marker array of the present invention. [Figure 12] 10 shows a system user interface showing marker array position errors. [Figure 13A] 1 conceptually illustrates a marker array within a radiological image volume. [Figure 13B] 10 conceptually illustrates a software method for identifying marker array elements. [Figure 14] 1 shows a flowchart of initial navigation image acquisition and registration. [Figure 15] 1 shows a flowchart of an automatic registration algorithm. [Figure 16] 1 shows a flowchart of an algorithm for locating a registration plate. [Figure 17] 1 shows a flowchart of an algorithm for identifying fiducials in a registration plate. [Figure 18] 1 shows a flowchart of an algorithm for removing points that are not in the plane of the plate. [Figure 19] 10 shows a flow chart of steps for determining instrument guide alignment. [Figure 20] 10 shows a flow chart of steps for determining inadvertent marker array movement. [Figure 21] 10 shows a flowchart of steps for determining instrument path alignment. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of components that may be used in a navigation system 1 of the present invention is presented in Figure 2A. These components include a system station or mobile cart 2. Station 2 is equipped with a display 5 and user interface components, such as a conventional keyboard 6 or mouse 7. Figure 2 also shows an optical sensor 10 (e.g., camera 11) and a path system 25 removably mounted on station 2, both of which are described in more detail below.

[0010] The navigation system includes a computer system 220, shown as a system diagram in FIG. 2B. The computer system 220 may include a memory 222, a processor 224, the display 5 described above, a network interface 228, and an input device 230. The memory 222 may store an application 236 and / or CT data 234. The application 236, when executed by the processor 224, may cause the display 5 to present a user interface 238. The processor 224 may be a general-purpose processor, a dedicated graphics processing unit (GPU) configured to perform specific graphics processing tasks while freeing the general-purpose processor to perform other tasks, and / or any number or combination of such processors. The keyboard 6 and mouse 7 serve as data input devices. Alternatively, the display 5 may be touch-sensitive and / or voice-activated, allowing the display 5 to function as both an input and output device. The network interface 228 may be configured to connect to a network via a wired or wireless connection. Through the network interface 228, the computing system 220 receives data from and sends commands to associated devices, such as the camera 11 and the pathway system 25. The computing system 220 may also receive patient image data, such as computed tomography (CT) image data, from a server, such as a hospital server, an internet server, or other similar server, for use during surgical planning and execution. The application 236 may be one or more software programs stored in the memory 222 and executed by the processor 224. The software programs perform computer-related functions, described below, including generating navigation images and various user interfaces.

[0011] While many aspects of the present invention relate to software manipulation of medical images, one embodiment of the present invention relates to improvements to OR tables. For reasons that will become apparent, one important feature of certain embodiments of the present invention is the immobilization of the anatomical structures of a patient undergoing surgery relative to the OR table. Figures 3A and 3B present an OR table modified to provide a stabilizing surface that helps to immobilize the patient during a surgical procedure. Figure 3A shows the top of a rail-type OR table 70 (e.g., an Allen Advanced or Jackson radiolucent OR table) having a series of patient support surfaces or pads, such as leg support pad 71, pelvic support pad 75, and head support pad 73. The support pads may sometimes be referred to as "table adapters." Figure 3B shows a cross-section of the pelvic or hip structure of a patient in a prone position resting on pelvic support pad 75, the structure of which is also shown. This embodiment of the pelvic support pad 75 includes at least three foam layers, each with a different density or stiffness, with a top layer (or "patient boundary layer") 76 being the least stiff layer and a bottom layer 78 being the most stiff layer. The bottom layer 78 is secured (e.g., by adhesive) to a very stiff backing layer 79, which is attached to a support bracket 74 that is connected to the rails of the OR table.

[0012] In one example embodiment, the top layer 76 has a stiffness or hardness of 30-40 on the Shore A scale, the middle layer 77 has a hardness of 50-70, the bottom layer 78 has a hardness of 70-90, and the backing layer 79 has an even higher hardness. Generally, the layer thicknesses range from 0.5 centimeters to 10 centimeters, preferably 1-5 centimeters. In certain embodiments, the layers increase in thickness from the top layer 76 to the bottom layer 78. In a preferred embodiment, the layers are formed of a viscoelastic polyurethane foam material.

[0013] Figures 4 and 5 show a patient in a prone position on an OR table 70. In this position, it can be seen that two opposing, inwardly and downwardly sloping pelvic support pads 75 serve to support the patient's pelvic or buttock structures and stabilize the patient's position with the patient's weight. Figure 4 also shows how a series of patient fixation sheets 85 extend over the patient and are secured to the OR table on either side to further immobilize the patient's pelvic and thoracic spine regions. Typically, fixation sheets are breathable, stretchable fabric sheets or bands 4 to 18 inches wide to distribute forces on the patient's body and avoid restricting blood circulation. In one example, the fixation sheets may be iFIX Fleece strips sold by Interventional Systems USA, Stamford, Connecticut. However, the fixation sheets may alternatively be flexible stretch bandages, custom-designed belts with sufficient padding, or virtually any means for safely and securely securing the relevant patient anatomy to the OR table without interfering with the surgical process.

[0014] FIG. 5A allows visualization of one method of the present invention, which involves using a patient secured to an OR table to secure an optical sensor (e.g., camera 11 in FIG. 5A) so that camera 11 does not move relative to the surgical field of view 90. In the embodiment of FIG. 5A, this relative fixation is achieved by securing the patient to the OR table as described above and then securing camera 11 to the OR table by any conventional means. FIG. 5A shows bed mounting bracket 13 that grips the rails of the OR table and multiple positioning arm segments 14A-14C that connect mounting bracket 13 to camera 11. Locking sleeve 16A may be loosened to allow positioning arm segment 14B to rotate relative to arm segment 14B, after which locking sleeve 16A is tightened to lock the relative position of the arm segments. Arm segment 14B may also extend telescopically from arm segment 14A, with locking sleeve 16A also securing this telescopic relationship. Similarly, locking hinge 15 allows positioning arm segment 14C to rotate relative to 14B and then lock in the desired relative position. Locking sleeve 16B allows camera 11 to rotate relative to arm segment 14C and then lock in place. Using this mechanical structure, camera 11 is positioned so as not to interfere with the surgical procedure and to maintain within its field of view any tracking arrays used during the surgical procedure (e.g., tracking array 40 in FIG. 5A, the function of which is explained in more detail below). In the illustrated embodiment, camera 11 may be a monocular localization camera sold by Intellijoint Surgical, Inc. of Kitchener, Canada.

[0015] While FIG. 5A illustrates the relative fixation of the surgical site and camera by securing the patient and camera to the OR table, this relative fixation may be ensured by other methods. For example, if the OR table is firmly secured to the operating room floor, the camera may likewise be secured to the OR floor. If the patient is immobile relative to the OR table, the camera and surgical site are fixed relative to one another. Also shown in FIG. 5B, in an additional embodiment, the camera 11 may be fixed to the patient's skeletal structure, such as by a rod 20 or similar structure pinned to the patient's hip bone or other pelvic bone or fastened directly to the patient's vertebrae. In this latter approach, for a surgical site that is relatively motionless relative to the pelvis (e.g., the lumbar spine), fixation of the patient to the OR table may be less important. In a preferred embodiment, the camera 11 is equipped with an accelerometer and / or a gravitational sensor. These sensors may serve to detect any inadvertent movement of the camera 11 (e.g., a surgical technician accidentally bumping into the camera). If these sensors detect camera movement, the system may generate a warning about the camera movement that may have changed the relative fixation of the camera to the surgical site.

[0016] As mentioned above, the patient's anatomy located in the operating room is registered to the radiological image data using a probe or registration array that can be tracked by a camera. Figure 4 shows an intraoperative x-ray imaging system 200 (e.g., an O-arm Imaging System sold by Medtronic, Inc., Minneapolis, Minnesota) positioned to produce radiological images of the patient's anatomy at the surgical site. A registration array 60 is positioned above the surgical site. A more detailed description of the registration array 60 is provided below. For now, it is sufficient to note that the registration process can transform the three-dimensional radiological image data to correlate with the three-dimensional coordinates of the corresponding patient's anatomy located on the OR table, using a camera 11 that is fixed relative to the patient's anatomy and detects the registration array 60.

[0017] Surgical navigation systems typically use some type of surgical instrument alignment guide. Alignment guides are often cylindrical tubes or sleeves through which surgical instruments, such as awls or driver / screw assemblies, are directed. Looking at FIG. 5A, another aspect of the present invention is a system and method (and associated software) for positioning an alignment guide over the surgical field in preparation for using the alignment guide with a surgical instrument. As shown in FIG. 5A, instrument guide 37 is attached to targeting platform 26, which is part of the overall pathway system 25. A robotic arm 27 extends from targeting platform 26 and couples to instrument guide 37. Using robotic arm 27, targeting platform 26 can extend, retract, and rotate instrument guide 37 over short distances (a few centimeters). Targeting platform 26 is secured to the OR table by a series of adjustable positioning arms similar to those supporting camera 11, as described above. An instrument guide tracking array 40 (sometimes referred to as the "first marker array") is fixed to the instrument guide 37, allowing the camera 11 to track the position and orientation of the instrument guide 37. In the illustrated embodiment, the targeting platform 26 may be a Micromate® personal robotic assistance system sold by iSYS Medizintechnik GmbH (d / b / a Interventional Systems) of Kitzbuhel, Austria.

[0018] To properly position the instrument guide 37, it must be aligned with the intended path of the surgical instrument or implant component (e.g., pedicle screw). Because the targeting platform 26 allows for only relatively small adjustments to the instrument guide 37, surgical personnel must manually position the targeting platform 26 so that the instrument guide 37 is positioned relatively close (within a few centimeters) to the intended path. This coarse location of the targeting platform can then be fixed in place by a locking hinge and locking sleeve on an adjustable positioning arm that supports the targeting platform. Figure 5A also shows a control unit 35 that can be used to wirelessly control various functions of the targeting platform 26 described herein.

[0019] 6A-6D illustrate a novel user interface for assisting the surgical personnel in achieving this initial rough positioning of the targeting platform 26 and instrument guide 37. As shown in FIG. 6A, the instrument guide tracking array 40 is located within the field of view of the camera 11. The general algorithm controlling this user interface is presented in FIG. 19. Step 301 of FIG. 19 considers input of a surgical plan through software that allows the surgical personnel to overlay implant component paths on the radiographic image. In step 302, the computer system converts the image coordinates of the planned path into camera coordinates. This allows the computer system to display, in step 303, disks or "path targets" representing each path in the surgical plan. FIG. 6B shows a representation of this path target 47, which represents the position and orientation of one of the planned implant paths (e.g., the first implant component in the surgical plan). Also shown on the display is a three-dimensional artifact 46 representing the position and orientation of the instrument guide detected by the camera. In the illustrated embodiment, the path target 47 is shown as a ring shape with an inner ring diameter 48. The three-dimensional artifact 46 is cylindrical in shape (hence also referred to as "cylinder 46") with a diameter approximating the inner annulus diameter 48. The software tracks the movement of the instrument guide tracking array 40 in three dimensions via data from the camera 11. In response to the camera data, the software updates the image of the cylinder 46 on the display as the instrument guide moves within the camera's field of view. This is represented in step 304 of FIG. 19, where the system receives the position and orientation of the instrument guide and converts this information to camera coordinates. In step 306, the updated position of the cylinder 46 is shown on the display. If the cylinder 46 is not aligned with the path target 47, the system receives the next position of the instrument guide.

[0020] As presented in Figure 6A, the instrument guide is positioned between the user and the display. The software displays and updates the position of the cylinder 46 so that movement of the instrument guide 37 results in the same movement of the user's viewpoint relative to the cylinder 46 on the display. For example, movement of the instrument guide to the left from the user's viewpoint is shown on the display as movement of the cylinder 46 to the left. Similarly, movement of the instrument guide toward the user is shown on the display as movement of the cylinder 46 downward on the display. Similarly, rotation of the instrument guide 37 is represented as a rotation of the cylinder 46 on the display in the same direction.

[0021] As described above, the robotic arm of the targeting platform 26 has a limited mechanical "reach," i.e., the distance over which the targeting platform itself can control the positioning of the instrument guide. As shown in FIG. 6C, when the system detects that the instrument guide's distance from the intended path is within the targeting platform's mechanical reach, the software shows an indicator to that effect on the display (e.g., the "within reach" indicator shown in FIG. 6C). After the user moves the instrument guide and robotic arm within the targeting platform's mechanical reach, the computer assumes control and provides instructions to the targeting platform and robotic arm that enable the robotic arm to automatically align the instrument guide with the intended implant path. This can be achieved essentially by following the same steps 305-306 of FIG. 19. If the instrument guide is aligned with the intended path, the condition at step 306 of FIG. 19 becomes positive. In FIG. 6D, a cylinder 46 is shown centered at a path target 47, and an indicator such as "aligned" is displayed.

[0022] Another embodiment of the present invention is a system and method for generating more accurate path estimates for a surgical instrument during a surgical procedure. FIG. 7 shows an instrument guide 37 and its path array 40 aligned over a surgical site. Also shown is a user inserting a surgical instrument 55 (e.g., an awl) through the instrument guide 37. A surgical instrument tracking array 50 (sometimes referred to as a "second tracking array" or "second marker array") is affixed to the surgical instrument 55. As will be appreciated, based on camera 11 detection of the medical instrument's tracking array, the navigation system software can determine the orientation of the tracking array (and therefore the orientation of the medical instrument), and the system can then display the estimated path of the medical instrument superimposed on a radiological image of the patient's anatomy at the surgical site (see, e.g., FIGS. 9A and 9B).

[0023] While calculating the instrument path based on the IR marker elements 52 of the instrument tracking array 50 may be acceptable for some applications, it is often desirable to obtain the most accurate instrument path possible. FIG. 8 illustrates a novel approach to estimating the instrument path, which uses the IR marker elements of the instrument tracking array 50 in conjunction with the IR marker elements 42 of the instrument guide tracking array 40. In the example of FIG. 8, the camera detects eight IR marker elements, essentially treating the two tracking arrays at the time of image capture as a single, rigid, eight-marker element array. Even if the instrument tracking array 50 changes position relative to the guide tracking array 40 (e.g., as the leading end of the surgical instrument moves through the instrument guide and into the patient's body), the system software calculates an updated path based on the changes in the "eight-marker element array" detected by the camera 11. In other words, the system calculates a path based on the positions of the IR marker elements of both the instrument tracking array 50 and the guide tracker array 40 in a first time frame, and then calculates an updated path based on the subsequent positions of the IR marker elements of the instrument tracking array 50 and the guide tracker array 40 in a second time frame.

[0024] As will be appreciated, a larger number of IR marker arrays and greater geographic spacing between IR marker elements are more likely to provide a more accurate path estimate. There are many variations on this technique. As an example, FIG. 8 shows that the IR marker elements 52 of the instrument tracking array 50 have a different spatial arrangement than the IR marker elements 42 of the guide tracking array 40. This allows the system to distinguish between the two tracking arrays and, in some cases, give more weight in the path calculation to the IR marker elements of one tracking array than the other if that is more likely to produce a more accurate path estimate given the particular circumstances of the instrument and / or surgical procedure.

[0025] An additional aspect of the present invention is a system and method for indicating acceptable positional alignment of a surgical instrument. Specifically, as shown schematically in FIG. 10, the system identifies the positions of both the instrument tip 56 and the instrument tail 57. In one example, this may be done solely based on the instrument tracking array 50. Because the system knows the dimensions of the instrument and how the tracking array 50 is secured to the instrument, by determining the position of the tracking array 50, the system can calculate the positions of the instrument tip and tail. In another example, the bottom of the instrument guide 37 is determined (using a similar method based on the position of the guide tracking array 40), and the position of the bottom of the instrument guide is used to estimate the position of the instrument tip. As yet another alternative, the method described in FIG. 8 may be used to determine the instrument tip and tail.

[0026] Once the instrument tip and tail positions have been calculated, an estimated instrument path 58 can be calculated based on the tip and tail positions. A comparison can then be made between the planned surgical path 45 and the estimated instrument path 58, as presented in FIG. 10 . An offset distance “d” between the planned surgical path 45 and the estimated instrument path 58, for example at the instrument tip 56, can be calculated to determine the variation of the instrument path from the planned path. Of course, this offset distance “d” can be calculated at the tail or at any possible location along the instrument. If the offset distance “d” exceeds a set threshold, the system will present a misalignment warning on the display (see, for example, FIG. 9B ). Similarly, the misalignment warning may also include presenting an image of the surgical instrument on the display in red. Similarly, the image of the surgical instrument can be presented on the display in green if the offset distance is less than a threshold. In certain example embodiments, this threshold is approximately 2 mm or less.

[0027] FIG. 21 is a flowchart of one method for determining and displaying acceptable alignment of a medical instrument with a planned surgical path. In steps 320 and 321, the system determines the positions of the leading and trailing ends of the instrument guide and surgical instrument (e.g., awl) based on their respective tracker arrays and the known dimensional relationships between the instrument guide and surgical instrument and their respective tracker arrays. In step 322, the leading and trailing ends of the instrument guide are used as two points to generate an extension line. Because the instrument guide is oriented to follow the planned surgical path, this extension line also follows the planned surgical path. In step 323, the system finds the closest point on this line to the leading end (and / or trailing end) of the surgical instrument. Step 324 calculates the distance between the awl tip and the closest point on the line. If this distance is less than a given threshold in step 325, the system displays the surgical instrument on the radiographic image using the closest point of the tip to the line. If this distance exceeds a threshold, step 326 displays a red status for the awl and instrument guide.

[0028] As discussed above with reference to Figure 4, certain embodiments of the present invention involve the use of a registration array 60 when generating radiographic images. Figure 11 shows one embodiment of a registration array 60 in more detail. This example registration array 60 is formed of a marker frame (or marker support structure) 61 extending outward from a plate portion 63. The marker frame 61 supports IR marker elements 62, and an array of openings in the plate portion 63 form "fiducials" 64, which will be described in more detail below.

[0029] FIG. 14 provides an overall overview of the software functions performed by one embodiment of the present invention. At start 250, the system displays a setup state in step 251. Generally, the next step (252) displays instructions to acquire radiographic images. The system receives the radiographic images in step 253 and automatically registers the images to the camera in step 254. Specific aspects of the registration process are described in more detail elsewhere herein. After the images are acquired and registered, they are displayed in step 255. Step 256 allows the surgical personnel to validate the surgical plan and display such plan. For example, the location of pedicle screws in the patient's anatomy may be determined and displayed on the radiographic image as part of the planning process. Step 257 displays a guidance view (an example of which is described above) to assist in aligning the instrument guide to the appropriate path over the surgical site. Step 258 displays the instrument in relation to the radiographic image as soon as the instrument is within the camera's field of view (e.g., FIG. 9A).

[0030] Prior to generating radiographic images, the registration array 60 is positioned on or adjacent to the surgical site. While FIG. 4 shows the registration array 60 secured to the patient at the surgical site by tape, other securement means may be used as long as the registration array 60 is easily removable after radiographic imaging is complete. In many imaging / registration techniques, it is important that the registration array does not move relative to the surgical site while the radiographic images are being generated. Therefore, another aspect of the present invention is a method for monitoring the registration array for movement during the imaging process. An example of this process is presented in the flowchart of FIG. 20. In step 310, the computer system begins monitoring the registration array 60 (or specifically, the IR marker elements 62 on the registration array 60) for movement by the camera 11. In step 311, the start position is set equal to the current position most recently detected by the camera 11. A dwell counter is set to 0. In the loop of steps 312-315, the computer system continuously (e.g., at the system's frame rate) determines the current position of the registration array and then determines the difference (displacement or distance) between the starting position and the current position. As used herein, "frame rate" refers to the rate at which the system can calculate the position of the tracker array and update the position of each instrument on the display. As long as the difference in distance in step 314 does not exceed the distance threshold, the dwell counter is incremented in step 315 and the loop continues. This distance threshold can vary, but in one example, the distance threshold is 1 mm.

[0031] If the distance exceeds the distance threshold, step 316 determines whether the dwell counter exceeds a counter threshold in step 316. If this condition is not met, the starting position is reset to the current position at that time. If this condition is met, the system displays an array movement error indicator. The counter threshold represents a given amount of time (expressed in terms of the number of loops the dwell counter is incremented) during which the distance threshold is not exceeded. Essentially, it is the time during which the registration array has not moved appreciably, i.e., to an extent that would introduce unacceptable error into the registration process. Of course, rather than counting software loops, the counter threshold could simply be a specified amount of time, and the dwell counter could be the elapsed time. In fact, the algorithm of FIG. 20 will not trigger an error or warning if there are a series of small changes in the position of the registration array, for example, if the registration array is constantly moving within the camera's field of view while OR personnel secure it over the surgical site. However, if the registration array moves after the remaining stationary period, it is likely that this movement was inadvertent (e.g., during the radiographic imaging process) and an error notification is generated. In one embodiment, the threshold time in step 316 is set to 3 seconds, although in other embodiments the threshold time may be as little as 1 second.

[0032] Figure 12 provides an example of an error message that may be generated if the system detects movement of the registration array during generation of a radiographic image. A similar error message may be generated if the camera fails to detect the array within its field of view or if the system fails to detect the registration array plate within the radiographic image.

[0033] Yet another aspect of the present invention relates to the structure of a registration array 60, shown in FIG. 11. It can be seen that the plate portion 63 is generally rectangular and has a series of "fiducials" 64 formed therein. In FIG. 11, the fiducials are openings or sharply angled depressions formed in the plate, but they may also be protrusions, i.e., bumps, or spheres attached to the plate portion. While FIG. 11 shows 12 fiducials, there may be fewer or more fiducials 64, and many embodiments have at least six. The marker support structure 61 extends upwardly from or near one of the short sides of the rectangular plate portion 63 (e.g., within 20% of the total length of the plate). The marker support structure 63 lies in a plane generally perpendicular to the plane containing the plate portion 63 (e.g., within + / - 30° of perpendicular). A preferred embodiment of the marker support structure includes at least four spherical IR marker elements 62.

[0034] 13A shows a registration array 60 for use in conjunction with the capture of a 3D radiographic image having an image volume 95. In this embodiment, the length of the plate portion 63 should generally match the length of the image volume 95, with the length of the plate portion 63 being 75% or greater of the length of the image volume. In many embodiments, the plate portion 63 has a length of 150 mm to 300 mm and a width of 50 mm to 150 mm.

[0035] If the image volume 95 of a radiographic image includes a plate portion 63, as shown in FIG. 13A, fiducials 64 may be used to assist in registering the radiographic image to the 3D space viewed by the camera 11. However, before the image can be registered, fiducials must be identified within the 3D image data by software running on a computer system. As shown in the flowchart of FIG. 15, an image volume is received in step 260, and then in step 261, the volume is cropped to include a posterior portion (e.g., half) of the volume where the registration plate is expected to be found. Cropping this volume (sometimes referred to as the "first" image volume) reduces the amount of data to be processed in searching for the plate, thereby reducing processing time. In step 262, an initial threshold is established for image features that are likely to distinguish image elements related to the plate portion 63 from other image elements (e.g., the bone structure shown in FIG. 13A). In one embodiment, the image feature used to calibrate the threshold is grayscale; that is, image features related to the plate are likely to have a different grayscale value than other image elements. Steps 263 and 264 determine whether the plate can be identified from other portions of the cropped image volume. If not, the threshold is increased (e.g., the grayscale value is increased) and another attempt is made to find the plate. Thus, the process for finding the registration plate involves iteratively increasing the image threshold.

[0036] An example of an algorithm for finding the plate in step 263 is shown in Figure 16. Step 273 in Figure 16 starts with a first cropped volume, and in step 274, a connected component analysis is performed to identify all objects within the volume. Next, in step 275, the geometric characteristics of each object are determined, including size, shape, and volume. Finally, in step 276, the object that best matches the geometric characteristics of the plate is selected as the result.

[0037] Returning to FIG. 15 and step 264, if a plate is found, step 266 crops a "second" image volume associated with the identified plate (e.g., approximately 120% of the plate volume) in preparation for searching for fiducials within the plate. In step 267, the program begins searching for fiducials, using, for example, the algorithm shown in FIG. 17. The second image volume in step 280 undergoes initial processing steps of dilation, erosion, and masking in steps 281, 282, and 283. As shown in FIG. 13B, an initial version of the second image volume (showing the fiducial opening) is dilated to generate an image showing a mostly filled fiducial opening. Erosion then removes any visible trace of the fiducial opening. The initial image is then used as a mask for the eroded image to create a final representation consisting of a mirror image of the fiducial opening, i.e., a series of small cylinders that replace the fiducial opening. Next, referring to FIG. 17, step 284 uses connected component analysis to find all objects (e.g., here the cylinders represent the fiducials) in this latest fiducial image. Step 285 calculates the geometry of all objects found in step 284, and step 286 uses geometric characteristics, such as elongation and volume, to remove objects that do not match the expected cylindrical size and shape of the reference (removing outliers). In step 287, inter-point distances (IDPs) are calculated for the remaining objects, and the positions of these objects are compared to the known grid spacing of the reference in the original plate to calculate an IDP error for each object. In step 288, all objects that do not correspond to the grid spacing (i.e., have an unacceptable IDP error) are removed as outliers.

[0038] Finally, in step 289, a random sample consensus (RANSAC) analysis is used to identify and remove all objects that are not in the same plane as the majority of the fiducials (i.e., the estimated plane of the plate).

[0039] FIG. 18 presents an example of RANSAC analysis. In step 291, a set of points representing objects present after IDP error analysis is established. In step 292, three points are randomly selected to define a plane, and then step 293 calculates the distance to the plane for all points. Step 294 removes points that are not at or near the plane. In step 295, if the number of remaining points is less than or equal to a given minimum value (i.e., the expected number of fiducials), the process is restarted and a new plane is defined. If the number of remaining points is greater than the minimum value, in step 296, the remaining points are presumed to correspond to fiducials. Returning to FIG. 15, in step 269, the positions of the fiducials are classified to correspond to the fiducial model in marker array coordinates (i.e., coordinates determined by a camera viewing marker elements extending above the registration plate). In other words, because the system knows the positional relationship between the fiducials and the IR marker elements on the registration array 60, the positions of the fiducials in the camera's coordinate system can be generated. Finally, in step 270 of FIG. 15, a reference registration is calculated that transforms the points from camera coordinates to image coordinates.

[0040] The term "about" generally refers to an approximate numerical value, and to a numerical value for which small variations do not significantly affect the practice of the disclosed embodiments. When a numerical limitation is used, unless the context dictates otherwise, "about" means that the numerical value may vary by + / - 5%, + / - 10%, or in certain embodiments, + / - 15%, and even + / - 20%. Similarly, "substantially" generally refers to at least 85% to 99% of the characteristic it modifies. For example, "substantially all" means 85% or more, 90% or more, or 95% or more, etc.

[0041] While preferred embodiments of the present disclosure have been described, it will be understood that the described embodiments are exemplary only, and that the scope of the invention is to be defined solely by the appended claims, which accord with the full range of equivalents along with the numerous variations and modifications which will naturally occur to those skilled in the art upon reading this specification.

Claims

1. 1. A system including a computer readable storage device for storing instructions and / or data, the instructions and / or data, when executed on a computer system, causing the computer system to: (a) receiving a 3D radiological image of a surgical site including a registration array including: (i) a plate having at least six fiducials formed thereon; and (ii) a marker support structure extending upward from a plane including the plate for placing a marker above the plate; (b) identifying coordinates of the fiducials within the 3D radiological image; (c) receiving an image of the registration array from an optical sensor; and (d) registering fiducials of the registration array detected by the optical sensor to coordinates of the fiducials in the 3D image; and configuring the device to: The criteria are: (i) cropping a first image volume of the 3D image in which the registration plate is expected to be located; (ii) detecting the registration plate in the first image volume and cropping a second image volume that includes the registration plate; (iii) detecting fiducials within the second image volume and classifying the fiducials; The system is detected by a method comprising:

2. The system of claim 1 , wherein detecting the registration plate comprises iteratively increasing an image threshold.

3. The system of claim 2 , wherein the image threshold is a grayscale.

4. The step of detecting the registration plate includes: (i) performing a connected component analysis on the second image volume; (ii) determining geometric features in the second image volume; and (iii) identifying an object that matches a geometric feature of the registration plate; The system of claim 1 , comprising:

5. The step of detecting the reference comprises: (i) dilating the second image volume to form a dilated image; (ii) eroding the dilated image to form an eroded image that closes the fiducial opening; (iii) masking the eroded image with an image from the second image volume to generate the reference image; The system of claim 1 , comprising:

6. The step of detecting the reference comprises: (i) performing connected component analysis on the reference image; (ii) computing a set of geometric features from the reference image; (iii) removing outliers from the set of geometric features; (iv) calculating Inter-Point Distance (IPD) errors between the remaining geometric features; (v) removing outliers using the IPD error; (iv) removing outliers using RANSAC; The system of claim 5 , comprising:

Images