Surgical system using a combination of sensor-based navigation and endoscopy

Abstract

To provide a comparative interface during a surgery by image-guided surgery (IGS) navigation.SOLUTION: A set of pre-operative images of an anatomical structure may be captured using an endoscopic camera. Each captured image is associated with a position and orientation of the camera at the moment of capture using image guided surgery (IGS) techniques. This image data and position data may be used to create a navigation map of captured images. During a surgical procedure on the anatomical structure, a real-time endoscopic view may be captured and displayed to a surgeon. The IGS navigation system may determine the position and orientation of the real-time image, and select an appropriate pre-operative image from the navigation map to display to the surgeon in addition to the real-time image.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] (Priority) This application claims priority to U.S. Provisional Patent Application No. 62 / 779,614, filed December 14, 2018, entitled "Surgical System with Combination of Sensor-Based Navigation and Endoscopy," the disclosure of which is incorporated herein by reference in its entirety. [Background technology]

[0002] Image-guided surgery (IGS) is a technology that uses a computer to perform real-time correlation of the positions of instruments inserted within a patient's body to a set of preoperatively acquired images (e.g., CT scans or MRI scans, 3D maps, etc.), allowing the computer system to overlay the current positions of the instruments on the preoperatively acquired images. An example of an electromagnetic IGS navigation system that can be used in IGS procedures is the CARTO® 3 System by Biosense-Webster, Inc. (Irvine, California). In some IGS procedures, a digital tomographic scan (e.g., CT or MRI, 3D map, etc.) of the surgical field is obtained before the surgery. A specially programmed computer then converts the digital tomographic scan data into a digital map. During the surgery, the procedure is performed using special instruments with sensors (e.g., electromagnetic coils that generate electromagnetic fields and / or respond to externally generated electromagnetic fields), which simultaneously send data to the computer indicating the current position of each surgical instrument. The computer correlates the data received from the sensors with the digital map created from the preoperative tomographic scan. The tomographic scan images are displayed on a video monitor along with indicators (e.g., crosshairs or illuminated dots) that indicate the real-time location of each surgical instrument relative to the anatomical structures shown in the scan image. Thus, even if the surgeon cannot directly visualize the instruments themselves in their current locations within the body, he or she can know the exact location of each sensor-equipped instrument by viewing the video monitor.

[0003] The use of IGS navigation can significantly improve the safety and success of surgical procedures. For example, surgeons may be able to more accurately position instruments within anatomy. The virtual view provided by an IGS navigation system may also be displayed along with actual video and images captured by an endoscope (e.g., a stand-alone endoscope or an endoscope integrated with other surgical instruments used at the surgical site). By having preoperatively acquired topographic images used to simulate virtual views and instrument positions, and real-time images of the surgical site during the procedure, surgeons have a wide range of inputs that can be considered during the procedure to guide their actions and determine when the procedure is complete. Even so, preoperative images are typically generated using indirect imaging techniques and are therefore limited in some respects. For example, IGS-navigated preoperative images are generated by interpreting various non-visual factors, such as magnetism, radio waves, x-rays, or ionizing radiation, rather than directly capturing reflected light from an object and generating a full-color, photorealistic image of the object, as a camera does.

[0004] While the resulting image sets are useful in many applications, they may not capture some aspects of the anatomy (e.g., color, texture, and other surface characteristics) and may have reduced resolution and clarity compared to direct image capture. While endoscopic images may provide such additional detail and clarity, the endoscopic view is limited to a real-time image of the anatomy in its current state. Any such limitations on the information available to the surgeon during surgery may contribute to poor outcomes, which may require follow-up and corrective action.

[0005] While several systems and methods have been developed and used in surgery, it is believed that no one prior to the present inventors has made or used the invention as set forth in the appended claims. [Brief explanation of the drawings]

[0006] While this specification concludes with claims particularly pointing out and distinctly claiming the invention, it is believed that the invention will be better understood from the following description of specific embodiments read in conjunction with the accompanying drawings, in which like reference numerals refer to like elements, and in which: [Figure 1] 1 shows a schematic diagram of an exemplary surgical navigation system in use with a patient seated in an exemplary medical procedure chair. [Figure 2] 1 is a schematic diagram of an exemplary endoscope that can be used with a surgical navigation system. [Figure 3] 10 shows a simulated screenshot of an exemplary comparison interface. [Figure 4] 10 shows a simulated screenshot of an exemplary comparison interface with a navigation image. [Figure 5] 10 shows simulated screenshots of an exemplary comparison interface with a time-series sequence of images. [Figure 6] 6 shows a flowchart of an exemplary set of high-level steps that may be performed to provide the comparison interface of any of FIGS. [Figure 7] 1 shows a flowchart of an exemplary set of steps that may be performed to create a pictorial navigation map. [Figure 8] 1 shows a flowchart of an exemplary set of steps that may be performed to capture live images during surgery. [Figure 9] 1 shows a flowchart of an exemplary set of steps that may be performed to create a comparison image. [Figure 10] 6 shows a flowchart of an exemplary set of steps that may be performed to display information via the comparison interface of any of FIGS.

[0007] The drawings are not intended to be limiting in any manner, and it is contemplated that various embodiments of the invention may be embodied in a variety of other ways, including those not necessarily depicted in the drawings. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate several aspects of the invention and, together with the description, serve to explain the principles of the invention. It should be understood, however, that the invention is not limited to the precise arrangements shown. DETAILED DESCRIPTION OF THE INVENTION

[0008] The following description of specific examples of the present invention should not be used for the purpose of limiting the scope of the present invention. Other examples, features / features, aspects, embodiments, and advantages of the present invention will become apparent to those skilled in the art from the following description, which illustrates, by way of example, one of the best modes contemplated for carrying out the invention. As will be understood, the present invention is capable of other different and obvious aspects, all without departing from the present invention. Accordingly, the drawings and description should be regarded as illustrative in nature, and not as restrictive.

[0009] It will be understood that the terms "proximal" and "distal" are used herein relative to a clinician holding the handpiece assembly; i.e., the end effector is distal relative to the more proximal handpiece assembly. It will be further understood that for convenience and clarity, spatial terms such as "upper" and "lower" are also used herein relative to a clinician holding the handpiece assembly. However, surgical instruments are used in many orientations and positions, and these terms are not intended to be limiting and absolute.

[0010] It is further understood that any one or more of the teachings, expressions, variations, examples, etc. described herein can be combined with any one or more of the other teachings, expressions, variations, examples, etc. described herein. Accordingly, the teachings, expressions, variations, examples, etc. described below should not be considered in isolation from one another. In light of the teachings herein, various suitable ways in which the teachings herein can be combined will be readily apparent to those skilled in the art. Such modifications and variations are intended to be included within the scope of the claims.

[0011] I. Exemplary Image-Guided Surgical Navigation System When performing a medical procedure within the head (H) of a patient (P), it may be desirable to have information regarding the location of an instrument within the head (H) of the patient (P), especially when the instrument is located in a location where it is difficult or impossible to obtain an endoscopic view of the instrument's working element within the head (H) of the patient (P). FIG. 1 illustrates an exemplary IGS navigation system (10) that enables the performance of ear, nose, and throat (ENT) surgery using image guidance. In addition to, or instead of, having the components and operability described herein, the IGS navigation system (10) may be configured and operative in accordance with at least some of the teachings of the following documents: No. 7,720,521, entitled "Methods and Devices for Performing Procedures within the Ear, Nose, Throat and Paranasal Sinuses," issued May 18, 2014, the disclosure of which is incorporated herein by reference, and U.S. Patent Application Publication No. 2014 / 0364725, entitled "Systems and Methods for Performing Image-Guided Procedures within the Ear, Nose, Throat and Paranasal Sinuses," published December 11, 2014, the disclosure of which is incorporated herein by reference.

[0012] The IGS navigation system (10) of this embodiment includes a magnetic field generator assembly (20) including a set of magnetic field generators (24) integrated into a horseshoe-shaped frame (22). The magnetic field generators (24) are operable to generate alternating magnetic fields of different frequencies around the head (H) of a patient (P). In this embodiment, a navigation guidewire (40) is inserted into the head (H) of the patient (P). The navigation guidewire (40) may be a stand-alone device or may be positioned on an end effector or elsewhere in a medical instrument, such as a surgical cutting or dilating instrument. In this embodiment, the frame (22) is attached to a chair (30), and the patient (P) is seated in the chair (30) such that the frame (22) is positioned adjacent to the patient's (P) head (H). By way of example only, chair (30) and / or magnetic field generator assembly (20) may be configured and operable in accordance with at least some of the teachings of U.S. patent application Ser. No. 15 / 933,737, entitled "Apparatus to Secure Field Generating Device to Chair," filed March 23, 2018, the disclosure of which is incorporated herein by reference.

[0013] The IGS navigation system 10 of this embodiment further includes a processing unit 12 that controls the magnetic field generator 24 and other elements of the IGS navigation system 10. For example, the processing unit 12 is operable to drive the magnetic field generator 24 to generate an alternating electromagnetic field and to process signals from the navigation guidewire 40 to determine the location of sensors in the navigation guidewire 40 within the head H of the patient P. The processing unit 12 includes a processing unit in communication with one or more memories. The processing unit 12 of this embodiment is mounted within a console 18 that includes a motion control unit 14, which may include a keypad and / or a pointing device, such as a mouse or trackball. A surgeon interacts with the processing unit 12 using the motion control unit 14 while performing a surgical procedure. In some implementations, the processing unit 12 may include one or more memory and processing units, which may be distributed across one or more separate computing devices located in proximity to the IGS navigation system 10 (e.g., one or more computers in the operating room), or located remotely from the IGS navigation system 10 (e.g., one or more remote, virtual, or cloud servers communicating with the IGS navigation system 10 via a network), or a combination thereof. In such implementations, different tasks may be handled by different subcomponents or subprocessors of the processing unit 12; for example, a computer, processing unit, and memory in the operating room may process and provide instrument tracking functionality, while a remotely located server may receive endoscopic, topographic, and other images and perform image processing, storage, encryption, or other processing on the received images.

[0014] The navigation guidewire 40 includes a sensor (not shown) that responds to positioning within the alternating electromagnetic field generated by the magnetic field generator 24. A coupling unit 42 is secured to the proximal end of the navigation guidewire 40 and is configured to provide communication of data and other signals between the console 18 and the navigation guidewire 40. The coupling unit 42 can provide wired or wireless communication of data and other signals.

[0015] In this example, the sensor of the navigation guidewire 40 includes at least one coil at the distal end of the navigation guidewire 40. When such a coil is placed in an alternating electromagnetic field generated by the magnetic field generator 24, the alternating magnetic field can generate an electric current in the coil, which can then be transmitted along conductive path(s) within the navigation guidewire 40 to the processing unit 12 via the coupling unit 42. This phenomenon enables the IGS navigation system 10 to determine the location of the distal end of the navigation guidewire 40 or other medical instrument (e.g., dilation instrument, surgical cutting instrument, etc.) in three-dimensional space (i.e., within the head (H) of the patient (P)). To accomplish this, the processing unit 12 executes an algorithm that calculates the position coordinates of the distal end of the navigation guidewire 40 from the position-related signals of the coil(s) within the navigation guidewire 40. In this embodiment, the position sensor is located within the guidewire (40), although such position sensors may be integrated into a variety of other types of devices, including those described in more detail below.

[0016] The processing unit 12 calibrates and operates the IGS navigation system 10 using software stored in the processing unit 12's memory. Such operations include driving the magnetic field generators 24, processing data from the navigation guidewire 40, processing data from the motion control unit 14, and driving the display screen 16. In some implementations, operations may also include monitoring and implementing one or more safety features or functions of the IGS navigation system 10. The processing unit 12 is further operable to provide real-time animation via the display screen 16 showing the position of the distal end of the navigation guidewire 40 relative to video camera images of the patient's head H, CT scan images of the patient's head H, and / or a computer-generated three-dimensional model of the anatomical structures within and adjacent to the patient's nasal cavity. The display screen 16 can display such images simultaneously and / or overlaid on each other during a surgical procedure. The images thus displayed may also include a graphical representation of an instrument inserted into the patient's head (H), such as a navigation guidewire (40), allowing the operator to view a virtual rendering of the instrument in its actual location in real time. By way of example only, the display screen (16) may provide images in accordance with at least some of the teachings of U.S. Patent Application Publication No. 2016 / 0008083, entitled "Guidewire Navigation for Sinuplasty," published January 14, 2016, the disclosure of which is incorporated herein by reference.

[0017] If the operator is also using an endoscope, endoscopic images can also be displayed on the display screen 16. For example, Figure 2 is a schematic diagram of an exemplary endoscope 100 that can be used with the IGS navigation system 10. The endoscope 100 includes a camera 102 operable to capture images and video, which may be located at the distal end of a rigid or flexible body (e.g., similar to the navigation guidewire 40) that can be navigated into the nasal cavity to capture images at the surgical site. The endoscope 100 also includes a position sensor 104 or tracking sensor proximate the camera 102 and configured to be tracked by the IGS navigation system 10 (e.g., similar to the navigation guidewire 40) to determine the position, orientation, or both of the camera 102. While the instrument tracking technique described above involves using a magnetic field generator assembly 20 to generate a magnetic field that interacts with a position sensor in the navigation guidewire 40 to enable position tracking, it should be understood that other types of position tracking exist and may be implemented with the techniques disclosed herein. For example, in some cases, wireless beacon triangulation may instead be used to implement a tracking region (e.g., a magnetic field) that communicates with and determines the location of the position sensor 104 in the endoscope 100, or other forms of wireless positioning and tracking capable of detection within the patient's body may be used.

[0018] Power and information may be transmitted via wires extending along the body of the endoscope 100, for example, to enable operation of the camera 102, to send and receive information to and from the position sensor 104, and to receive images from the camera 102 as digital data (e.g., via a data connection) or optical data (e.g., via optical fiber). The endoscope 100 of this embodiment also includes a connection unit 106, similar to the connection unit 42, which may be configured to enable the endoscope 100 to be used with the IGS navigation system 10. This may include, for example, receiving signals from the position sensor 104 and communicating them to the processing unit 12 so that the position of the position sensor 104 can be determined, and receiving signals or data from the camera 102 and communicating them to the processing unit 12 (e.g., in raw form as received or after conversion, such as converting optical image data received via optical fiber to digital image data before communication) so that they can be displayed via the display screen 16.

[0019] While the position sensor 104 is described as being usable to determine position within three-dimensional space, it should be understood that some implementations of the position sensor 104 may also be usable to determine orientation within three-dimensional space (e.g., by using a gyroscopic element, a combination of two or more independently tracked sensors such as quadrature coils with a fixed relationship to each other, or other similar means). Because the position sensor 104 may be usable to determine both position and orientation within a tracking region, it may sometimes be referred to as a tracking sensor with one or both capabilities. In implementations in which the position sensor 104 or another connected device or component is not capable of determining the orientation of the endoscope 100, the IGS navigation system 10 may automatically determine orientation by assuming that the endoscope 100 is always pointed at a particular anatomical target or other point within the surgical site.

[0020] The images provided via display screen 16 may help guide the operator as he or she performs technical and other manipulating operations within the patient's head H using instruments, if such instruments incorporate position sensors, such as position sensor 104. It should also be understood that sensors, such as the sensor of navigation guidewire 40 or sensor 104 of endoscope 100, may be incorporated into other components of surgical instruments and other types of surgical instruments, as described below.

[0021] II. Exemplary Systems for Comparative Image Navigation As described, an endoscope, such as endoscope 100, may be used in conjunction with IGS navigation system 10 to capture endoscopic images and videos that are displayed on display screen 16. Such endoscopic images may be displayed alone, in combination with other images (e.g., virtual endoscopic views, preoperative CT image views, etc.), or with other information and interfaces as may be desired. One example of an interface that may be advantageously displayed via display screen 16 may show a combination of preoperative endoscopic images and either intraoperative or postoperative endoscopic images to show both the current appearance of the surgical site and one or more previous appearances of the surgical site. In this manner, the surgeon may view the current appearance of the surgical site along with additional context (e.g., one or more previous appearances of the surgical site) that may be useful in determining whether the surgical procedure currently being performed is successful or complete.

[0022] By way of example only, as will be apparent to those skilled in the art in view of the teachings herein, it may be desirable to compare pre-operative endoscopic images of various anatomical structures within the nasal cavity with intra- or post-operative endoscopic images after performing a medical procedure, such as functional endoscopic sinus surgery (FESS), turbinate reduction, sinuplasty, or various other medical procedures. Such visual comparisons can easily enable an operator to compare tissue structures before, during, and after a medical procedure, in cases where the medical procedure has resulted in remodeling or other alterations to the tissue structures. By having endoscope position data incorporated directly into the pre-operative image data, the most appropriate pre-operative images can be selected based on real-time positioning of instrumentation during or after the medical procedure, or based on user selections made after the medical procedure.

[0023] For example, Figures 3-5 each illustrate interfaces that may be displayed via a display, such as display screen 16, during IGS navigation to provide such additional context, and Figures 6-10 illustrate sets of steps that may be performed to configure and provide one or more of the interfaces of Figures 3-5. Specifically, Figure 3 illustrates a simulated screenshot of an exemplary comparison interface 200. The comparison interface 200 includes a preoperative endoscopic image 202 and a preoperative endoscopic image status 204, as well as a live endoscopic image 206 and a live endoscopic image status 208. As used herein, describing an endoscopic image or information as being associated with a "pre-operative" timing of capture or status may indicate that the endoscopic image or information was captured prior to the start of a particular procedure, such as during an examination or exploratory imaging of the surgical site days or weeks prior to the procedure, but may also indicate that the endoscopic image or information was captured at any time prior to the current or immediately preceding stage of the procedure, such as immediately prior to a first tissue modification made during a first stage of the procedure, or immediately prior to a second tissue modification made during a second stage of the procedure, or at some other time. While Figure 3 shows the live endoscopic image (206) and the pre-operative endoscopic image (202) displayed side-by-side, it should be understood that they may be displayed or overlaid in a variety of other ways relative to one another (e.g., the pre-operative endoscopic image (202) may be modified to be semi-transparent and overlaid on top of the live endoscopic image (206)).

[0024] The preoperative endoscopic image (202) may represent an endoscopic image captured from a previous time point (e.g., before the start of the current procedure or during a previous stage of the current procedure), and the preoperative endoscopic image status (204) may represent one or more pieces of information associated with the preoperative endoscopic image (202). For example, the preoperative endoscopic image status (204) may include the three-dimensional position of the position sensor (104) at the time the preoperative endoscopic image (202) was captured, which may be displayed as information (e.g., x, y, and z coordinates) or as a simulated position relative to the surgical site at the time of capture (e.g., a rendered three-dimensional position of the position sensor (104) relative to the surgical site).

[0025] The live endoscopic image (206) may show an endoscopic image captured from the current time, such as one captured by the endoscope (100) during the current surgical procedure, while the live endoscopic image status (208) may show one or more pieces of information associated with the live endoscopic image (206). The live endoscopic image status (208) may show information similar to that shown in the pre-operative image status (204) and may include the three-dimensional position of the position sensor (104) as coordinates (e.g., x, y, and z coordinates) at the time the live endoscopic image (206) was captured, or may be displayed as a simulated position relative to the surgical site.

[0026] The live endoscopic image (206) represents a real-time image or video of the surgical site captured via the endoscope (100), while the pre-operative endoscopic image (202) represents a previous image or video of the surgical site captured from the same or similar field of view as the live endoscopic image (206) by comparing previous and current position data from the position sensor (104) associated with each endoscopic image or video, as described in further detail below in Figures 6-10. Using such an interface, as the surgeon moves the endoscope (100) (e.g., advances, retreats, rotates in any direction, mechanically articulates in any direction), not only does the real-time position data provided by the position sensor (104) change, but the real-time image shown in the live endoscopic image (206) also changes. As the real-time position data changes, the pre-operative endoscopic image (202) is also updated based on the real-time position data and may represent a previously captured image from the same or similar field of view as that indicated by the real-time position data.

[0027] As an example of the above, in some implementations, the preoperative endoscopic image (202) may be automatically updated as the position data changes, or may be presented as a sequence of images that appear to be a moving image of the preoperative endoscopic image, or as a still image when the position data is not changing. In this manner, the preoperative endoscopic image (202) can provide a preoperatively captured field of view and view similar to that captured in real time by the endoscope (100). In some implementations, the preoperative endoscopic image (202) may be presented as a still image that is manually updated by the user (e.g., based on a button input, voice input, or other input) as desired, so that the user can position the endoscope (100) and then manually update the displayed preoperative endoscopic image. In some implementations, the preoperative endoscopic image (202) may be presented as a still image that is automatically updated based on changing position data only when the endoscope (100) is not substantially moved or positioned, so that the still image is automatically updated after intentional movement of the endoscope (100) occurs and then stops.

[0028] In some implementations, the preoperative endoscopic images (202) may be presented as a looped sequence of images selected from preoperatively acquired images or videos based on changing position data. This may include, for example, displaying a series of images that includes the preoperatively acquired endoscopic image associated with that position data as well as several other preoperatively acquired endoscopic images (e.g., from slightly different positions or orientations) associated with nearby position data, thereby providing a subtle sense of movement surrounding and relative to the surgical site displayed by the preoperative endoscopic images. Other techniques for displaying the preoperative endoscopic images (202) exist and are described below or will become apparent to those skilled in the art in light of the teachings herein.

[0029] In some implementations of such an interface, the preoperative image status (204), the live endoscopic image status (208), or both, may provide information describing their perspectives (e.g., positions and orientations in three-dimensional space) relative to one another as a numerical offset, or directional movements or rotations that may be performed to more precisely align the images. As an example, if a live endoscopic image (206) is captured from a field of view having positions x1, y1, and z1 and an orientation having yaw 1, pitch 1, and roll 1, a previously captured image from that exact field of view may not exist. Instead, a best match having a field of view of x1, y2, and z1 and an orientation of yaw 1, roll 1, and pitch 2 may be displayed. In such a case, the live endoscopic image status (208) may show numbers, text, or directional information indicating changes that must be made to reduce or eliminate the difference between y1 and y2 (e.g., a directional arrow pointing up) and the difference between pitch 1 and pitch 2 (e.g., a directional arrow indicating an upward rotational pitch).

[0030] 4 shows a simulated screenshot of an exemplary alternative comparison interface (201) that may be displayed to a user of a system such as the IGS navigation system (10). The comparison interface (201) includes a live endoscopic image (206) and a live endoscopic image status (204), as well as a preoperative image status (208) and a panel of preoperative images (205) that includes the preoperative endoscopic image (202). The preoperative image (205) panel also includes a set of digitally scanned images (e.g., CT or MRI, 3D maps, etc.) that include a face-down image (210), a forward image (212), and a side image (214), each with an overlaid instrument position (211). The overlaid instrument position (211) may represent the real-time position of the endoscope (100) based on the position sensor (104), and each of the digital scan images may be navigated together using conventional means or controls (e.g., keyboard, mouse, touch screen, or navigation arrows or controls), or may be automatically selected and displayed based on data from the position sensor (104) (e.g., an image from above (210) may be selected based on determined x and z coordinates, a front image (212) may be selected based on determined y and z coordinates, and a side image (214) may be selected based on determined y and z coordinates).

[0031] 3, a pre-operative endoscopic image (202) may be selected and displayed based on data from the position sensor (104), such that the field of view of the displayed endoscopic image matches or substantially matches the current field of view of the live endoscopic image (206) captured by the endoscope (100) based on data from the position sensor (104). In this manner, as the surgeon moves the endoscope (100) and receives endoscopic images and video as the live endoscopic image (206), one or more images in the panel of pre-operative images (205) may also be updated to display a similar field of view, as described in more detail below.

[0032] 5 shows a simulated screenshot of an exemplary alternative comparison interface (203) that includes, in addition to the live endoscopic image (206) and the live endoscopic image status (204), a time series endoscopic image panel (217). The time series image panel (217) may include a set of ordered (e.g., ordered by capture time, surgical stage) endoscopic images showing the surgical site at different moments in time between an earlier endoscopic image (e.g., a pre-operative endoscopic image (202), which may be captured prior to processing or prior to a first tissue modification or treatment step) and a more recent endoscopic image (e.g., an endoscopic image (207), which may be the live endoscopic image (206) or the most recent endoscopic image captured or saved prior to the live endoscopic image (206)). The set of intervening endoscopic images (216a, 216b, 216c, 216d) may show one or more endoscopic images of the surgical site at various times from similar or identical views as those shown in the pre-operative endoscopic image (202) and the live endoscopic image (206), and may include, as shown, a first endoscopic image (216a), a second endoscopic image (216b), a third endoscopic image (216c), and a fourth endoscopic image (216d). As in the previous example, the selection and display of the endoscopic images in the time series image panel (217) may be determined based on data from the position sensor (100), such that the endoscopic images show the surgical site from a view that matches or substantially matches the live endoscopic image (206) currently being captured by the endoscope (100).

[0033] For example, if a particular surgical procedure has five major steps, each intervening endoscopic image may show the surgical site after completion of a step. In such an example, pre-operative endoscopic image (202) may show the surgical site before any tissue modifications, first endoscopic image (216a) may show the surgical site after a first set of tissue modifications during a first stage of the procedure, while fourth endoscopic image (216d) may show the surgical site after a next set of tissue modifications during a fourth stage of the procedure, and endoscopic image (207) may show the surgical site after a set of tissue modifications in the final stage of the procedure.

[0034] As another example, one or more of the interventional endoscopic images may show the surgical site at various times manually determined by the surgeon. In such an embodiment, the surgeon may mark a current live endoscopic image (206) to be added to the set of interventional endoscopic images by interacting with a button or control on the IGS navigation system (10), which may allow the surgeon to select, via the comparison interface (203), a particular milestone or other reference endoscopic image to save for later reference. As yet another example, one or more interventional endoscopic images may show the surgical site at various times automatically determined by the IGS navigation system (10) or another tool or device used during surgery. In such an example, endoscopic images may be automatically added to the interventional endoscopic images based on information or activity from a surgical instrument, a surgical instrument generator, or another device. This may include capturing an endoscopic image each time a cutting tool is activated, so that a set of interventional endoscopic images is automatically built to show the surgical site before each tissue modification or activation. As yet another example, endoscopic images may be captured and added to the intervening endoscopic image over time (e.g., endoscopic images may be captured and added to the intervening endoscopic image periodically based on a configurable schedule).

[0035] In either case, the set of intervening endoscopic images may be sized, configured, and displayed in various ways, including according to one or more of the following examples. In some implementations, the endoscopic images may be sized and configured to be simultaneously displayed via the comparison interface (203). In some implementations, the endoscopic images may be scrolled or traversed horizontally or laterally based on user input, thereby accommodating a large set of intervening endoscopic images. In some implementations, the endoscopic images may be displayed using various digital processes, including, for example, as a video sequence (e.g., each endoscopic image may be displayed briefly in sequence, creating the visual illusion of watching a moving image rather than viewing multiple individual images), or using image morphing (e.g., where a first endoscopic image and a second endoscopic image are used to generate and display multiple intervening endoscopic images, creating the visual illusion of gradually morphing from a preoperatively acquired endoscopic image to a current endoscopic image or other more recent endoscopic image).

[0036] The comparison interface (203) also includes a timeline status (215), which may show information similar to the preoperative endoscopic image status (208) and may additionally show timeline-related information, such as the time each of the interventional endoscopic images was captured (e.g., hour and minute, time since capture, stage of surgery at which the image was captured), and the circumstances of capture (e.g., whether added manually or automatically, and under what circumstances or in response to what event).

[0037] III. Exemplary Methods of Comparative Navigation While Figures 3-5 illustrate variations of comparison interfaces that may be provided by a system such as the IGS navigation system 10, Figures 6-10 illustrate steps that may be performed by such a system to construct and display those interfaces. For example, Figure 6 illustrates a flowchart of an exemplary set of high-level steps 300 that may be performed to provide any of the comparison interfaces of Figures 3-5. These steps include generating an endoscopic image map (block 302) of pre-operative endoscopic images (e.g., past, previously captured, or historical endoscopic images), which may involve using the endoscope 100 before the start of the procedure to capture endoscopic images of the surgical site from multiple different views (e.g., different positions and orientations relative to the surgical site) for the purpose of generating a set of endoscopic images and associated position or view data. The system may also receive live endoscopic images (block 304) during the procedure via the endoscope 100, which are also associated with position or view data. The system can then generate one or more comparison endoscopic images based on the endoscopic image map and the live endoscopic image (block 306), which may include selecting a matching endoscopic image from the endoscopic image map and using it without modification, or modifying or combining one or more endoscopic images from the endoscopic image map to provide as a comparison image. Using the one or more generated (block 306) comparison endoscopic images, the system can then provide a comparison interface via a display, such as display screen 16, that includes any combination of the functionality shown and described herein (block 308).

[0038] FIG. 7 shows a flowchart of an exemplary set of steps (400) that may be performed during a process such as creating an endoscopic image map (block 302). The endoscope (100) may be navigated to a surgical site, and the system may display the target anatomical structure at the surgical site (block 402) as the endoscope (100) captures a set of endoscopic images of the target anatomical structure (block 404). As each endoscopic image is captured by the endoscope (100), the system may receive field of view data (e.g., position, orientation, or both) from the position sensor (104) (block 406) and associate the field of view data with the corresponding endoscopic image (block 408). The result is a set of endoscopic images of the target anatomical structure, with each endoscopic image associated with a field of view of the target anatomical structure at the moment of capture. The endoscopic images may be captured by the endoscope as still endoscopic images or as a video sequence of endoscopic images / frames, as desired to achieve a particular resolution, clarity, or focus in the resulting endoscopic image set.

[0039] In some implementations, the system may identify and discard duplicate endoscopic images (e.g., two endoscopic images captured from the exact same field of view) or substantially duplicate endoscopic images (e.g., two endoscopic images captured from two different fields of view that are visually indistinguishable from each other) from the resulting endoscopic image set in order to reduce the overall size and content of the endoscopic image map.

[0040] In some implementations, the system may determine that the endoscopic image from a first field of view may be modified to provide an endoscopic image from a second field of view that is substantially overlapping. For example, if the second field of view is located linearly or substantially linearly between the first field of view and the target anatomical structure, the endoscopic image from the second field of view may appear to be a zoomed, enlarged, or closer endoscopic image of the target anatomical structure from the first field of view. In other words, the endoscopic image of the target anatomical structure captured from the most distal field of view may be cropped and enlarged (e.g., modified before being added to the endoscopic image map or modified in real time as it is selected from the endoscopic image map) to simulate endoscopic images from each field of view between the most distal field of view and the target anatomical structure, which may further reduce the size and content of the endoscopic image map and may also reduce the need to navigate the endoscope (100) to each individual field of view during creation of the endoscopic image map (e.g., if endoscopic images are captured from multiple distal fields of view, more proximal fields may be simulated without the need for the endoscope (100) to advance toward the target anatomical structure).

[0041] During capture of pre-operative endoscopic images (block 404), the system may provide instructions to the user of the endoscope 100 indicating fields of view that have already been captured and fields of view that have not yet been captured. This may include, for example, directional prompts indicating that the endoscope 100 should be moved or rotated to capture endoscopic images from a desired field of view (e.g., if the system is configured to capture endoscopic images from 10 different fields of view surrounding the surgical site, either by the user or based on the specific procedure). As another example, this may include displaying a three-dimensional sphere or other object having a representation of the target anatomical structure located at a fixed point and rendering areas of the object where fields of view have been captured visually distinct from areas where fields of view have not been captured. In such a system, the user of the endoscope 100 may refer to the rendered three-dimensional object to assist in navigating the endoscope 100 to a different field of view for the purpose of capturing endoscopic images from that field of view.

[0042] Once captured and associated with viewpoint data, the set of endoscopic images may be stored in various ways, such as in a database where endoscopic images may be selected based on an exact or partial match of the viewpoint data used in the query, or may be compiled into an object or software module that can receive the viewpoint data as input and provide matching or similar endoscopic images accordingly.

[0043] 8 shows a flowchart of an exemplary set of steps (410) that may be performed during a procedure such as receiving a live endoscopic image map (block 304) during a medical procedure. Once the endoscope (100) is positioned during the medical procedure, live endoscopic images of the surgical site may be captured in real time (block 412), and such information may be displayed via the display screen (16). The IGS navigation system (10) may be configured to track the position of the endoscope (100) via the position sensor (104) during the procedure to provide normal IGS navigation tracking of the instrument (e.g., to provide features such as overlaid instrument position (211)). Such data, which may traditionally only be used for tracking, may be received as field of view data (block 414) and retained or stored to provide one or more functions of the comparison interface.

[0044] The system may also receive (block 416) one or more metadata associated with the captured (block 412) endoscopic image, including, for example, the time of capture, the stage of the procedure during capture, the instrument status during capture (e.g., whether a cutting device was activated during capture), and manual input from the surgeon to tag or otherwise associate the captured (block 412) endoscopic image with a particular characteristic (e.g., marking the captured endoscopic image for display as an interventional endoscopic image in the time series image panel (217)). If the received (block 416) endoscopic image metadata indicates that a particular live endoscopic image has been tagged or saved (e.g., as a result of user input or automatically based on instrument activation, completion of a stage of the procedure, or other circumstances as described), the endoscopic image may be added to the endoscopic image map for future matching and display (e.g., via the comparison interface (203)). The system may then associate (block 418) the captured (block 412) live endoscopic image with field of view data, metadata, or both, which may include storing or converting the endoscopic image and data in a form that can be efficiently compared, queried, or otherwise evaluated against the image map created (block 302) for the purpose of identifying previous endoscopic images that match or substantially match the captured (block 412) live endoscopic image.

[0045] 9 shows a flowchart of an exemplary set of steps (420) that may be performed during a process such as creating a comparison endoscopic image (block 306). Once associated with field of view data (block 418), the live endoscopic image may be compared to the endoscopic image map created (block 302) to determine whether the endoscopic image map includes one or more endoscopic images whose fields of view match or substantially match the field of view from which the current live endoscopic image was captured. Fields of view that match or substantially match the field of view of the live endoscopic image may be determined to exceed a configured match threshold (block 422). The match threshold may be manually set to allow for differences in field of view when determining whether a sufficient endoscopic image match exists for the live endoscopic image in the endoscopic image map. The allowable difference in field of view may be expressed on a coordinate-by-coordinate basis (e.g., a difference in the x, y, or z coordinate may be considered an acceptable match if the difference is 5 mm or less, or a difference in the yaw, pitch, or roll may be considered an acceptable match if the difference is 10 degrees or less), or it may be considered as a collection of differences across all coordinates (e.g., a small difference in the x coordinate may be acceptable, while small differences in the x, y, and z coordinates may be unacceptable).

[0046] If none of the endoscopic images in the image map meet the match threshold, the system can determine that there is not a sufficient match (block 422) and may determine adjustments to the field of view that can be made to place the endoscope (100) in a field of view where the endoscopic image map contains a good endoscopic image match (block 424). This may include, for example, identifying one or more endoscopic images in the endoscopic image map that are below the match threshold (block 422) but are a suboptimal match, and then providing instructions via the display screen (16) for navigating the endoscope (100) to one or more of these fields of view (e.g., by providing numerical information or directional arrows indicating a change in position, orientation, or both).

[0047] If one or more endoscopic images are determined to be within the endoscopic image map that exceed the match threshold (block 422), the system may determine the best or closest match (e.g., if several images exceed the threshold, the image whose field of view has the smallest coordinate-by-coordinate difference or the smallest overall coordinate difference) (block 426). In some cases, the system may perform one or more endoscopic image processing or image interpolation processes in preparation for displaying the determined (block 426) best match (block 428). This may include modifying the pre-operatively acquired endoscopic image in one or more ways to provide a better comparison image for the live endoscopic image, and may include, for example, adjusting the size or resolution of the pre-operatively acquired endoscopic image, smoothing or blurring the pre-operatively acquired endoscopic image, cropping the pre-operatively acquired endoscopic image, adjusting the color, brightness, or hue of the pre-operatively acquired endoscopic image, and other modifications to better visually match the live endoscopic image captured via the endoscope 100. This may also include combining one or more equal or nearly equal endoscopic images that are determined to be the best match (block 426) (e.g., using image morphing, blending, or other techniques for combining two images from slightly different fields of view). After performing any necessary adjustments (block 428), the system may then create a comparison output endoscopic image suitable for display via a comparison interface (block 430), which may include, for example, converting the final adjusted endoscopic image into a particular format suitable for display.

[0048] Figure 10 shows a flowchart of an exemplary set of steps (432) that may be performed to display information via a comparison interface having one or more features, such as those shown in Figures 3-5. When providing the comparison interface, the system may display a live endoscopic image (block 434), such as may be captured via the endoscope (100) and displayed as a live endoscopic image (206). If a comparative endoscopic image acquired preoperatively is available (block 436) or may be created as described in Figures 6-9 (e.g., by identifying a best match (block 426) and preparing it for display), the system may also display a comparative endoscopic image (block 440), which may be displayed as a preoperative endoscopic image (202). If a matching suitable endoscopic image is not available (block 436), the system may display a warning indicating that the endoscopic image map does not include preoperatively acquired endoscopic images from a field of view suitable for display (block 438), may provide information (e.g., directional arrows) to assist the user in navigating the endoscope (100) to a field of view where a suitable endoscopic image is available, or may do both.

[0049] When displaying live endoscopic images (block 434) and comparative endoscopic images acquired preoperatively (block 440), the system may provide additional features and information, such as those shown in the comparison interface in Figures 3-5. Such features may be statically enabled, configured for a particular interface, or enabled or disabled by the user during surgery. For example, status information may be enabled (block 442) to display status information (block 444) regarding the live endoscopic images, the comparative endoscopic images, or both, and the status information may include one or more of the preoperative endoscopic image status (204), the live endoscopic image status (208), and the timeline status (215). As another example, a navigation image may be enabled (block 446) to display comparative endoscopic images (block 440), the live endoscopic images (block 434), and a set of associated navigation images, such as a panel of preoperative endoscopic images (205), (block 448).

[0050] As another example, time series image display may be enabled (block 450). Once enabled, as described, the system may determine a set of time series endoscopic images associated with the current live image field of view (e.g., by performing steps such as those shown in FIG. 9 to match and prepare multiple endoscopic images from different time points) (block 452) and then display these endoscopic images as a video sequence, morphed endoscopic images, or as interventional endoscopic images in the time series image panel (217) (block 454).

[0051] IV. Exemplary Combinations The following examples illustrate various, non-exhaustive, ways in which the teachings herein can be combined or applied. It is understood that the following examples are not intended to limit the scope of any claims that may be presented at any time in this application or in a later filing related to this application. No disclaimer is intended. The following examples are provided for illustrative purposes only. It is contemplated that the various teachings herein may be configured and applied in many other ways. It is also contemplated that certain features referred to in the following examples may be omitted in some variations. Accordingly, none of the aspects or features referred to below should be considered critical unless later expressly indicated to be so by the inventor or the inventor's successor in interest. If a claim including additional features beyond those referred to below is presented in this application or in a later filing related to this application, those additional features should not be considered added for any reasons related to patentability. [Example]

[0052] an endoscope having a guidance camera and a tracking sensor configured to interact with a tracking region; a processing unit configured to track the camera based on the tracking sensor's interaction with the tracking region; and a display, wherein the processing unit is further configured to: receive past images of a patient's surgical site, the past images being captured by the camera at a first time; determine a set of past image tracking data for the camera at the first time; add the past images and the set of past image tracking data to an image map as past views, the image map including a plurality of past views of the surgical site, each past view of the plurality of past views including an image of the surgical site captured from a view and a set of tracking data associated with the view; receive surgical images of the surgical site captured by the camera at a second time; determine a set of live image tracking data for the camera at the second time; determine a matching view of the plurality of past views based on the set of live image tracking data and the set of tracking data of the matching view; and display the surgical images and the image of the surgical site of the matching view via the display. [Example]

[0053] The processing device includes a computer located in an operating room where the IGS navigation system is available and a remote computer located outside the operating room, the tracking area includes a magnetic field provided by a tracking magnetic field generator, and the set of past image tracking data describes the position and orientation of the camera, an IGS navigation system as described in Example 1. [Example]

[0054] An IGS navigation system as described in any one or more of Examples 1 or 2, wherein the processing device is further configured to receive a next surgical image of the surgical site, determine a next set of live image tracking data for the next surgical image, redetermine a matching field of view based on the next set of live image tracking data, and display each next surgical image and an image of the surgical site of the matching field of view in near real time via the display in response to receiving one or more next surgical images. [Example]

[0055] An IGS navigation system described in any one or more of Examples 1 to 3, wherein the processing device is further configured, when determining a matching field of view, to query an image map based on each position and orientation coordinate of the set of live image tracking data to determine a best matching field of view, determine an offset of each position and orientation coordinate of the best matching field of view relative to the set of live image tracking data, and select the best matching field of view as the matching field of view when the offset is within a configured acceptable matching threshold. [Example]

[0056] An IGS navigation system as described in any one or more of Examples 1 to 4, wherein the processing device is further configured to display an image status, the image status including a comparison of a set of live image tracking data and a set of tracking data of the matching field of view, indicating an offset of the live field of view of the surgical image relative to the field of view of the matching field of view. [Example]

[0057] An IGS navigation system according to any one or more of Examples 1 to 5, wherein the processing device is further configured to simultaneously display the surgical image and an image of the surgical site in a corresponding field of view. [Example]

[0058] The IGS navigation system of Example 6, wherein the processing device is further configured to display an image of the surgical site of a corresponding field of view within a panel of preoperative images, the panel of preoperative images further including at least three digital scan images of the patient, each of the at least three digital scan images including an overlaid instrument position. [Example]

[0059] An IGS navigation system described in any one or more of Examples 1 to 7, wherein the processing device is further configured to determine that the surgical image should be retained as an intervening image, add the surgical image, the set of live image tracking data, and the second time to the image map as intervening views, determine one or more matching intervening views in the image map based on the set of live image tracking data and the set of intervening tracking data associated with each intervening view, and display a set of intervening images of the one or more matching intervening views. [Example]

[0060] The IGS navigation system of Example 8, wherein the processing device is further configured to determine that the surgical image should be retained as an interventional image based on the activation status of the surgical cutting instrument. [Example]

[0061] An IGS navigation system as described in any one or more of Examples 8 or 9, wherein each intervening image of the set of intervening images is displayed individually as a video sequence ordered by the capture time associated with each intervening image. [Example]

[0062] An IGS navigation system as described in any one or more of Examples 8 to 10, wherein each intervening image of the set of intervening images is displayed individually as a progressive image morph ordered by the capture time associated with each intervening image. [Example]

[0063] An IGS navigation system as described in any one or more of Examples 1 to 11, wherein the processing device is further configured to display, via the display, a message indicating a change in the camera's field of view that would result in a matching field of view becoming available in the image map if a matching field of view cannot be determined. [Example]

[0064] An IGS navigation system as described in any one or more of Examples 1 to 12, wherein the processing device is further configured to receive a second past image of the surgical site, determine a field of view associated with the second past image based on a second set of past image tracking data, determine whether the field of view associated with the second past image is identical to a field of view already in the multiple past views of the surgical site, and add the second past image to the multiple past views only if the fields of view are not identical. [Example]

[0065] 1. A method for providing a comparison interface during image-guided (IGS) navigation surgery, the method comprising: receiving, in a processing device, past images of a patient's surgical site, the past images being captured by a camera of an endoscope at a first time; determining a set of past image tracking data for the camera at the first time based on a tracking sensor of the endoscope; adding the past images and the set of past image tracking data to an image map as past views, the image map including a plurality of past views of the surgical site, each past view of the plurality of past views including an image of the surgical site captured from a view and a set of tracking data associated with the view; receiving surgical images of the surgical site captured by the camera at a second time and determining a set of live image tracking data for the camera at the second time; determining a matching view of the plurality of past views based on the set of live image tracking data and the set of tracking data of the matching view; and displaying the surgical images and the image of the surgical site of the matching view via a display. [Example]

[0066] The method of Example 14, further comprising receiving a next surgical image of the surgical site, determining a next set of live image tracking data for the next surgical image, redetermining a matching field of view based on the next set of live image tracking data, and displaying each next surgical image and an image of the surgical site of the matching field of view in real time via a display in response to receiving one or more next surgical images. [Example]

[0067] The method of any one or more of Examples 14 or 15, further comprising, when determining a matching field of view, querying an image map based on each position and orientation coordinate of the set of live image tracking data to determine a best matching field of view, determining an offset of each position and orientation coordinate of the best matching field of view relative to the set of live image tracking data, and selecting the best matching field of view as the matching field of view when the offset is within a configured acceptable matching threshold. [Example]

[0068] 17. The method of any one or more of Examples 14-16, further comprising simultaneously displaying the surgical image and an image of the surgical site in a corresponding field of view. [Example]

[0069] The method of Example 17, further comprising displaying an image of the surgical site of a corresponding field of view within a panel of preoperative images, the panel of preoperative images further comprising at least three digital scan images of the patient, each of the at least three digital scan images including an overlaid instrument position. [Example]

[0070] 19. The method of any one or more of Examples 14-18, further comprising, if a matching field of view cannot be determined, displaying a message via the display indicating a change in the camera's field of view that will result in a matching field of view becoming available in the image map. [Example]

[0071] 1. The processing device is configured to: receive past images of a patient's surgical site, the past images being captured by a camera at a first time; determine a set of past image tracking data for the camera at the first time; add the past images and the set of past image tracking data to an image map as past views, the image map including a plurality of past views of the surgical site, each past view of the plurality of past views including an image of the surgical site captured from a view and a set of tracking data associated with the view; receive surgical images of the surgical site captured by the camera at a second time; determine a set of live image tracking data for the camera at the second time; determine a matching view of the plurality of past views based on the set of live image tracking data and the set of tracking data for the matching view; and display the surgical images and the image of the surgical site of the matching view via the display.

[0072] V. Other It should be understood that any of the embodiments described herein may include various other features in addition to or in place of those described above. By way of example only, any of the embodiments described herein may include one or more of the various features disclosed in any of the various references incorporated herein by reference.

[0073] It should be understood that any one or more of the teachings, expressions, embodiments, examples, etc. described herein can be combined with any one or more of the other teachings, expressions, embodiments, examples, etc. described herein. Thus, the above teachings, expressions, embodiments, examples, etc. should not be considered in isolation from one another. In light of the teachings herein, various suitable ways in which the teachings herein can be combined will be readily apparent to those skilled in the art. Such modifications and variations are intended to be included within the scope of the claims.

[0074] Any patent, publication, or other disclosure referred to herein as being incorporated by reference should be understood to be incorporated herein, in whole or in part, only to the extent that the incorporated content does not contradict current definitions, views, or other disclosures set forth in this disclosure. As such, and to the extent necessary, the disclosures explicitly set forth herein shall prevail over any conflicting statements incorporated herein by reference. Any content, or portions thereof, that is referred to herein as being incorporated by reference but that contradicts current definitions, views, or other disclosures set forth herein shall be incorporated only to the extent that no conflict arises between the incorporated content and the current disclosures.

[0075] Device variations disclosed herein can be designed to be disposed of after a single use, or they can be designed to be used multiple times. In either case, the variations can be reconditioned for reuse after at least one use. Reconditioning can include any combination of the steps of disassembly of the device, followed by cleaning or replacement of particular pieces, and subsequent reassembly. In particular, device variations can be disassembled, and any number of particular pieces or parts of the device can be selectively replaced or removed in any combination. Upon cleaning and / or replacement of particular parts, the device variation can be reassembled for subsequent use either at a reconditioning facility, or by a surgical team immediately prior to a surgical procedure. Those skilled in the art will appreciate that reconditioning of a device can utilize a variety of techniques for disassembly, cleaning / replacement, and reassembly. Use of such techniques, and the resulting reconditioned device, are all within the scope of the present invention.

[0076] By way of example only, the variations described herein can be processed before surgery. First, new or used instruments can be obtained and, if necessary, cleaned. The instruments can then be sterilized. In one sterilization technique, the instruments are placed in a closed and sealed container, such as a plastic bag or a TYVEK bag. The container and instruments can then be placed in a radiation field that can penetrate the container, such as gamma radiation, x-rays, or high-energy electrons. The radiation can kill bacteria on the instruments and in the container. The sterilized instruments can then be stored in the sterile container. The sealed container can keep the instruments sterile until opened at the surgical facility. Any other technique known in the art can also be used to sterilize the device, including, but not limited to, beta or gamma radiation, ethylene oxide, or steam.

[0077] While various variations of the present invention have been shown and described, further applications of the methods and systems described herein may be realized by those skilled in the art through appropriate modifications without departing from the scope of the present invention. While some of such possible modifications have been described, other modifications will be apparent to those skilled in the art. For example, the above-described embodiments, variations, geometries, materials, dimensions, proportions, steps, etc. are illustrative and not required. Accordingly, the scope of the present invention should be considered in light of the following claims and should not be understood to be limited to the details of structure and operation shown and described in this specification and drawings.

[0078] [Embodiment] (1) An image-guided surgery (IGS) navigation system, comprising: (a) an endoscope comprising a camera and a tracking sensor, the tracking sensor configured to interact with a tracking region; (b) a processing unit configured to track the camera based on an interaction of the tracking sensor with the tracking region; (c) a display; The processing device includes: (i) receiving a prior image of a surgical site on a patient, the prior image being captured by the camera at a first time; (ii) determining a set of historical image tracking data for the camera at the first time; (iii) adding the set of past images and the past image tracking data to an image map as past views, wherein the image map includes a plurality of past views of the surgical site, and each past view of the plurality of past views comprises: (A) An image of the surgical site captured from a field of view; (B) a set of tracking data associated with the field of view; and (iv) receiving a surgical image of the surgical site captured by the camera at a second time; and (v) determining a set of live image tracking data for the camera at the second time; and (vi) determining a matching field of view from the plurality of past fields of view based on the set of live image tracking data and the set of tracking data of the matching field of view; (vii) displaying the surgical image and the image of the surgical site in the corresponding field of view via the display. (2) the processing device comprises a computer located in an operating room in which the IGS navigation system is available and a remote computer located outside the operating room; the tracking region includes a magnetic field provided by a tracking field generator; An IGS navigation system as described in embodiment 1, wherein the set of past image tracking data describes the position and orientation of the camera. (3) The processing device comprises: (i) receiving a next surgical image of the surgical site; (ii) determining a next set of live image tracking data for the next surgical image; (iii) redetermining the matching field of view based on a next set of the live image tracking data; (iv) An IGS navigation system as described in embodiment 1, further configured to display each next surgical image and the image of the surgical site in the corresponding field of view in near real time via the display in response to receiving one or more next surgical images. (4) when determining the matching field of view, the processing device: (i) querying the image map based on each position and orientation coordinate of the set of live image tracking data to determine a best match field of view; (ii) determining an offset of each position and orientation coordinate of the best match field of view relative to the set of live image tracking data; (iii) An IGS navigation system as described in embodiment 1, further configured to select the best match field of view as the match field of view when the offset is within a configured acceptable match threshold. (5) The IGS navigation system of embodiment 1, wherein the processing device is further configured to display an image status, the image status including a comparison between the set of live image tracking data and the set of tracking data of the matching field of view, indicating an offset of the live field of view of the surgical image relative to the field of view of the matching field of view.

[0079] (6) An IGS navigation system as described in embodiment 1, wherein the processing device is further configured to simultaneously display the surgical image and the image of the surgical site in the corresponding field of view. (7) The IGS navigation system of embodiment 6, wherein the processing device is further configured to display the image of the surgical site of the matching field of view within a panel of preoperative images, the panel of preoperative images further including at least three digital scan images of the patient, each of the at least three digital scan images including an overlaid instrument position. (8) The processing device comprises: (i) determining that the surgical image should be retained as an interventional image; (ii) adding the surgical image, the set of live image tracking data, and the second time to the image map as an intervening field of view; (iii) determining one or more matching intervening fields of view of the image map based on the set of live image tracking data and a set of intervening tracking data associated with each intervening field of view; (iv) An IGS navigation system as described in embodiment 1, further configured to display a set of intervening images of the one or more matching intervening fields. (9) An IGS navigation system as described in embodiment 8, wherein the processing device is further configured to determine that the surgical image should be retained as an intervention image based on the activation status of a surgical cutting instrument. (10) An IGS navigation system as described in embodiment 8, wherein each intervening image of the set of intervening images is displayed individually as a video sequence ordered by the capture time associated with each intervening image.

[0080] (11) An IGS navigation system as described in embodiment 8, wherein each intervening image of the set of intervening images is displayed individually as a progressive image morphing ordered by the capture time associated with each intervening image. (12) An IGS navigation system as described in embodiment 1, wherein the processing device is further configured to display, via the display, a message indicating a change in the camera's field of view that would result in the matching field of view becoming available in the image map if the matching field of view cannot be determined. (13) The processing device comprises: (i) receiving a second prior image of the surgical site and determining a field of view associated with the second prior image based on a second set of prior image tracking data; (ii) determining whether the field of view associated with the second prior image is identical to a field of view already in the plurality of prior views of the surgical site; (iii) An IGS navigation system as described in embodiment 1, further configured to add the second past image to the plurality of past views only if the views are not exactly the same. (14) A method for providing a comparison interface during an image-guided (IGS) navigated surgery, comprising: (a) In a processing device: (i) receiving a prior image of a surgical site on a patient, the prior image being captured by a camera of an endoscope at a first time; (ii) determining a set of historical image tracking data for the camera at the first time based on a tracking sensor of the endoscope; and (b) adding the set of past images and the past image tracking data to an image map as past views, the image map including a plurality of past views of the surgical site, each past view of the plurality of past views comprising: (i) an image of the surgical site captured from a field of view; (ii) a set of tracking data associated with the field of view; (c) receiving surgical images of the surgical site captured by the camera at a second time and determining a set of live image tracking data for the camera at the second time; (d) determining a matching field of view from the plurality of past fields of view based on the set of live image tracking data and the set of tracking data of the matching field of view; (e) displaying the surgical image and the image of the surgical site in the corresponding field of view via a display. (15) (a) receiving a next surgical image of the surgical site and determining a next set of live image tracking data for the next surgical image; (b) redetermining the matching field of view based on a next set of the live image tracking data; The method of embodiment 14, further comprising: (c) displaying, in real time via the display, each subsequent surgical image and the image of the surgical site in the corresponding field of view in response to receiving one or more subsequent surgical images.

[0081] (16) When determining the corresponding field of view, (i) querying the image map based on each position and orientation coordinate of the set of live image tracking data to determine a best match field of view; (ii) determining an offset of each position and orientation coordinate of the best match field of view relative to the set of live image tracking data; 15. The method of claim 14, further comprising: (iii) selecting the best matching field of view as the matching field of view when the offset is within a configured acceptable matching threshold. (17) The method of embodiment 14, further comprising simultaneously displaying the surgical image and the image of the surgical site in the corresponding field of view. (18) The method of embodiment 17, further comprising displaying the image of the surgical site of the corresponding field of view within a panel of preoperative images, the panel of preoperative images further comprising at least three digital scan images of the patient, each of the at least three digital scan images including an overlaid instrument position. (19) The method of embodiment 14, further comprising, if the matching field of view cannot be determined, displaying a message via the display indicating a change in the field of view of the camera that would result in the matching field of view becoming available in the image map. (20) A processing device, (a) receiving a prior image of a surgical site on a patient, the prior image being captured by a camera at a first time; (b) determining a set of historical image tracking data for the camera at the first time; (c) adding the set of past images and the past image tracking data to an image map as past views, the image map including a plurality of past views of the surgical site, each past view of the plurality of past views comprising: (i) an image of the surgical site captured from a field of view; (ii) a set of tracking data associated with the field of view; (d) receiving a surgical image of the surgical site captured by the camera at a second time; (e) determining a set of live image tracking data for the camera at the second time; and (f) determining a matching field of view from the plurality of past fields of view based on the set of live image tracking data and the set of tracking data of the matching field of view; (g) displaying the surgical image and the image of the surgical site in the corresponding field of view via the display.

Claims

1. 1. A method for providing a comparison interface during an image-guided (IGS) navigated procedure, comprising: (a) in a processing device, (i) receiving a prior image of a surgical site on a patient, the prior image being captured by a camera of an endoscope at a first time; (ii) associating with the past image a set of past image tracking data indicating the position and orientation of the camera at the first time based on a tracking sensor of the endoscope; (b) adding the set of past images and past image tracking data to an image map as a past view; (c) receiving a surgical image of the surgical site captured by the camera at a second time and associating with the surgical image a set of live image tracking data indicative of a position and orientation of the camera at the second time; (d) determining a best match set of historical image tracking data from the set of historical image tracking data that most closely matches the set of live image tracking data; (e) displaying via a display the surgical image and a past image associated with the best-matched set of past image tracking data; When determining the set of best matching historical image tracking data: (i) querying the image map based on the live image tracking data set to determine the best matching candidate historical image tracking data set whose offset relative to the position and orientation of the live image tracking data set is within a configured acceptable match threshold; (ii) selecting the set of past image tracking data having the smallest offset from among the candidates as the best-matching set of past image tracking data.

2. (a) receiving a next surgical image of the surgical site captured by the camera at a time after the second time and determining a next set of live image tracking data for the next surgical image; (b) reselecting the best matching candidate set of past image tracking data based on the next set of live image tracking data; The method of claim 1, further comprising: (c) in response to receiving one or more next surgical images, displaying in real time via the display each next surgical image and a past image associated with the reselected set of best-matching past image tracking data.

3. The method of claim 1 , further comprising simultaneously displaying the surgical image and the past image associated with the best-matching set of past image tracking data.

4. 4. The method of claim 3, further comprising displaying the past images associated with the best-matched set of past image tracking data within a panel of preoperative images, the panel of preoperative images further comprising at least three digital scan images of the patient acquired preoperatively, and overlaying the position of the camera at the second time on each of the at least three digital scan images.

5. 2. The method of claim 1, further comprising: if the candidate cannot be determined, determining a next-best matching set of past image tracking data among the sets of past image tracking data having the smallest offset; and displaying a message indicating the offset to the next-best matching set of past image tracking data via the display.

6. A processing device, (a) receiving a prior image of a surgical site on a patient, the prior image being captured by a camera at a first time; (b) associating with the past image a set of past image tracking data indicative of the position and orientation of the camera at the first time; (c) adding the set of past images and past image tracking data to an image map as a past view; (d) receiving a surgical image of the surgical site captured by the camera at a second time; (e) associating a set of live image tracking data with the surgical image that indicates the position and orientation of the camera at the second time; (f) determining a best match set of historical image tracking data from the set of historical image tracking data that most closely matches the set of live image tracking data; (g) displaying, via a display, the surgical image and a past image associated with the best-matched set of past image tracking data; When the processing device determines the set of best matching historical image tracking data, (i) querying the image map based on the live image tracking data set to determine the best matching candidate historical image tracking data set whose offset relative to the position and orientation of the live image tracking data set is within a configured acceptable match threshold; (ii) a processing device further configured to select the set of past image tracking data having the smallest offset from among the candidates as the best-matching set of past image tracking data.

Images