Path Planning System and Method

Abstract

The present disclosure relates to path planning systems and methods. A system and method for planning a path through a network of anatomical lumens of a patient, comprising: a memory storing a computer program configured to analyze and manipulate CT images; a processor configured to run the computer program; and a display device configured to display a user interface of the computer program. The user interface includes an airway finder window that displays a CT image including a target. The CT image can be rotated about a predetermined axis of rotation to assist a user in identifying airways of the network of anatomical lumens.

Description

[0001] This application is a divisional application of the application with the application date of March 14, 2014, application number 201410093840.7, and invention name "Path Planning System and Method". Technical Field

[0002] The present disclosure relates to systems and methods for planning a path through a network of anatomical lumens of a patient. Background Art

[0003] During a surgical procedure, clinicians often use CT images to determine a path through a network of lumens of a patient. However, clinicians typically have difficulty effectively planning a path based solely on CT images, particularly in the smaller branches of the bronchial tree, where CT images typically do not provide sufficient resolution for accurate navigation.

[0004] To assist clinicians in planning a path through a network of lumens, automated path planning systems and methods have been implemented that automatically generate a path from a specified target on a CT image to an entry point of the patient (e.g., the patient's mouth, nose, other natural entry points, or an artificial entry point, such as an incision). An example of an automated path planning system and method can be found in U.S. Patent No. 8,218,846, which is incorporated herein by reference in its entirety.

[0005] However, when using a fully automated path planning system, a medical device may reach the end of the path in a direction where the working end of the medical device is not oriented towards the target. In this example, the side of the medical device may be oriented towards the target rather than the working end, and it may be difficult or impossible for the clinician to reach the target with the working end. In particular, when navigating through the small airways of the bronchial tree, when the target is perpendicular to the path along which the medical device travels in the small airway, it may be difficult or even impossible to bend or turn the working end of the medical device towards the target. Summary of the Invention

[0006] Systems and methods for planning a path through a network of anatomical lumens of a patient are provided.

[0007] In one aspect of the present disclosure, a system for planning a path through a network of anatomical lumens of a patient is disclosed, including: a memory storing a computer program configured to analyze and manipulate CT images; a processor configured to execute the computer program; and a display device configured to display a user interface of the computer program. The user interface includes an airway discovery window that displays a CT image including a target. The CT image can be rotated about a predefined axis of rotation to assist a user in identifying the airways of the network of anatomical lumens.

[0008] In one aspect of the present disclosure, the predefined axis is defined as the axis from the target to a known airway of the anatomical lumen network.

[0009] In one aspect of the present disclosure, the predefined axis is defined as the axis from the target to a part of the trachea in the anatomical lumen network.

[0010] In one aspect of the present disclosure, the predefined axis is an axis defined by a path from the target to an intermediate point.

[0011] In one aspect of the present disclosure, the predefined axis is an axis defined by a path from a first intermediate point to a second intermediate point.

[0012] In one aspect of the present disclosure, the user interface further includes a rotation interface configured to identify the amount of rotation of the CT image about the predefined axis relative to an initial rotational orientation of the CT image.

[0013] In one aspect of the present disclosure, the processor is configured to receive an input from a user identifying the location of a new intermediate point, generate a new intermediate point at the identified location, and output the new intermediate point for display on the user interface. The processor may be configured to generate a path from at least one of the target or a previous intermediate point to the new intermediate point and output the path for display on the user interface. If the new intermediate point is located in a known airway of the anatomical lumen network, the processor may be configured to automatically complete a path from the new intermediate point to an entry point of the anatomical lumen network and output the automatically completed path for display on the user interface.

[0014] In one aspect of the present disclosure, a method for planning a path through a patient's anatomical lumen network is disclosed, including the steps of: defining a rotation axis; rotating a CT image about the rotation axis; receiving an input from a user indicating the location of a new intermediate point in the rotated CT image's airway; setting a new intermediate point in the identified airway; and generating a new path to the new intermediate point.

[0015] In one aspect of the present disclosure, the step of defining a rotation axis includes defining a rotation axis from a target to a known airway of the anatomical lumen network, and the step of generating a new path includes generating a new path from the target to the new intermediate point.

[0016] In one aspect of the present disclosure, the step of defining a rotation axis includes defining a rotation axis from a target to a part of the trachea in the anatomical lumen network, and the step of generating a new path includes generating a new path from the target to the new intermediate point.

[0017] In one aspect of the present disclosure, the step of defining the axis of rotation includes defining the axis of rotation along a path from a target to an intermediate point, and the step of generating a new path includes generating a new path from the intermediate point to a new intermediate point.

[0018] In one aspect of the present disclosure, the step of defining the axis of rotation includes defining the axis of rotation along a path from a first intermediate point to a second intermediate point, and the step of generating a new path includes generating a new path from the second intermediate point to a new intermediate point.

[0019] In one aspect of the present disclosure, the method further includes: if the new intermediate point is set within a known airway of the anatomical lumen network, automatically completing the path from the new intermediate point to the inlet opening of the anatomical lumen network.

[0020] In one aspect of the present disclosure, a non-transitory computer-readable storage medium encoded with a program is disclosed, the program causing a user interface to perform the following steps when run by a processor: defining an axis of rotation; rotating a CT image about the axis of rotation; receiving an input from a user indicating the position of a new intermediate point in an airway of the rotated CT image; setting the new intermediate point in the identified airway; and generating a new path to the new intermediate point.

[0021] In one aspect of the present disclosure, the step of defining the axis of rotation includes defining the axis of rotation from a target to a known airway of the anatomical lumen network, and the step of generating a new path includes generating a new path from the target to the new intermediate point.

[0022] In one aspect of the present disclosure, the step of defining the axis of rotation includes defining the axis of rotation along a path from a target to an intermediate point, and the step of generating a new path includes generating a new path from the intermediate point to a new intermediate point.

[0023] In one aspect of the present disclosure, the step of defining the axis of rotation includes defining the axis of rotation along a path from a first intermediate point to a second intermediate point, and the step of generating a new path includes generating a new path from the second intermediate point to a new intermediate point.

[0024] In one aspect of the present disclosure, the program further causes the user interface to perform the following steps: if the new intermediate point is set within a known airway of the anatomical lumen network, automatically completing the path from the new intermediate point to the inlet opening of the anatomical lumen network.

[0025] Any of the above aspects and embodiments of the present disclosure may be combined without departing from the scope of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The objects and features of the presently disclosed systems and methods will become apparent to those skilled in the art when the description of the embodiments is read with reference to the accompanying drawings, in which:

[0027] Figure 1 Schematic diagram of a computing device for path planning according to an embodiment of the present disclosure;

[0028] Figure 2A View of a CT scan image of a patient's lung obtained from the axial direction according to an embodiment of the present disclosure;

[0029] Figure 2B Perspective view showing the patient's body in the axial direction according to an embodiment of the present disclosure;

[0030] Figure 2C View of a CT scan image of a patient's lung obtained from the coronal direction according to an embodiment of the present disclosure;

[0031] Figure 2D Perspective view showing the patient's body in the coronal direction according to an embodiment of the present disclosure;

[0032] Figure 2E View of a CT scan image of a patient's lung obtained from the sagittal direction according to an embodiment of the present disclosure;

[0033] Figure 2F Perspective view showing the patient's body in the sagittal direction according to an embodiment of the present disclosure;

[0034] Figure 3 Flowchart showing four stages of path planning according to an embodiment of the present disclosure;

[0035] Figure 4 Illustration of a user interface for selecting patient data according to an embodiment of the present disclosure;

[0036] Figure 5 Flowchart of a method for selecting patient data according to an embodiment of the present disclosure;

[0037] Figure 6 Illustration of a user interface for adding a target to path planning according to an embodiment of the present disclosure;

[0038] Figure 7 Flowchart of a method for adding a target to path planning according to an embodiment of the present disclosure;

[0039] Figure 8 Illustration of a user interface for editing the target details of the added target according to an embodiment of the present disclosure;

[0040] Figure 9 Illustration of a user interface for viewing a 3D graph according to an embodiment of the present disclosure;

[0041] Figure 10 is a flowchart of a method for inspecting a 3D map according to an embodiment of the present disclosure;

[0042] Figure 11A is an illustration of a user interface for finding a path from a target to an entry point to a patient;

[0043] Figure 11B is after the CT image in the user interface has been rotated about an initial axis Figure 11A of the user interface;

[0044] Figure 11C is a perspective view of a 3D model of a patient's bronchial tree showing an initial rotation axis according to an embodiment of the present disclosure;

[0045] Figure 11D is after a midpoint has been added and a path between the target and the midpoint has been created Figure 11B of the user interface, where the CT image has been rotated about the path as an axis;

[0046] Figure 11E is after a second midpoint has been added and a second path between the midpoint and the second midpoint has been created Figure 11D of the user interface;

[0047] Figure 11F is after the CT image has been rotated about the second path as an axis to display a known airway Figure 11E of the user interface;

[0048] Figure 11G is after a third midpoint has been added within the known airway and the path has been automatically completed Figure 11F of the user interface;

[0049] Figure 12 is a flowchart of a method for finding a known airway and creating a path according to an embodiment of the present disclosure;

[0050] Figure 13 is an illustration of a user interface for viewing a path according to an embodiment of the present disclosure;

[0051] Figure 14 is a flowchart of a method for viewing a path according to an embodiment of the present disclosure;

[0052] Figure 15 is an illustration of a user interface for viewing a target and a path and for creating additional targets and paths according to an embodiment of the present disclosure;

[0053] Figure 16is an illustration of a user interface for viewing and exporting path planning according to an embodiment of the present disclosure; and

[0054] Figure 17 is a flowchart of a method for viewing and exporting targets, paths, and path planning according to an embodiment of the present disclosure. Detailed implementation manners

[0055] Although the present invention will be described with respect to specific embodiments, it will be readily appreciated by those skilled in the art that various modifications, rearrangements, and substitutions can be made without departing from the spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

[0056] Now refer to Figure 1 , the present disclosure generally relates to a path planning system 10 and method for planning a path through a network of a patient's anatomical lumens for use during a surgical procedure. The path planning system 10 can include a computing device 100, such as, for example, a laptop computer, a desktop computer, a tablet computer, or other similar device, having a display 102, a memory 104, one or more processors 106, and / or other types of components common to computing devices. The display 102 can be touch-sensitive and / or voice-activated, enabling the display 102 to serve as an input and output device. Alternatively, a keyboard (not shown), a mouse (not shown), or other data input devices can be used.

[0057] The memory 104 includes any non-transitory computer-readable storage medium for storing data and / or software that can be executed by the processor 106 and control the operation of the computing device 100. In one embodiment, the memory 104 can include one or more solid-state storage devices such as flash memory chips. In an alternative embodiment, the memory 104 can be a mass storage connected to the processor 106 via a mass storage controller (not shown) and a communication bus (not shown). Although the description of computer-readable media included herein refers to solid-state memory, those skilled in the art should understand that the computer-readable storage medium can be any available medium accessible by the processor 106. That is, the computer-readable storage medium includes non-transitory, volatile and non-volatile, removable and non-removable media implemented in any information storage method and technology, such as computer-readable instructions, data structures, program modules, or other data. For example, the computer-readable storage medium includes RAM, ROM, EPROM, EEPROM, flash memory or other solid-state storage technologies, CD-ROM, DVD or other optical storage, magnetic cassette, tape, disk storage or other magnetic storage devices, or any other medium that can be used to store the required information and accessible by the computing device 100.

[0058] The computing device 100 may also include a network module 108 connected to a distributed network or the Internet via a wired or wireless connection for data transceiver with other sources. For example, the computing device 100 may receive computed tomography (CT) images from a server (such as a hospital server, an Internet server, or other similar servers) for use during path planning. The CT images of a patient may also be provided to the computing device 100 via the removable memory 104.

[0059] The path planning module 200 includes a software program stored in the memory 104 and executed by the processor 106 of the computing device 100. As will be described in more detail below, the path planning module 200 guides the clinician through a series of steps to develop a path plan for later use during a medical procedure. The path planning module 200 communicates with the user interface module 202 to display visual interaction features to the clinician on the display 102 and receive input from the clinician.

[0060] As used herein, the term "clinician" refers to any medical professional (i.e., doctor, surgeon, nurse, etc.) or other user of the path planning system 10 involved in planning, performing, monitoring, and / or supervising a medical procedure involving the use of the embodiments described herein.

[0061] Temporary reference Figures 2A - 2F , in fact, the most effective method of identifying a target involves using computed tomography (CT) images. As an introduction, the use of CT images as a diagnostic tool has become routine, and CT results are usually the primary source of information for clinicians regarding the size and location of lesions, tumors, or other similar targets of concern. Clinicians use this information to plan surgical procedures such as biopsies, but it is only available as "offline" information and generally must be remembered to the best of the doctor's ability before starting the surgical procedure. CT images are typically obtained by digitally imaging a patient in slices along each of the axial, coronal, and sagittal directions. For example, Figure 2A shows a slice of a CT image obtained in the axial direction, i.e., as if looking parallel to the patient's spine, as Figure 2B shown. Figure 2C shows a slice of a CT image obtained in the coronal direction, i.e., as a bird's-eye view of the patient as Figure 2D shown. Figure 2E shows a slice of a CT image obtained in the sagittal direction, i.e., as a side view of the patient as Figure 2F shown. Clinicians can view the CT image data slice by slice from each direction when attempting to identify or locate a target.

[0062] Now refer to Figure 3, in one embodiment, the path planning using path planning module 200 can be performed in four separate stages. In the first stage S1, the clinician selects a patient for path planning. In the second stage S2, the clinician adds a target. In the third stage S3, the clinician creates a path to the target. Finally, in the fourth stage S4, the clinician views and accepts the plan and can export the plan for use during the medical procedure. The clinician can repeat either or both of the second and third stages S2 and S3 as needed to select additional targets and / or create additional paths for a particular patient. For example, the clinician can select additional targets and can create paths to each target. The clinician can also or alternatively create multiple paths to the same target. Each of stages S1 - S4 will now be described in more detail with reference to Figures 4 - 16 hereinbelow.

[0063] As used herein, the term "window" refers to any screen, image, layer, user interface, or combination thereof projected or provided by user interface 202 on display 102 and encompasses components, assemblies, devices, modules, etc. that implement the functionality of these elements.

[0064] Now referring to Figure 4 and 5 , in stage S1, user interface 202 presents a window 210 to the clinician for selecting patient data 212 on which to perform path planning. Figure 4 A user interface 202 including window 210 is shown, Figure 5A method of selecting patient data according to an embodiment of the present disclosure is shown. The user interface 202 initially begins the method of selecting patient data in step S500 by opening a window 210 for a clinician to view. The window 210 includes a selectable source location menu 214 that provides the clinician with the ability to select a source from which to receive patient data 212 for path planning. In step S510, the clinician selects from a number of storage devices or storage media, including for example CDs, DVDs, Blu-ray discs, other insertable optical media, universal serial bus (USB) storage devices, external or internal hard drives, solid state storage devices, or any other type of memory or storage 104 that is connected to or in data communication with the computing device 100, as described above. The window 210 may also provide access to patient data 212 stored at a remote location such as a server on a network or the Internet. The source location menu 214 may allow the clinician to select a single patient data source, or may allow the clinician to select multiple patient data sources simultaneously. The source location menu 214 may also include an option to list patients from all sources. In step S504, the clinician may search the list of patients, or may enter a search term in the search box 216 to narrow the list of patients to those that meet the selection criteria (e.g., patient's last or first name, ID number, date of birth, or other similar criteria). Once the clinician has selected the desired patients, the clinician proceeds to step S506.

[0065] In step S506, once the clinician selects a patient, a drop-down menu 218 of the patient is displayed, including a list of the patient's available CT images 220, and any path plans 222 for the selected patient previously created for each CT image 220. The clinician can choose to create a new plan based on the CT image by selecting the create new plan option 224 and proceeding to step S510, or if there is a previously created plan for the selected CT image 220, the clinician can open the previously created plan by selecting the open plan option 226 and proceeding to step S514. When the create new plan option 224 is selected, the CT image 220 is preferably imported into the path planning module 200 in DICOM format. The computing device 100 processes the CT image 220 and assembles the CT image 220 into a three-dimensional CT volume by arranging the CT images 220 in the order in which they were taken and separating them according to the distance between slices set on the CT scanning device when they were taken. The path planning module 200 may perform a data filling function to create a seamless three-dimensional (3D) model or CT volume of the patient's bronchial tree. The path planning module 200 uses the newly constructed CT volume to generate a three-dimensional map of the airways in the bronchial tree. The three-dimensional map may be simplified such that each airway is represented as a line, or it may include airways with dimensions, where the dimensions represent the corresponding diameters of the airways. Preferably, when generating the three-dimensional map, the airways are labeled with the air flow direction (inhalation, exhalation, or separate arrows for each) for later use. Techniques for generating three-dimensional CT volumes and models are described in co-owned U.S. Patent Nos. 6,246,784 and 6,345,112 to Summers et al. and the references cited therein, all of which are incorporated herein by reference.

[0066] The window 210 also includes a capture screen option 228 that may allow the clinician to capture an image of the current screen displayed on the display 102, such as the window 210, and save the captured image to memory. The capture screen option 228 may also be configured to remove patient-specific data from the captured image to protect patient privacy. The removal of patient-specific data may be an option selectable by the clinician and may be set to "selected" by default.

[0067] Now referring Figures 6 - 8 , if the clinician selects the create new plan option 224 from the window 210, the method proceeds to step S512 and stage S2, adding a target. Figure 6 and 8 shows a user interface 202 including windows 230 and 244, Figure 7A method of adding a target according to an embodiment of the present disclosure is shown. When stage S2 starts, the method proceeds to step S700, where the user interface 202 opens a window 230 for identifying and selecting a target 232 for which path planning will be performed. In the window 230, slices 234 of CT image data are provided to the clinician in the main window 236. The slices 234 can be obtained from the CT image data in any one of the axial direction, the coronal direction, and the sagittal direction. The clinician can freely switch the slices 234 displayed in the main window 236 between the slices 234 from the axial direction, the coronal direction, and the sagittal direction at any time. In the example shown, slices 234 from the axial direction CT image data are provided. It should be noted that by showing only a single slice and direction at a time, for example, only showing the slices 234 from the axial direction CT image data, a simple and clean interface for the clinician to select a target from is provided. The clinician can manipulate and reposition the image of the selected slice 234 in the main window 236 and can zoom in or out of the selected slice 234 to obtain a magnified or reduced view of a specific portion of the selected slice 234.

[0068] The window 230 also includes a locator 238 that provides a general overview of the patient's CT image data for the clinician. In the example shown, the locator 238 provides a locator window 240 that includes a general view of the patient's lungs in the coronal direction. The locator window 240 can display, for example, a CT image in the coronal direction, a fluoroscopy-like image, or other similar images that provide a view of the patient's lungs to the clinician. The locator 238 includes a position element 242 that extends across the locator window 240, such as a line or a bar, and the position element 242 provides the clinician with the position of the selected slice 234 displayed in the main window 236 relative to the patient's lungs as shown by the locator 238. The position element 242 is selectable by the clinician and is movable or slidable relative to the locator window 240 to allow the clinician to scroll through the CT image slices of the patient's lungs displayed on the main window 236. For example, the CT image slices can be scrolled through or displayed in the sequential order defined by the CT image data. The clinician can also alternatively click or select a portion of the locator window 240 to move the locator 238 to a selected position in the patient's lungs. The clinician can also alternatively scroll through the CT image slices of the patient's lungs displayed in the main window 236 by an input device such as a mouse wheel or other device without directly interacting with the user interface 202. When another direction (e.g., the coronal direction) is selected to be displayed on the main window 236, the locator 238 can display a general view of one of the other directions (e.g., the axial direction or the sagittal direction). The locator 238 provides the clinician with a general reference as to where a particular lesion or other target 232 is located in the patient's lungs. The locator 238 can also display one or more previously selected targets for the clinician's reference.

[0069] In step S702, the clinician scrolls through the CT image slices 234 to identify the target 232 on the CT image. In step S704, once the target 232 in the current CT slice 234 is identified, the clinician can use a target selection element 243 (such as a crosshair, mouse pointer, hand, or other similar selection element) to click or otherwise select the target 232 from the main window 236. For example, the clinician can drag the CT image displayed on the main window 236 such that the target selection element 243 is positioned over the target 232, or alternatively, can directly select the target 232 by clicking on the target 232 using a mouse (not shown) or other input device. If the display 102 is touch-sensitive, the clinician can touch the target 232 on the display 102 to select the target 232. Then, the target 232 can be added to the plan in step S706 by selecting the add target option 245.

[0070] Now refer to Figure 8, once the target 232 is added, the target detail window 244 is displayed via the user interface 202. The target detail window 244 can overlay the window 230 or can replace the window 230. The target detail window 244 provides the clinician with an enlarged or close-up version of the selected target 232, as shown in the axial view 246, the coronal view 248, and the sagittal view 250. In step S708, the clinician can input the width, height, and depth dimensions of the target 232, name the target 232, and add additional comments related to the target 232. Additionally, a target sizing element 252 (e.g., a crosshair or other similar element) is positioned over the target 232 in each of the views 246, 248, 250 and can be manipulated or moved by the clinician to center the target 232 within the target sizing element 252 in each of the views 246, 248, 250. The target sizing element 252 also includes an adjustable boundary ring 254 that can be manipulated by the clinician to adjust the size of the target 232. For example, the clinician can adjust the size of the boundary ring 254 in each of the axial view 246, the coronal view 248, and the sagittal view 250 to accurately define the size of the target 232. The boundary ring 254 can be circular, elliptical, or other similar geometric shapes, and the shape of the boundary ring 254 can be adjusted to generally match the gross size of the target 232. In one embodiment, the boundary ring 254 can be adjusted by the clinician in a non-geometric manner, such as by free-form manipulation of the boundary ring 254, to conform to the non-geometric size of the target 232. It should be noted that since the target 232 is a three-dimensional object, such as a lesion, a tumor, etc., and the views 246, 248, 250 are taken from different directions, manipulation and adjustment of the boundary ring 254 on one of the views 246, 248, 250 by the clinician may result in changes or adjustments to the boundary ring 254 in one or both of the remaining views 246, 248, 250. In this way, the clinician can accurately select the target size and location of the target 232 in all three views, effectively mapping the target to specific coordinates and dimensions in the 3D coordinate space. In step S710, once the clinician has selected the size and location of the target 232, the clinician selects the save target option 256 and proceeds to view the generated three-dimensional map of the patient's bronchial tree in step S712.

[0071] Now refer to Figure 9 and 10, after the clinician selects the save target option 256 of window 230, the method proceeds to step S1000 to view a 3D map of the bronchial tree. In step S1000, the user interface 202 opens window 260 for viewing the three-dimensional map generated by the path planning module 200. Window 260 includes a three-dimensional map window 262 that displays a three-dimensional model 264 of the patient's bronchial tree and a scan window 266 that displays a CT image of one of the axial direction, the coronal direction, and the sagittal direction for the clinician's reference. In the illustrated embodiment, a CT image in the coronal direction is displayed. The CT image in the coronal direction is displayed because the coronal direction provides an aerial view or a front view of the patient's bronchial tree and is more likely to show the main recognizable features of the bronchial tree to the clinician, such as the main trunk and branches of the main airway. By comparing the CT image with the three-dimensional model 264, the clinician can determine or verify that the three-dimensional model 264 includes the main recognizable features of the patient's bronchial tree and that there are no severe abnormalities in the three-dimensional model 264 when compared with the CT image. In step S1002, the clinician rotates the three-dimensional model as needed by manipulating the rotation slider 268 of the three-dimensional map window 262 to determine whether the 3D map is acceptable. In step S1004, if the clinician is satisfied that the three-dimensional model 264 is substantially accurate, for example, sufficiently showing the main airway or the central airway, the clinician selects the approve map option 270 and proceeds to phase S3 and the airway finder. If the 3D map is unacceptable, the clinician proceeds to step S1006 and returns to step S500 to select new patient data, such as a new patient or a new CT scan of the same patient.

[0072] Now refer to Figures 11A - 11G and 12, after the clinician selects the approve map option 270 of window 260, the method proceeds to phase S3 and step S1200 to activate the airway finder. Figures 11A - 11B and 11D - 11G illustrate a user interface 202 including window 272, Figure 11C illustrating an initial axis "A" for use by the airway finder, Figure 12 illustrating a method of finding an airway and completing a path according to an embodiment of the present disclosure. In step S1200, initially refer to Figures 11A - 11C, the user interface 202 opens a window 272 for creating a path to an entry point (e.g., a natural orifice such as the mouth or nose, or an artificial entry point such as an incision) from the target 232 to the patient. The window 272 includes an airway finder 274 that displays a CT image 276 including the target 232 and a rotation interface 278 that describes the rotation of the CT image 276 about a specified axis. In one embodiment, when the window 272 is initially opened, only the target 232 is displayed on the CT image 276 of the airway finder window 274, and on the rotation interface 278, the rotation indicators 280 (e.g., arrow 280a and rotation bar 280b) are aligned. The rotation interface 278 provides rotation information about the relative rotation of the CT image 276 about the specified axis. Now refer to Figure 11B , a target marker 232A is displayed on the CT image 276 and positioned above the target 232 to show the clinician the location of the target 232. A lead line 232B extends from the center of the target marker 232A and can be moved by the clinician by moving a pointer, mouse, or other input device. For example, movement of the input device by the clinician moves the end 232C of the lead line 232B that extends away from the target 232. The clinician uses the lead line 232B to select an appropriate airway, as will be described in more detail below with reference to step S1208.

[0073] In one embodiment, briefly refer to Figure 11C , when the window 272 is initially opened, an initial axis "A" is set in step S1202, where the axis "A" is defined as the axis taken along the center portion from the target 232 to the lumen of the patient's trachea. By defining the initial axis "A" as the axis along from the target 232 to the trachea, the clinician has an increased likelihood of finding an airway near the target 232 that connects the target 232 to the entry point. This is due to the tree-like or branching nature of the bronchial tree. In other embodiments, the initial axis "A" can be defined as the axis taken along from the target 232 to any other part of the patient's bronchial tree (e.g., to the center portion of the nearest main branch of the bronchial tree, or to the nearest known airway in a 3D map of the patient's bronchial tree).

[0074] In step S1204, the clinician rotates the CT image 276 about the initial axis by, for example, rolling a mouse wheel, manipulating another input device, and / or by manipulating a portion of the user interface 202 (e.g., rotating the interface 278). When the clinician rotates the CT image 276 about the initial axis, indicators 280 on the rotation interface 278 move relative to each other in a corresponding direction along the rotation interface 278 to indicate the amount of rotation relative to the initial view. For example, the rotation bar 280b moves relative to the arrow 280a. When the rotation bar 280b reaches the end of the rotation interface 278 after the clinician continues to rotate in the same direction, the rotation bar 280b will disappear from the end of the rotation interface 278, reappear at the opposite end of the rotation interface 278, and continue to slide along the rotation interface 278 in the same direction. When the clinician rotates the CT image 276 a full circle about the initial axis, the indicators 280 (e.g., arrow 280a and rotation bar 280b) will align again at the center of the rotation interface 278.

[0075] Now refer Figure 11B and 11D , when rotating the CT image 276 about the initial axis, the clinician evaluates the CT image 276 in step S1206 to determine whether there is an airway 282 near the target 232. For example, a dark area extending away from or near the target 232 in the CT image 276 can be an indication of the presence of the airway 282. If the clinician determines that there is an airway 282, the method proceeds to step S1208, where the clinician positions the end 232C of the lead 232B at a determined position within the airway 282 on the CT image 276 to create a path midpoint 282a on the CT image 276. The path planning module 200 draws a path line 284 on the CT image 276 between the target 232 and the path midpoint 282a, and proceeds to step S1210. In this way, the clinician defines the portion of the airway 282 that is closest to the target 232. If no airway 282 is present, the clinician returns to step S1204 and continues to rotate the CT image about the specified axis. If the path midpoint 282a is not correctly positioned or the clinician wishes to find another airway, the clinician can remove the path midpoint 282a and return to either step S1204 or S1208.

[0076] In step S1210, the path planning module 200 determines whether the path midpoint 282a selected by the clinician is located within a known airway in the three-dimensional map generated by the path planning module 200. If the path midpoint 282a is located within a known airway in the three-dimensional map, the method proceeds to step S1214, where the path planning module 200 automatically completes the path from the path midpoint 282a through the known airway in the three-dimensional map to the patient's trachea and the entry point, as described below in Figure 11Gas further shown for waypoint 282a in the middle of the path.

[0077] If the path waypoint 282 is not located within a known airway in the three-dimensional map, the method proceeds to step S1212. Now refer to Figure 11D and 11E , the airway finder 274 displays the CT image 286 including the target 232, the target marker 232A, the path waypoint 282a, the path line 284, and the rotation interface 278, as described above. As Figure 11D shown, a lead 282b with an end 282c now extends from the path waypoint 282a.

[0078] In step S1212, the specified axis is set to the axis defined by the path line 284. The CT image 286 is rotated about the path line 284 instead of the initial axis "A", and the rotation interface 278 displays the relative rotation of the CT image 286 about the axis defined by the path line 284. By defining the axis of rotation about the path line 284, the possibility for the clinician to find an airway connected to the airway including the path waypoint 282a on the CT image 286 is increased. After the specified axis has been set to the path line 284, the method returns to step S1204. When rotating the CT image 286 about the axis defined by the path line 284, the clinician evaluates the CT image 286 to determine whether there is an airway 288 connected to the path including the path waypoint 282a, as described above. If in step S1206 the clinician determines that there is an airway 288, the method proceeds to step S1208, where the clinician positions the end 282C of the lead 282B at a determined position within the airway 288 on the CT image 286 to create a path waypoint 288a on the CT image 286. The path planning module 200 draws a path line 290 from the path waypoint 282a to the path waypoint 288a on the CT image 286, as Figure 11E shown. If the path waypoint 288a is not correctly positioned or the clinician wishes to find another airway, the clinician can remove the path waypoint 288a and return to either step S1204 or S1208.

[0079] In step S1210, now refer to Figure 11F , the path planning module 200 determines whether the path waypoint 288a selected by the clinician is located within a known airway (e.g., airway 291) in the three-dimensional map generated by the path planning module 200. If the path waypoint 288a is not located within the known airway in the three-dimensional map, the method proceeds to step S1212, and the clinician continues to set additional path waypoints, as described above, until the path waypoint is located within the known airway in the three-dimensional map.

[0080] Now refer toFigure 11G , the path midway point 291a has been added in the airway 291 in the above-described manner. In this illustration, the airway 291 is a known airway in the three-dimensional view. The method proceeds to step S1214, where the path 291b is automatically completed by the path planning module 200 from the path midway point 291a through the known branches of the airway 291 and the three-dimensional view of the bronchial tree to the entry point of the patient. Once the path is automatically completed, the method proceeds to step S1216, where the path completion window 293 is displayed by the user interface 202, providing the clinician with the options to proceed to view the path option 293a and the option to undo the automatic completion 293b. The clinician can select the option to proceed to view the path 293a to proceed to step S1218 and start viewing the path. Alternatively, if the clinician wants to continue to draw midway points using the airway finder 274, the clinician can select the option to undo the automatic completion 293b and return to step S1212 to further create path midway points as described above.

[0081] The path planning is created in this way for later use by the clinician during the surgical procedure. Since the clinician can manually select and create the path midway point and path closest to the target 232 before the automatic completion, the clinician is able to create such a path planning that directly controls the final orientation of the medical device relative to the target 232 at the end of the path planning. This allows the clinician to create a path planning for the medical device that will allow the medical device to travel along the small airways of the patient in a direction that will allow the working end of the medical device to be substantially oriented towards the target 232, where substantially towards the target 232 includes any orientation that the clinician can effectively reach the target 232 within the limitations of the medical device being used.

[0082] Now refer to Figure 13 and 14 , after the clinician has completed the path, the method proceeds to step S1400, where the user interface 202 opens a window 294 for viewing the path from the target 232 to the entry point of the patient. Figure 13 The user interface 202 including the window 294 is shown, Figure 14A method of viewing a path according to an embodiment of the present disclosure is shown. Window 294 includes a virtual window 295 and a three-dimensional (3D) map window 296. The 3D map window 296 displays a 3D model 298 of a patient's bronchial tree similar to the 3D map window 262. The virtual window 295 displays a CT-based "virtual bronchoscopy" that depicts a simulated view similar to an actual bronchoscope view and includes view selection labels 295a for selecting between the virtual bronchoscopy view, the local view, and the radial view. During virtual bronchoscopy, a clinician can switch between the virtual, local, and radial views as needed to view the path. The virtual bronchoscopy view displays a virtual visualization of the airway obtained from CT data, which approximates a video image from a bronchoscope. The local view displays an enhanced perspective view of a cross-section of the CT volume at the current navigation position. The radial view displays a cross-section of the CT volume perpendicular to the navigation position and the local path segment. Virtual bronchoscopy techniques are described in commonly assigned U.S. Patent Nos. 6,246,784 and 6,345,112 to Summers et al. and the references cited therein, which are hereby incorporated by reference in their entirety.

[0083] In step S1402, once the clinician has created a path, the user views the plan, target, and path by subsequent through-flight virtual bronchoscopy on the virtual window 295. The user interface 202 generates a line 300 in the virtual window 295 that represents the created path. The clinician follows the line 300 from the entry point through the trachea and the airways of the patient's bronchial tree until the line 300 reaches the target 232. It will be appreciated that as the clinician follows the line 300 through the increasingly smaller airways of the patient's bronchial tree, the ability of the path planning module 200 to resolve the smaller airways becomes increasingly difficult due to the insufficient resolution of the imported CT images. Due to this lack of resolution, the simulated view of the virtual bronchoscopy displayed in the virtual window 295 may eventually fail to display a clear airway lumen. In any case, the target 232 and the line 300 will be displayed in the virtual window 295 to allow the clinician to use the system for path planning purposes.

[0084] As the clinician moves along line 300 through the patient's bronchial tree to reach the target 232, the corresponding marker 298a moves along the three-dimensional model 298 to the target 232, indicating the position of the simulated view of the virtual window 295 relative to the three-dimensional model 298. In step S1404, after viewing the virtual bronchoscopy, the clinician determines whether the path is acceptable. If the path is acceptable, the clinician can select the approval option 299a, and the method proceeds to step S1408. If the path is not acceptable, the method proceeds to step S1406, where the clinician can select the abandon path and restart option 299b, thereby returning to the airway finder window 272 to edit the path or create a new path.

[0085] Now referring to Figures 15 - 17 , once the clinician has viewed and accepted the path, the method proceeds to phase S4 and step S1700. Figure 15 and 16 respectively illustrate a user interface 202 including windows 302 and 316, Figure 17 illustrating a method of viewing a plan according to an embodiment of the present disclosure. In step S1700, the user interface 202 opens a window 302 that includes a three-dimensional map window 304 showing the selected path and views of each of the axial direction 306, the coronal direction 308, and the sagittal direction 310. The window 302 includes a target label 312 and a path list 314. The target label 312 allows the clinician to add additional targets and select already identified targets for further examination. The path list 312 allows the clinician to examine the paths associated with the selected target label 312 and add paths for the selected target label 312. In step S1704, the clinician determines whether the target is acceptable. If the target is not acceptable, the method proceeds to step S1706, and the path planning module 200 returns the clinician to the add target window 230 to add a new target as described above. If the target is acceptable, the method proceeds to step S1708, where the clinician determines whether the path is acceptable. If the path is not acceptable, the method proceeds to step S1710, and the path planning module 200 returns the clinician to the airway finder window 272 to create additional paths as described above. If both the target and the path are acceptable, the clinician selects the complete and export option 315 in step S1716 and proceeds to view the plan.

[0086] Now referring to Figure 16, in step S1716, the user interface 202 opens a window 316 that includes a 3D graph window 318 and a list of targets 320 identified for the selected plan. Each target 320 can be selected by the clinician to display an associated path 322, and each path 322 can be viewed by the clinician by selecting a viewing option 324. The window 316 also provides an indication of whether the 3D graph has been viewed and approved and whether the current plan has been exported. In step S1712, if the 3D graph has not been approved, the clinician can re-view the 3D graph by selecting the view 3D graph option 326. If the view 3D graph option 326 has been selected, the method proceeds to step S1714, and the path planning module 200 returns the clinician to the view 3D graph window 260 described above. If the 3D graph has been approved, the method proceeds to step S1716, and the clinician determines whether the overall plan is acceptable. If the plan is not acceptable, the method proceeds to step S1718, and the path planning module 200 returns the clinician to the patient selection window 210 described above. If the clinician is satisfied with the plan, the method proceeds to step S1720, and the clinician can export the plan for use during the surgical procedure by selecting the export option 328. The plan can be exported to any form of non-transitory computer-readable medium, memory, or storage device, such as that described above for the memory 104, including, for example, the memory on the device 100, a removable memory device, or exported to a remote or server memory via transmission across a wired or wireless connection, etc.

[0087] Reference Figure 4 , 6 , 15, and 16, the user interface 202 can include one or more navigation bars that can be manipulated by the clinician to return to or repeat any of the above stages and / or steps. For example, as Figure 4 shown, the clinician can manipulate the navigation bar 330 to switch between the stages. Options can also be provided to the clinician to return to a previous step or window in any user interface 202 window.

[0088] As noted above, the present disclosure uses CT images for path planning. CT images are also typically used by clinicians during a medical procedure for navigation purposes. CT images are superior to other imaging forms because they have their own coordinate system. Matching two coordinate systems, such as the coordinate system of a CT image and the coordinate system of a patient, is generally referred to as registration. Registration is generally performed by identifying positions on or within the CT image and the body and measuring their coordinates in both systems.

[0089] Methods for manual and semi - automated registration of CT data and patient data are described in detail in, for example, U.S. Patent No. 7,233,820, assigned to Covidien LP, which is incorporated herein by reference. Since manual registration is somewhat time - consuming and requires multiple steps, many practitioners rely on the automated registration techniques described below. However, in some cases, particularly when the quality of the CT image data is not high enough, manual registration may still be required or desired.

[0090] For most procedures, automated registration has become the norm because, although the manual fiducial point designation of the registration techniques cited above is highly effective, the choice of the number of sampling points necessarily represents a trade - off between accuracy and efficiency. Similarly, although semi - automated techniques are a viable option, they require an image sensor at the distal end of the catheter assembly, which increases the complexity of the system.

[0091] Automated registration techniques are described in detail in commonly assigned U.S. Patent Application No. 12 / 780,678, which is incorporated herein by reference. Automated registration between a digital image of a bifurcated structure and a real - time indicator representative of the position of a sensor within the bifurcated structure is achieved by using the sensor to "draw" the digital image inside the structure. Once sufficient position data has been collected, registration is achieved. Registration is "automated" in the sense that navigating through the bifurcated structure will necessarily result in the collection of additional position data and, as a result, the registration is continuously refined.

[0092] Although the embodiments have been described in detail with reference to the respective drawings for purposes of illustration and explanation, it is understood that the processes and apparatus of the present invention should not be construed as being limited thereby. It will be apparent to those skilled in the art that various modifications can be made to the foregoing embodiments without departing from the scope of the present disclosure.

Claims

1. A system for planning a path through a network of anatomical lumens of a patient, the system comprising: a memory storing a computer program configured to analyze and manipulate CT images; a processor configured to run the computer program; and a display device configured to display a user interface of the computer program, the user interface comprising: an airway finder window that displays a CT image of a network of anatomical lumens including a target, the CT image being rotatable about a predetermined axis of rotation to assist a user in identifying airways of the network of anatomical lumens near the target, wherein the predetermined axis of rotation extends outside of the airways of the network of anatomical lumens, and when the CT image of the network of anatomical lumens is rotated about the predetermined axis, a simulated CT image of the network of anatomical lumens depicting a slice of the CT image along the predetermined axis of rotation is displayed, wherein after the user indicates a position of a midpoint in the airway of the rotated CT image, the predetermined axis includes an axis defined by a path from the target to the midpoint.

2. The system according to claim 1, wherein, The predetermined axis of rotation is an axis defined as extending from the target to a portion of the trachea of the network of anatomical lumens.

3. The system according to claim 1, wherein The predetermined axis of rotation is an axis defined by a path from a first midpoint to a second midpoint.

4. The system according to claim 1, wherein, The user interface further includes a rotation interface configured to identify an amount of rotation of the CT image about the predetermined axis of rotation relative to an initial rotational orientation of the CT image.

5. The system according to claim 1, wherein The processor is configured to: receive an input from the user identifying a position of a sensed airway; generate a new midpoint at the position of the sensed airway; and output the new midpoint for display on the user interface.

6. The system according to claim 5, wherein, The processor is configured to: generate a path from at least one of the target or a previous midpoint to the new midpoint; and output the path for display on the user interface.

7. The system according to claim 5, wherein, The processor is configured to: automatically complete a path from the new midpoint to an entry point of the network of anatomical lumens, and output the automatically completed path for display on the user interface.

8. The system according to claim 5, wherein, The user interface is configured to allow the user to identify a position of a sensed airway in the simulated CT image.

9. The system according to claim 8, wherein The sensed airway is depicted as a dark region in the simulated CT image.

10. A method for planning a path through a network of anatomical lumens of a patient, the method comprising the steps of: defining a rotation axis from a target to a known airway of the network of anatomical lumens, wherein the rotation axis extends outside of the airways of the network of anatomical lumens; rotating a CT image about the rotation axis such that non-displayed airways of the anatomical network become displayed airways of the anatomical network; receiving, from a user, an input indicating a position of a new midpoint in the airway of the rotated CT image based on the displayed airways of the anatomical network; and generating a new path from the target to the new midpoint, wherein after the user indicates a position of a midpoint in the airway of the rotated CT image, the predetermined axis includes an axis defined by a path from the target to the midpoint.

11. The method according to claim 10, wherein, The step of defining the rotation axis includes defining a rotation axis from the target to a portion of the trachea in the network of anatomical lumens, and wherein the step of generating the new path includes generating a new path from the target to the new midpoint.

12. The method according to claim 10, wherein, The step of generating a new path includes generating a new path from the midway point to the new midway point.

13. The method according to claim 10, wherein, The step of defining a rotation axis includes defining a rotation axis along the path from the first midway point to the second midway point, and the step of generating a new path includes generating a new path from the second midway point to the new midway point.

14. The method according to claim 10, further comprising the following steps: If the new midway point is set within a known airway of the anatomical lumen network, automatically complete the path from the new midway point to the inlet opening of the anatomical lumen network.

Images