Graphical User Interface for Defining Anatomical Boundaries - Patent application

Abstract

To provide a graphical user interface for supporting an intuitive plan of a medical treatment including a low invasive medical technology.SOLUTION: A medical system includes a display system and a user input device. The medical system also includes a control system communicably connected to the display system and the user input device. The control system is configured so as to display image data corresponding to a three-dimensional anatomical region through the display system, receive a first user input through the user input device, and generate a first curve in the three-dimensional anatomical region. The control system is configured so as to receive a second user input through the user input device, generate a second curve in the three-dimensional anatomical region, and determine an anatomical border determined by the first curve and the second curve. The anatomical border indicates the surface of an anatomical structure in the three-dimensional anatomical region.SELECTED DRAWING: Figure 3B

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Provisional Application No. 62 / 741,157, filed October 4, 2018, which is incorporated by reference in its entirety.

[0002] The present disclosure relates to systems and methods for planning and performing image-guided procedures, and more particularly, to systems and methods for defining anatomical boundaries using a graphical user interface. [Background technology]

[0003] Minimally invasive medical techniques aim to reduce the amount of tissue damaged during medical procedures, thereby reducing patient recovery time, discomfort, and adverse side effects. Such minimally invasive techniques can be performed through natural orifices in the patient's anatomy or through one or more surgical incisions. Through these natural orifices or incisions, clinicians can insert minimally invasive medical instruments (including surgical, diagnostic, therapeutic, or biopsy instruments) to reach the location of the target tissue. One such minimally invasive technique is the use of flexible elongated devices, such as catheters, which may be steerable, that can be inserted into anatomical passages and navigated toward the region of interest in the patient's anatomy. Control of such elongated devices by medical personnel during image-guided procedures involves management of several degrees of freedom, including at least the insertion and retraction of the elongated device, as well as management of the steering or bending radius of the device. Additionally, various modes of operation may also be supported.

[0004] Therefore, it would be advantageous to provide a graphical user interface that supports intuitive planning of medical procedures, including minimally invasive medical techniques. Summary of the Invention

[0005] Embodiments of the present invention are best summarized by the claims that follow the detailed description.

[0006] In one embodiment, the medical system includes a display system and a user input device. The medical system also includes a control system communicatively coupled to the display system and the user input device. The control system is configured to display image data corresponding to the three-dimensional anatomical region via the display system, receive a first user input via the user input device to generate a first curve in the three-dimensional anatomical region. The control system is configured to receive a second user input via the user input device to generate a second curve in the three-dimensional anatomical region, and determine an anatomical boundary bounded by the first curve and the second curve. The anatomical boundary represents a surface of an anatomical structure in the three-dimensional anatomical region.

[0007] In another embodiment, a method for planning a medical procedure includes displaying, via a display system, image data corresponding to a three-dimensional anatomical region and receiving, via a user input device, a plurality of user inputs to generate a plurality of curves in the three-dimensional anatomical region. The method also includes determining an anatomical boundary from the plurality of curves. The anatomical boundary defines a boundary of a weakened portion of the three-dimensional anatomical region. The method also includes displaying, via the display system, the anatomical boundary overlaid on the image data.

[0008] In another embodiment, a non-transitory machine-readable medium includes a plurality of machine-readable instructions that, when executed by one or more processors associated with a planning workstation, are adapted to cause the one or more processors to perform the steps of: displaying CT image data corresponding to a lung via a display system; receiving a plurality of user inputs via a user input device to generate a plurality of curves in different slices of the CT image data; interpolating between the plurality of curves to determine an anatomical boundary indicative of a location of the pleura of the lung in the CT image data; and displaying, via the display system, the anatomical boundary overlaid on the CT image data.

[0009] It is to be understood that both the foregoing general description and the following detailed description are exemplary and explanatory in nature and are intended to provide an understanding of the present disclosure without limiting the scope of the disclosure. In that regard, additional aspects, features, and advantages of the present disclosure will become apparent to those skilled in the art from the following detailed description. [Brief description of the drawings]

[0010] The patent file or application file contains at least one drawing executed in color. Copies of this patent or patent application publication and any color drawing(s) will be provided by the Office upon request and payment of the necessary fee. [Figure 1] 1 is a simplified diagram of a medical system according to some embodiments. [Figure 2A] FIG. 1 is a simplified diagram of a side view of a patient coordinate space including a medical instrument attached to an insertion assembly, according to some embodiments. [Figure 2B] FIG. 1 is a simplified diagram of a side view of a patient coordinate space including a medical instrument attached to an insertion assembly, according to some embodiments. [Figure 3A] 1 is a simplified diagram of a method for defining anatomical boundaries according to some embodiments. [Figure 3B] 11 is an illustration of a method for defining anatomical boundaries according to another embodiment. [Figure 3C] FIG. 1 is an illustration of a method for presenting anatomical boundaries according to some embodiments. [Figure 3D] 1 is an illustration of a method for providing guidance information to guide the determination of anatomical boundaries, according to some embodiments. [Figure 3E] FIG. 1 is an illustration of a method for providing guidance information according to some embodiments. [Figure 3F] FIG. 1 is an illustration of a method for providing guidance information according to some embodiments. [Figure 3G] FIG. 1 is an illustration of a method for providing guidance information according to some embodiments. [Figure 4A] 1 is a simplified diagram of a graphical user interface during the execution of a method for defining anatomical boundaries, according to some embodiments. [Figure 4B] 1 is a simplified diagram of a graphical user interface during the execution of a method for defining anatomical boundaries, according to some embodiments. [Figure 4C] 1 is a simplified diagram of a graphical user interface during the execution of a method for defining anatomical boundaries, according to some embodiments. [Figure 4D] 1 is a simplified diagram of a graphical user interface during the execution of a method for defining anatomical boundaries, according to some embodiments. [Figure 4E] 1 is a simplified diagram of a graphical user interface during the execution of a method for defining anatomical boundaries, according to some embodiments. [Figure 4F] 1 is a simplified diagram of a graphical user interface during the execution of a method for defining anatomical boundaries, according to some embodiments. [Figure 5A] 1 is a simplified diagram illustrating range guidance relative to anatomical boundaries according to some embodiments. [Figure 5B] 1 is a simplified diagram illustrating range guidance relative to anatomical boundaries according to some embodiments. [Figure 5C]FIG. 13 illustrates a graphical user interface presenting range guidance along with two-dimensional image data. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The embodiments of the present disclosure and their advantages are best understood by referring to the following detailed description, in which like reference numerals are used to identify like elements shown in one or more of the figures, and it should be understood that the designations therein are for purposes of illustrating embodiments of the present disclosure and not for purposes of limiting the same.

[0012] In the following description, specific details are given to describe some embodiments consistent with the present disclosure. Numerous specific details are given to provide a thorough understanding of the embodiments. However, it will be apparent to one skilled in the art that some embodiments may be practiced without some or all of these specific details. The specific embodiments disclosed herein are intended to be illustrative, but not limiting. One skilled in the art may recognize other elements that are not specifically described herein but are within the scope and spirit of the present disclosure. Furthermore, to avoid unnecessary repetition, one or more features shown and described in connection with one embodiment may be incorporated in other embodiments, unless otherwise specifically described or unless one or more features render the embodiment non-functional.

[0013] In some instances, well known methods, procedures, components, and circuits have not been described in detail as not to unnecessarily obscure aspects of the embodiments.

[0014] This disclosure describes various instruments and parts of instruments in terms of their state in three-dimensional space. As used herein, the term "position" refers to the position of an object or part of an object in three-dimensional space (e.g., three translational degrees of freedom along Cartesian x, y, and z coordinates). As used herein, the term "orientation" refers to the rotational configuration of an object or part of an object (e.g., three rotational degrees of freedom such as roll, pitch, and yaw). As used herein, the term "pose" refers to the position of an object or part of an object in at least one translational degree of freedom and the orientation of that object or part of an object in at least one rotational degree of freedom (up to a total of six degrees of freedom). As used herein, the term "shape" refers to a set of poses, positions, or orientations measured along an object.

[0015] As shown in FIG. 1, the medical system 100 generally includes a manipulator assembly 102 for manipulating a medical instrument 104 in performing various procedures on a patient P. The medical instrument 104 can be extended to an internal surgical site within the patient P's body through an orifice in the patient P's body. The medical system 100 can be teleoperated, non-teleoperated, or a hybrid of the two. The manipulator assembly 102 can be a teleoperated, non-teleoperated, or hybrid teleoperated and non-teleoperated assembly with selected degrees of freedom of movement that can be motorized and / or teleoperated, and selected degrees of freedom of movement that can be non-motorized and / or non-teleoperated. The manipulator assembly 102 is mounted at or near a surgical table T. A master assembly 106 allows an operator O (e.g., a surgeon, clinician, or physician as shown in FIG. 1) to view the intervention site and control the manipulator assembly 102.

[0016] The master assembly 106 may be located in an operator console that is typically located in the same room as the operating table T, such as to the side of the operating table on which the patient P is positioned. However, it should be understood that the operator O may be located in a different room than the patient P or in a different building entirely. The master assembly 106 generally includes one or more controls for controlling the manipulator assembly 102. The controls may include any number of a variety of input devices, such as a joystick, trackball, data glove, trigger gun, hand controller, voice recognition device, motion or presence sensors, etc.

[0017] The manipulator assembly 102 supports the medical instrument 104 and may include one or more non-servo controlled links (e.g., one or more links that can be manually positioned and locked into place, e.g., commonly referred to as a setup), and / or one or more servo controlled links (e.g., one or more links that can be controlled in response to commands from a control system) and kinematic structure of the manipulator. The manipulator assembly 102 may optionally include multiple actuators or motors that drive the inputs of the medical instrument 104 in response to commands from a control system (e.g., control system 112). The actuators may optionally include a drive system that, when coupled to the medical instrument 104, can advance the medical instrument 104 into a natural or surgically created anatomical orifice. Other drive systems can move the tip of the medical instrument 104 in multiple degrees of freedom, which can include three linear degrees of freedom (e.g., linear motion along the X, Y, and Z Cartesian axes) and three rotational degrees of freedom (e.g., rotation about the X, Y, and Z Cartesian axes). Additionally, actuators can be used to actuate articulated end effectors of the medical instrument 104 to grasp tissue with jaws, such as a biopsy device.

[0018] The medical system 100 may include a sensor system 108 that includes one or more subsystems for receiving information regarding the manipulator assembly 102 and / or the medical instrument 104. Such subsystems may include a position / location sensor system (e.g., an electromagnetic (EM) sensor system), a shape sensor system for determining the position, orientation, speed, velocity, attitude, and / or shape of the tip and / or one or more segments along a flexible body that may comprise the medical instrument 104, a visualization system for capturing images from the tip of the medical instrument 104, and actuator position sensors, such as resolvers, encoders, potentiometers, etc., that describe the rotations and orientation of motors that control the instrument 104.

[0019] The medical system 100 also includes a display system 110 for displaying images or representations of the surgical site and the medical instruments 104. The display system 110 and the master assembly 106 may be oriented such that an operator O can control the medical instruments 104 and the master assembly 106 with telepresence awareness.

[0020] In some embodiments, the medical instrument 104 may include a visualization system that may include an image capture assembly that records concurrent or real-time images of the surgical site and provides the images to the operator O via one or more displays of the display system 110. The concurrent images may be, for example, two-dimensional or three-dimensional images captured by an endoscope positioned within the surgical site. In some embodiments, the visualization system includes an endoscope component that may be integrally or removably coupled to the medical instrument 104. However, in some embodiments, a separate endoscope attached to a separate manipulator assembly may be used with the medical instrument 104 to image the surgical site. The visualization system may be implemented as hardware, firmware, software, or a combination thereof that interacts with or executes on one or more computer processors, which may include a processor of the control system 112.

[0021] The display system 110 may also display images of the surgical site and medical instruments captured by the visualization system. In some examples, the medical system 100 may configure the controls of the medical instruments 104 and master assembly 106 such that the relative positions of the medical instruments are similar to the relative positions of the eyes and hands of the operator O. In this way, the operator O may manipulate the medical instruments 104 and hand controls as if he were viewing the workspace in substantially true presence. True presence means that the presentation of the images is a true perspective image that simulates the perspective of a physician physically manipulating the medical instruments 104.

[0022] In some examples, the display system 110 may present images of a surgical site recorded pre- or intra-operatively using image data from imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), fluoroscopy, thermography, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube x-ray imaging, etc. The pre- or intra-operative image data may be presented as two-dimensional, three-dimensional, or four-dimensional (e.g., including time-based or velocity-based information) images and / or as images from models formed from the pre- or intra-operative image datasets.

[0023] In some embodiments, often for purposes of image-guided medical procedures, the display system 110 can display virtual navigation images in which the actual position of the medical instrument 104 is registered (i.e., dynamically referenced) to a pre-operative or contemporaneous image / model. This can be done to present the operator O with a virtual view of the internal surgical site from the perspective of the medical instrument 104.

[0024] The medical system 100 may also include a control system 112. The control system 112 includes at least one computer processor (not shown) and at least one memory for effecting control between the medical instruments 104, the master assembly 106, the sensor system 108, and the display system 110. The control system 112 also includes programmed instructions (e.g., a non-transitory machine-readable medium storing instructions) for implementing some or all of the methods described in accordance with aspects disclosed herein, including instructions for providing information to the display system 110. While the control system 112 is shown as a single block in the simplified schematic diagram of FIG. 1, the system may include two or more data processing circuits, with some of the processing optionally performed on or adjacent to the manipulator assembly 102, and another portion of the processing performed on the master assembly 106, etc. The processor of the control system 112 may execute instructions, including instructions corresponding to processes disclosed herein and described in more detail below. In some embodiments, the control system 112 may receive force and / or torque feedback from the medical instruments 104. In response to the feedback, the control system 112 can send signals to the master assembly 106. In some examples, the control system 112 can send signals directing one or more actuators of the manipulator assembly 102 to move the medical instrument 104.

[0025] The control system 112 may optionally further include a virtual visualization system for providing navigational assistance to the operator O when controlling the medical instrument 104 during an image-guided medical procedure. Virtual navigation using the virtual visualization system may be based on reference to acquired pre-operative or intra-operative data sets of anatomical passageways. Software, which may be used in combination with operator input, is used to convert the recorded images into a segmented two-dimensional or three-dimensional composite representation of a portion or the entire anatomical organ or anatomical region. The image data set is associated with the composite representation. The virtual visualization system acquires sensor data from the sensor system 108, which is used to calculate the approximate position of the medical instrument 104 with respect to the anatomy of the patient P. The system may implement the sensor system 108 to register and display the medical instrument with pre-operative or intra-operative recorded surgical images. For example, International Publication No. WO 2016 / 191298 (published December 1, 2016) (disclosing “Systems and Methods of Registration for Image Guided Surgery”) discloses one such system, which is incorporated by reference in its entirety into this specification.

[0026] The medical system 100 may further include optional operation and support systems (not shown), such as a lighting system, a navigation control system, an irrigation system, and / or an aspiration system. In some embodiments, the medical system 100 may include two or more manipulator assemblies and / or two or more master assemblies. The exact number of manipulator assemblies will depend on factors such as the medical procedure and space constraints within the operating room, among others. The master assemblies 106 may be co-located or located in separate locations. The use of multiple master assemblies allows multiple operators to control one or more manipulator assemblies in various combinations.

[0027] 2A and 2B are simplified diagrams of a side view of a patient coordinate space including a medical instrument attached to an insertion assembly, according to some embodiments. As shown in FIGS. 2A and 2B, a surgical environment 300 including a patient P is positioned on the table T of FIG. 1. The patient P may be stationary within the surgical environment in the sense that the patient's overall movement is restricted by administration of sedation, restraints, and / or other means. Periodic anatomical motion including respiratory and cardiac motion of the patient P may continue. Within the surgical environment 300, a medical instrument 304 is used to perform a medical procedure that may include, for example, surgery, biopsy, cauterization, illumination, irrigation, aspiration, or system alignment procedures. The medical instrument 304 may be, for example, the instrument 104. The instrument 304 includes a flexible elongated device 310 (e.g., a catheter) coupled to an instrument body 312. The elongated device 310 includes one or more channels (not shown) sized and shaped to receive a medical tool (not shown).

[0028] The elongated device 310 may also include one or more sensors (e.g., components of the sensor system 108). In some embodiments, the fiber optic shape sensor 314 is fixed to a proximal point 316 on the instrument body 312. In some embodiments, the proximal point 316 of the fiber optic shape sensor 314 is movable with the instrument body 312, but the position of the proximal point 316 may be known (e.g., via a tracking sensor or other tracking device). The shape sensor 314 measures the shape of the elongated device 310 from the proximal point 316 to another point, such as the distal end 318. The shape sensor 314 may be aligned with the flexible elongated device 310 (e.g., provided within an internal channel (not shown) or attached externally). In one embodiment, the optical fiber has a diameter of about 200 micrometers (μm). In other embodiments, the dimensions may be larger or smaller. The shape sensor 314 may be used to determine the shape of the flexible elongated device 310. In one alternative, optical fibers containing fiber Bragg gratings (FBGs) are used to provide strain measurements in one or more dimensions within the structure. Various systems and methods for monitoring the shape and relative position of optical fibers in three dimensions are described in U.S. patent application Ser. No. 11 / 180,389, filed July 13, 2005 ("Fiber optic position and shape sensing device and method relating thereto"); U.S. patent application Ser. No. 12 / 047,056 (filed July 16, 2004) (disclosing "Fiber-optic No. 6,389,187, filed Jun. 17, 1998 (disclosing an “Optical Fibre Bend Sensor”), all of which are incorporated herein by reference in their entireties. The sensors in some embodiments may use other suitable strain sensing techniques, such as Rayleigh scattering, Raman scattering, Brillouin scattering, and fluorescence scattering. Various systems for registering and displaying surgical instruments with surgical images using fiber optic sensors are provided in WO 2016 / 191298, published Dec. 1, 2016 (disclosing “Systems and Methods of Registration for Image Guided Surgery”), all of which are incorporated herein by reference in their entireties.

[0029] In various embodiments, position sensors, such as electromagnetic (EM) sensors, may be incorporated into the medical instrument 304. In various embodiments, a series of position sensors may be positioned along the elongated device 310 and then used for shape sensing. In some embodiments, the position sensors may be configured and positioned to measure six degrees of freedom, e.g., three position coordinates X, Y, Z, and three orientation angles indicative of the pitch, yaw, and roll of a base, or five degrees of freedom, e.g., three position coordinates X, Y, Z, and two orientation angles indicative of the pitch and yaw of a base. Further description of position sensor systems may be found in U.S. Pat. No. 6,380,732, filed Aug. 11, 1999, entitled "Six-Degree of Freedom Tracking System Having a Passive Transponder on the Object Being Tracked,” which is incorporated herein by reference in its entirety.

[0030] The elongated device 310 may house cables, linkages, or other steering controls (not shown) extending between the instrument body 312 and the tip 318 to controllably bend the tip 318. In some examples, at least four cables are used to provide independent "up / down" steering to control the pitch of the tip 318 and "left / right" steering to control the yaw of the tip 318. Steerable elongated devices are described in detail in U.S. Patent Application Serial No. 13 / 274,208, filed October 14, 2011 (disclosing "Catheter with Removable Vision Probe"), which is incorporated herein by reference in its entirety. The instrument body 312 may include a drive input that removably couples to and receives power from a drive element, such as an actuator, of a manipulator assembly.

[0031] The instrument body 312 may be coupled to an instrument carriage 306. The instrument carriage 306 is attached to an insertion stage 308 that is fixed within the surgical environment 300. Alternatively, the insertion stage 308 may be movable but have a known position within the surgical environment 300 (e.g., via a tracking sensor or other tracking device). The instrument carriage 306 may be a component of a manipulator assembly (e.g., manipulator assembly 102) that couples to the medical instrument 304 and controls the insertion motion (i.e., motion along the A-axis) and, optionally, the motion in multiple directions including yaw, pitch, and roll of the tip 318 of the elongate device 310. The instrument carriage 306 or insertion stage 308 may include actuators such as servo motors (not shown) that control the movement of the instrument carriage 306 along the insertion stage 308.

[0032] A sensor device 320, which may be a component of the sensor system 108, provides information regarding the position of the instrument body 312 as it moves along the insertion axis A on the insertion stage 308. The sensor device 320 may include resolvers, encoders, potentiometers, and / or other sensors that determine the rotation and / or orientation of actuators that control the movement of the instrument carriage 306 and thus the instrument body 312. In some embodiments, the insertion stage 308 is linear. In some embodiments, the insertion stage 308 may be curved or have a combination of curved and linear sections.

[0033] FIG. 2A shows the instrument body 312 and instrument carriage 306 in a retracted position along the insertion stage 308. In this retracted position, the proximal end point 316 is at position L0 on axis A. At this position along the insertion stage 308, the position of the proximal end point 316 can be set to zero and / or another reference value to provide a base reference for describing the position of the instrument carriage 306, and thus the proximal end point 316, on the insertion stage 308. At this retracted position of the instrument body 312 and instrument carriage 306, the distal end 318 of the elongated device 310 can be positioned just inside the entrance orifice of the patient P. Also, at this position, the sensor device 320 can be set to zero and / or another reference value (e.g., I=0). In FIG. 2B, the instrument body 312 and instrument carriage 306 advance along the linear track of the insertion stage 308 and the distal end 318 of the elongated device 310 advances into the patient P. In this advanced position, proximal point 316 is at position L1 on axis A. In some examples, encoders and / or other position data from one or more actuators controlling the motion of instrument carriage 306 along insertion stage 308, and / or one or more position sensors associated with instrument carriage 306 and / or insertion stage 308 are used to determine the position L of proximal point 316 relative to position L0. x In some examples, the position L x may further be used as an indicator of the distance or insertion depth that the tip 318 of the elongate device 310 is inserted into the passageway of the patient P's anatomy.

[0034] In an exemplary application, a medical system, such as medical system 100, may include a robotic catheter system for use in a lung biopsy procedure. The catheter of the robotic catheter system provides a conduit for a tool, such as an endoscope, an endobronchial ultrasound (EBUS) probe, and / or a biopsy tool, to be delivered to a location within the airway where one or more anatomical targets for the lung biopsy, such as a lesion, nodule, tumor, etc., are present. Typically, the endoscope is positioned so that a clinician, such as a surgeon O, can monitor a live camera feed of the tip of the catheter as the catheter is driven through the anatomy. The live camera feed and / or other real-time navigation information may be displayed to the clinician via a graphical user interface. An example of a graphical user interface for monitoring a biopsy procedure is described in U.S. Provisional Application No. 62 / 486,879, filed April 18, 2017, entitled "Graphical User Interface for Monitoring an Image-Guided Procedure," which is incorporated herein by reference in its entirety.

[0035] Prior to performing a biopsy procedure using the robotic catheter system, a pre-operative planning step may be performed to plan the biopsy procedure. The pre-operative planning step may include segmenting image data, such as a patient CT scan, to create a 3D model of the anatomy, selecting an anatomical target within the 3D model, determining airways within the model, growing the airways to form a connection tree of the airways, and planning a trajectory between the target and the connection tree. One or more of these steps may be performed on the same robotic catheter system used to perform the biopsy. Alternatively or additionally, the planning may be performed on a different system, such as a workstation dedicated to pre-operative planning. The plan for the biopsy procedure may be saved (e.g., as one or more digital files) and transferred to the robotic catheter system used to perform the biopsy procedure. The saved plan may include the 3D model, identification of the airways, target location, trajectory to the target location, path through the 3D model, etc.

[0036] Provided below are exemplary embodiments of a graphical user interface for planning medical procedures, including but not limited to the lung biopsy procedure described above. The graphical user interface may include multiple modes, including a data selection mode, a hybrid segmentation and planning mode, a preview mode, a save mode, a manage mode, and a review mode. Some aspects of the graphical user interface are described in detail in “Graphical User Interface for Displaying Guidance Information During Planning,” filed June 30, 2016. No. 62 / 357,217, entitled “Graphical User Interface for Displaying Guidance Information in a Computer-Aided System and Image-Guided Procedure,” filed June 30, 2016. No. 62 / 357,258, entitled "Plurality of Modes During and Image-Guided Procedure," which are incorporated herein by reference in their entireties.

[0037] In planning and performing medical procedures, anatomical boundaries or virtual "hazard fences" may be defined by identifying surfaces that a medical instrument must not cross during a medical procedure. The anatomical boundaries may prevent weak or other portions of interest of an anatomical structure in the vicinity of a target location from being inadvertently penetrated by a medical instrument. Portions of interest that include weak anatomical structures or surfaces may include, for example, lung pleura, lung fissures, large blisters, and blood vessels. For example, puncturing a lung pleura during a medical procedure may cause a dangerous pneumothorax in a patient. Consistent with such an embodiment, defining an anatomical boundary corresponding to a lung pleura allows an operator to constrain the path of a medical instrument to avoid weak portions of the anatomy. For example, a candidate path may be invalid when the candidate path passes within a threshold distance of a weak portion of the anatomy, when the candidate path violates a weak portion of the anatomy, etc.

[0038] Figure 3A is a simplified diagram of a method 400A for defining anatomical boundaries according to some embodiments. Figures 4A-4F are corresponding simplified diagrams of a graphical user interface 500 during the performance of method 400A according to some embodiments. In some embodiments consistent with Figures 1-2B, the graphical user interface 500 may be displayable on a display system, such as display system 110 and / or on a display system of a separate planning workstation.

[0039] The graphical user interface 500 displays information related to planning of a medical procedure in one or more views viewable by a user, such as an operator O. Although exemplary arrangements of views are shown in FIGS. 4A-4F, it should be understood that the graphical user interface 500 may display any suitable number of views, in any suitable arrangement and / or on any suitable number of screens. In some examples, the number of views displayed simultaneously may be changed by opening / closing views, minimizing and maximizing views, moving views between the foreground and background of the graphical user interface 500, switching screens, and / or otherwise fully or partially hiding views. Similarly, the arrangement of the views (including their size, shape, orientation, order (if views overlap), etc.) may vary and / or may be user configurable.

[0040] The methods disclosed herein are illustrated as a series of operations or processes. Not all illustrated processes may be performed in all embodiments of the illustrated methods. Additionally, one or more processes not explicitly illustrated may be included before, after, during, or as part of an illustrated process. In some embodiments, one or more processes may be implemented, at least in part, in the form of executable code stored on a non-transitory, tangible, machine-readable medium, such that the code, when executed by one or more processors (e.g., processors of a control system), can cause the one or more processors to perform one or more processes. In one or more embodiments, the processes may be performed by the control system 112.

[0041] In process 410, image data 510 corresponding to a three-dimensional anatomical region of patient P is displayed via the graphical user interface 500. As shown in FIGS. 4A-4F, the image data 510 may include, for example, computed tomography (CT) image data. The image data 510 may include multiple images of the three-dimensional anatomical region of FIG. 4A showing a single plane or "slice" of image data. Additionally or alternatively, the image data 510 may include a 3D anatomical model, such as shown in a thumbnail view 512 of the graphical user interface 500. In some embodiments, the image data 510 may include segmentation data 514 indicating the location of anatomical features, such as lung airways, blood vessels, etc., identified from the CT image data. In some embodiments, the image data 510 may include an anatomical target 516 of a medical procedure, such as a biopsy site. In various alternative embodiments, image data may be generated using other imaging techniques, such as magnetic resonance imaging (MRI), fluoroscopy, thermography, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube x-ray imaging, and the like.

[0042] In process 420, a first user input for generating or defining a curve 520 in the three-dimensional anatomical region is received via a user input device. The curve 520 is generated in one plane of the image data 510. In some embodiments, the first user input may be provided by an operator via a mouse, touch screen, stylus, etc. As shown in FIG. 4B, the curve 520 may be displayed via a graphical user interface 500. In this embodiment, the curve 520 may correspond to a surface identified by the operator as part of the pulmonary pleura.

[0043] In process 430, a second user input is received via a user input device to generate or define a second curve 530 in the three-dimensional anatomical region. Curve 530 is generated in a plane of image data 510 different from the image plan in which curve 520 was defined. Curve 530 may be displayed via a graphical user interface 500, as shown in FIG.

[0044] Additional user inputs may optionally be received in process 440, with each additional user input generating or defining an additional curve (e.g., additional curve 532, FIG. 4D ) in the three-dimensional anatomical region. Generally, curves 530, 532 and any additional curves are defined in a similar manner as curve 520. Any additional curves may be positioned in different planes of the image data 510 (e.g., different slices of the CT image data) relative to curves 520, 530, and in any order relative to each other.

[0045] In process 450, an anatomical boundary 540 is determined that is bounded by curve 520, curve 530, and any additional curves, as shown in FIGURE 4E. In some embodiments, the anatomical boundary is determined by interpolating or otherwise identifying intermediate curves that define boundary 540. According to some embodiments, anatomical boundary 540 represents a surface or a surface of weakness or interest of a three-dimensional anatomical region that will not be crossed by a medical instrument during a medical procedure.

[0046] Optionally, in process 460, the anatomical boundary 540 is displayed via the graphical user interface 500. According to some embodiments, a visual representation of the anatomical boundary 540 may be overlaid on the image data. As shown in Figures 4E and 4F, a cross-sectional representation of the anatomical boundary 540 may be displayed as a curve overlaid on a CT slice, and a three-dimensional representation of the anatomical boundary 540 may be displayed as a semi-transparent or gridded wire mesh on a 3D anatomical model, such as the thumbnail view 512.

[0047] In some cases, the interpolated portion of the anatomical boundary 540 may not accurately track the actual anatomical boundary that the operator is attempting to define. For example, in the illustrative example shown in FIG. 4E, the interpolated portion of the anatomical boundary 540 between the curve 520 and the curve 530, which is intended to track the pleura of the lung, is visibly misaligned from the pleura. To correct this misalignment, the method 400A can return to processes 420-460 to receive additional user input defining additional curves that are used to update the anatomical boundary 540 to more closely align with the desired anatomical boundary (shown in FIG. 4F). In this manner, the processes 420-450 can be performed iteratively until a satisfactory alignment is achieved. In a similar manner, the range of the anatomical boundary 540 can be expanded by returning to processes 420-450 to receive additional user input defining additional curves that are outside the current range of the anatomical boundary 540.

[0048] 3B is a diagram of a method 400B for defining anatomical boundaries according to some embodiments. Some processes of method 400B are the same as those identified in diagram 400A and are indicated with the same reference numbers.

[0049] Before or after display of the image data in process 410, in optional process 412, the user may be presented with selectable choices of curve drawing options including freehand and polyline formats. In some embodiments, the curve 520 may be drawn freehand, in polyline format, with a series of plotted points, etc. In the case of a polyline input (e.g., a series of straight line segments) or a series of plotted points, the curve 520 may be determined, for example, by spline fitting. Optionally, spline fitting may be performed as all points are received. Optionally, spline fitting may be performed on all received points and updated as new points are received. Optionally, spline fitting may be performed with all received points and the current mouse position, allowing the user to see the shape of the fitted curve in real time before receiving the points. According to some embodiments, a first user input may be received in response to receiving a selection of the anatomical boundary tool 518 by the operator. Selection of the anatomical boundary tool 518 indicates that the operator intends to define an anatomical boundary via the graphical user interface 500 .

[0050] In process 450, the anatomical boundary 540 may be determined based on being stored or represented as a three-dimensional surface mesh including a plurality of vertices. FIG. 3C illustrates a method 470 for presenting an anatomical boundary according to some embodiments. In process 472, the anatomical boundary 540 may be generated as a three-dimensional surface mesh including a plurality of vertices. In one optional technique, in process 473, the vertices of the three-dimensional surface mesh may be determined by resampling each of the curves 520, 530, and any additional curves to the same number of sample points. In process 474, spline fitting is performed between matching sample points from each curve to generate a plurality of splines. In process 475, each of the plurality of splines is resampled to generate the vertices of the three-dimensional surface mesh. In another optional technique, in process 476, the vertices of the three-dimensional surface mesh may be determined by fitting a three-dimensional spline surface to the curves 520, 530, and any additional curves. In process 477, the 3D spline surface is resampled to generate the vertices of a 3D surface mesh.

[0051] 3B, in optional process 452, anatomical boundaries 540 may be further determined based on characteristics of the image data 510. For example, anatomical boundaries 540 may be snapped to regions of the image data 510 having high intensity gradients, as such gradients indicate the presence of surfaces of interest (e.g., pulmonary pleura, blood vessel walls, etc.). Similarly, computer vision techniques, including machine learning algorithms, may be applied to the image data 510 to identify candidate anatomical boundaries. Consistent with such embodiments, anatomical boundaries 540 may be snapped to candidate anatomical boundaries determined by such computer vision or machine learning techniques.

[0052] In optional process 462, the anatomical boundaries 540 may deform based on patient motion. During navigation, the patient's anatomy, and therefore the model, may move or deform due to forces, for example, from the medical instrument, the inhalation and exhalation of the lungs, and the beating heart. The deformations may be measured, for example, by a shape sensor on the medical instrument, or predicted by simulation, and the deformations may be applied to the model. The anatomical boundaries 540 may similarly be adjusted or deformed to accommodate the deformations of the model.

[0053] 3D is an illustration of a method 400C for defining an anatomical boundary according to some embodiments. Some processes of method 400C are the same as those identified in method 400A and are indicated with the same reference numbers. In processes 414, 422, and 452, various guidance information and visualization aids may be displayed via the graphical user interface 500 to assist the operator in defining or adjusting the anatomical boundary 540.

[0054] In process 414, guidance information and visualization aids may further be displayed to suggest the extent or shape that should be covered by anatomical boundary 540. Thus, extent guidance information may be displayed to improve the extent of protection provided by anatomical boundary 540, as discussed in further detail below with reference to Figures 5A-5B.

[0055] 5A and 5B are simplified diagrams illustrating range guidance 600 in relation to an anatomical boundary, such as anatomical boundary 540, according to some embodiments. FIG. 5C illustrates a graphical user interface 670 presenting range guidance 600 along with two-dimensional image data 510 to assist and guide a user in drawing a curve. As previously discussed, anatomical boundary 540 generally identifies a surface 610, such as a lung pleura, that must not be punctured or otherwise contacted or crossed by a medical instrument during a medical procedure at the site of target 620. As shown in FIGS. 5A and 5B, the medical procedure may correspond to a biopsy procedure in which a catheter 630 is inserted near target 620. During the biopsy procedure, a needle is directed from an exit point 635 of catheter 630 to target 620. Thus, in a biopsy procedure (and various other types of procedures in which an instrument may be extended from the catheter 630 towards the target 620), the anatomical boundary 540 can be used to identify the portion of the surface 610 that is behind the target 620 relative to the exit point 635 and therefore at risk of being punctured if the needle (or other instrument) extends too far beyond the target 620.

[0056] 5A, a three-dimensional risk portion 640 of the surface 610 is determined based on the intersection of the anatomical surface 610 with a three-dimensional zone. The zone may be, for example, a cone-shaped projection 642 extending from the exit point 635 through the target 620. In some embodiments, the risk portion 640 may include an additional margin 644 directly beyond the area within the projection 642. In some embodiments, the risk portion 640 may be determined in a binary manner (e.g., a given portion is considered to be either at risk or not at risk) or in a stepped or continuous manner to reflect various levels of risk at different locations.

[0057] Based on the determination of the portion at risk 640 of the surface 610, guidance information may be provided to the operator in the image data 510 to ensure that the anatomical boundary 540 defined during the method 400A provides sufficient protection for the portion at risk 640 of the surface 610. For example, a visual representation of the portion at risk 640, the projection 642, or both, may be displayed via the graphical user interface 500.

[0058] FIG. 3E illustrates one embodiment of the guidance process 414 in more detail by illustrating a method 414a for providing guidance information. In process 480, a three-dimensional zone (e.g., a cone-shaped projection 642) is generated extending from the instrument exit point 635 toward the target 620. In process 482, a two-dimensional projection of the three-dimensional zone is displayed with the image data 510. As shown in FIG. 5C, a two-dimensional projection region 650 of the three-dimensional zone (e.g., the cone 642) is provided overlaid on the two-dimensional image data 510 depicting the target 620. In process 484, using the projection 650 as a guide, the user can generate a curve 652 as described in processes 420 and 430. The curve 652 can be drawn within the region 650 and optionally extending beyond the region 650 to define the boundary of the risk portion 640. As previously described, additional curves may be drawn on additional slices of the two-dimensional image data 510 to generate multiple curves that are used to generate the anatomical boundary 540. In some embodiments, pixels in the risk region may be displayed with a different shade, color, or semi-transparent color overlay. Guidance may be turned on or off automatically, by user selection, or in combination. Additionally or alternatively, an indicator of whether the anatomical boundary 540 completely protects the risk portion 640 may be displayed or otherwise communicated to the operator via the graphical user interface 500.

[0059] 5B, one factor that may result in different levels of risk is the uncertainty associated with the medical procedure (e.g., uncertainty in the location of the exit point 635, uncertainty in the location of the target 620, or both). Other factors that may result in different levels of risk include the distance between the surface 610 and the exit point 635. Locations further from the exit point 635 generally have a lower risk than locations closer to the exit point 635.

[0060] During the planning procedure, a safety score may be calculated and provided to the operator, indicating the likelihood that the instrument will violate the boundary 540. Based on the score, the planned navigation path may be adjusted or modified to achieve a safer route. Various paths with different safety scores may be provided to the operator to choose from.

[0061] Referring again to FIG. 3D, additional guidance information and visualization aids may be provided in process 422. FIG. 3F illustrates one embodiment of guidance process 422 in more detail by illustrating a method 422a for providing guidance information. In process 486, a projection or shadow of curve 520 may be displayed in other planes of image data 510 where curve 520 would not otherwise be displayed (e.g., in CT slices of image data 510 other than the slice containing curve 520). Thus, the projection or shadow of curve 520 provides guidance in the form of a reminder to the operator of the characteristics of curve 520 (e.g., start point, end point, length, etc.) when defining curve 530. Without such a reminder, the operator may inadvertently define curve 530 having significantly different characteristics (e.g., significantly different start location, end location, or length) than curve 520. In such cases, anatomical boundary 540 may have an irregular shape or may not otherwise correspond to a desired anatomical boundary.

[0062] In process 488, the guidance information may include the start and end points of the first curve. In some embodiments, the anatomical boundary 540 may also have an irregular shape when the curve 530 is inadvertently reversed relative to the curve 520 (e.g., when the respective start and end points are at opposite ends of the curve). For example, the anatomical boundary 540 may have a twisted shape when the direction is reversed. Thus, guidance information may be displayed to indicate that the curve 530 should be oriented to match the curve 520. For example, with respect to the projection or shading of the curve 520 described above (or similarly, the projection of the anatomical boundary 540), the start point may be displayed in a manner that is visually distinct from the end point (e.g., using a different color, pattern, texture, etc.).

[0063] Referring again to FIG. 3D, instrument or boundary adjustment guidance information may be provided in process 452. FIG. 3G illustrates one embodiment of guidance process 452 in more detail by illustrating a method 452a for providing guidance information. For example, in optional process 490, a projection or shadow of anatomical boundary 540 may be extrapolated and displayed in areas outside the current extent of anatomical boundary 540 to provide guidance to the operator when extending the extent of anatomical boundary 540. In some embodiments, the projection or shadow of anatomical boundary 540 may be displayed in a manner that is visually distinct from anatomical boundary 540 itself (e.g., using a different color, pattern, texture, etc.) to alert the operator if the currently displayed cross-section is within or outside the current extent of anatomical boundary 540. As previously described, the plan of the biopsy procedure, including anatomical boundary 540, may be stored and used by the control system to provide automatic navigation or operator navigation assistance of the medical instrument to perform the biopsy procedure. During navigation, the boundary 540 may be displayed along with a three-dimensional anatomical model of the anatomical region (e.g., view 512), an endoluminal view, or other anatomical view presented on a user display. The boundary 540 may also, or instead, be displayed (e.g., overlaid) along with registered images from other imaging techniques, such as fluoroscopic images obtained during a medical procedure.

[0064] In optional process 491, suggested deployment locations for medical instruments may be provided. For example, during a registration procedure to align the three-dimensional model to the patient's anatomy, a point-collecting medical instrument may be used to touch a recommended cloud of points within the patient's anatomy. The recommended cloud of points may be determined based on their location relative to the boundary 540. For example, points may be recommended only if they are within a threshold distance from the boundary 540. Similarly, during a biopsy procedure, recommended biopsy locations may be determined based on their location relative to the boundary 540. For example, biopsy points may be recommended only if they are within a threshold distance from the boundary 540.

[0065] In optional process 492, the position and orientation of the medical instrument relative to the anatomical boundary 540 can be monitored during the medical procedure. The distance between the medical instrument and the anatomical boundary 540 can be measured, for example, from the tip of the instrument or from the portion of the instrument closest to the anatomical boundary 540. In process 493, an indicator can be provided to the operator when the distance between the instrument and the anatomical boundary 540 falls below a predetermined threshold distance value. For example, the visual indicator on the graphical user interface 500 can be provided in the form of a color change, a textual warning, a highlighted instrument, a highlighted boundary 540, or other visual warning signal. The indicator may also be provided in the form of an audible, tactile, or other operator perceptible signal. Additionally or alternatively, in process 494, the control system 112 can monitor the distance and slow the instrument speed or stop the instrument completely as it approaches the surface corresponding to the boundary 540. Additionally or alternatively, in process 495, an operator may provide user input (e.g., a button press) to orient the tip of the medical instrument away from the surface corresponding to boundary 540. Additionally or alternatively, distance-based indicators may be used in planning procedures with virtual medical instruments.

[0066] One or more elements in the embodiments of the present disclosure may be implemented in software to run on a processor of a computer system such as a control processing system. When implemented in software, the elements of the embodiments of the present invention are essentially code segments to perform the necessary tasks. The programs or code segments may be stored in a processor-readable storage medium or device, which may be downloaded via a computer data signal embodied in a carrier wave over a transmission medium or a communication link. The processor-readable storage device may include any medium capable of storing information, including optical media, semiconductor media, and magnetic media. Examples of the processor-readable storage device include electronic circuits, semiconductor devices, semiconductor memory devices, read-only memories (ROMs), flash memories, erasable programmable read-only memories (EPROMs), floppy disks, CD-ROMs, optical disks, hard disks, or other storage devices. The code segments may be downloaded over a computer network such as the Internet, an intranet, or the like. Any of a variety of centralized or distributed data processing architectures may be used. The programmed instructions may be implemented as several separate programs or subroutines, or the instructions may be integrated into several other aspects of the system described herein. In one embodiment, the control system supports wireless communication protocols such as Bluetooth, IrDA, Home RF, IEEE802.11, DECT, and Wireless Telemetry.

[0067] Medical tools that may be delivered via the flexible elongate devices or catheters disclosed herein may include, for example, image capture probes, biopsy instruments, laser ablation fibers, and / or other surgical, diagnostic, or therapeutic tools. The medical tools may include end effectors having a single actuating member, such as scalpels, blunt blades, optical fibers, electrodes, etc. Other end effectors may include, for example, forceps, graspers, scissors, clip appliers, etc. Other end effectors may further include electrically activated end effectors, such as electrosurgical electrodes, transducers, sensors, etc. The medical tools may include image capture probes including stereoscopic or monoscopic cameras for capturing images (including video images). The medical tools may further contain cables, linkages, or other actuation controls (not shown) extending between their proximal and distal ends to controllably bend the distal end of the medical instrument 304. A steerable instrument is described in U.S. Pat. No. 7,316,681, filed Oct. 4, 2005, entitled “Articulated Surgical Instrument for Performing Minimally Invasive No. 12 / 286,644, filed Sep. 30, 2008 (disclosing “Passive Preload and Capstan Drive for Surgical Instruments”), which are incorporated herein by reference in their entireties.

[0068] The systems described herein may be suitable for navigation and treatment of anatomical tissue via natural or surgically created connecting passageways in any of a variety of anatomical systems, including the lungs, colon, intestines, kidneys and renal calyces, brain, heart, circulatory system including vasculature, etc.

[0069] It should be noted that the presented processes and displays may not be inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the operations described. The required structure for these various systems appears as elements of the claims. Moreover, embodiments of the present invention are not described with reference to any particular programming language. It will be understood that a variety of programming languages ​​may be used to implement the teachings of the present invention as described herein.

[0070] Although specific exemplary embodiments of the present invention are described and illustrated in the accompanying drawings, it is to be understood that such embodiments are merely illustrative of the broad invention and are not limiting, and that embodiments of the present invention are not limited to the specific constructions and arrangements shown and described, since various other modifications may occur to those skilled in the art.

[0071] The contents of the claims as originally filed are described below as examples. [Example 1] 1. A medical system, comprising: A display system; A user input device; a control system communicatively coupled to the display system and the user input device; The control system includes: displaying image data corresponding to the three-dimensional anatomical region via the display system; receiving a first user input via the user input device to generate a first curve in the three-dimensional anatomical region; receiving a second user input via the user input device to generate a second curve in the three-dimensional anatomical region; determining an anatomical boundary bounded by the first curve and the second curve; the anatomical boundary represents a surface of an anatomical structure in the three-dimensional anatomical region; Healthcare system. [Example 2] 2. The medical system of example 1, wherein the anatomical boundary is determined from an intermediate curve between the first curve and the second curve. [Example 3] The medical system of example 1, wherein the control system is further configured to display the anatomical boundaries together with the image data via the display system. [Example 4] The medical system of example 3, wherein the anatomical boundaries are displayed overlaid on the image data via the display system. [Example 5] The control system includes: receiving a third user input via the user input device to generate a third curve in the three-dimensional anatomical region; adjusting the anatomical boundary so that it is bounded by the first curve, the second curve, and the third curve; 2. The medical system of claim 1, further configured to display the adjusted anatomical boundary together with the image data via the display system. [Example 6] The medical system of Example 1, wherein the control system is further configured to provide user-selectable options for providing the first user input and the second user input in freehand format or polyline format. [Example 7] 2. The medical system of claim 1, wherein the anatomical boundary corresponds to a three-dimensional surface mesh comprising a plurality of vertices. [Example 8] The control system includes: resampling each of the first curve and the second curve to an equal number of sample points; performing spline fitting between pairs of sample points at corresponding locations along the first curve and the second curve to generate a plurality of splines; The medical system of Example 7, configured to determine the plurality of vertices, including resampling each of the plurality of splines to generate the plurality of vertices. [Example 9] The control system includes: fitting a 3D spline surface to the first curve and the second curve; The medical system of Example 7, configured to determine the plurality of vertices, including resampling the 3D spline surface to generate the plurality of vertices. [Example 10] 2. The medical system of claim 1, wherein the control system is configured to determine the anatomical boundary based on an intensity gradient associated with the image data. [Example 11] The medical system of example 1, wherein the control system is further configured to apply computer vision to the image data to identify candidate anatomical boundaries, and the anatomical boundaries are snapped to the candidate anatomical boundaries. [Example 12] The medical system of Example 1, wherein the control system is further configured to display guidance information via the display system during placement of the second curve. [Example 13] 13. The medical system of example 12, wherein the guidance information includes a projection of the first curve in a plane of the image data other than a plane containing the first curve. [Example 14] 13. The medical system of claim 12, wherein the guidance information includes a start point and an end point of the first curve. [Example 15] 13. The medical system of example 12, wherein the guidance information includes an extrapolated projection of the anatomical boundary in an area outside the current extent of the anatomical boundary. [Example 16] The medical system of example 1, wherein the control system is further configured to display, via the display system, the anatomical boundaries overlaid on an anatomical model derived from the image data. [Example 17] The medical system of Example 16, wherein the control system is further configured to deform the anatomical boundaries to match deformations of the anatomical model based on movement of the patient's anatomical structure. [Example 18] The medical system of example 1, wherein the control system is further configured to display the anatomical boundaries overlaid on fluoroscopic image data obtained during a patient procedure. [Example 19] The control system includes: receiving a third user input while the medical instrument is positioned within the three-dimensional anatomical region; The medical system of example 1, further configured to: in response to the third user input, orient the tip of the medical instrument away from the anatomical boundary. [Example 20] The medical system of example 1, wherein the control system is further configured to determine a distance between a tip of a virtual medical instrument and the anatomical boundary. [Example 21] 2. The medical system of claim 1, wherein the control system is further configured to determine a distance between a tip of the medical instrument and the anatomical boundary. [Example 22] The medical system of Example 21, wherein the control system is further configured to provide a visual, auditory, or tactile indicator when the distance between the tip of the medical instrument and the anatomical boundary is less than a predetermined threshold distance. [Example 23] The medical system of Example 21, wherein the control system is further configured to modify an advancement speed of the medical instrument based on the determined distance. [Example 24] The medical system of Example 1, wherein the control system is further configured to provide one or more proposed deployment positions for the medical device, the one or more proposed deployment positions being located at least a threshold distance from the anatomical boundary. [Example 25] The medical system of Example 1, wherein the control system is further configured to generate a three-dimensional zone based on the instrument exit point and the target. [Example 26] The medical system of Example 25, wherein the control system is further configured to display a two-dimensional projection of the zone together with the image data and determine a risk portion of the surface based on an intersection between the surface and the zone. [Example 27] 27. The medical system of claim 26, wherein the first curve is generated at least partially along the intersection to mark the risk portion on the image data. [Example 28] 1. A method for planning a medical procedure, the method comprising: displaying, via a display system, image data corresponding to the three-dimensional anatomical region; receiving a plurality of user inputs via a user input device to generate a plurality of curves in the three-dimensional anatomical region; determining an anatomical boundary from the plurality of curves, the anatomical boundary delimiting a portion of interest of the three-dimensional anatomical region; and displaying, via the display system, the anatomical boundary overlaid on the image data. method. [Example 29] 29. The method of example 28, further comprising providing a user-selectable option for providing the plurality of user inputs in a freehand format or in a polyline format. [Example 30] 29. The method of example 28, wherein the anatomical boundary corresponds to a three-dimensional surface mesh comprising a plurality of vertices. [Example 31] resampling each of said plurality of curves to the same number of sample points; performing spline fitting between pairs of sample points at matching locations along the curves to generate a plurality of splines; 31. The method of example embodiment 30, further comprising: resampling each of the plurality of splines to generate the plurality of vertices. [Example 32] fitting a 3D spline surface to the plurality of curves; 31. The method of example embodiment 30, further comprising: resampling the cubic spline surface to generate the plurality of vertices. [Example 33] 29. The method of example 28, wherein the anatomical boundary is further determined based on a brightness gradient associated with the image data. [Example 34] 29. The method of example 28, further comprising applying computer vision to the image data to identify candidate anatomical boundaries, wherein the anatomical boundaries are snapped to the candidate anatomical boundaries. [Example 35] 29. The method of example embodiment 28, further comprising displaying guidance information via the display system while receiving the plurality of user inputs. [Example 36] 36. The method of example 35, wherein the guidance information comprises a projection of a first curve of the plurality of curves in a plane of the image data other than a plane containing the first curve. [Example 37] 36. The method of example embodiment 35, wherein the guidance information includes a start point and an end point of a first curve of the plurality of curves. [Example 38] 36. The method of example 35, wherein the guidance information comprises an extrapolated projection of the anatomical boundary in a region outside the current extent of the anatomical boundary. [Example 39] 29. The method of example 28, further comprising the step of deforming the anatomical boundary to match a deformation of an anatomical model generated from the image data, the deformation of the anatomical model being based on movement of a patient's anatomical structure. [Example 40] 29. The method of example 28, further comprising overlaying the anatomical boundary on fluoroscopic image data obtained during a patient procedure. [Example 41] 29. The method of example 28, further comprising determining a distance between a tip of a virtual medical instrument and the anatomical boundary. [Example 42] 29. The method of example 28, further comprising determining a distance between a tip of the medical instrument and the anatomical boundary. [Example 43] 43. The method of example 42, further comprising orienting the tip of the medical instrument away from the anatomical boundary based on the determined distance. [Example 44] The method of example 42, further comprising providing a visual, audio, or tactile indicator if the distance between the tip of the medical instrument and the anatomical boundary is less than a predetermined threshold distance. [Example 45] 43. The method of example 42, further comprising the step of altering a speed of advancement of the medical instrument based on the determined distance. [Example 46] 29. The method of example 28, further comprising the step of providing one or more proposed deployment locations for the medical device, the one or more proposed deployment locations being located at least a threshold distance from the anatomical boundary. [Example 47] 30. The method of example 28, further comprising generating a three-dimensional zone based on the instrument exit point and the target. [Example 48] The method of example 47, further comprising displaying a two-dimensional projection of the zone together with the image data to determine risk portions of the surface based on an intersection between a surface of an anatomical structure and the zone. [Example 49] 49. The method of claim 48, wherein one of the plurality of curves is generated at least partially along the intersection to mark the risk portion relative to the image data. [Example 50] A non-transitory machine-readable medium comprising a plurality of machine-readable instructions, the instructions being adapted, when executed by one or more processors associated with a planning workstation, to cause the one or more processors to perform the steps of: displaying, via a display system, the CT image data corresponding to the lungs; receiving a plurality of user inputs via a user input device to generate a plurality of curves for different slices of the CT image data; interpolating between the curves to determine an anatomical boundary indicative of a location of the pulmonary pleura in the CT image data; and displaying, via the display system, the anatomical boundary overlaid on the CT image data. Non-transitory machine-readable media.

Claims

1. 1. A method for planning a medical procedure, the method comprising: displaying, via a display system, image data corresponding to the three-dimensional anatomical region; receiving a plurality of user inputs via a user input device to generate a plurality of curves in the three-dimensional anatomical region; determining an anatomical boundary from the plurality of curves, the anatomical boundary delimiting a portion of interest of the three-dimensional anatomical region; displaying, via the display system, the anatomical boundary overlaid on the image data; generating a three-dimensional zone based on an instrument exit point and a target through which the medical tool is configured to extend; displaying a two-dimensional projection of the three-dimensional zone together with the image data to determine risk portions of the surface of the anatomical structure based on an intersection between the surface of the anatomical structure and the three-dimensional zone; the risk portion includes a portion that is at risk of puncture if the medical tool extends beyond the target; method.

2. The method of claim 1 , wherein the anatomical boundary corresponds to a three-dimensional surface mesh comprising a plurality of vertices.

3. resampling each of said plurality of curves to the same number of sample points; generating a plurality of splines by performing spline fitting and identifying respective sets of matching sample points from respective curves, each set of matching sample points including sample points from a respective curve; The method of claim 2 , further comprising: resampling each of the plurality of splines to generate the plurality of vertices.

4. fitting a cubic spline surface to the plurality of curves; The method of claim 2 , further comprising: resampling the cubic spline surface to generate the plurality of vertices.

5. The method of claim 1 , wherein the anatomical boundaries are further determined based on a brightness gradient associated with the image data.

6. The method of claim 1 , further comprising applying computer vision to the image data to identify candidate anatomical boundaries, wherein the anatomical boundaries are snapped to the candidate anatomical boundaries.

7. The method of claim 1 , further comprising the step of displaying guidance information via the display system while receiving the plurality of user inputs.

8. The method of claim 7 , wherein the guidance information comprises a projection of a first curve of the plurality of curves in a plane of the image data other than a plane containing the first curve.

9. The method of claim 7 , wherein the guidance information includes a start point and an end point of a first curve of the plurality of curves.

10. The method of claim 7 , wherein the guidance information comprises an extrapolated projection of the anatomical boundary in a region outside the current extent of the anatomical boundary.

11. 10. The method of claim 1, further comprising: deforming the anatomical boundary to match a deformation of an anatomical model generated from the image data, the deformation of the anatomical model being based on movement of a patient's anatomy.

12. The method of claim 1 , further comprising determining a distance between a tip of a virtual medical instrument and the anatomical boundary.

13. 10. The method of claim 1, further comprising providing one or more proposed deployment locations for a medical device, the one or more proposed deployment locations being located at least a threshold distance from the anatomical boundary.

Images