Robotic catheter system and method for displaying corrective procedures therefor

Abstract

A robotic catheter system and method for performing corrective actions during its use, comprising: providing a catheter having one or more bendable segments and a catheter tip; inserting the catheter into a body lumen and navigating the catheter through the lumen along an insertion trajectory while an actuator applies an actuation force to the one or more bendable segments; recording navigation parameters associated with the insertion trajectory as the catheter moves through the lumen; displaying on a display a graphical representation of the navigation parameters relative to thresholds indicative of locations on the insertion trajectory where catheter navigation errors may or have occurred; and outputting on the display an interactive user guide indicating one or more corrective actions that may be taken to correct and / or prevent the navigation errors.

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Application No. 63 / 305,567, filed February 1, 2022. The disclosure of the provisional application is incorporated herein by reference in its entirety for all purposes. The benefit of priority is claimed under 35 U.S.C. § 119(e).

[0002] FIELD OF THE DISCLOSURE The present disclosure relates to medical devices and, more particularly, to a robotic catheter system and a method for displaying information regarding corrective procedures during use of the robotic catheter system. [Background technology]

[0003] Robotic catheters and endoscopes include a flexible tubular shaft that is actuated by actuation forces (pull or push) applied through drive wires controlled by an actuator. The flexible tubular shaft (referred to herein as a "steerable catheter") may include multiple articulated sections configured to bend and rotate continuously like a snake. Typically, a steerable catheter is inserted through a natural orifice or small incision in the patient's body and deployed through the patient's body cavity (e.g., airway or blood vessel) to a target site (e.g., a site within the patient's anatomy designated for an intraluminal procedure such as an ablation or biopsy). A handheld controller (e.g., a joystick or gamepad controller) may be used as an interface for interaction between a user and the robotic system to control catheter navigation within the patient's body.

[0004] Navigation of the steerable catheter can be guided by a live view of a camera or videoscope located at the distal tip of the catheter shaft. To this end, a display device, such as a console-based or wall-mounted liquid crystal display (LCD) monitor, displays an image of the camera's field of view (FOV image) to assist the user in navigating the steerable catheter through the patient's anatomy (through the body lumen) to reach the target site. The camera view orientation, the controller coordinates, and the catheter's pose and shape are mapped (calibrated) before the catheter is inserted into the patient's body. As the user manipulates the catheter within the patient's anatomy, the camera transfers the camera's FOV image to the display device. Ideally, the displayed image should allow the user to interact with the endoscopic image as if the user's own eyes were actually inside the cavity of the endoscope.

[0005] In some robotic systems, sensors are used to detect catheter advancement and prevent the robot from harming the patient (e.g., exerting excessive force on soft tissue or blocking blood flow through prolonged compression, causing ischemia). In this regard, the current state of the art provides commercially available robotic catheter systems that can keep a record of catheter force and / or trajectory, display a graph of catheter trajectory along with the history of applied force and the current position of the catheter, and even issue warnings to the user. Such capabilities of the catheter system allow the user to take preventive measures, such as stopping catheter movement, when a collision between the catheter and the patient's anatomy occurs or is imminent. See, for example, U.S. Patent Nos. 9,333,650, 9,770,216, 10,111,723, and 10,271,915, U.S. Pregrant Patent Application Publication Nos. 2022 / 0125527 and 2020 / 0054399, and WO 2018 / 005861. These publications are incorporated herein by reference in their entireties.

[0006] The above documents disclose a system for verifying the indication of the robot catheter situation and warning the operator of catheter collision, but there is no related technology that provides an indication of corrective action to get out of the warning situation or to avoid a critical failure situation. For example, one of the above documents can form a virtual barrier based on force measurements at a specific position, but the related technology does not necessarily guarantee that the user can continue navigation normally because it does not address the root cause of the critical situation.

[0007] Therefore, there is a need for a robotic catheter system and method for displaying information to a user in a manner that guides the user to take corrective action to address the root cause of a critical situation. Summary of the Invention

[0008] According to at least one embodiment, a method of operating a robotic catheter system configured to manipulate a steerable catheter is provided having a catheter body including one or more bendable segments and a catheter tip, and including an actuation portion coupled to the bendable segments via one or more drive wires disposed along a wall of the catheter body. The method includes inserting at least a portion of the catheter body into a body lumen along an insertion trajectory that follows a length of the lumen; recording navigation parameters of the catheter while at least the catheter tip is within the lumen; and displaying a graphical representation of the catheter to indicate the navigation parameters relative to a threshold at which a navigation error (e.g., catheter collision or system malfunction) may or has occurred. The area or volume of the graphical representation indicates one or more corrective actions to correct and / or prevent the navigation error.

[0009] Various embodiments disclosed in this disclosure provide several advantages over conventional robotic catheter systems. According to one embodiment, it is advantageous to provide a graphical representation of a force timeline diagram that maps points, events, and flags throughout the procedure to several thresholded force levels (e.g., different thresholds can be set for each bendable segment of the catheter). A force timeline diagram with several thresholded force levels allows the user to identify locations where force levels are acceptable (below thresholds) and resolve navigation errors (catheter collisions and system malfunctions) by retracting the catheter tip to those locations. By mapping catheter tip poses throughout the procedure to specific thresholded force levels, the user can avoid future collisions by identifying poses that minimize the likelihood of collisions. A graphical representation of the scales and ranges of monitored catheter parameters (wire force, catheter tip position, catheter pose, catheter tip bend angle, etc.) is communicated to the user in real time. A graphical representation of various levels of warning allows the user to confidently approach parameter thresholds with greater precision to avoid undesired navigation events (catheter collisions and system malfunctions). In particular, if the graphical representation shows navigation parameters associated with the state of individual drive wires, the user or the system can take more finely tuned corrective actions for specific navigation errors.

[0010] It is to be understood that the foregoing summary and detailed description are exemplary and explanatory in nature and are intended to provide a thorough understanding of the present disclosure without limiting the scope of the disclosure. Further objects, features and advantages of the present disclosure will become apparent to those skilled in the art upon reading the following detailed description of illustrative embodiments in conjunction with the accompanying drawings and the appended claims. [Brief description of the drawings]

[0011] [Figure 1]FIG. 1 illustrates a robotic catheter system 1000 configured to manipulate a steerable catheter 100 having one or more bendable segments and a catheter tip that can be used in a typical medical environment, such as an operating room. [Diagram 2] FIG. 2 illustrates a functional block diagram and components of a robotic catheter system 1000. [Diagram 3] FIG. 3 shows details of a steerable catheter 100 having one or more bendable segments according to an embodiment of the present disclosure. [Figure 4] 4A, 4B, and 4C illustrate various examples of insertion trajectories of a steerable catheter 100 inserted into a lumen, including potential points of catheter collision where a user may perform one or more corrective procedures, according to embodiments of the present disclosure. [Diagram 5] FIG. 5 illustrates a graphical representation of a force timeline diagram associated with the insertion trajectory and current position of a steerable catheter 100 according to an embodiment of the present disclosure. [Figure 6] FIG. 6 illustrates a graphical representation of a concentric force diagram associated with an endoscopic live view image surrounded by a radar plot showing the forces applied to the steerable catheter 100 to generate a catheter tip pose in accordance with an embodiment of the present disclosure. [Figure 7] 7A and 7B illustrate example graphs of a generalized catheter parameter X relative to a threshold at which catheter collision and / or system malfunction may occur, according to an embodiment of the present disclosure. [Figure 8] FIG. 8 illustrates a graphical user interface for enabling a user to perform corrective procedures during navigation of the steerable catheter 100. [Figure 9] 9A and 9B illustrate a graphical user interface for enabling a user to perform corrective actions based on parameters of two or more segments of the steerable catheter 100. [Figure 10] FIG. 10 illustrates a graphical user interface for enabling a user to perform one or more corrective actions to correct or prevent catheter collisions and / or system failures according to an embodiment of the present disclosure. [Figure 11] FIG. 11 shows an example process 1100 of a method of operation of the robotic catheter system 1000 for displaying a graphical representation of navigation parameters relative to thresholds at which navigation errors may occur (areas of the graphical representation indicate one or more corrective actions that may be taken to correct and / or prevent navigation errors). DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0012] Aspects of the present disclosure can be understood by reading the following detailed description in light of the accompanying drawings. It should be noted that, according to standard practice, the various features of the drawings are not drawn to scale, and are not intended to represent actual components. Some details, such as dimensions of various features, may be arbitrarily increased or decreased for ease of illustration. Furthermore, reference numbers, labels and / or letters may be repeated in various instances to depict similar components and / or functions. This repetition is for the purposes of simplicity and clarity, and is not, in itself, intended to limit the various embodiments and / or configurations of the same components described.

[0013] Before the various embodiments are described in further detail, it should be understood that the present disclosure is not limited to particular embodiments. It should also be understood that the terminology used herein is for the purpose of describing example embodiments only and is not intended to be limiting. The embodiments of the present disclosure may have many applications within the field of medical procedures or minimally invasive surgery (MIS).

[0014] Throughout the figures, the same reference numerals and characters, unless otherwise stated, are used to denote like features, elements, components or portions of the illustrated embodiments. Moreover, while the present disclosure will be described in detail with reference to the enclosed figures, it is done so in connection with the illustrated embodiments. It is intended that changes and modifications can be made to the illustrated exemplary embodiments without departing from the true scope of the present disclosure, which is defined by the appended claims. While the drawings represent some possible configurations and approaches, the drawings are not necessarily to scale, and certain features may be exaggerated, omitted or partially cut away in order to more clearly illustrate and explain certain aspects of the present disclosure. The description set forth herein is not intended to be exhaustive or otherwise limit or restrict the scope of the claims to the exact forms and configurations shown in the drawings and disclosed in the following detailed description.

[0015] Those skilled in the art will appreciate that the terms used herein, in general, and in the appended claims, particularly (e.g., the body of the appended claims), are generally intended as "open" terms (e.g., the term "including" should be interpreted as "including, but not limited to," the term "having" should be interpreted as "having at least," "includes" should be interpreted as "including, but not limited to," etc.). Moreover, those skilled in the art will appreciate that, where specific numbers of introduced claim recitations are intended, such intent will be expressly set forth in the claims, and, in the absence of such recitation, no such intent exists. For example, as an aid to understanding, the following appended claims may include the use of the introductory phrases "at least one" and "one or more" to introduce claim recitations. However, the use of such phrases should not be construed as meaning that the introduction of a claim statement with the indefinite article "a" or "an" means that a particular claim containing such an introduced claim statement is limited to claims containing only one such statement, even if the same claim contains both the introductory phrase "one or more" or "at least one" and an indefinite article such as "a" or "an" (e.g., "a" and / or "an" should typically be construed to mean "at least one" or "one or more"). The same is true for the use of definite articles used to introduce claim statements.

[0016] Furthermore, even if specific numbers in the claims are explicitly stated, it is understood by those skilled in the art that such a statement should typically be interpreted to mean at least the stated number (e.g., when only "two statements" is stated without other modifiers, it typically means at least two statements, or two or more statements). Furthermore, when a definition similar to "at least one of A, B, and C, etc." is used, such a syntax is generally intended in the sense that a person skilled in the art can understand the definition (e.g., "a system having at least one of A, B, and C" may include a system having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together). When a definition similar to "at least one of A, B, or C, etc." is used, such syntax is generally intended in the sense that one of ordinary skill in the art would understand the definition (e.g., "a system having at least one of A, B, or C" can include a system having A alone, B alone, C alone, A and B together, A and C together, B and C together, and / or A, B, and C together, etc.). Furthermore, one of ordinary skill in the art will appreciate that typically disjunctions and / or phrases (whether in the specification, claims, or drawings) that express two or more alternative terms should be understood to contemplate the possibility of including one of the terms, either of the terms, or both of the terms, unless the context indicates otherwise. For example, the expression "A or B" is typically understood to include the possibilities of "A" or "B" or "A and B."

[0017] In this specification, when a feature or element is referred to as being "on" another feature or element, it may be directly on the other feature or element, or intervening features and / or elements may also be present. In contrast, when a feature or element is referred to as being "directly on" another feature or element, there are no intervening features or elements present. It should also be understood that when a feature or element is referred to as being "connected," "attached," "coupled," or the like, to another feature or element, it may be directly connected, attached, or coupled to the other feature, or there may be intervening features or elements present. In contrast, when a feature or element is referred to as being "directly connected," "directly attached," or "directly coupled" to another feature or element, there are no intervening features or elements present. Although described or illustrated with respect to one embodiment, features and elements so described or illustrated in one embodiment may be applicable to other embodiments. Additionally, as will be understood by one of ordinary skill in the art, a reference to a structure or feature being located "adjacent" another feature may have portions that overlap or underlie the adjacent feature.

[0018] In this specification, terms such as first, second, third, etc. may be used to describe various elements, components, regions, parts, and / or portions. It is to be understood that these elements, components, regions, parts, and / or portions are not limited by these designated terms. These designated terms are used only to distinguish one element, component, region, part, or portion from another region, part, or portion. Thus, a first element, component, region, part, or portion described below can be referred to as a second element, component, region, part, or portion merely for purposes of distinction, but without limitation, and without departing from the structural or functional meaning.

[0019] As used herein, the singular is intended to include the plural unless the context clearly dictates otherwise. Furthermore, it should be understood that the terms "comprise," "comprising," and "consisting," when used in this specification and claims, specify the presence of the stated features, integers, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof that are not expressly recited. Furthermore, in this disclosure, the transitional phrase "consisting of" excludes any element, step, or component not specified in the claim. It should also be noted that some claims or some features of the claims may be drafted to exclude any element, and such claims may use exclusive language such as "solely," "only," or "negative" limitations in connection with the recitation of the claim elements.

[0020] The term "about" or "approximately" as used herein means, for example, within 10%, within 5% or less. In some embodiments, the term "about" may mean within measurement error. In this regard, when described or claimed, all numerical values ​​may be read as if they were preceded by the word "about" or "approximately", even if the term is not explicitly indicated. The phrase "about" or "approximately" may be used when describing a size and / or location to indicate that the stated value and / or location is within a reasonable expected range of value and / or location. For example, a numerical value may include values ​​that are ±0.1% of the stated value (or range of values), ±1% of the stated value (or range of values), ±2% of the stated value (or range of values), ±5% of the stated value (or range of values), ±10% of the stated value (or range of values), etc. As described herein, any numerical range is intended to include the end value and includes all subranges therein, unless specifically stated otherwise. As used herein, the term "substantially" means allowing for deviations from the descriptor that do not adversely affect the intended purpose. For example, deviations resulting from limitations in measurements, differences within manufacturing tolerances, or variations of less than 5% can be considered to be within substantially the same range. The specified descriptors can be absolute (e.g., substantially spherical, substantially vertical, substantially concentric, etc.) or relative terms (e.g., substantially similar, substantially the same, etc.).

[0021] Unless specifically stated otherwise, and as will be apparent from the disclosure that follows, discussions throughout this disclosure using terms such as "processing," "computing," "calculating," "determining," "displaying," and the like are understood to refer to operations and processes of a computer system, or similar electronic computing device, or data processing device that manipulates and converts data represented as physical (electronic) quantities in the registers and memory of a computer system into other data that is also represented as physical quantities in the memory or registers of a computer system or other device for storing, transmitting, or displaying such information. The computer or electronic operations described herein or recited in the appended claims can generally be performed in any order, unless the context dictates otherwise. Also, while various operational flow diagrams are presented in a sequential order, it will be understood that the various operations can be performed in orders other than those shown or claimed, or operations can be performed simultaneously. Examples of such alternative orderings include overlapping, interleaving, interrupting, reordering, incremental, preparatory, supplementary, simultaneous, reverse, or other variant orderings, unless the context dictates otherwise. Moreover, terms such as "in response to," "in response to," "in connection with," "based on," or other similar past tense adjectives generally do not intend to exclude such variations unless the context indicates otherwise.

[0022] As used herein, the term "real-time" refers to describing a process or event, for example, being communicated, shown, or presented, substantially simultaneously as the process or event actually occurs. Real-time refers to a level of computer responsiveness that a user perceives as being sufficiently immediate or that the computer can follow some external process. For example, in computer technology, the term real-time refers to the actual time that something happens, and a computer can at least partially process data in real-time (as the data is received). As another example, "real-time" processing in signal processing refers to systems such as missile guidance, airline reservation systems, and real-time quotes (RTQs) on the stock market, where input data is processed within milliseconds and is available as feedback almost immediately.

[0023] The present disclosure relates generally to medical devices and illustrates embodiments of an endoscope or catheter, and more specifically, to a steerable catheter controlled by a medical continuum robot (MCR). The embodiments of the endoscope or catheter and portions thereof are described with respect to their state in three-dimensional space. As used herein, the term "position" refers to the position of an object or part of an object in three-dimensional space (e.g., three translational degrees of freedom along Cartesian X, Y, Z coordinates), the term "orientation" refers to the rotational configuration of an object or part of an object (three degrees of rotational freedom - e.g., roll, pitch, yaw), the term "pose" refers to the position of an object or part of an object in at least one translational degree of freedom and the orientation of an object or part of an object in at least one rotational degree of freedom (up to six degrees of freedom in total), and the term "shape" refers to a set of poses, positions and / or orientations measured along the elongated body of an object.

[0024] As is known in the medical device art, the terms "proximal" and "distal" are used in reference to the operating end of an instrument that extends from a user to a surgical or diagnostic site. In this regard, the term "proximal" refers to the portion of the instrument that is closer to the user (e.g., the handle) and the term "distal" refers to the portion of the instrument that is further from the user and closer to the surgical or diagnostic site (the tip). It will further be appreciated that for convenience and brevity, spatial terms such as "vertical," "parallel," "above," and "below" may be used herein with respect to the drawings. However, surgical instruments are used in many orientations and positions, and these terms are not intended to be limiting and / or absolute. In that regard, all directional references (e.g., superior, inferior, upward, downward, left, right, leftward, rightward, top, bottom, above, below, vertical, horizontal, clockwise, counterclockwise) are used for identification purposes only to aid the reader in understanding this disclosure, and are not intended to create limitations with respect to the location, orientation, or use of this disclosure in particular.

[0025] As used herein, the term "catheter" generally refers to a flexible, thin, tubular instrument made from medical grade materials designed to be inserted through a narrow opening into an anatomical body cavity (e.g., an airway or a blood vessel) to perform a wide variety of medical functions. The more specific term "steerable catheter" refers to a medical instrument comprising an elongated, flexible shaft having at least one tool channel extending through a plurality of bendable segments that are driven by an actuator that applies an actuation force via a drive wire disposed along the wall of the shaft.

[0026] As used herein, the term "endoscope" refers to a rigid or flexible medical instrument that uses light guided by an optical probe to view the inside of a body cavity or organ. A medical procedure in which an endoscope is inserted through a natural orifice is called endoscopy. Specialized endoscopes are generally named for how or where they are used, such as bronchoscope (mouth), sigmoidoscope (rectum), cystoscope (bladder), nephroscope (kidney), bronchoscope (bronchi), pharyngoscope (pharynx), otoscope (ear), arthroscope (joint), laparoscope (abdomen), and gastrointestinal endoscope.

[0027] In this disclosure, terms such as "optical fiber" or simply "fiber" refer to an elongated, flexible, light-conducting waveguide capable of conducting light from one end to the other by an effect known as total internal reflection. The terms "light-guiding component" or "waveguide" may also refer to or function as optical fibers. The term "fiber" may refer to one or more light-conducting fibers.

[0028] <Robotic Catheter System> An example configuration of a robotic catheter system 1000 is described with reference to Figures 1 and 2. Figure 1 illustrates an example representation of a medical environment, such as an operating room, in which the robotic catheter system 1000 may be used. The system 1000 may include a robotic catheter 110 that is maneuverable by a user 10 (e.g., a physician) to perform an intraluminal procedure on a patient 80. The system 1000 may include a computer 400 operatively connected to the robotic catheter 110 via a robotic platform 90. The robotic platform 90 includes one or more robotic arms 92 and a translational linear stage 91. 。The computer 400 (e.g., a system console) includes at least a central processing unit (CPU) 410, which may be comprised of one or more processors, and a display screen 420 (display device), such as a liquid crystal display (LCD), OLED, QLED display, etc. As shown in Figure 2, the CPU 410 is operatively connected to a storage memory 411 (ROM and RAM memory), a system interface 412 (e.g., an FPGA card), a user interface 413 (e.g., a mouse and keyboard), and the display screen 420.

[0029] The robotic catheter 110 includes a handle 200 and a steerable catheter 100. The steerable catheter 100 is removably attached to the handle 200 via a connector assembly 50 (connector hub). The steerable catheter 100, sometimes referred to as a continuum robotic catheter or a snake robotic catheter, is configured to form a continuous curve based on actuation principles known in the art. A well-known approach to forming a continuous curve using a continuum robotic catheter is the follow-the-leader (FTL) technique. The handle 200 is connected to an actuator system 300, which receives electronic commands from a computer 400 to mechanically actuate the steerable catheter 100. The handle 200 is configured to be removably attached to a robotic platform 90, which is configured to robotically guide the steerable catheter 100 through a body cavity 81 toward a target 181 within a subject or patient 80. When the handle 200 is not attached to the robotic platform 90, the handle 200 allows a user 10 to control the steerable catheter 100. but One or more knobs 252 Using2. For treatment or examination of the patient 80, the robotic catheter 110 may include one or more access ports 250 located on or about the handle 200. The access ports 250 are used to introduce end effector tools and pass fluids to and from the patient 80. A tracking system (e.g., having an electromagnetic (EM) field generator 60 and one or more EM sensors 190 located on the steerable catheter 100) is used to track the position, shape, attitude, and / or orientation of the steerable catheter 100 during insertion through the body cavity 81 toward a target 181. The target 181 is a region of interest (e.g., the center of a tumor or lesion) located at or near the lumen 81 of the patient 80. As an alternative or in addition to an EM component, the tracking system may include magnetic and / or radiopaque markers.

[0030] During an intraluminal procedure, the system's processor or CPU 410 is configured to execute (process) computer executable code pre-stored in the system's memory 411 to perform actions based on user input. The display screen 420 may include a graphical user interface (GUI) configured to display a graphical representation 421 of catheter navigation parameters and patient information, an endoscopic image 422 (live view image), an intraoperative guided image 423, and a preoperative image 424 (e.g., 3D or 2D slice image) of the region of interest of the patient 80. The intraoperative guided image 423 may include conventional fluoroscopic, sonographic, or ultrasound images. The preoperative images 424 may include 2D or 3D computed tomography (CT) or magnetic resonance imaging (MRI) images.

[0031] As shown in Figure 2, the steerable catheter 100 is comprised of a proximal section 140, a distal section 130, and a stiff catheter tip 120, which are arranged in that order from the proximal end to the distal end along the catheter axis (Ax). The distal section 130 is a steerable section made up of multiple bendable segments. The proximal section 140 includes a flexible, non-steerable tubular shaft, which serves to connect the steerable section 130 to a handle 200. At least the catheter tip 120 includes a tracking sensor 190 (e.g., one or more EM sensors), which is tracked by the system 1000 based on the EM field generator 60.

[0032] The steerable catheter 100 is controlled by an actuation system, which consists of a handle 200, an actuator system 300, a robotic platform 90, and / or a handheld controller 205 (e.g., a gamepad controller or joystick), which are in electronic communication with a computer 400 via a cable or a network connection 425. The actuator system 300 includes a microcontroller 320 and an actuator 310, which are operatively connected to the computer 400 via the network connection 425. The microcontroller 320 may include a proportional-integral-derivative (PID) controller or other similar digital signal processor (DSP) circuit. The actuator 310 includes a number of actuator servo motors (or piezoelectric actuators) M1-Mn, where "n" may be equal to the number of drive wires 210 required to steer the steerable catheter 100.

[0033] ActuatorThe system 300 also includes one or more sensors 304. The sensors 304 may include strain sensors and / or position sensors. The strain sensors may be implemented, for example, by strain gauges or piezoresistors. The strain sensors function to detect and / or measure the compressive or tensile forces on each drive wire 210. In this case, the strain sensors output a signal 305 corresponding to the amount of compressive or tensile forces (amount of strain) being applied to each drive wire 210 during actuation of the steerable catheter 100. The sensors 304 may output a signal 305 corresponding to the amount of movement (displacement distance) of each actuated drive wire 210. The sensors 304 measuring the amount of displacement of the drive wires may also be implemented by Hall effect sensors. The sensors 304 may also be part of a tracking system implemented by electromagnetic (EM) sensors configured to measure and / or detect the position and orientation (pose) of the catheter tip 120. Signals 305 from sensors 304 (strain sensors, displacement sensors, and / or attitude or position sensors) on one or more drive wires 210 are sent to the controller 320 and / or computer 400 to provide real-time feedback and form closed-loop control for each of the motors or actuators. In this manner, each drive wire 210 can be actively controlled to provide appropriate shaft guidance to safely navigate the steerable catheter 100 through the lumen 81.

[0034] The computer 400 includes appropriate software, firmware, and peripheral hardware operated by one or more processors of the CPU 410. The computer 400, the actuator system 300, and the handle 200 are operatively connected to each other by a network connection 425 (e.g., a cable bundle or a wireless link). Additionally, the computer 400, the actuator system 300, and the handle 200 are operatively connected to each other by a robotic platform 90. In some embodiments, the actuator system 300 may include or be connected to a handheld controller (such as a gamepad controller) or a portable computing device (such as a smartphone or tablet). Among other functions, the computer 400 and the actuator system 300 may provide a graphical user interface (GUI) and navigation information to a surgeon or other operator via a display screen 420 for operating the steerable catheter 100.

[0035] FIG. 3 shows an exemplary embodiment of a steerable catheter 100. The proximal section 140 is configured to be attached to the handle 200 via a connector assembly 50. The steerable distal section 130 includes a number of bendable segments configured to be actuated by drive wires 210 disposed along the wall of the catheter. The bendable segments of the steerable catheter 100 may include a distal bendable segment 130A, a middle bendable segment 130B, and a proximal bendable segment 130C. Each bendable segment is formed by a number of annular components (rings). The annular components are defined as wire guiding members 108 or anchoring members 109 depending on their function in the catheter. The anchoring members 109 are annular components to which the distal ends of one or more drive wires 210 are attached. The wire guiding members 108 are annular components through which some of the drive wires 210 slide (without being attached to them).

[0036] Detail A of FIG. 3 illustrates an exemplary embodiment of the annular components (wire guide member 108 or anchor member 109). Each annular component includes a central opening forming a tool channel 105 and multiple conduits 104 (sub-channels or through holes) formed lengthwise along the annular wall of each annular component, spaced equidistant from the central opening. The non-steerable proximal section 140 is a tubular shaft made from an extruded polymeric material. The tubular shaft of the proximal section 140 also has a central opening or tool channel 105 and multiple conduits 104 along the wall of the shaft surrounding the tool channel. Thus, at least one tool channel 105 formed in the steerable catheter 100 allows for the passage of an imaging device 180 and / or an end effector tool from the access port 250 to the distal end of the catheter.

[0037] Imaging devices 180 that can be inserted through the tool channel include an endoscopic camera (videoscope) with illumination optics (e.g., fiber optics and LEDs) that provide illumination to illuminate a lesion target 181, which is an area of ​​interest within the patient. End effector tools refer to endoscopic surgical instruments such as clamps, forceps, scissors, staplers, ablation or biopsy needles, and other similar tools that function to manipulate a body part (organ or tumor tissue) during examination or surgery.

[0038] An example of robotically controlled navigation of the steerable catheter 100 will now be described. In general, when inserting or retracting the steerable catheter 100 through the body cavity 81, the centerline of the lumen (e.g., the centerline of the airway in the lungs) is considered as the desired trajectory to follow during active control of the bendable segments of the steerable section 130 (see FIG. 3). To this end, the steerable catheter 100 is robotically steered using various kinematic techniques with the goal of controlling the bendable segments and guiding the catheter tip 120 along the desired trajectory to reach the target. In one such example, during robotically controlled navigation, the steerable catheter 100 advances through the lumen 81 while sensors measure the insertion depth of the catheter shaft, monitor the force applied to the catheter, and measure the angular position of the catheter tip to obtain trajectory information. The trajectory information is stored in the system's memory and is continuously updated. After a small advance in the insertion depth, the shape of the steerable catheter is updated by steering (rotating, twisting, or bending) one or more of the bendable segments of the catheter such that the new shape closely matches the desired trajectory. This process is repeated until the target is reached. The same process applies (in reverse) when the steerable catheter is withdrawn from the patient's lumen. The segments of the distal steerable section 130 can be controlled individually to direct the catheter tip 120 by combined actuation of all the bendable segments. Alternatively, the catheter tip can be steered in a follow-the-leader (FTL) approach by controlling the most distal segment and the remaining segments that follow the path taken by the most distal segment. A reverse FTL (rFTL) process can be implemented when withdrawing the catheter.

[0039] In robotic catheter systems such as those described above, catheter collisions against the patient's anatomy can occur if the catheter trajectory is not maintained within the lumen constraints. In general, when navigating along straight sections of the lumen, it is desirable to keep the catheter along the centerline of the lumen. In particular, when turning, a stiff catheter tip requires that the trajectory be offset from the centerline in order to navigate "around corners" with sharp turns. Thus, when navigating through tortuous anatomy, the approach path followed by the catheter tip may deviate from the intended trajectory for various reasons (e.g., patient motion or user intervention). Deviations from the ideal path (insertion trajectory) guided by the user may propagate from the most distal section to subsequent bendable segments, depending on the control algorithm used. Furthermore, the position of the robotically controlled bendable segments may deviate from the user-guided path for various reasons, such as differences in the design of the sections, differences in the tolerances for each bendable segment, differences in the location of the base of the section, etc.

[0040] This challenge can occur frequently during navigation for many reasons, as described in the literature disclosed by the related art. The first solution to improve catheter navigation is to address the root cause of what is causing the error during navigation. In other words, according to the present disclosure, one solution is to retract the catheter by a certain amount of the insertion trajectory and try an alternative trajectory approach that takes into account the catheter force timeline.

[0041] Correlating navigation forces with catheter position timeline The first step to enable catheter retraction and the initiation of a new catheter approach is to relate the forces applied to the various sections of the catheter to the insertion trajectory followed by the catheter during the initial insertion. More specifically, the present disclosure proposes to relate the navigation force timeline to the current (live) position of the catheter tip 120 in order to perform corrective actions when a particular error has occurred or is about to occur. In this regard, the current state of the art knows that force-time graphs are used to track the position of steerable catheters. See, for example, the patents and patent documents listed in the Background section. However, the information provided by conventional force-time graphs is not sufficient to allow the user to take corrective actions that can resolve the root cause of the navigation error (e.g. catheter collision, system malfunction, catheter damage, etc.).

[0042] According to an embodiment of the present disclosure, by relating (e.g., mapping or registering) the current (real-time) insertion position and orientation of the catheter to a timeline of forces applied to the catheter (force timeline), the system can continuously monitor the catheter movement and provide actionable feedback in real-time. In this way, the user can instantly understand how the forces applied to the catheter tip 120 have changed based on the history of recorded points / positions and the corresponding history of forces applied to the catheter while the catheter is moving through the patient's anatomy. In other words, the system can prompt the user to retract the catheter, for example, to a position / point where no error or collision occurred or where the catheter was within a predefined range of safe force values. After the catheter is returned to a position with minimal interference or no collision, the system prompts the user to repeat the insertion with a different (new) trajectory approach. This different trajectory approach can be calculated by the system based on the navigation force timeline recorded just before a particular error. Alternatively, the system can calculate a different (new) trajectory based on the difference between the planned insertion trajectory and the trajectory recorded just before the occurrence of a navigation error.

[0043] Here, examples of navigational error occurrences include catheter collisions and system malfunctions of the type known in the art (see, e.g., U.S. Pat. No. 1,011,1723). However, the various embodiments disclosed herein are equally applicable to any other errors associated with catheter navigation. To the extent that the system can provide a user-guiding display that allows the user to at a glance see the forces of one or more drive wires relative to the root cause of the error (crash, malfunction, damage threshold, etc.), it will be appreciated by those skilled in the art that being able to see the wire forces and interactively take corrective action is highly beneficial. In particular, seeing the wire forces relative to the position / orientation of the catheter tip relative to the lumen and relative to each other in real time is advantageous. For example, if most of the virtual representations of the wire forces are close to a threshold (i.e., close to maximum tensile or compressive forces), the system can provide a useful early warning to the user that something may be wrong.

[0044] According to one embodiment, the system uses an FTL approach for navigation, recording and displaying the history of forces applied to various segments of the catheter (historical force timeline) in relation to the insertion trajectory, thereby monitoring the current location where a navigation error, such as a collision, has occurred or is likely to occur. One way this mapping can be displayed to the user is through a force timeline (force-time graph), for example as shown in Figures 5 and 7B. Advantageously, in addition to the force-time graph, the system provides a user guide with specific corrective actions to address the root cause of the navigation error.

[0045] 4A-4C illustrate a hypothetical example in which a steerable catheter 100 is inserted into a lumen 81 and a user takes corrective action to address a catheter-lumen collision. The system can record all three parameters: force, time, and position. In one embodiment, the force data is displayed relative to a threshold value along the insertion trajectory. In FIG. 4A, the steerable catheter 100 is initially inserted into a first position P1 and advanced by the linear stage 91 along the center of the lumen 81 in a linear direction to a second position P2. After position P2, the robotic system controls one or more bendable segments of the steerable catheter 100 to bend one or more segments of the catheter and steer the catheter tip 120 toward a third position P3. To advance to position P3, both the linear stage 91 and one or more bendable segments of the catheter are actuated. If the trajectory taken by the catheter tip is incorrect (e.g., the catheter is bent too much in one direction), the catheter tip 120 may experience a first collision C1 at a position between the second position P2 and the third position P3. In this case, a solution to guide the catheter tip to the correct trajectory (e.g., center of the lumen) is to calculate the inverse kinematic values, retract the catheter tip to position P2 (last known safe position) and try again. Another solution is to store the forces applied to the drive wire with respect to the catheter's orientation at the position of collision C1, then put the catheter into a relaxed mode (i.e., stop the actuation forces) and automatically retract the catheter until the catheter tip returns to position P2. At position P2, the system recalculates the forward kinematic values ​​to guide the distal tip 120 to position P3 ("around the corner" of the first curved portion of the lumen 81). The system can then use the recorded forces or position coordinates of collision C1 to calculate a new / different trajectory to reach position P3 and continue forward.

[0046] 4B illustrates the case where the catheter tip 120 experiences a second collision C2 between the third position P3 and the fourth position P4. Similar to dealing with the previous collision C1, the solution to deal with the second collision C2 is to retract the catheter tip to a safe (error-free) position where the catheter was before the collision C2. For example, the catheter tip 120 can be retracted to position P3 using the trajectory previously recorded by the system. At position P3, the forward kinematic values ​​are recalculated by the system to keep the catheter tip 120 along the correct trajectory within the lumen 81 until the catheter tip reaches the fourth position P4.

[0047] FIG. 4C illustrates the case where the catheter tip 120 makes a third collision C3 after passing through position P4, before reaching the first target TG1 at position P5 or the second target TG2 at position P6. When navigating the catheter tip 120 to the second target TG2 through the sublumen 81b, the catheter tip 120 can be slightly deflected from the collision C3 without the need to retract. This solution would be sufficient, since the second target TG2 is already approximately aligned with the distal tip 120 of the steerable catheter 100. On the other hand, when navigating the catheter tip 120 to the first target TG1 through the sublumen 81a, the catheter tip 120 needs to be retracted at least to position P4 (the last known error-free or safe position). After the catheter tip retracts to position P4, the system will recalculate the forward kinematics of the catheter to keep the catheter tip along the approximate center of the sublumen 81 until the catheter tip reaches the first target TG1 without any further collision. This process is accomplished by continuously monitoring the catheter's movement and providing actionable feedback in real time.

[0048] It should be appreciated that collision of the catheter against the lumen may occur not only when the catheter tip navigates through a tortuous portion of the lumen, but also when one or more of the bendable segments get stuck on the lumen wall. If the catheter body gets stuck on the lumen wall, the system puts the catheter into a relax mode, retracts the catheter tip to the last known safe position (error-free position), and recalculates the navigation trajectory from the last known safe position to the target. In one embodiment, the system provides a segmented model of the lung and displays force data on the segmented model to display the applied forces for each bifurcation (each tracheal bifurcation of the lung airway) along the insertion trajectory of the catheter tip. If a corrective action is required (e.g., a relax mode is implemented), the system can record in the force history timeline the insertion depth and catheter pose at the position where the corrective action was performed. The system can then use the recorded data to return the catheter tip to the last known error-free position and / or avoid navigation through error-prone positions.

[0049] 5 illustrates an embodiment of a graphical representation of a force timeline 500 displayed on the display screen 420 by the robotic system 1000. In this embodiment, the left-right direction of the drawing represents insertion depth, and the colored border of each section along the timeline represents the maximum force experienced by the tip at that insertion location. In FIG. 5, the graphical representation of the force timeline 500 includes a number of associated points or tick marks 502-508 that represent navigational events on the insertion trajectory of the steerable catheter 100. For example, force The first tick mark 502 on the timeline 500 represents the initial point or starting position P1 where the catheter tip is inserted into the patient's body. The final point or last tick mark 508 represents the current position (live position) of the catheter tip during the procedure. The intermediate tick marks (second tick mark 504 and third tick mark 506) represent navigation events between the starting position P1 and the current position P4 on the catheter trajectory (insertion depth). In some embodiments: forceThe timeline 500 can be labeled with particular colors or patterns, each of which can indicate a distinct navigation event controlled by a different threshold. For example, the force timeline 500 can be color-coded such that green represents an error-free catheter trajectory, yellow or orange represents catheter trajectories where the catheter parameters are close to a threshold (i.e., a range of force values ​​that are conservatively applied), and red or flashing red represents events where a catheter collision is imminent or has occurred.

[0050] In FIG. 5, the force timeline 500 can be expanded by adding one or more flags representing specific navigation parameters of the catheter as it advances through the lumen 81. One or more flags can be added during catheter insertion automatically by the system or manually by the user. During catheter insertion, the user can interact with the force timeline 500 and / or flags to learn more about the status / condition of the catheter parameters at any point along the history of the insertion trajectory. In FIG. 5, for example, the system automatically adds a first flag (S-Flag-1) when the catheter is inserted into the patient's body cavity. S-Flag-1 can be associated with the initial orientation of the catheter tip 120 (e.g., provided by the EM sensor 190), the velocity of the linear stage 91, etc. The system can also add a second flag (S-Flag-2) when the catheter tip 120 makes a first collision C1, and a third flag (S-Flag-3) when the catheter tip 120 makes a second collision C2, for example. Similarly, a user may manually add a first user flag (U-Flag-1) and a second flag (U-Flag-2) to record, for example, the amount of force that caused a collision and the force that caused the system to change color on the force timeline 500. The flags added by the user and / or the system may be associated with the number of lumen bifurcations that the catheter tip 120 passed through during insertion.

[0051] Flags added by the user or the system are not limited to the above examples. Other flags may be added to record or display other parameters such as, but not limited to: the force value applied at a particular point on the insertion trajectory (e.g., the force value used to navigate the catheter tip through the tortuous curves and bifurcations of the lumen); the stage position or stage speed when the catheter collision occurred; the EM sensor position / orientation when the error occurred; the orientation of the endoscopic view (e.g., a live view photo can be recorded as part of the flag); the catheter pose (angle and orientation at the time of the collision); the force difference between the current applied force and the next threshold level; the EM position (or a derivative thereof, such as an estimated insertion volume); the percentage progressed along the pre-planned insertion trajectory; the number of bifurcations traversed (e.g., each flag can mark a bifurcation in the lumen), etc.

[0052] In one embodiment, the final point on the force timeline 500 may represent 100% of the planned or expected catheter insertion trajectory, and the 'current' position will move along the timeline representing the percentage of insertion as the catheter moves (inserted or retracted) relative to the lumen. The percentage of insertion along the catheter trajectory may be calculated, for example, by dividing the amount of insertion (or the estimated amount of insertion) by the total length of the path, and / or by projecting the position of the EM sensor 190 onto the planned trajectory and calculating the projected position of the EM sensor 190 along the total length of the catheter trajectory.

[0053] Alternative force values ​​for color-coded sections and points / flags of the force timeline 500 include the aggregate of all forces applied to the drive wire 210 of each bendable segment, derived external forces, and the maximum force encountered at the collision location where a collision occurred (rather than a threshold level). Derived external forces include forces experienced by the catheter tip exerted by external tissue (e.g., due to organ movement). In particular, if the catheter does not have a dedicated force sensor at its tip, the external forces acting on the catheter are "derived" from known empirical formulas using the forces of the drive wire (e.g., forces sensed by sensor 304). An example of derivation of external forces includes the following method: 1. As a calibration process, record the forces predicted to be exerted on the drive wire when bending a particular pose (air, no collisions). 2. During the procedure, the system records the forces exerted on the drive wire during actual catheter insertion. 3. Calculating the difference between 1 and 2 should correlate to the forces exerted externally on the catheter (i.e., from anatomical collisions between tissue and the catheter tip). As another example, the force history timeline can be used to determine system malfunction or catheter damage. To that end, the force history timeline can also track the operating force limits of the motor that operates the drive wire to inform the user how close the motor is to its maximum pulling / pushing force. This can serve as a kind of proxy in determining the maximum amount of bending possible. This feature can be important to monitor inadvertent catheter bending that does not necessarily lead to system malfunction, catheter collision with the patient's anatomy, or catheter damage. If the inadvertent catheter bending is extreme (e.g., above a certain threshold), it can be detected by a force sensor (e.g., sensor 304 shown in FIG. 2). Potential catheter bending or collision events can also be detected using motor encoders and other sensors such as EM sensors and shape sensors. Shape sensing and shape reconstruction techniques for continuum robots can be done by utilizing fiber Bragg grating (FBG) sensing, electromagnetic (EM) tracking, and intraoperative imaging.Even if an extreme bend is detected and no navigation error is determined, the system can store the position of the catheter tip along the force timeline.

[0054] Additionally, depending on user preference, the color scale of force timeline 500 can be a range of discrete values ​​representing particular colors, or the entire force timeline 500 (or a section thereof) can be displayed in grayscale (shades of gray) with values ​​ranging, for example, from white representing no force (zero force) to black representing the maximum force that can be applied to the catheter. Additionally, force timeline 500 can be displayed in many alternative ways besides a color-coded timeline or a grayscale one-dimensional timeline.

[0055] For example, the force timeline 500 can be alternatively or simultaneously displayed as a table or list of force points below (or above) a certain threshold and the mapped values, as a 3D map (e.g., a 3D map with the x-axis being time, the y-axis being the number of drive wires, and the z-axis being the drive wire force). Either form of force timeline can display values ​​using the true position of the EM sensor or using the projected position of the EM sensor on a pre-planned trajectory. Instead of using color to represent force, the various force values ​​can be displayed in other ways, such as size / shape (e.g., if the force of each wire is represented by a dot, the size of the dot can be correlated to the amount of force being applied to each drive wire; similarly, if the drive wire forces are represented by bars along a linear force timeline). The size / shape is displayed as a glyph in the 3D map. For example, a small sphere represents a small force and a large sphere represents a large force. Similarly, something like a symmetrical sphere can indicate no navigation error, while a star with N points can indicate N navigation errors (catheter tip collision, one or more curved segments getting caught in the anatomy, etc.).

[0056] Once the force timeline has been established using the recorded points / flags / colors, the user can interact with the force timeline 500, such as by interactively touching these points / flags / colors (e.g., with a mouse pointer or manually) to move the catheter tip back to a desired position. For example, the user can force A point on the timeline 500 can be selected and the system can automatically start the linear stage 91 to retract the catheter to a safe position. At the same time, the actuator system adjusts the catheter's pose using inverse kinematics (e.g., using the rFTL algorithm). This process can be performed repeatedly until the catheter tip reaches the selected point (position) on its trajectory within the lumen.

[0057] When the system retracts the catheter to a previous position (the retracted position), it is distal to the current position of the catheter. force The "registered" points on the timeline 500 may be removed from the recorded catheter trajectory since they are no longer relevant or useful to the user to continue driving the catheter towards the target. However, in at least some embodiments, the user may choose to save the erroneous trajectory information since it may be useful for prediction and future collision avoidance calculations. In particular, the discarded information includes data about position / position combinations that are known to cause navigation errors, and the system may use this information to improve catheter guidance during subsequent insertions and future interventions. In this case, the system may avoid navigating the catheter at positions (coordinates) or orientations that are known to cause errors.

[0058] <Relate navigation forces to historical force radar plots> Although the above-mentioned embodiment addresses the problem of navigation errors resulting from insertion along the insertion trajectory, collisions may also occur due to the catheter tip bending instead of moving along the insertion trajectory. In fact, if a user advances (inserts) the catheter while significant collisions are occurring, the collisions during bending may lead to collisions during insertion. Collisions during bending of the distal section may be propagated to the previous (proximal) section by FTL control, which may cause subsequent sections to further collide against the anatomical structure due to subsequent crosstalk. For example, if the catheter tip is bent toward the lumen wall and the catheter is pushed forward, the middle section will also follow the attitude and start colliding with the wall, causing the tip to exert even greater force against the anatomical structure.

[0059] In this scenario, the force values ​​can be mapped to the catheter tip position as well as the insertion trajectory. The force values ​​applied to the various drive wires that control the catheter tip position can be measured by sensor 304, and the catheter tip pose can be measured by EM sensor 190. The measured force values ​​can be presented to the user as a radar plot surrounding the endoscopic view to better determine corrective action to be taken.

[0060] <Displays measured parameters of the catheter tip and corresponding corrective actions> 6 illustrates a graphical representation 600 of a radar-like force plot 610 for displaying measured parameters of the applied forces to generate a posture of the catheter tip 120, according to an embodiment of the present disclosure. In FIG. 6, the central circle represents an endoscopic view 602 acquired by the imaging device 180 located at the catheter tip 120. Around the endoscopic view 602 (the periphery of the endoscopic view 602), Radar-like There is a force plot 610 . Radar-like The force plot 610 displays the force values ​​and may include color-coded thresholds. Radar-likeThe color-coded thresholds of the force plot 610 may be similar to the color-coding scheme used for the force timeline 500. In FIG. 6, the light areas have a first color as the color of section 612C, the dark areas have a second color as the color of section 612B, and the dark areas with small stars have a third color as the color of section 612A. The endoscopic view 602 may display the approximate orientation (pose) of the catheter tip 120 at a particular location (current position) along the insertion trajectory. For example, the endoscopic view 602 may be mapped to up-down and left-right directions in which the catheter tip 120 may be steered to reach a desired target location.

[0061] Radar-like The force plot 610 (i.e., the area around the endoscopic view 602) shows the direction (orientation) of the forces applied during navigation and bending of the catheter tip as it progresses from the insertion point (coordinates: x=0, y=0, z=0) to its current (live) position, relative to the catheter axis. Radar-like The distance to the circles around the force plot 610 can represent the distance from the current position to the position where the catheter tip encountered the force. For example, a first circle 612 can represent a first distance (e.g., from P4 to P3 in FIG. 4C where the catheter tip is bent and moved within the lumen 81), a second circle 614 can represent a second distance (e.g., from P4 to P2) that is greater than the first distance, and a third circle 616 can represent a third distance (P4 to P1) that is greater than the sum of the first and second distances. Additional circles can be provided from the current position of the catheter tip to the insertion point along the insertion trajectory that the catheter has already traveled.

[0062] The force values ​​in each distance circle may be strictly relative to a specific insertion position (as in embodiment 1) or may represent navigation error events that occurred during catheter insertion. For example, inside the first circle 612, the force values ​​experienced by the catheter tip between a current position (e.g., P4) and a previous position (e.g., P3) are color-coded as a first color in section 612A, a second color in second section 612B, and a third color in section 612C. An uncolored section inside circle 612 indicates that there is zero force being applied in that direction. If the force values ​​are represented in red in the first section 612A, Radar-like The graphical representation in section 612A of force plot 610 indicates forces above a threshold that could cause a catheter collision (e.g., C2 in FIG. 4B) or other system malfunction (such as excessive force on the drive wire that actuates the catheter tip upwards). When the force values ​​are represented in green in the second section 612B, Radar-like The graphical representation of section 612B of force plot 610 indicates that the catheter tip is operating safely below the threshold. When the force values ​​in the third section 612C are represented in yellow or orange, Radar-like The graphical representation of section 612C of force plot 610 indicates that the force being applied to the catheter tip is close to a force threshold above which navigation errors may occur.

[0063] The force values ​​can be reset as the stage position moves forward (or redisplayed as the stage position retracts the catheter tip), or the force values ​​can be algorithmically expanded to cover multiple points along the insertion trajectory (e.g., ±5 mm increments from a particular position). These values ​​can be measured by the change in position of the linear stage 91 and the bending of the catheter tip as the steerable catheter 100 is inserted into the lumen 81. Radar-like The size and extent of the circles in the force plot 610 can be adjusted manually or automatically to suit the user's needs.

[0064] (rather than color-coded thresholds) Radar-like The graphical representation of the force plot 610 can also be varied, Radar-like The force plot 610 can be displayed in the same alternative manner as proposed in the embodiment of FIG. Radar-like One can interact with the force plot 610 to see additional details about the state of the system when the forces were mapped. Details of corrective actions can also be similar to those described with reference to FIG.

[0065] In addition, the present system Radar-like The information provided by the force plot 610 can be used in a number of ways. For example, the system can completely limit bending of the catheter tip in a particular direction when the force in that direction exceeds a certain threshold. Alternatively, the user can select one of the distance circles (e.g., one of circles 612, 614, 616) and the linear stage 91 can automatically return the catheter tip to the position and orientation corresponding to the selected circle. Clicking on a point does not necessarily move the stage; instead, the system may be programmed to change the orientation of at least one bendable segment of the catheter to match the orientation when that force value was recorded (although the system may also move the stage if the stage position was different when that data point was recorded).

[0066] this Radar-like The force plot 610 may be displayed in a number of other ways, such as as a pure radar plot without centering the endoscopic view 602, or as a 3D field in a virtual view, which may be a series of color-coded points or lines in 3D space (e.g., color-coded points or lines superimposed on a pre-operative CT or MRI 3D image).

[0067] Around the endoscopic view 602 Radar-like forceThe position of each circle in plot 610 can be calculated in a variety of ways, such as by tracking the position of EM sensor 190 along the insertion trajectory and / or by projecting the mapped position of EM sensor 190 onto the plane of the current pose. Additionally, forward kinematics can be used to position catheter tip 120 in a position such that bending the catheter in a given direction will achieve the desired pose. Then, Radar-like force When one of the circles in plot 610 is selected by the user, the system uses reverse kinematics to retract the catheter along the insertion trajectory to the given point, bringing the catheter tip into the pose / orientation recorded at that location.

[0068] A graphical user interface for taking one or more corrective actions. Returning to Figure 2, recall that the robotic catheter system 1000 comprises a steerable catheter 100 having multiple bendable segments, an actuator 310 for bending the bendable segments, and a controller 320 for commanding the actuator 310 based on operator input via a joystick or gamepad controller (205 in Figure 1). The system 1000 also includes a display screen 420 for outputting necessary information to the operator 10.

[0069] As shown in FIG. 3, each of the three bendable segments of the steerable catheter 100 has multiple drive wires 210. When each bendable segment is actuated by three drive wires 210, the steerable catheter 100 has nine drive wires arranged along the walls of the catheter. The actuation unit 310 pushes and pulls at least one of these nine drive wires 210 to bend each bendable segment of the catheter. Forces are applied to the individual drive wires to steer / maneuver the catheter to a desired position. The actuation unit 310 includes servo motors for individually pushing and pulling each of the drive wires 210. The actuation unit 310 also includes force sensors 304 for individually measuring the force on the drive wires. The drive wires 210 have operating limits because the steerable catheter 100 has operating limits (such as bending angle limits, catheter failure limits, distal force limits applied by the catheter tip to the anatomy, etc.). The operating limits are stored and / or programmed in the control unit 320 or computer 400. Thus, during operation, the control unit 320 compares the force measured by the force sensors 304 to the operating limits of each drive wire 210 .

[0070] As the operating limits are approached or reached, the controller warns the operator that a navigation error may occur. In addition to the warning, the present disclosure provides one or more specific corrective actions for the user to take. Specifically, for drive wire operating limits, the controller provides a "relax mode," in which the controller uses measurements of the drive wire force to perform feedback control to reduce the force on the drive wire. In the relax mode, the force applied to the drive wire is reduced so that the steerable catheter 100 can be repositioned to a position or orientation where the force on the drive wire is reduced. According to at least one embodiment, in addition to providing a warning, the controller 320 interactively guides the operator to take corrective actions.

[0071] According to one embodiment, if there is a generalized parameter X that represents a catheter condition including a critical failure, the controller monitors the parameter X as a function of the independent variable and compares the parameter to a threshold value. FIG. 7A illustrates an example of how to display the parameter X versus the independent variable, with a threshold line indicating when corrective action needs to be taken. The threshold value may be a specific value of the parameter X or a calculated value derived from the parameter X. It should be understood that the parameter X is a catheter parameter that changes dynamically during catheter insertion or catheter bending, e.g., as a function of the time or distance the catheter travels through the lumen toward a particular target. In other words, the system monitors the historical state of the catheter parameter X with respect to the independent variables (other parameters) that may cause navigation errors (e.g., catheter collision or excessive bending force, and / or system malfunction). When the real-time value of the parameter X approaches a threshold value, the system can display this condition in an appropriate manner. For example, the controller can display a graphical representation of at least a portion of the catheter and the state of the parameter X on the display screen 420 to inform the user. When the controller determines that parameter X has reached a threshold, the system provides interactive guidance to the operator to take corrective action.

[0072] In one embodiment, parameter X is the force measured for each drive wire 210 during insertion or manipulation (bending or rotating movements) of the catheter, and the corrective action can be activation of a relaxed (compliant) mode of the steerable catheter 100. FIG. 7B illustrates an example of how to display force parameters versus time (a force-time graph), with a force threshold indicating when corrective action needs to be taken. According to FIG. 7B, the graph displays force on the y-axis and time on the x-axis. Over time, catheters are inserted for intraluminal procedures, A force is applied to the catheter drive wire 210 to drive the catheter tip 120 but Bending direction (orientation) To Manipulated RTo prevent catheter collisions and system malfunctions and / or to allow the catheter tip to move properly without damaging the tissue in the lumen, the applied force needs to be kept below a certain threshold. At some point, if the force applied to one or more of the drive wires reaches or exceeds the threshold, the system issues a warning. For example, a message "Enter Relax Mode" can be displayed on the display, and the user or the system takes corrective action to reduce or release the force applied to the drive wires. According to another example, when the linear stage 91 is controlled to move (insert or withdraw) the catheter through the lumen 81, one or more of the force sensor 304 or EM sensor 190 (position sensor) can track the external force applied to the catheter tip 120 during the intraluminal procedure. At some point, if the external force applied to the catheter tip exceeds a threshold, the system issues a warning. For example, the display can display "Stop," and the user or the system takes corrective action, such as reversing the drive direction of the linear stage or reversing the direction of the force applied to a particular drive wire that bends the catheter.

[0073] 8, 9A-9B, and 10 illustrate further examples of catheter graphical representations to indicate catheter parameters with respect to thresholds at which navigation errors (catheter collision, excessive force on the drive wire, and / or system malfunction) may or have occurred. FIG. 8 illustrates a graphical representation 800 of drive wire force monitoring results displayed on display screen 420. According to this embodiment, the drive wire force monitoring results can be plotted as a spider web force map. In FIG. 8, circle 802 represents the maximum allowable force (threshold force) applied to each of the drive wires 210 of steerable catheter 100. Center C of circle 802 represents the center of the catheter (i.e., catheter axis Ax). Each of the lines radiating from center C to circle 802 represent the allowable range of motion for various drive wires 210. In this example, the spider web force map includes WR1, WR2, WR3, WR4, WR5, WR6, WR7, WR8, WR9, WR10, WR11, WR12, WR13, WR14, WR15, WR16, WR17, WR18, WR19, WR20, WR21, WR22, WR23, WR24, WR25, WR26, WR27, WR28, WR29, WR30, WR31, WR32, WR33, WR34, WR35, WR36, WR37, WR38, WR39, WR40, WR41, WR42, WR43, WR44, WR45, WR46, WR47, WR48, WR49, WR50, WR51, WR52, WR53, WR54, WR55, WR56, WR57, WR58, WR59, WR6 andThe diagram includes nine radial line markers labeled F1, F2, F3, F4, F5, F6, F7, F8, and F9, each representing a range of motion for the drive wire 210. In this graphical representation, the radial direction indicates the magnitude of force applied to each drive wire. The center C of the circle 802 indicates zero applied force (i.e., the relaxed mode of the catheter). marker Dynamic force markers located on WR1-WR9 indicate the force measured for that particular drive wire. Rings or circles 802 represent the force threshold that each drive wire must not exceed. When force is applied to any drive wire 210, the corresponding dynamic force marker (F) appears as a radial line marker The force marker F moves along the WR. Drive Wire The dynamic force markers (F values) are proportional to the force applied to the wires 802. If the force F applied to one of the drive wires crosses the circle of maximum allowable force 802, the system will issue a warning to the user. If any one of the dynamic force markers (F values) enters the outer zone of circle 802, the user must take corrective action. Alternatively, the system can be programmed to automatically apply the required corrective action. For example, the system can be programmed to automatically go into a relax mode where the actuation forces on all wires are stopped.

[0074] Using this graphical user interface, the controller can inform the user about a situation where the applied force is approaching a threshold. If the applied force is equal to or greater than the threshold, the system can guide the operator to activate a relaxation mode. Specifically, the wheel-shaped indicator can indicate a particular direction in which the force is increasing and approaching a threshold. The wheel-shaped indicator can also be aligned with the bending direction of the robotic catheter on the display. For example, if the operator tilts the joystick upward, the catheter tip 120 of the robotic catheter can be bent toward the top of the display. Similarly, for the downward, left, and right directions. This configuration can indirectly provide preventive measures. For example, if the operator sees a large force being applied in one direction, the operator can understand to bend the robotic catheter in the opposite direction. Furthermore, by providing a graphical representation of the force applied to each drive wire, the system can more easily determine system malfunctions. For example, one or more force markers (F) can be aligned with a corresponding Radial Line Marker If the user observes that the actuator does not move along (WR), the user can understand that the drive wire may be cut, broken, or stuck and therefore not moving.

[0075] In other embodiments, the system may include multiple thresholds for different bendable segments. Figures 9A and 9B illustrate an embodiment of a graphical user interface configured to display a spider-like force map of multiple bendable segments of the steerable catheter 100. In Figure 9B, force map 910 (right side of the image) illustrates an example force value (navigation parameter) applied to a first bendable segment (e.g., the most distal bendable segment), the force value being limited by a first threshold circle 802A. In Figure 9A, force map 920 (left side of the image) illustrates an example force value (navigation parameter) applied to a second bendable segment (e.g., the bendable segment proximal to the most distal bendable segment), the force value being limited by a second threshold circle 802B. Of course, there is no limit to the number of bendable segments, and similar graphical representations of navigation parameters and thresholds for each bendable segment may be displayed as multiple force maps equal to the number of bendable segments. Alternatively, the force maps of all bendable segments may be combined into a single graphical representation. In that case, the drive wires of each bendable segment can be represented radially and concentrically with each other with their respective thresholds simultaneously displayed. For example, the graphical representations shown in Figures 9A and 9B can be combined into a single display. In that case, the most distal bendable segment can be shown in the center, and other bendable segments proximal to the most distal segment can be shown radially and concentrically surrounding the most distal segment, thereby effectively creating a true spider web graphical representation of two or more navigation parameters with their respective thresholds.

[0076] In normal operation of the robotic catheter system 1000, as the steerable catheter 100 advances through the lumen 81 toward the target 181, a force is applied to the drive wire 210 of each bendable segment to steer the catheter tip 120 in a desired direction. In a typical procedure, the force on the drive wires varies so that each axial radial line Marker WRThe measured force F above will move towards or away from the center C of the force map. For example, in FIG. 9B, Radial Lines The bendable segment 1 can be actively controlled by three drive wires, represented by markers WR2, WR5, and WR8, where an actuation force is applied to each drive wire, which will move the dynamic force markers F2, F5, and F8 along their respective axes accordingly. Radial Lines The bendable segment 2 can be actively controlled by three drive wires, represented by markers WR1, WR4, and WR7, where an actuation force is applied to each drive wire, which causes the dynamic force markers F1, F4, and F7 to move along their respective axes accordingly. To display a graphical representation of this embodiment, different colors can be used to represent the actuated drive wires in each bendable segment, and circles 802A and 802B can represent different threshold values, although circles 802A and 802B can be drawn with the same color or pattern to help intuitively understand that the actuation forces are approaching the threshold values.

[0077] In other embodiments, the data displayed in the web force maps of FIGS. 9A and 9B can represent the difference between the currently measured force applied to the drive wire and the calculated (ideal) force for the push-pull actuation of the entire catheter or individual bendable segments. In certain embodiments, the calculated (ideal) force can be used as a reference for the amount of push-pull force being applied to the catheter in real time during the procedure (assuming bending the catheter in air). According to this embodiment, by using the force difference between the ideal force value and the real-time measured force value, the system can calculate the magnitude of the external force acting on the catheter from the interaction of the catheter with the patient's anatomy by suppressing the normal operating force. As a result, the system can highlight more fundamental information about the root cause of errors or failures. Thus, the system can provide more intuitive information to the user to make it easier to take corrective action.

[0078] FIG. 10 shows a graphical user interface including a graphical representation of navigation parameters displayed as a spider web force plot 1010 illustrating error or failure events and corrective actions therefor. According to this embodiment, the system is configured to control the display device to output the spider web force plot 1010 during an intervention procedure. During normal operating conditions, the force plot 1010 outputs the real-time (current) applied force, which is represented by the dynamic movement of one or more force markers F1-F9, as shown in FIG. 8 or FIG. 9. If at least one drive wire 210 is actuated beyond its operating limits and / or if the catheter tip experiences an impact inside the lumen 81 (e.g., impact with delicate tissue), the system automatically outputs a pop-up window 1030 and a warning indicator 1020. The pop-up window 1030 provides the user with one or more user-selectable corrective actions, and the warning indicator 1020 provides an indication of the exact location or element that is the root cause of the alert.

[0079] For example, as shown in FIG. markerIf the drive wire 210, represented by WR5, is actuated in such a way that the dynamic force marker F5 extends beyond the circle 802, a warning indicator 1020 (in this example, an arrow) is prominently displayed and a pop-up window 1030 is immediately displayed. The user can then select one or more corrective actions, such as a first corrective action 1032 ("go to relax mode"), a second corrective action 1034 ("pull back the catheter"), or a third corrective action 1036 ("bend the catheter in the opposite direction"). Once the measured forces return to within the operational limits, the pop-up window 1030 disappears from the display screen and normal operation resumes. For example, the system reverts to displaying force plots for each of the bendable segments, as illustrated in Figures 9A-9B. It should be noted that the warning indicator 1020 and the pop-up window 1030 correspond to areas of the graphical representation that indicate one or more corrective actions that the user can take to correct and / or prevent catheter navigation errors. In that regard, the warning indicator 1020 and pop-up window 1030 apply to all graphical representations of the force history timeline shown in Figures 4A-4C, 5, 7A-7B and the concentric radar force plots shown in Figures 6, 8, 9A-9B and 10.

[0080] <Software Implementation> At least certain aspects of the exemplary embodiments disclosed herein may be implemented by a computer in a system or device that reads and executes computer executable instructions (e.g., one or more programs or executable codes) stored in a memory or storage medium (sometimes referred to as a "non-transitory computer readable storage medium"), such as a solid state drive (SSD), to perform the functions of one or more of the block diagrams, systems, or flowcharts described above. FIG. 11 illustrates a flowchart of an exemplary process 1100 of a method of operation of a robotic catheter system 1000. The robotic catheter system 1000 includes an actuation portion 310 (actuator) configured to manipulate a steerable catheter 100 having one or more bendable segments and a catheter tip 120, and coupled to the bendable segments via one or more drive wires 210 disposed along the wall of the steerable catheter 100. The process 1100 includes at least a registration process S1110, a navigation process S1120, and an intraluminal process S1130.

[0081] The registration process S1110 may include catheter-to-patient registration or instrument-to-image registration, where registration of the catheter coordinates to the tracking system coordinates can be performed by any known procedure. Examples of registration processes are as described in U.S. Patent Nos. 10,898,057 and 10,624,701, which are incorporated herein by reference for all purposes.

[0082] The navigation process S1120 includes at least the following sub-processes according to various embodiments of the present disclosure. More specifically, after the steerable catheter 100 is registered with the patient and / or the EM tracking system in step S1121, the system 1000 enters the navigation process S1120. Here, the CPU 410 of the computer 400 (FIG. 2) continuously records the catheter navigation parameters in the storage memory 411 and outputs on the display screen 420 a graphical representation of the navigation parameters relative to a threshold indicating a location where a catheter navigation error may occur or has occurred. The navigation parameters relative to the threshold are displayed in real time as a graphical representation 421 (FIG. 1) via the display screen 420. As mentioned above, the navigation parameters can be recorded and displayed as a linear force timeline or a linear force history timeline as shown in FIG. 5. Here, the system automatically performs force-time data collection during catheter insertion. In other embodiments, as shown in Figures 6 and 8-10, the navigation parameters can be recorded and displayed as concentric force plots representing the bending attitude of the catheter tip or the force levels applied to the individual drive wires during actuation. In step S1122, the system and / or the user can add flags or markers to the recorded navigation parameters. The flags or markers can be recorded automatically by the system (e.g., S-Flag-1, S-Flag-2, etc.) as shown in Figure 5, or manually by the user (e.g., U-Flag-1, U-Flag-2, etc.). Such flags, markers, or points can be added at predetermined time or distance intervals (e.g., every few seconds or every few millimeters of catheter insertion).

[0083] In step S1123, the system continuously monitors the navigation parameters to determine if a navigation error has occurred. As described in detail above, the system continuously compares the navigation parameters to thresholds to determine if a navigation error has occurred (e.g., catheter collision, system malfunction, etc.). If no navigation error has occurred (NO in S1123), the process proceeds to step S1126.

[0084] If a navigation error is likely or has occurred (YES in S1123), the system may stop catheter insertion, record the location of the error, and output a guide for corrective actions to be taken by the user in step S1124. For example, a pop-up window 1030 in the graphical representation of the navigation parameters may provide one or more corrective actions that the user may take to correct the navigation error in step S1125. In step S1125, the user may take one or more of several corrective actions, as described above with reference to FIG. 10. Alternatively or additionally, the system may be programmed to automatically perform a corrective action, such as making the steerable catheter compliant by entering a relaxed mode by releasing the actuation force applied to the drive wires.

[0085] After the corrective action is performed, the system continues monitoring the catheter navigation in step S1126. In this step, if the action performed by the user includes corrective action 1034 (pushing back the catheter), the system will advance the catheter using the corrected insertion trajectory. If the action performed by the user includes corrective action 1036 (bending the catheter in the opposite direction), the system will display the effect of the corrective action. For example, display screen 420 of FIG. 10 will display when force marker F5 returns inside the maximum allowable force circle. The system will then advance the catheter using the corrected insertion trajectory.

[0086] In step S1127, the system determines whether the catheter has reached the intended target. If the catheter has not reached the intended target (NO in S1127), the process returns to step S1121 and real-time monitoring of navigation parameters continues. If the catheter has safely reached the intended target (YES in S1127), the process proceeds to step S1130, where the user may perform the desired interventional procedure (e.g., ablation, biopsy, etc.).

[0087] The computer 400 may include various components known to those skilled in the art. For example, the computer may include a signal processor implemented by one or more circuits (e.g., a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC)) for performing one or more functions of the above-described embodiments, for example, by reading and executing computer-executable instructions from a storage medium to perform one or more functions of the above-described embodiments and / or by controlling one or more circuits for performing one or more functions of the above-described embodiments. The computer may have one or more processors, microprocessors (MPUs), as the CPU 410, and may also include a distributed remote computer or a network of separate processors (cloud computing) for reading and executing computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a cloud-based network or from a storage medium. Examples of storage media include one or more of a hard disk, a random access memory (RAM), a read-only memory (ROM), storage of a distributed computing system, an optical disk (such as a compact disk (CD), a digital versatile disk (DVD), or a Blu-ray Disc (BD)™), a flash memory device, a memory card, etc. A computer may include an input / output (I / O) interface for sending and receiving communication signals (data) to and from an input / output device, such as a keyboard, a display, a mouse, a touch screen, a touchless interface (e.g., a gesture recognition device), a printing device, a stylus, an optical storage device, a scanner, a microphone, a camera, a network drive, a wired or wireless communication port, etc.

[0088] Various embodiments disclosed in this disclosure provide several advantages over conventional robotic catheter systems. According to one embodiment, it is advantageous to provide a graphical representation of a force timeline diagram that maps points throughout the procedure to several thresholded force levels (e.g., different thresholds can be set for each bendable segment of the catheter). For example, the most distal bendable segment can be set with the most sensitive (e.g., lowest) threshold since the catheter tip is most likely to collide with the patient's lumen. Bendable segments other than the most distal catheter segment can have less sensitive thresholds (e.g., increasingly higher thresholds) to monitor for potential events where the curved portion catches on the luminal tissue even when the distal tip is correctly positioned. A force timeline diagram with several thresholded force levels allows a user to resolve navigation errors (e.g., catheter collision or system malfunction) by identifying locations on the insertion trajectory where the force levels are acceptable (below the threshold). By mapping the catheter tip poses throughout the procedure to specific thresholded force levels, a user can avoid future collisions by identifying poses that minimize the likelihood of collisions. A graphical representation of the scale or range of monitored catheter parameters (wire force, insertion depth, catheter tip collision, etc.) is communicated to the user in real time. Graphical representations showing various levels of warning allow the user to approach parameter thresholds with greater precision to avoid catheter collision, system malfunction, system failure, and / or patient harm. In particular, when the graphical representations show catheter parameters associated with the state of individual drive wires, the user or system can take more finely tuned corrective actions to prevent potential system malfunction or patient harm.

[0089] The present application also discloses various aspects of a robotic catheter system. According to aspect (1), the robotic catheter system includes: a catheter having one or more bendable segments and a catheter tip; an actuator coupled to the bendable segments of the catheter via one or more drive wires disposed along a wall of the catheter; a processor in operative communication with the actuator; and a memory having stored thereon instructions. The instructions, when executed by the processor, cause the processor to: continuously record navigation parameters while the catheter is inserted through the lumen along an insertion trajectory; and cause a display device to display a graphical representation of the navigation parameters relative to thresholds indicative of locations on the insertion trajectory where catheter navigation errors may or have occurred. Regions of the graphical representation indicate one or more corrective actions that may be taken to correct and / or prevent catheter navigation errors.

[0090] Other aspects of the catheter system of the present application include instructions that, when executed by the processor, further configure the processor to perform the following: in aspect (2), recording, as the navigation parameters, a position parameter associated with a real-time position or orientation of the catheter tip relative to the lumen; in aspect (3), controlling the actuator to apply a force to the drive wires to bend at least one of the one or more bendable segments of the catheter, and the processor recording, as the navigation parameters, a force parameter associated with a push or pull force applied by the actuator to the one or more drive wires to bend the one or more bendable segments of the catheter; in aspect (4), the processor recording, as the navigation parameters, one or more of aspect (4a) a force parameter associated with a location of the catheter collision, aspect (4b) a percentage of a distance the catheter tip had traveled along a pre-planned insertion trajectory until the catheter collision occurred, and aspect (4c) a number of lumen bifurcations traversed by the catheter tip before the navigation error occurred; in aspect (5), the processor recording, as the navigation parameters, a linear force timeline associated with an insertion trajectory traversed by the catheter tip during insertion into the lumen. The linear force timeline includes force levels of push and pull forces applied by the actuation portion to one or more drive wires to bend one or more bendable segments of the catheter; and, in aspect (6), the processor controls the display to display one or more markers or flags throughout the linear force timeline, the one or more markers or flags indicating force levels approaching or exceeding a threshold at which a navigation error may occur or has occurred.

[0091] Aspect 7: A system according to aspect (6), wherein the processor is further configured to: control the display device to display a guide for a user to take corrective action to resolve the navigation error by identifying at least one of a point or marker or flag in the linear force timeline where the force level falls below a threshold; retracting the catheter tip to a retraction position within the lumen that corresponds to the identified at least one of a point or marker or flag in the linear force timeline; and continuing to insert the catheter into the lumen from the retraction position using the modified insertion trajectory.

[0092] Aspect 8: A system according to aspect (1), wherein the processor is further configured to record a position of the catheter tip along an insertion trajectory during insertion of the catheter into the lumen, and the recorded position of the catheter tip is associated with a force level of a force applied by the actuation portion to one or more drive wires to bend the catheter tip.

[0093] Aspect 9. A system according to aspect (8), wherein the processor is further configured to display as a graphical representation a radar-like force plot including one or more circles indicating positions of the catheter tip where the force level is approaching or exceeding a threshold where a navigation error may occur, and the force level is displayed as a different color-coded value or a different pattern in at least one of the one or more circles.

[0094] Example 10. A system according to example 1, wherein the processor is further configured to: control an imaging device disposed in the tip of the catheter to acquire a live view image of the lumen; and display the live view image of the lumen on a display device together with a graphical representation of the navigation parameters.

[0095] Example 11. A system according to example 10, wherein the processor displays a live view image together with a graphical representation of the navigation parameters and surrounds the live view image with a radar-like force plot associated with the position and / or orientation of the catheter tip within the lumen.

[0096] Example 12. A system according to example 10, wherein the processor displays a radar-like force plot indicating the force applied to one or more drive wires by the actuation portion to bend the catheter tip in one or more directions as a live view image accompanied by a graphical representation of the navigation parameters, the one or more directions including upward, downward, left, right, or a combination thereof with respect to the live view image.

[0097] Example 13. The system according to example 1, wherein the processor records the position of the catheter tip relative to the wall of the lumen and / or records the orientation of the catheter tip relative to the catheter axis as the navigation parameter.

[0098] Example 14. A system according to example 13, wherein the processor displays a virtual representation of one or more drive wires in a radial arrangement as the graphical representation of the navigation parameter, and displays the threshold value as a threshold circle surrounding the radial arrangement of the one or more drive wires.

[0099] Aspect 15. A system according to aspect 14, wherein a virtual representation of the one or more drive wires is displayed as a force marker on each drive wire of the steerable catheter, and each of the force markers is configured to dynamically move radially in response to an amount of force applied to the one or more drive wires by an actuation portion to bend the catheter tip.

[0100] Example 16. A system according to Example 14, wherein the one or more bendable segments have a first bendable segment and a second bendable segment in order from the distal end to the proximal end of the catheter, and the processor displays a first radial arrangement of force markers of one or more drive wires of the first bendable segment and a second radial arrangement of force markers of one or more drive wires of the second bendable segment as virtual representations of the one or more drive wires, each of the force markers of the first radial arrangement moving radially in response to a force applied by an actuation unit to bend the first bendable segment, and each of the force markers of the second radial arrangement moving radially in response to a force applied by an actuation unit to bend the second bendable segment.

[0101] Aspect 17. A system according to aspect 14, wherein the virtual representation of the one or more drive wires shows each of the one or more drive wires as a force marker that moves radially in response to a force applied by an actuation portion to bend the catheter tip in an upward or downward or left or right direction or combinations thereof relative to the catheter axis.

[0102] Example 18. In a system according to Example 17, when the amount of force applied by the actuator to one or more drive wires to bend one or more bendable segments is greater than or equal to a threshold value, a portion of the graphical representation shows a pop-up window listing one or more corrective actions to correct and / or prevent navigation errors.

[0103] Example 19. The system of example 1, wherein the processor is further configured to: detect that a navigation error has occurred when the value of the navigation parameter is greater than or equal to a threshold; and perform corrective action to correct the navigation error.

[0104] Example 20. A system according to example 19, wherein the processor is further configured to: record the location of occurrence of the navigation error on the insertion trajectory; and display the location of occurrence of the navigation error on the insertion trajectory together with a graphical representation of the navigation parameters.

[0105] Example 21. A system according to example 20, wherein the processor displays a linear force timeline of the insertion trajectory as a graphical representation of the navigation parameters, and the processor automatically records a flag or marker on the linear force timeline as a location where a navigation error occurred, or prompts the user to record a flag or marker on the linear force timeline, the flag or marker indicating the location where the navigation error occurred.

[0106] Aspect 22. A system according to aspect 20, wherein the processor is further configured to: place the catheter in a relaxed mode based on detection of a navigation error; retract the catheter tip to a position on the insertion trajectory that is proximal to the flag or marker; and navigate the catheter along the corrected insertion trajectory using an actuation force below a threshold.

[0107] <Modifications and / or combinations of the embodiments> When referring to the description, specific details are set forth to provide a thorough understanding of the disclosed examples. In other instances, well-known methods, procedures, components, and circuits are not described in detail so as not to unnecessarily lengthen the present disclosure. Unless otherwise defined herein, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which the present disclosure belongs. In that regard, the breadth and scope of the present disclosure is not limited by the specification or drawings, but rather only by the plain meaning of the claim terms employed.

[0108] In describing the exemplary embodiments shown in the drawings, specific terminology is used for clarity, however, the disclosure of this patent specification is not intended to be limited to the specific terminology so selected, and it is understood that each specific element includes all technical equivalents that function in a similar manner.

[0109] Although the present disclosure has been described with reference to exemplary embodiments, it should be understood that the present disclosure is not limited to the disclosed exemplary embodiments. All embodiments can be modified and / or combined to improve and / or simplify the anti-twist features as applicable to a particular application. The scope of the following claims should be accorded the broadest interpretation so as to encompass all such modifications and equivalent structures and functions.

[0110] Any patent, pre-grant patent publication, or other disclosure that is incorporated by reference herein, in whole or in part, is incorporated only to the extent that the incorporated material does not conflict with standard definitions, terms, or descriptions and explanations set forth in this disclosure, and therefore, to the extent necessary, the disclosure expressly set forth herein supersedes any conflicting material incorporated by reference.

Claims

1. providing a catheter having one or more bendable segments and a catheter tip, the one or more bendable segments being actuable by an actuator disposed proximally relative to a proximal end of the catheter, the actuator applying an actuation force to the one or more bendable segments via a drive wire disposed along a wall of the catheter, one end of the drive wire being attached to the actuator and the other end of the drive wire being attached to one of the one or more bendable segments; inserting the catheter tip into a body lumen and navigating the catheter through the lumen along an insertion trajectory while the actuation portion applies the actuation force to the one or more bendable segments; recording navigation parameters associated with the insertion trajectory while the catheter tip is inserted into the lumen; displaying on a display a graphical representation of the navigation parameters relative to thresholds indicative of positions along the insertion trajectory where navigation errors of the catheter may or have occurred; outputting on said display an interactive user guide indicating one or more corrective actions that can be taken to correct and / or prevent said navigation error; The method includes:

2. and recording the navigation parameters includes recording a real-time position or orientation of the catheter tip relative to the lumen during navigation of the catheter through the lumen. The method of claim 1.

3. steering the catheter tip by bending the one or more bendable segments by transmitting the actuation force from the actuation section to the one or more bendable segments of the catheter; Further comprising:

2. The method of claim 1, wherein recording the navigation parameters comprises recording force parameters associated with push and pull forces applied by the actuation portion to the one or more drive wires to bend the one or more bendable segments of the catheter.

4. or wherein recording the navigation parameters includes recording a force parameter associated with a location of occurrence of a catheter collision with the lumen; or recording the navigation parameters includes recording the percentage of the distance the catheter tip had traveled along a pre-planned insertion trajectory when a catheter collision occurred; or and recording the navigation parameters includes recording a number of lumen bifurcations traversed by the catheter tip before a navigation error occurs. The method of claim 1.

5. recording navigation parameters includes recording a force timeline associated with the insertion trajectory followed by the catheter tip during insertion into the lumen; the force timeline includes force levels of push and pull forces applied by the actuation portion to the one or more drive wires to bend the one or more bendable segments of the catheter. The method of claim 1.

6. the force timeline is displayed as a linear force timeline; displaying the graphical representation includes displaying one or more markers or flags throughout the linear force timeline; the one or more markers or flags indicate force levels approaching or exceeding the threshold at which a navigation error may or has occurred. The method according to claim 5.

7. displaying guidance for the user to take corrective action to resolve the navigation error by identifying at least one of a point or a marker or a flag in the linear force timeline where the force level falls below the threshold; retracting the catheter tip within the lumen to a retraction position that corresponds to the identified at least one of the points or markers or flags in the linear force timeline; calculating a modified insertion trajectory based on the linear force timeline; continuing insertion of the catheter tip into the lumen from the retracted position using the modified insertion trajectory; The method of claim 6 further comprising:

8. recording navigation parameters includes recording a position of a catheter tip along the insertion trajectory during insertion of the catheter into the lumen; the recorded catheter tip position is associated with a force level applied by the actuation portion to the one or more drive wires that bends the catheter tip. The method of claim 1.

9. displaying the graphical representation includes displaying a radar-like force plot including one or more circles indicating locations of the catheter tip where the force level is approaching or exceeding the threshold where the navigation error may occur; the force levels are displayed with different color-coded values ​​or different patterns in at least one of the one or more circles; The method according to claim 8.

10. controlling an imaging device disposed in the tip of the catheter to obtain a live view image of the lumen; displaying the live view image of the lumen along with the graphical representation of the navigation parameters; The method of claim 1 further comprising:

11. displaying the live view image along with the graphical representation of the navigation parameters comprises displaying the live view image surrounded by a radar-like force plot associated with a position and / or orientation of the catheter tip within the lumen. The method of claim 10.

12. displaying the live view image along with the graphical representation of the navigation parameters includes displaying a radar-like force plot indicative of forces applied by the actuation portion to the one or more drive wires to bend the catheter tip in one or more directions; The one or more directions include an upward direction, a downward direction, a leftward direction, a rightward direction, or a combination thereof with respect to the live view image. The method of claim 10.

13. and recording the navigation parameters includes recording a position of the catheter tip relative to a wall of the lumen and / or recording an orientation of the catheter tip relative to the catheter axis. The method of claim 1.

14. displaying the graphical representation of the navigation parameters includes displaying a virtual representation of the one or more drive wires in a radial arrangement and displaying the threshold as a threshold circle surrounding the radial arrangement of the one or more drive wires. The method of claim 13.

15. the virtual representation of the one or more drive wires is displayed as a force marker for each drive wire of the steerable catheter; each of the force markers is configured to dynamically move radially in response to an amount of force applied to the one or more drive wires by the actuation portion to bend the catheter tip; The method of claim 14.

16. the one or more bendable segments comprising, in order from the distal end to the proximal end of the catheter, a first bendable segment and a second bendable segment; displaying the virtual representation of the one or more drive wires includes displaying a first radial arrangement of force markers for one or more drive wires of the first bendable segment and displaying a second radial arrangement of force markers for one or more drive wires of the second bendable segment; each of the force markers in the first radial arrangement moves radially in response to the force applied by the actuator to bend the first bendable segment, and each of the force markers in the second radial arrangement moves radially in response to the force applied by the actuator to bend the second bendable segment. The method of claim 14.

17. the virtual representation of the one or more drive wires shows each of the one or more drive wires as a force marker that moves radially in response to the force applied by the actuation portion to bend the catheter tip in an upward or downward or left or right direction or combinations thereof relative to the catheter axis. The method of claim 14.

18. if an amount of force applied by the actuator to the one or more drive wires to bend the one or more bendable segments is equal to or greater than the threshold, the portion of the graphical representation shows a pop-up window listing one or more corrective actions to correct and / or prevent the navigation error.

20. The method of claim 17.

19. detecting that a navigation error has occurred when the value of the navigation parameter is equal to or greater than the threshold value; performing corrective action to correct the navigation error; The method of claim 1 further comprising:

20. recording a position where the navigation error occurs on the insertion trajectory; displaying the location of the navigation error on the insertion trajectory together with the graphical representation of the navigation parameters; 20. The method of claim 19, further comprising:

21. displaying the graphical representation of the navigation parameters includes displaying a linear force timeline of the insertion trajectory; recording the location of the navigation error includes automatically or manually adding a flag or marker to the linear force timeline, the flag or marker indicating one or more of the location of the navigation error and the amount of force applied to the drive wire at the location of the navigation error; 21. The method of claim 20.

22. transitioning the catheter into a relaxed mode in which no force is applied to the drive wire; retracting the catheter tip to a position on the insertion trajectory that is proximal to the location where the error occurred; navigating the catheter along a modified insertion trajectory using an actuation force below the threshold; 21. The method of claim 20, further comprising:

23. a steerable catheter having one or more bendable segments and a catheter tip; an actuation section coupled to the bendable segment of the catheter via one or more drive wires disposed along a wall of the catheter; a processor in operative communication with the actuator; A memory storing instructions; A system comprising: The instructions, when executed by the processor, cause the processor to: recording navigation parameters while the catheter is inserted along an insertion trajectory through a body lumen; causing a display device to display a graphical representation of the navigation parameters with respect to thresholds indicative of locations along the insertion trajectory where navigation errors of the catheter may or have occurred; outputting on said display device an interactive user guide indicating one or more corrective actions that can be taken to correct and / or prevent said navigation error; Configure it to run, system.

24. The instructions, when executed by the processor, cause the processor to: recording as the navigation parameter a position parameter related to a real-time position or orientation of the catheter tip relative to the lumen; controlling the actuator to apply a force to the drive wire to bend at least one of the one or more bendable segments of the catheter, wherein the processor records as the navigation parameter a force parameter related to a push or pull force applied by the actuator to the one or more drive wires to bend the one or more bendable segments of the catheter; [0033] 2. The method according to claim 1, further comprising:

24. The system of claim 23.

25. The instructions, when executed by the processor, cause the processor to: recording as the navigation parameters one or more of a force parameter associated with the location of the catheter collision, a percentage of the distance the catheter tip had traveled along the preplanned insertion trajectory before the catheter collision occurred, and a number of lumen bifurcations traversed by the catheter tip before a navigation error occurred; [0033] 2. The method according to claim 1, further comprising:

24. The system of claim 23.

26. The instructions, when executed by the processor, cause the processor to: recording as the navigation parameter a linear force timeline associated with the insertion trajectory followed by the catheter tip during insertion into the lumen; and further configured to execute the linear force timeline includes force levels of push and pull forces applied by the actuation portion to the one or more drive wires to bend the one or more bendable segments of the catheter; the processor controls the display device to display one or more markers or flags throughout the linear force timeline, the one or more markers or flags indicating force levels approaching or exceeding the threshold at which a navigation error may occur or has occurred.

24. The system of claim 23.

27. The instructions, when executed by the processor, cause the processor to: detecting that a navigation error has occurred when a value of the navigation parameter is equal to or greater than the threshold; prompting the user to take corrective action to correct the navigation error; and further configured to execute The corrective action may include actions to manually or automatically cause one or more of: (a) activating a relaxation mode of the catheter; (b) stopping insertion of the catheter; (c) reversing a direction of movement of the catheter; and (d) reversing a direction of a force applied to at least one of the one or more drive wires that bends the catheter.

24. The system of claim 23.

28. The instructions, when executed by the processor, cause the processor to: Recording a location of a navigation error on the insertion trajectory; and displaying the location of the navigation error on the insertion trajectory; and automatically adding or prompting the user to add a flag or marker to the linear force timeline, the flag or marker representing the location of the occurrence of the navigation error; [0033] 2. The method according to claim 1, further comprising:

24. The system of claim 23.

29. When an occurrence of a navigation error is detected, the processor: transitioning the catheter into a relaxed mode by causing the actuator to reduce the actuation force applied to the one or more drive wires; retracting the catheter tip to a position on the insertion trajectory that is proximal to the flag or marker; navigating the catheter along a modified insertion trajectory with an actuation force below the threshold; and and further configured to execute 30. The system of claim 28.

30. a catheter having one or more bendable segments and at least one sensor disposed at a tip of the catheter; A processor; A system comprising: The processor, controlling insertion of the catheter along an insertion trajectory; applying forces to the one or more bendable segments to generate a pose at the catheter tip based on input from a user; recording force data at multiple time points and / or multiple positions along the insertion trajectory with the at least one sensor; displaying on a display a graphical representation of the force data with respect to insertion depth and the pose of the catheter tip; configured to execute system.

31. The processor is further configured to display, on the display, at least one flagged point on the insertion trajectory that represents an event associated with the force data.

31. The system of claim 30.

32. the processor is further configured to record additional flagged points along the insertion trajectory based on the user input.

31. The system of claim 30.

33. the processor is further configured to automatically record additional flagged points at predetermined time or distance intervals along the insertion trajectory.

31. The system of claim 30.

34. The processor is further configured to determine whether insertion of the catheter tip causes a navigation error; If insertion of the catheter tip causes a navigation error, the processor stops insertion of the catheter tip, records the location of the navigation error, and performs corrective action.

31. The system of claim 30.

35. the processor performs the corrective action by retracting the catheter tip to a flagged point that is proximal to the location of the navigation error.

35. The system of claim 34.

36. a catheter having one or more bendable segments and at least one sensor disposed at a tip of the catheter; A processor; A system comprising: The processor, controlling insertion of the catheter along an insertion trajectory based on input from a user; generating a pose at the catheter tip based on actuation forces applied to the one or more bendable segments; recording navigation parameters at multiple time points and / or multiple positions along the insertion trajectory with the at least one sensor; detecting a navigation error when the navigation parameter exceeds a threshold; displaying on a display a graphical representation of the navigation parameter relative to the threshold value or relative to a navigation parameter recorded at a previous time and / or position along the insertion trajectory; providing correction functionality including one or more of: displaying an interactive user guide on the display indicating catheter movements that can be performed to correct the navigation error; and displaying flagged points associated with insertion depth; configured to execute system.

37. The flagged points include error-free positions on the insertion trajectory.

37. The system of claim 36.

38. When a navigation error is detected, the processor transitions the catheter into a relax mode, retracts the catheter tip to the error-free position on the insertion trajectory, and controls reinsertion of the catheter tip from the error-free position along a corrected insertion trajectory.

38. The system of claim 37.

39. the flagged points include error-prone locations on the insertion trajectory; 37. The system of claim 36.

40. When a navigation error is detected, the processor transitions the catheter into a relax mode, retracts the catheter tip to a position on the insertion trajectory that is proximal to the error-prone location, and controls reinsertion of the catheter tip along a modified insertion trajectory that avoids the error-prone location.

40. The system of claim 39.

41. a catheter body having one or more bendable segments and a catheter tip; an actuation section coupled to the bendable segment of the catheter body via one or more drive wires disposed along a wall of the catheter body; At least one sensor provided at the tip of the catheter; a processor in operative communication with the actuator; A memory storing instructions; A system comprising: The instructions, when executed by the processor, cause the processor to: moving the catheter body along an insertion trajectory within a body cavity; applying an actuation force to the one or more bendable segments via the one or more drive wires to bend the catheter tip based on input from a user; receiving information from the at least one sensor during movement of the catheter body within the body lumen and / or during bending of the catheter tip; recording navigation parameters at a plurality of positions along the insertion trajectory and an orientation of the catheter tip relative to the body lumen based on the information received from the at least one sensor; displaying on a display a graphical representation of the insertion trajectory with respect to the navigation parameters recorded at the multiple positions and a graphical representation of the pose of the catheter tip with respect to the body lumen; Configure it to run, system.

42. the navigation parameters include force data corresponding to the forces applied to the one or more bendable segments to bend the catheter tip; the processor records the force data as a radar-like plot corresponding to one or more directions the catheter tip is bent to generate the pose of the catheter tip.

42. The system of claim 41.

43. the processor displays on the display at least one flagged point on the insertion trajectory that represents a navigation event relative to a threshold.

42. The system of claim 41.

44. If the value of the force data is greater than the threshold, the processor determines that the navigation event includes a navigation error.

44. The system of claim 43.

45. the processor is further configured to retract the catheter body in a direction opposite to the insertion trajectory such that a position of the catheter tip is proximal to the at least one flagged point representing the navigation event; The processor is further configured to control reinsertion of the catheter body through the body lumen along a modified insertion trajectory.

44. The system of claim 43.

Images