Systems and methods for responsive insertion and retraction of robotic endoscopes

Abstract

A method for controlling a tip speed of an articulated flexible endoscope is provided. The method includes: generating a command to move a tip of an elongate member of the articulated flexible endoscope at an expected speed; receiving sensor data acquired by a sensor disposed at a distal tip portion of the elongate member to calculate a tip velocity; calculating a difference between the expected speed and the tip speed; and control the tip speed based on the difference.

Description

[0001] This application is a divisional application of the Chinese patent application with the application date of March 24, 2023, application number 202380030119.0, and invention name “System and method for responsive insertion and retraction of a robotic endoscope” (the corresponding PCT application with the application date of March 24, 2023 and application number PCT / US2023 / 016286).

[0002] References

[0003] This application claims priority to U.S. Provisional Patent Application No. 63 / 324,746, filed on March 29, 2022, and U.S. Provisional Patent Application No. 63 / 480,502, filed on January 18, 2023, each of which is incorporated herein by reference in its entirety. Background Art

[0004] Endoscopy procedures use an endoscope to examine the interior of a hollow organ or body cavity. Unlike many other medical imaging techniques, an endoscope is inserted directly into an organ. Flexible endoscopes, which rely on intuitive manipulation and control, can be used to diagnose and treat diseases accessible through any natural orifice in the body. Depending on the clinical indication, endoscopes can be designated as bronchoscopes, ureteroscopes, colonoscopes, gastroscopes, otolaryngoscopes, and various other endoscopes. For example, flexible bronchoscopes can be used for lung cancer diagnosis and / or surgical treatment. However, one challenge with bronchoscopy is reaching the upper lobes of the lungs while navigating through the airways. In another example, flexible endoscopy has been used to examine and treat gastrointestinal (GI) tract disorders without creating an opening in the patient. The endoscope is introduced into the upper or lower GI tract via the mouth or anus, respectively. A miniature camera at the distal end captures images of the GI wall, which helps clinicians diagnose GI diseases. Simple surgical procedures (such as polypectomies and biopsies) can be performed by introducing a flexible tool through a working channel to reach the site of interest at the distal end.

[0005] Traditionally, endoscopes are reusable and may need to be thoroughly cleaned, disinfected, and / or sterilized after each procedure. In most cases, cleaning, disinfection, and sterilization can be aggressive processes that kill germs and / or bacteria. Such procedures can also be harsh on the endoscope itself. As a result, the design of such reusable endoscopes can often be complex, especially to ensure they can withstand such harsh cleaning, disinfection, and sterilization regimens. Periodic maintenance and repairs may also be required.

[0006] For instruments that are difficult to clean properly, low-cost disposable medical devices designated for single use have become popular. Single-use disposable devices can be packaged in sterile wrapping to avoid the risk of pathogenic cross-contamination of diseases such as HIV, hepatitis and other pathogens. Hospitals generally welcome the convenience of single-use, disposable products because they no longer have to worry about aging, overuse, breakage, malfunction and sterilization of the product. Traditional endoscopes typically include a handle that the operator uses to manipulate the endoscope. For single-use endoscopes, the handle typically encloses a camera, expensive electronics and mechanical structure at the proximal end to transmit video and allow the user to manipulate the endoscope via a user interface. This can lead to a high cost for the handle of a single-use endoscope.

[0007] Continuous robotics and endoscopes are typically long and flexible. The shaft of an endoscope can be restricted by bending or kinking during device insertion into the anatomy. Passive deformation of the endoscope shaft can be a result of insertion forces and contact with the anatomy, and this deformation is difficult to model. Prolapse or kinking can lead to potential damage by exposing sharp edges of the kinked, elongated device and complicating the surgical procedure. Furthermore, a bent or kinked elongated device can cause the system to lose control of the device's position / shape and potentially obstruct instrument passage. Summary of the Invention

[0008] The present invention recognizes the need for a robotic endoscope that allows surgical or diagnostic procedures to be performed with improved performance and cost-efficiency. It is also recognized herein that devices and systems including endoscopes can be disposable and do not require extensive cleaning procedures. The present disclosure provides low-cost, single-use, articulated endoscopes for diagnosis and treatment in various applications such as bronchoscopy, urology, gynecology, arthroscopy, orthopedics, otolaryngology, gastrointestinal endoscopy, neurosurgery, colonoscopy, and various other applications. In some embodiments, the present disclosure provides single-use, robotically controlled, disposable bronchoscopes for use with robotic systems to enable diagnostic assessment of injuries anywhere in the pulmonary anatomy. It should be noted that the provided endoscopic system can be used in various minimally invasive surgical procedures, therapeutic or diagnostic procedures involving various types of tissue including heart, bladder, and lung tissue, and can be used in other anatomical regions of the patient's body such as the digestive system including but not limited to the esophagus, liver, stomach, colon, urinary tract, or the respiratory system including but not limited to the bronchi, lungs, and various other regions.

[0009] In one aspect of the present disclosure, a method for controlling the tip velocity of an articulating flexible endoscope is provided. The method includes: generating a command to move the tip of an elongated member of the articulating flexible endoscope at a desired velocity; receiving sensor data acquired by a sensor disposed at a distal tip portion of the elongated member to calculate the tip velocity; calculating a difference between the desired velocity and the tip velocity; and controlling the tip velocity based on the difference.

[0010] In one aspect, a method for controlling the motion of a tip of an articulated flexible endoscope is provided. The method includes generating a command to drive an elongated member of the articulated flexible endoscope along an anatomical path via an instrument drive mechanism (IDM); receiving sensor data acquired by a position sensor disposed at a distal tip portion of the elongated member; upon determining that the distal tip portion is at a preselected position within the anatomical path, setting the motion of the distal tip to zero and calculating the motion of the distal tip portion within a time window; calculating a difference between the motion of the distal tip portion and the motion of the IDM within the same time window; and detecting a buckling event by comparing the difference to a threshold.

[0011] In some embodiments, the preselected location is the main carina. In some embodiments, the method further comprises determining that the distal tip portion is located at the preselected location based at least in part on the sensor data and the 3D model of the anatomical path. In some cases, the position sensor comprises an electromagnetic sensor.

[0012] In some embodiments, the method further comprises determining that the distal tip portion is located at a preselected position based at least in part on image data acquired by a camera located at the distal tip portion. In some embodiments, the threshold value is dynamically determined based on the target anatomical region toward which the articulated flexible endoscope is being moved. In some embodiments, the threshold value is a function of the tortuosity of the anatomical pathway.

[0013] In some embodiments, the size of the time window is between 4 seconds and 8 seconds. In some cases, the size of the time window is determined based on empirical data. In some embodiments, the movement of the distal tip portion within the time window is the accumulation of the distance traveled per time step.

[0014] In some embodiments, the method further includes setting the insertion force applied by the IDM to zero upon determining that the distal tip portion is at a preselected position within the anatomical path and comparing the insertion force to a force threshold. In some embodiments, the method further includes generating and displaying a warning message on the user interface upon determining that the insertion force is above the force threshold.

[0015] In some embodiments, the method further includes controlling the velocity of the distal tip portion of the elongated member based on the difference between the expected velocity and the measured tip velocity. In some cases, the measured tip velocity is calculated as a filtered time derivative of the sensor data projected in the forward direction. In some cases, the expected velocity is based on the input command. In some cases, controlling the velocity of the distal tip portion of the elongated member includes closed-loop control. For example, the measured tip velocity is processed through a low-pass filter to serve as a feedback signal for the closed-loop control.

[0016] In some embodiments, the distal tip portion includes structure for housing an imaging device, a position sensor, and an illumination device. In some embodiments, the proximal end of the elongated member of the articulated flexible endoscope is coupled to the IDM for applying force to one or more pull wires to articulate the distal tip portion of the elongated member for inserting or retracting the articulated flexible endoscope. In some embodiments, the method further includes displaying a message on a user interface indicating a buckling event and a recommendation for action to be taken in response to the buckling event.

[0017] In a related but independent aspect, a system for controlling the motion of the tip of an articulated flexible endoscope is provided. The system includes: a memory storing computer-executable instructions; one or more processors in communication with the articulated flexible endoscope and configured to execute the computer-executable instructions to: generate commands to drive an elongated member of the articulated flexible endoscope along an anatomical path via an instrument drive mechanism (IDM); receive sensor data acquired by a position sensor disposed at a distal tip portion of the elongated member; upon determining that the distal tip portion is at a preselected position within the anatomical path, set the motion of the distal tip to zero and calculate the motion of the distal tip portion within a time window; calculate the difference between the motion of the distal tip portion and the motion of the IDM within the same time window; and detect a buckling event by comparing the difference to a threshold.

[0018] It should be noted that the individual components of the provided modular endoscopic assemblies and devices can be used in various minimally invasive surgical procedures, therapeutic or diagnostic procedures involving various types of tissue including heart, bladder, and lung tissue, and can be used in other anatomical regions of the patient's body (such as the digestive system including but not limited to the esophagus, liver, stomach, colon, urethra or the respiratory system including but not limited to the bronchi, lungs, and various other regions).

[0019] Additional aspects and advantages of the present disclosure will become readily apparent to those skilled in the art from the following detailed description, wherein only illustrative embodiments of the present disclosure are shown and described. As will be appreciated, the present disclosure is capable of other and different embodiments, and its several details are capable of modification in various obvious respects, all without departing from the present disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not restrictive.

[0020] Incorporation by reference

[0021] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. In the event that a publication, patent, or patent application incorporated by reference contradicts the disclosure contained in the specification, the specification is intended to supersede and / or take precedence over any such contradictory material. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The novel features of the present invention are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present invention will be obtained by referring to the following detailed description of illustrative embodiments utilizing the principles of the present invention and the accompanying drawings (also referred to herein as "Figures"), in which:

[0023] Figure 1 An example of a patient's luminal network is schematically shown.

[0024] Figure 2 An example of a flexible elongate member undergoing extension and flexion in an airway / passageway is shown.

[0025] Figure 3 An example of a dead zone occurring during controlled retraction of an endoscope is shown.

[0026] Figure 4 An exemplary algorithm for controlling the velocity of an endoscope tip through a safety check operation is shown, in accordance with some embodiments of the present disclosure.

[0027] Figure 5 An exemplary warning algorithm is shown.

[0028] Figure 6 Examples of IDM insertion distance and endoscope tip insertion distance at different time points are shown.

[0029] Figure 7 An example of the dynamic tracking error threshold (DTET) as a function of insertion distance is shown.

[0030] Figure 8 An example of a user interface module is shown.

[0031] Figure 9 An example of a robotic endoscopy (eg, bronchoscopy) system according to some embodiments of the present invention is shown.

[0032] Figure 10 An example of a flexible endoscope according to some embodiments of the present disclosure is illustrated.

[0033] Figure 11 An example of a robotic bronchoscope including a handle portion and a flexible elongated member is shown.

[0034] Figure 12A An example of an instrument drive mechanism (IDM) is shown, which provides the mechanical interface to the handle portion of a robotic bronchoscope.

[0035] Figure 12B An example of a disposable endoscope removably coupled to an IDM is shown.

[0036] Figure 13 An example of the distal tip of an endoscope is shown.

[0037] Figure 14 An example distal portion of a catheter with integrated imaging and lighting is shown.

[0038] Figure 15 An example of a distal portion of a catheter with integrated imaging and lighting is shown.

[0039] Figure 16 An example of a "zero position" is shown.

[0040] Figure 17 An example of calculating motion difference or motion mismatch within a predetermined time window is shown.

[0041] Figure 18 An exemplary algorithm for controlling the motion of the endoscope tip is shown.

[0042] Figure 19 An example of a threshold value that varies based on anatomical region is shown.

[0043] Figure 20 and Figure 21 An example of a GUI displaying a warning message is shown. DETAILED DESCRIPTION

[0044] Although various embodiments of the present invention have been shown and described herein, it will be readily understood by those skilled in the art that such embodiments are provided by way of example only. Many variations, modifications, and substitutions may occur to those skilled in the art without departing from the present invention. It should be understood that various alternatives to the embodiments of the present invention described herein may be employed.

[0045] The embodiments disclosed herein can be combined in one or more of many ways to provide improved diagnosis and treatment for patients. The disclosed embodiments can be combined with existing methods and devices to provide improved treatment, for example, with known lung diagnostics, surgical methods, and surgical methods for other tissues and organs. It should be understood that any one or more of the structures and steps described herein can be combined with any one or more additional structures and steps of the methods and devices described herein, and the accompanying drawings and supporting text provide a description according to the embodiments.

[0046] While the exemplary embodiments will primarily be directed to devices or systems for bronchoscopy, those skilled in the art will appreciate that this is not intended to be limiting and that the devices described herein may be used for other therapeutic or diagnostic procedures and various anatomical regions of a patient's body. The provided devices or systems may be used in urology, gynecology, rhinology, otology, laryngoscopy, gastroenterology using an endoscope, a combination device including an endoscope and an instrument, an endoscope with positioning capabilities, and those skilled in the art will appreciate that this is not intended to be limiting and that the devices described herein may be used for other therapeutic or diagnostic procedures and other anatomical regions of a patient's body, such as the brain, heart, lungs, intestines, eyes, skin, kidneys, liver, pancreas, stomach, uterus, ovaries, testicles, bladder, ears, nose, mouth, soft tissues such as bone marrow, adipose tissue, muscle, glandular and mucosal tissue, spinal cord and nerves. tissue, cartilage), hard biological tissue (such as teeth, bones, etc.), as well as body cavities and ducts (such as sinuses, ureters, colon, esophagus, lungs, blood vessels and throat), and various other tissues in the form of: neuroendoscopes, encephaloscopes, ophthalmoscopes, otoscopes, rhinoscopes, laryngoscopes, gastroscopes, esophagoscopes, bronchoscopes, thoracoscopes, pleuroscopes, angioscopes, mediastinoscopes, nephroscopes, gastroscopes, duodenoscopes, choledochoscopes, bile ductoscopes, laparoscopes, amniographs, ureteroscopes, hysteroscopes, cystoscopes, rectoscopes, colonoscopes, arthroscopes, sialendoscopy, orthopedic endoscopes, etc., which are combined with various tools or instruments.

[0047] The systems and devices herein can be combined in one or more of many ways to provide improved diagnosis and treatment for patients. The systems and devices provided herein can be combined with existing methods and devices to provide improved treatment, for example, with known lung diagnostics, surgical methods, and surgical methods for other tissues and organs. It should be understood that any one or more of the structures and steps described herein can be combined with any one or more additional structures and steps of the methods and devices described herein, and the accompanying figures and supporting text provide a description according to the embodiments.

[0048] Whenever the term "at least," "greater than," or "greater than or equal to" precedes the first value in a series of two or more values, the term "at least," "greater than," or "greater than or equal to" applies to every value in the series. For example, greater than or equal to 1, 2, or 3 is equivalent to greater than or equal to 1, greater than or equal to 2, or greater than or equal to 3.

[0049] Whenever the term "not more than," "less than," or "less than or equal to" precedes the first value in a series of two or more values, the term "not more than," "less than," or "less than or equal to" applies to every value in the series. For example, less than or equal to 3, 2, or 1 is equivalent to less than or equal to 3, less than or equal to 2, or less than or equal to 1.

[0050] As used herein, the terms distal and proximal can generally refer to a position with reference to the device and can be opposite to an anatomical structure. For example, a distal position of a main shaft or main catheter can correspond to a proximal position of a patient's elongated member, and a proximal position of a main sheath or main catheter can correspond to a distal position of a patient's elongated member.

[0051] Answer insertion and retraction control

[0052] As mentioned above, one challenge with navigating endoluminal devices (eg, bronchoscopy) is reaching difficult-to-reach areas (eg, upper lobes of the lungs) while navigating through the airways. Figure 1 An example of a patient's luminal network 100 is schematically shown. In the illustrated embodiment, the luminal network 100 is a bronchial network of airways (i.e., lumens, branches) of a patient's lungs. Although the luminal network 100 is shown as a bronchial network of airways within a patient's lungs, the present disclosure is not limited to the illustrated example. The systems and methods described herein can be used to navigate any type of luminal network, such as a bronchial network, a renal network, a cardiovascular network (e.g., arteries and veins), the gastrointestinal tract, the urinary tract, as described elsewhere herein.

[0053] Accessing the periphery of the right upper lobe 101 of the lung through the apical segment (Ap) has always been a challenge for conventional bronchoscopes. Figure 1In the illustrated example 110, the 180-degree sharp angle between the trachea and the AP, combined with the large space at the junction of the bronchus intermedius, right mainstem bronchus, and right upper lobe bronchus, can cause a bronchoscope to kink and / or prolapse downward through the bronchus intermedius. As segments of the elongated device flex as it moves through one or more passageways, the flexed segments are compressed. This compression causes the elongated device to bend along its length and contact the walls of one or more passageways. In areas surrounding the elongated device with one or more large passageways, when the distal end of the device encounters tissue resistance, the elongated device may bend within the large airway and move toward an unintended anatomical region rather than in the intended direction ("prolapse"). Prolapse is more likely when tissue resistance is high (e.g., due to an airway that is too small, an incompletely inflated lung, tissue pathology, blind insertion, etc.). Example 110 illustrates that deflection / kinking of the scope shaft can even deflect into the contralateral bronchus (e.g., when the tip is unable to move forward but the operator continues to attempt to drive the scope further).

[0054] Figure 2 An example of prolapse 210 and buckling 220 during insertion of a flexible device is shown. As the flexible endoscope is advanced proximally, the flexible endoscope can deform 210 as it passes through turns and buckles 220 during insertion of the flexible device into the anatomy. Deformation can occur during insertion because the flexible device can assume a minimum energy shape, which can be a shaft "snapping" against tissue. Buckling can occur when the distal portion of the shaft encounters resistance.

[0055] During retraction of a flexible device, the distance lost to flexion can become slack when the system actuation direction is reversed. This phenomenon can result in a perceptible dead zone or system delay. For example, a user input commanding the robotic endoscope tip to move does not directly map to the robotic endoscope tip motion. Figure 3 An example of a dead zone for controlling endoscope retraction is shown. When the endoscope is flexed, retraction of the endoscope can result in robotic actuation with little translational motion of the endoscope tip.

[0056] Prolapse or kinking can cause potential damage because it can expose the sharp edges of the kinked elongated device and complicate the surgical procedure. In addition, a bent or kinked elongated device can cause the system to lose control of the position / shape of the device during insertion and retraction and can block the passage of instruments. In addition, a prolapsed or kinked device may not provide sufficient ability to reach the target anatomical structure to perform the intended task. Current approaches to addressing the kink / prolapse problem can include detecting prolapse, using shape sensing, force sensing, medical imaging of the device shape and position (comparing the detected position to the expected position) to detect when prolapse may occur. However, this approach requires additional imaging or sensing methods to determine shape and deformation, which may not provide real-time control of the movement of the endoscope tip and may further complicate the device or increase cost.

[0057] In one aspect of the present disclosure, a method and system for responsive insertion and retraction speed control of a flexible endoscope are provided. The speed control method herein can automatically correct for discrepancies in motion between the endoscope tip and speed commands (e.g., instrument drive mechanism (IDM) commands). The method for controlling the tip motion of a flexible endoscope can have an integrated safety check for detecting buckling / deformation.

[0058] Unlike conventional methods of buckling detection based on discrepancies in endoscopic device position (e.g., expected position and measured tip position), the methods and systems herein can automatically correct for buckling / deformation during insertion and retraction based on velocity and velocity control commands measured at the endoscope tip. This advantageously avoids the need for shape sensing or the use of additional imaging methods to determine the shape or position of the endoscopic device. Furthermore, velocity-based insertion and retraction control allows for more responsive automatic buckling correction with minimal delay.

[0059] Figure 4 An exemplary algorithm 400 for controlling the speed of an endoscope tip with integrated safety checks, according to some embodiments of the present invention, is shown. During insertion of an endoscopic device 401, the endoscopic device may receive a speed command. The speed command may be provided by user input 403. For example, a user may provide a control instruction via a control interface of the endoscopic device indicating a desired / expected speed of the endoscope tip.

[0060] The control interface of the endoscopic device may be part of a user interface module for an operator or user to interact with an endoscope (eg, a bronchoscope) during a surgical procedure. Figure 8An example of a user interface module 933 is shown. The user interface module can be a handheld controller. In some cases, the user interface module can include a proprietary user input device and one or more add-on elements that can be removably coupled to an existing user device to improve the user input experience. For example, a physical trackball or scroll wheel can replace or supplement the functionality of at least one of the virtual graphical elements displayed on a graphical user interface (GUI) (e.g., navigation arrows displayed on a touchpad) by assigning it functionality similar to the replaced graphical element. Examples of user devices can include, but are not limited to, mobile devices, smartphones / cellular phones, tablet computers, personal digital assistants (PDAs), laptop computers or notebook computers, desktop computers, media content players, and the like.

[0061] The user interface may include various devices, such as a touch screen display, a joystick, a keyboard, and other interactive devices. The user may use the user input device to navigate and / or control the movement of the robotic arm 911 and the movement of the catheter 920 (e.g., tip speed). The user input device may have any type of user interaction component, such as a button, a mouse, a joystick, a trackball, a touchpad, a stylus, an image capture device, a motion capture device, a microphone, a touch screen, a handheld wrist gimbal, an exoskeleton glove, or other user interaction system, such as a virtual reality system, an augmented reality system, or the like.

[0062] User input for commanding the velocity of the endoscope tip can be received via an input device. For example, pressing a joystick can be mapped to an analog value indicating the velocity / speed of the tip. For example, pressing the joystick halfway can be mapped to 3 mm / s, pressing it fully can be mapped to 6 mm / s, and not pressing it can be mapped to 0 mm / s. The input velocity can be any value (e.g., a continuous number) when the joystick is in various positions.

[0063] Speed ​​input can be provided via any suitable user input device. In some cases, the user input device can be a tactile stylus device in physical contact with a touch-sensitive display screen, and the user can control the robot system by moving the tactile stylus device on the display screen. For example, one or more physical user input devices or additional elements (e.g., trackballs, joysticks, or rollers) can be coupled to a graphical user interface (GUI) provided on a user device via tactile sensing or Bluetooth. For example, a trackball, joystick, or roller can replace or supplement the function of at least one of the virtual graphical elements (e.g., navigation arrows, speed) displayed on a graphical user interface (GUI) by giving it a function similar to the replaced graphical element. Additional elements can be coupled to the GUI via physical contact in a touch screen, via IO ports, wired or wireless communication, so that the user input received via the additional elements can be mapped to the input received by the virtual graphical elements presented on the GUI. Examples of user devices can include, but are not limited to, mobile devices, smart phones / cellular phones, tablet computers, personal digital assistants (PDAs), laptop or notebook computers, desktop computers, media content players, etc.

[0064] Return Reference Figure 4 Based on the particular user input device, the user input can be processed to convert it into a desired / commanded velocity for the tip of the catheter / endoscope 405. Next, a tip velocity error is calculated 407. The tip velocity error is the difference between the desired / commanded tip velocity and the velocity of the endoscope tip. In some embodiments, the velocity of the endoscope tip can be measured based on sensor data. In some cases, the sensor data can include position and orientation information of the distal tip of the endoscope.

[0065] In some cases, the sensor signal can be acquired by a positioning sensor. For example, the sensor signal can be acquired by an electromagnetic coil located at the distal end, which is used in conjunction with an electromagnetic tracking system to detect the position and orientation of the distal end of the endoscope 417. For example, a positioning sensor (e.g., an electromagnetic (EM) sensor) can be embedded in the distal tip of a catheter, and an EM field generator can be placed next to the patient's torso during the procedure. The EM field generator can locate the EM sensor position in 3D space, or can locate the EM sensor position and orientation in 5D or 6D space. The endoscope tip position p measured by the EM sensor e =EM sensor (tip) position, which can be expressed in terms of the field generator frame.

[0066] Next, the linear velocity of the endoscope tip can be calculated based on the measured position data 419. The linear velocity can be calculated using a time derivative method (e.g., backward Euler derivative). In some cases, the endoscope tip velocity can be calculated as the filtered time derivative of the tip position projected onto the forward direction 423 (e.g., as measured by the EM sensor represented in the field generator frame). This projection may be required because the velocity command provided by the user is based on the integrated position change occurring in the direction of n. The EM sensor data can be obtained at a specific frequency (e.g., 10-60 Hz). In some cases, a low-pass filter can be applied to generate filtered time derivative data 421.

[0067] The endoscope tip velocity can be calculated using the projection matrix v n =nn T filt(dp e The filtered time derivative of the tip position (e.g., expressed in the field generator frame and measured by the EM sensor) projected in the forward direction of n is dp / dt, where n is a unit vector indicating the forward direction of the endoscope tip expressed in the field generator frame. There can be a mechanical offset between the EM sensor and the endoscope tip, and the mechanical offset can be calibrated for each endoscopic device. e / dt represents the time derivative of the endoscope tip, nn T is the projection matrix that maps the above velocities to the direction of travel of the endoscope tip (i.e., ignoring velocities that are not in the direction of travel). n It is not affected by the joint because the endoscope tip translates orthogonally to n due to the joint.

[0068] After calculating the endoscope tip velocity, an endoscope tip velocity error 407 can be calculated and can be further processed for safety checks 409. In some cases, the safety checks 409 can include multiple checks. For example, the multiple checks can include determining whether the endoscope tip is stationary (e.g., tip velocity is approximately zero) while the insertion distance of the handle portion (e.g., IDM) of the endoscope exceeds a distance threshold. In another example, the multiple checks can include determining whether the endoscope tip is retracted (e.g., negative tip velocity error) when the handle portion (e.g., IDM) of the endoscope is inserted, and if so, prolapse may have occurred. In another example, the multiple checks can include determining whether the insertion force exceeds a force threshold. As used herein, the term "insertion distance" can refer to a distance along a navigation path.

[0069] The method 400 may include closed-loop control of the tip velocity to reduce the tip velocity error 411. The tip velocity of the endoscope may be controlled based on the tip velocity error calculated at operation 407, which serves as a feedback signal. Figure 8The closed loop control of the tip velocity by commanding the movement of the IDM is schematically shown. The user velocity input can be mapped to a command to control the movement of the IDM (e.g., the speed of moving the IDM along the insertion axis). The command can be a control signal to control the motor of the manipulator, thereby controlling the movement of the IDM (i.e., the proximal end of the endoscope). The endoscope tip velocity can be calculated based on the inverse kinematics of the manipulator. The endoscope tip velocity is then used to calculate the tip velocity in the forward direction by projecting in the forward direction of n using the projection matrix as described above. In some cases, the feedback signal can be a projection of the tip velocity that has been low-pass filtered to filter out noisy data.

[0070] return Figure 4 Based on the control algorithm, motion commands are generated to activate the robotic arm 413, thereby affecting the tip velocity of the endoscope 415. Due to the tortuosity, buckling, and / or prolapse of the navigation path (as described above), the effects on the motion of the IDM (e.g., insertion speed, insertion distance) and the effects on the motion of the endoscope tip (e.g., tip velocity) may not be perfectly matched.

[0071] In some embodiments, multiple security checks may be combined, ordered, and selected according to an alert algorithm to trigger an alert or suspend operation. Figure 5 An exemplary warning algorithm 500 is shown. The warning algorithm can trigger an action such as a warning or suspending operation based on one or more of a number of safety checks. In the illustrated method, a warning can be triggered based on a comparison between the endoscope tip insertion distance and the IDM insertion distance.

[0072] The process can begin by acquiring EM sensor data (e.g., EM position and orientation signals) 501. The endoscope tip insertion distance can be calculated based on the acquired EM sensor data (e.g., EM position and orientation signals) 503. For example, the endoscope tip insertion distance can be calculated based on sensor orientation information, which is used to distinguish between lateral motion and joint motion and motion resulting from robotic insertion. Robotic joint position data 511 can also be acquired. As described later in this document, the endoscopic device can be attached to the robotic arm via an IDM. The position and orientation of the IDM can be calculated based on the joint position data using forward kinematics 513. The IDM insertion distance is calculated at a series of time points 515 and compared to the tip insertion distance 505.

[0073] Figure 6 shows the IDM insertion distance X at time t 601 IDM (t) and the IDM insertion distance X at time t+1 IDM Example of (t+1). Tip insertion distance is also calculated as Xtip. IDM insertion distance X IDMThe difference between the tip insertion distance and the tip insertion distance can be the deformation loss. The deformation loss can be caused by buckling, prolapse, kinking of the flexible catheter and / or the tortuosity of the navigation path. For example, a tortuous path can result in a greater deformation loss than navigating through a straight path.

[0074] Return Reference Figure 5 The difference / discrepancy can be compared to a threshold value to determine the presence of buckling. The threshold value can be dynamic based on one or more factors, such as the anatomical region and / or the tortuosity of the navigation path.

[0075] In some cases, the dynamic threshold can be referred to as a dynamic tracking error threshold (DTET), which can trigger a warning and / or system safety operation when the tracking error (TE) exceeds the DTET. The tracking error can be a deformation loss as described above. As described above, the DTET can be dynamic based at least in part on the anatomical region traversed by the flexible catheter. In some cases, the DTET can be a function of the navigation path, the insertion distance, or the tortuosity of the path. The tortuosity of the path can be calculated, for example, as the ratio of the length of a segment of the path to the distance between its two ends.

[0076] Figure 7 Examples of the dynamic tracking error threshold (DTET) as a function of insertion distance are shown. Example A shows the DTET for a less tortuous path, so the total DTET value is a function of insertion distance, with a maximum value below 4 cm. The tracking error (TE) is compared with the DTET at various insertion distances, and a warning can be triggered when the TE exceeds the DTET for a given insertion distance, for example, TE1. Example B shows the DTET for a more tortuous path, so the total DTET value is a function of insertion distance, with a value greater than 4 cm at a specific insertion distance (or specific anatomical region).

[0077] During retraction of the medical device, the previously determined deformation loss during insertion can be used as a feedforward term to inversely compensate for the retraction control. This reduces the deformation loss that occurred during insertion before the tip moves during retraction.

[0078] The dynamic tracking error threshold may be determined using any suitable method, such as based on historical data, computed tomography (CT) scans of the anatomy, a 3D model of the navigation path, etc. For example, the tortuosity of the path may be calculated for various anatomical regions, segments, and insertion lengths along the path, and the DTET may vary based on the anatomical region.

[0079] In some cases, in addition to the anatomical region, DTET can also consider the effects of applying the insertion force within a safety range or acceptable deformation range. The safety range describes the range within which buckling will not damage the contacting tissue or organs in the patient's body. This safety range can be determined based on historical endoscopic data and patient data about previous similar operations. Any suitable method can be used to determine this safety range. For example, a machine learning algorithm can be used to train a model to determine the safety range. Historical endoscopic data and patient data can be used to create a training data set for training the model. As new sensor data is collected, the model can be continuously updated and improved.

[0080] Return Reference Figure 5 , a dynamic tracking error threshold can be used to determine whether the difference triggers a tip tracking warning 507. Next, the process can continue to determine whether the tip moves backward as the IDM moves forward, i.e., inserted 509. If so, prolapse may be present and a prolapse warning 510 can be triggered.

[0081] The warning algorithm 500 is for illustrative purposes only. It should be noted that fewer or more safety checks may be performed. For example, in addition to prolapse and flexion detection, the algorithm may also include checking whether the insertion force exceeds a threshold. The insertion force may be detected by one or more force sensors coupled to the robotic arm of the surgical robot system. When the insertion force approaches an insertion force threshold or approaches an insertion force threshold within a predefined range, the surgical robot system sends visual, audio, and / or tactile feedback to the user via the system GUI. For example, a warning indicating that the insertion force is approaching an insertion force threshold or is close to an insertion force threshold may be displayed on the GUI or delivered to the user using a suitable UI function.

[0082] In some embodiments, instead of or in addition to detecting a speed difference, the methods herein can also track the difference between the IDM insertion motion and the measured tip motion within a time window. For example, the motion of the endoscope tip and the motion of the IDM can be summed within the time window to determine the difference. Figure 17 An example of calculating a motion difference or motion mismatch within a predetermined time window is shown. For example, the distance traveled in each time step in the forward direction (e.g., d1 between T0 and T1, d2 between T1 and T2, etc.) can be summed for seven seconds and used as the total travel distance of the endoscope tip and the IDM within the time window, respectively. The motion of the endoscope tip can be measured using an EM sensor as described elsewhere herein. The EM sensor data can be acquired at a specific frequency (e.g., 10-60 Hz).

[0083] The motion mismatch tracked within the time window can be compared to a threshold value to determine the presence of a buckling event. In some cases, the time window is a window of four seconds, five seconds, six seconds, seven seconds, eight seconds, a number less than five seconds, a number greater than eight seconds, or any number between integers. The length of the time window can be selected (e.g., seven seconds) so that the motion mismatch tracked within the time window is not overly sensitive while allowing real-time detection of buckling events. In some cases, the size of the time window can be determined based on empirical data. In some cases, the length of the time window (window size) can be adjusted based on the application of use (e.g., body region), the desired sensitivity level, etc. The length of the time window can remain unchanged throughout the process. Alternatively, the length of the time window can vary as the endoscope navigates to different regions in the body.

[0084] In some cases, when the endoscope is inserted using a controller command, only the forward motion of the IDM and the endoscope tip can be tracked. In some cases, if the user-commanded amount of endoscope retraction exceeds a retraction threshold, the tracking of the motion mismatch between the IDM and the endoscope tip can be reset to zero.

[0085] As described above, the force threshold and / or the endoscope tip movement threshold can be based on the tortuosity of the path or anatomical region. Figure 19 An example of a threshold value that varies based on the anatomical region is shown. In some cases, the dynamic tracking error threshold, motion mismatch threshold, and / or force threshold can be determined based on the tortuosity of the path. The amount of force applied by the endoscope tip to the anatomical structure can be determined at least in part based on the force applied by the IDM in the direction of insertion and the friction between the endoscope and the path to the tip. Higher tortuosity can result in greater friction and, therefore, less force applied by the endoscope tip to the anatomical structure, while the amount of force sensed at the IDM is fixed. Figure 19 An example of different thresholds selected depending on the part of the lung within which the endoscope is driven is shown. For example, because the upper right region is highly tortuous, a higher force threshold and a higher dynamic tracking error threshold / motion mismatch threshold may be selected; because the upper left region is moderately tortuous, a medium force threshold and a medium dynamic tracking error threshold may be selected; and because the lower lobe region is less tortuous, a lower force threshold and a lower dynamic tracking error threshold may be selected. In some cases, the threshold is a function of the tortuosity of the path, and this function may be determined at least in part based on an anatomical model of the subject.

[0086] Detection using thresholds (e.g., force thresholds and dynamic tracking error thresholds) can be challenging because the amount of motion mismatch and the end effector force reaching the main lung carina can be variables that are affected by the initial settings. Figure 16As shown, the amount of motion mismatch and end effector force to reach the main carina point 1600 can depend on variables in the setup, such as friction within the endotracheal tube, the mechanical interface with the endobronchial adapter, the angle of the robotic arm after retraction, the endoscope stiffness curve, and / or the amount of lubricant applied to the endoscope. The algorithms and methods herein can advantageously avoid noise or uncertainty introduced by initial setup conditions by setting the force and / or motion mismatch values ​​to zero as the endoscope tip moves along the endotracheal tube 1601 to the main carina 1600. It should be noted that the zero position can be selected to be another suitable location, such as a location within a certain distance from the main carina or a location that depends on the anatomical structure.

[0087] In some cases, the main carina position 1600 can be pre-selected as the zero position. For example, when the endoscope is inserted into the trachea and reaches the main carina point 1600, the system can initiate tracking of motion mismatch, starting at zero and comparing it to the motion mismatch (or dynamic tracking error threshold), as described elsewhere herein. Similarly, when the tip of the endoscope reaches the main carina position 1600, the force can also be set to zero, and the system can begin tracking the force and comparing it to the force threshold to detect a safety event.

[0088] In some cases, upon determining that the endoscope tip is at the main carina 1600, the system may automatically set the force and / or motion mismatch between the IDM and the endoscope tip motion to zero. The systems and methods herein may use various suitable methods to detect the zero position at the main carina. In some cases, the system may utilize navigation data (e.g., EM sensor data) to determine whether the endoscope tip has reached the main carina. For example, by tracking EM sensor data of the tip position within a CT scan model of the patient's lungs, the system may automatically detect whether the endoscope tip has reached the main carina. In some cases, the system may employ an image recognition algorithm to detect the exit of the endoscope tip from the endotracheal tube 1601 into the trachea. For example, the method may use an image recognition algorithm to process a camera image (e.g., Figure 20 or Figure 21 ) to identify the bifurcation at the main carina. Based on the detection of the bifurcation, it can be determined that the endoscope tip is located at the main carina 1600. Various other methods can also be used to automatically detect the zero position. For example, after the IDM is aligned with the bronchial adapter and automatically retracted to the starting position, it can be determined based on human factors that the endoscope tip is located at the main carina position 1600 at a fixed insertion distance. The above methods for detecting the zero position can be used individually or in any combination.

[0089] Figure 18An exemplary algorithm 1800 for controlling the motion of an endoscope tip is shown. The algorithm 1800 can begin by detecting a zero position (e.g., the main carina 1801). The zero position can be detected using navigation data (e.g., EM sensor data), image recognition methods, and / or other methods as described above. After detecting the zero position, the force and / or motion mismatch of the endoscope tip can be set to zero 1803. The algorithm can also identify which lobe of the lung the endoscope is being driven towards. Based on the target area, a force threshold and a motion threshold (or DTET) can be determined 1805. In some cases, the algorithm can begin tracking the motion mismatch within a sliding window (e.g., a window size of seven seconds) and continuously compare it to the motion mismatch threshold. For example, the sliding window can have a window size of seven seconds, and the sliding size can be one second so that comparisons can be made every second. In some cases, the sliding size can be adjusted to one second, two seconds, three seconds, etc. to adjust the detection frequency.

[0090] When a motion mismatch greater than a motion mismatch threshold is detected, a warning message can be displayed on the GUI notifying the user that a kink or buckling event may occur. The user can choose to follow the advice in the message or continue driving the endoscope. In some cases, when a force greater than a force threshold is detected, a warning message can be displayed on the GUI and the robotic arm can stop moving. In some cases, the warning message can provide the user with instructions 1807 to remove the endoscope and restore the system (e.g., the system can automatically retract the IDM). In some cases, when it is detected that the endoscope is being driven toward a different anatomical region (e.g., a lobe) 1809, the algorithm can repeat the operation 1803 of setting the motion mismatch and force to zero and repeat the process. In the example shown, during IDM / endoscope retraction, the algorithm can track the IDM retraction distance and use the IDM retraction distance as the endoscope tip retraction distance.

[0091] Detected buckling, prolapse, or large insertion force can trigger a warning message and / or a control signal to the system. The warning or feedback message can be delivered to the user in various forms. In some cases, the warning or feedback message can indicate the type of problem, such as buckling, prolapse, or unsafe insertion force. For example, a message or warning can be provided for display on a graphical user interface (GUI) presented on a display used by the operator to control the operation. In some cases, the system can also generate suggestions for the user to take action in response to the detected buckling event. For example, the system can generate suggestions for the user to move the endoscope backward, adjust the movement of the tip, reduce / adjust the insertion force provided by the IDM, etc. Figure 20 An example of a GUI displaying a warning message is shown. When endoscope buckling is detected, a notification or warning message can be displayed. In the example shown, the message can also include a suggestion, such as retracting the endoscope or taking a fluoroscopic image. Figure 21An example of a warning message triggered when a high insertion force is detected is shown. In this example, a warning message can be displayed on the GUI when an insertion force exceeding a force threshold is detected. The warning message can include information that the robot's motion has stopped and a suggestion to the user, such as "Remove the endoscope and press Resume." In some cases, when the user clicks the "Resume" button, the system can automatically retract the IDM.

[0092] In some cases, control signals can be generated to modify the operation of the system, the robotic arm, the IDM, and / or the endoscope. For example, control signals can be automatically generated based on the detection of buckling, prolapse, or large insertion forces to adjust the motion of the endoscope, stop the movement of the endoscope, etc. For example, the control signal can automatically command the actuator of the IDM and / or the robotic arm to move the tip of the endoscope to reduce buckling or insertion force. The speed, movement, and trajectory path of the moving endoscope tip can be automatically determined using an algorithm based on sensor data (e.g., EM sensor data).

[0093] Flexible endoscope systems and equipment

[0094] In some embodiments, the endoscope tip speed control methods and systems described herein can be used to improve the reliability and stability of flexible endoscopes. The provided endoscope tip speed control mechanism can be used by any device or apparatus. In one aspect of the present invention, a flexible endoscope with improved performance (e.g., improved reliability) is provided.

[0095] Figure 9 Examples of robotic endoscope (e.g., bronchoscope) systems 900, 930 are shown according to some embodiments of the present invention. Figure 9As shown, a robotic endoscope (e.g., bronchoscope) system 900 may include a steerable catheter assembly 920 and a robotic support system 910 for supporting or carrying the steerable catheter assembly. The steerable catheter assembly may be a bronchoscope. The steerable catheter assembly may be the same as the endoscopic device described above. In some embodiments, the steerable catheter assembly may be a disposable robotic bronchoscope. In some embodiments, the robotic endoscope (e.g., bronchoscope) system 900 may include an instrument drive mechanism (IDM) 913 attached to an arm of the robotic support system. The instrument drive mechanism may be provided by any suitable controller device (e.g., a handheld controller), which may or may not include a robotic system. The instrument drive mechanism may provide a mechanical and electrical interface for the steerable catheter assembly 920. The mechanical interface may allow the steerable catheter assembly 920 to be releasably coupled to the instrument drive mechanism. For example, the handle portion of the steerable catheter assembly may be attached to the instrument drive mechanism via a quick installation / release device (e.g., a magnet, a spring-loaded level, etc.). In some cases, the steerable catheter assembly can be manually coupled to or released from the instrument drive mechanism without the use of tools.

[0096] The steerable catheter assembly 920 may include a handle portion 923, which may include components configured to process image data, provide power, or establish communication with other external devices. For example, the handle portion 923 may include circuits and communication elements that enable electrical communication between the steerable catheter assembly 920 and the instrument drive mechanism 913 and any other external systems or devices. In another example, the handle portion 223 may include circuit elements, such as a power supply for powering the endoscope's electronic devices (e.g., a camera and LED light). In some cases, the handle portion may be in electrical communication with the instrument drive mechanism 913 via an electrical interface (e.g., a printed circuit board) so that image / video data and / or sensor data can be received by the communication module of the instrument drive mechanism and can be transmitted to other external devices / systems. As an alternative or in addition, the instrument drive mechanism 913 may only provide a mechanical interface. The handle portion may be in electrical communication with a modular wireless communication device or any other user device (e.g., a portable / handheld device or controller) to transmit sensor data and / or receive control signals.

[0097] Steerable catheter assembly 920 can include flexible elongated member 911, which is coupled to the handle portion. In certain embodiments, the flexible elongated member can include a shaft, a steerable tip and a steerable portion. The steerable catheter assembly can be a disposable robotic bronchoscope. In some cases, only the elongated member can be disposable. In some cases, at least a portion of the elongated member (e.g., a shaft, a steerable tip etc.) can be disposable. In some cases, the entire steerable catheter assembly 920 including the handle portion and the elongated member can be disposable. The flexible elongated member and the handle portion are designed so that the entire steerable catheter assembly can be arranged at a low cost.

[0098] In some embodiments, the provided bronchoscopic system may further include a user interface. As shown in example system 930, the bronchoscopic system may include a treatment interface module 931 (user console side) and / or a treatment control module 933 (patient and robot side). The treatment interface module may allow an operator or user to interact with the bronchoscope during the surgical procedure. In some embodiments, treatment control module 933 may be a handheld controller. Treatment control module 933 may allow the user to control the speed of the bronchoscope tip, as described elsewhere herein. In some cases, the treatment control module may include a proprietary user input device and one or more add-on elements that can be removably coupled to an existing user device to improve the user input experience. For example, a physical trackball or scroll wheel may replace or supplement the functionality of at least one virtual graphical element displayed on a graphical user interface (GUI) (e.g., navigation arrows displayed on a touchpad) by assigning it functionality similar to the replaced graphical element. Examples of user devices may include, but are not limited to, mobile devices, smartphones / cellular phones, tablet computers, personal digital assistants (PDAs), laptop computers, desktop computers, media content players, and the like. Details regarding the user interface device and user console are described later herein.

[0099] Figure 10 An example of a flexible endoscope 1000 according to some embodiments of the present disclosure is shown. Figure 10As shown, flexible endoscope 1000 may include a handle / proximal portion 1009 and a flexible, slender member to be inserted into an object. The flexible, slender member may be the same as the flexible, slender member described above. In some embodiments, the flexible, slender member may include a proximal shaft (e.g., insertion shaft 1001), a steerable tip (e.g., tip 1005), and a steerable segment (active bending segment 1003). The active bending segment and the proximal shaft segment may be the same as the active bending segment, anti-prolapse passive segment, and proximal shaft segment described elsewhere herein. Endoscope 100 may also be referred to as a steerable catheter assembly, as described elsewhere herein. In some cases, endoscope 100 may be a single-use robotic endoscope. In some cases, the entire catheter assembly may be disposable. In some cases, at least a portion of the catheter assembly may be disposable. In some cases, the entire endoscope may be released from the instrument drive mechanism and may be discarded. In some embodiments, the endoscope may include varying levels of stiffness along the shaft to improve functional operation.

[0100] The endoscope or steerable catheter assembly 1000 can include a handle portion 1009 that can include one or more components configured to process image data, provide power, or establish communications with other external devices. For example, the handle portion can include circuitry and communication elements that enable electrical communication between the steerable catheter assembly 1000 and an instrument drive mechanism (not shown) and any other external systems or devices. In another example, the handle portion 1009 can include circuitry elements, such as a power supply for powering the electronics of the endoscope (e.g., a camera, an electromagnetic sensor, and an LED light).

[0101] One or more components located at the handle can be optimized so that expensive and complex components can be allocated to a robot support system, a handheld controller, or an instrument drive mechanism, thereby reducing costs and simplifying the design of a disposable endoscope. The handle portion or proximal portion can provide an electrical and mechanical interface to allow electrical and mechanical communication with the instrument drive mechanism. The instrument drive mechanism can include a group of motors that are actuated to rotationally drive a group of pull wires of the catheter. The handle portion of the catheter assembly can be assembled to the instrument drive mechanism so that its pulley / capstan assembly is driven by the group of motors. The number of pulleys can vary based on the pull wire configuration. In some cases, one, two, three, four, or more pull wires can be used to articulate a flexible endoscope or catheter.

[0102] The handle portion can be designed to allow the robotic bronchoscope to be disposable at a reduced cost. For example, a classic manual bronchoscope and a robotic bronchoscope can have a cable at the proximal end of the bronchoscope handle. The cable typically includes an illumination fiber, a camera video cable, and other sensor fibers or cables (such as electromagnetic (EM) sensors or shape sensing fibers). Such complex cables can be expensive, increasing the cost of the bronchoscope. The provided robotic bronchoscope can have an optimized design so that a simplified structure and components can be employed while retaining mechanical and electrical functionality. In some cases, the handle portion of the robotic bronchoscope can be designed without cables while providing a mechanical / electrical interface for the catheter.

[0103] An electrical interface (e.g., a printed circuit board) can allow image / video data and / or sensor data to be received by the communication module of the instrument drive mechanism and can be transmitted to other external devices / systems. In some cases, the electrical interface can establish electrical communication without cables or wires. For example, the interface can include pins soldered to an electronic board such as a printed circuit board (PCB). For example, a socket connector (e.g., a female connector) is provided on the instrument drive mechanism as a mating interface. This can advantageously allow the endoscope to be quickly inserted into the instrument drive mechanism or robotic support without the use of additional cables. This type of electrical interface can also be used as a mechanical interface so that when the handle portion is inserted into the instrument drive mechanism, both mechanical and electrical coupling are established. Alternatively or in addition, the instrument drive mechanism can provide only a mechanical interface. The handle portion can be in electrical communication with a modular wireless communication device or any other user device (e.g., a portable / handheld device or controller) for transmitting sensor data and / or receiving control signals.

[0104] In some cases, the handle portion 1009 may include one or more mechanical control modules, such as a Luer interface 1011, for connecting an irrigation system / aspiration system. In some cases, the handle portion may include a lever / knob for articulation control. Alternatively, the articulation control may be located at a separate controller attached to the handle portion via the instrument drive mechanism.

[0105] The endoscope can be attached to a robotic support system or a handheld controller via an instrument drive mechanism. The instrument drive mechanism can be provided by any suitable controller device (e.g., a handheld controller), which may or may not include a robotic system. The instrument drive mechanism can provide a mechanical and electrical interface to the steerable catheter assembly 1000. The mechanical interface can allow the steerable catheter assembly 1000 to be releasably coupled to the instrument drive mechanism. For example, the handle portion of the steerable catheter assembly can be attached to the instrument drive mechanism via a quick installation / release tool (such as a magnet, a spring-loaded level, etc.). In some cases, the steerable catheter assembly can be manually coupled to the instrument drive mechanism or released from the instrument drive mechanism without the use of tools.

[0106] In the illustrated example, the distal tip of the catheter or endoscope shaft is configured to articulate / bend in two or more degrees of freedom to provide a desired camera field of view or to control the direction of the endoscope. As illustrated in this example, an imaging device (e.g., a camera), a position sensor (e.g., an electromagnetic sensor) 1007 is located at the tip of the catheter or endoscope shaft 1005. For example, the line of sight of the camera can be controlled by controlling the articulation of the active bending segment 1003. In some cases, the angle of the camera can be adjustable so that the line of sight can be adjusted without or in addition to articulating the distal tip of the catheter or endoscope shaft. For example, the camera can be oriented at a certain angle (e.g., tilted) relative to the axial direction of the endoscope tip by means of an optical assembly.

[0107] The distal tip 1005 can be a rigid component that allows sensors (such as electromagnetic (EM) sensors), imaging devices (e.g., cameras), and other electronic components (e.g., LED light sources) to be positioned embedded at the distal tip.

[0108] In real-time EM tracking, an EM sensor, consisting of one or more sensor coils embedded in one or more locations and orientations within a medical device (e.g., the tip of an endoscopic tool), measures changes in the EM field generated by one or more static EM field generators positioned near the patient. The positional information detected by the EM sensor is stored as EM data. An EM field generator (or transmitter) can be placed near the patient to generate a low-intensity magnetic field that the embedded sensor can detect. This magnetic field induces a small current in the sensor coils of the EM sensor, which can be analyzed to determine the distance and angle between the EM sensor and the EM field generator. For example, an EM field generator can be positioned near the patient's torso during surgery to locate the EM sensor position in 3D space, or it can locate the EM sensor position and orientation in 5D or 6D space. This can provide a visual guide to the operator as they maneuver the bronchoscope toward the target site.

[0109] The endoscope can have a unique design in the elongated member. In some cases, the active bending section 1003 and the proximal shaft of the endoscope can be composed of a single tube that includes a series of cutouts (e.g., notches, slits, etc.) along its length to allow for improved flexibility, desired stiffness, and anti-prolapse features (e.g., features for defining a minimum bend radius).

[0110] As described above, the active bending segment 1003 can be designed to allow bending (e.g., articulation) in two or more degrees of freedom. Greater degrees of bending, such as 180 degrees and 270 degrees (or other articulation parameters for clinical indications), can be achieved through the unique structure of the active bending segment. In some cases, a variable minimum bending radius along the axial axis of the elongated member can be provided, such that the active bending segment or the passive segment can include two or more different minimum bending radii.

[0111] The articulation of the endoscope can be controlled by applying a force to the distal end of the endoscope via one or more pull wires. One or more pull wires can be attached to the distal end of the endoscope. In the case of multiple pull wires, pulling one wire at a time can change the orientation of the distal tip to tilt it up, down, left, right, or in any desired direction. In some cases, the pull wire can be anchored at the distal tip of the endoscope, travel through the curved segment, and enter the handle where the pull wire is coupled to a drive assembly (e.g., a pulley). The handle pulley can interact with an output shaft from the robotic system.

[0112] In some embodiments, the proximal end or proximal portion of one or more pull wires can be operably coupled to various mechanisms (e.g., gears, pulleys, capstans, etc.) in the handle portion of the catheter assembly. The pull wire can be a metal wire, cable, or filament, or it can be a polymer wire, cable, or filament. The pull wire can also be made of natural or organic materials or fibers. The pull wire can be any type of suitable wire, cable, or filament that can support various loads without deforming, significantly deforming, or breaking. The distal end / distal portion of one or more pull wires can be anchored or integrated into the distal portion of the catheter so that operation of the pull wire by the control unit can apply a force or tension to the distal portion that can manipulate or articulate (e.g., up, down, pitch, yaw, or any direction therebetween) at least the distal portion (e.g., the flexible segment) of the catheter.

[0113] The pull wire can be made of any suitable material, such as stainless steel (e.g., SS316), metal, alloy, polymer, nylon, or a biocompatible material. The pull wire can be a wire, cable, or filament. In some embodiments, different pull wires can be made of different materials to change the load-bearing capacity of the pull wire. In some embodiments, different sections of the pull wire can be made of different materials to change the stiffness and / or load-bearing capacity along the pull wire. In some embodiments, the pull wire can be used for the transmission of electrical signals.

[0114] The proximal design can improve the reliability of the device without introducing additional cost, thereby allowing for low-cost single-use endoscopes. In another aspect of the present invention, a single-use robotic endoscope is provided. The robotic endoscope can be a bronchoscope and can be the same as the steerable catheter assembly described elsewhere herein. Conventional endoscopes can be complex in design and are typically designed to be reused after surgery, which requires thorough cleaning, disinfection, or sterilization after each surgery. Existing endoscopes are often designed with complex structures to ensure that the endoscope can withstand the cleaning, disinfection, and sterilization processes. The provided robotic bronchoscope can be a single-use endoscope, which can beneficially reduce cross-contamination between patients and infected persons. In some cases, the robotic bronchoscope can be delivered to the medical practitioner in pre-sterilized packaging and is intended to be discarded after a single use.

[0115] like Figure 11 As shown, the robotic bronchoscope 1120 may include a handle portion 1113 and a flexible elongated member 1111. In some embodiments, the flexible elongated member 1111 may include a shaft, a steerable tip, and a steerable / active bending section. Figure 10 The robotic bronchoscope can be a single-use robotic endoscope. In some cases, only the catheter can be disposable. In some cases, at least a portion of the catheter can be disposable. In some cases, the entire robotic bronchoscope can be released from the instrument drive mechanism and can be discarded. In some cases, the bronchoscope can include different levels of stiffness along its axis to improve functional operation. In some cases, the minimum bend radius along the axis can vary.

[0116] The robotic bronchoscope can be releasably coupled to an instrument drive mechanism 1120. The instrument drive mechanism 1120 can be mounted to an arm of a robotic support system, or can be mounted to any actuated support system as described elsewhere herein. The instrument drive mechanism can provide a mechanical and electrical interface to the robotic bronchoscope 1110. The mechanical interface can allow the robotic bronchoscope 1110 to be releasably coupled to the instrument drive mechanism. For example, a handle portion of the robotic bronchoscope can be attached to the instrument drive mechanism via a quick install / release tool (such as a magnet and a spring-loaded level). In some cases, the robotic bronchoscope can be manually coupled to or released from the instrument drive mechanism without the use of tools.

[0117] Figure 12A An example of an instrument drive mechanism (IDM) 1220 is shown that provides a mechanical interface to a handle portion 1213 of a robotic bronchoscope. As shown in this example, the instrument drive mechanism 1220 can include a set of motors that are actuated to rotationally drive a set of pull wires of a flexible endoscope or catheter. The handle portion 1213 of the catheter assembly can be assembled to the instrument drive mechanism so that its pulley assembly or capstan is driven by the set of motors. The number of pulleys can vary based on the pull wire configuration. In some cases, one, two, three, four, or more pull wires can be used to articulate the flexible endoscope or catheter.

[0118] The handle portion can be designed to allow the robotic bronchoscope to be disposable at a reduced cost. For example, a classic manual bronchoscope and a robotic bronchoscope can have a cable at the proximal end of the bronchoscope handle. The cable typically includes an illumination fiber, a camera video cable, and other sensor fibers or cables (such as electromagnetic (EM) sensors or shape sensing fibers). Such complex cables can be expensive, increasing the cost of the bronchoscope. The provided robotic bronchoscope can have an optimized design so that a simplified structure and components can be employed while retaining mechanical and electrical functionality. In some cases, the handle portion of the robotic bronchoscope can be designed without cables while providing a mechanical / electrical interface for the catheter.

[0119] Figure 12BAnother example of a disposable endoscope 1200 detachably coupled to an IDM 1201 is shown. One or more components located at the handle 1209 can be optimized, allowing expensive and complex components to be distributed to the robotic support system 1203, the handheld controller, or the instrument drive mechanism 1201, thereby reducing costs and simplifying the design of the disposable endoscope. The handle portion or proximal portion 1209 can provide electrical and mechanical interfaces to allow for electrical and mechanical communication with the instrument drive mechanism 1201. The instrument drive mechanism 1201 can include a set of motors that are actuated to rotationally drive a set of puller wires of a catheter. The handle portion 1209 of the catheter assembly can be mounted to the instrument drive mechanism 1201 so that its pulley / capstan assembly is driven by the motors. For example, the handle pulleys can interact with the output shaft 1203 of the IDM supported by the robotic system. The number of pulleys can vary based on the puller wire configuration. In some cases, one, two, three, four, or more puller wires can be used to articulate the flexible endoscope or catheter.

[0120] Figure 13 An example of a distal tip 1300 of an endoscope is shown. In some cases, the distal portion or tip of the catheter 1300 can be substantially flexible, allowing it to be steered in one or more directions (e.g., pitch, yaw). The catheter can include a tip portion, a curved segment, and an insertion axis. In some embodiments, the catheter can have variable bending stiffness along the longitudinal axis. For example, the catheter can include multiple segments with different bending stiffnesses (e.g., flexible, semi-rigid, and rigid). Bending stiffness can be varied by selecting materials with different stiffnesses / rigidities, changing the structure in different segments (e.g., cutouts, patterns), adding additional support components, or any combination of the above. In some embodiments, the catheter can have a variable minimum bending radius along the longitudinal axis. Selecting different minimum bending radii at different locations along the catheter can beneficially provide anti-prolapse capabilities while still allowing the catheter to reach hard-to-reach areas. In some cases, the proximal end of the catheter does not need to be highly curved, so the proximal portion of the catheter can be reinforced with additional mechanical structures (e.g., additional layers of material) to achieve greater bending stiffness. This design can provide support and stability to the catheter. In some cases, variable bending stiffness can be achieved by using different materials during the extrusion of the catheter. This can advantageously allow different stiffness levels along the axis of the catheter during the extrusion manufacturing process without requiring additional fastening or assembly of the different materials.

[0121] The distal portion of the catheter can be manipulated by one or more pull wires 1305. The distal portion of the catheter can be made of any suitable material (such as a copolymer, polymer, metal, or alloy) so that it can be bent by the pull wires. In some embodiments, the proximal or terminal ends of the one or more pull wires 1305 can be coupled to a drive mechanism (e.g., a gear, pulley, capstan, etc.) via an anchoring mechanism as described above.

[0122] The pull wire 1305 can be a metal wire, cable or filament, or it can be a polymer wire, cable or filament. The pull wire 1305 can also be made of natural or organic materials or fibers. The pull wire 1305 can be any type of suitable wire, cable or filament that can support various loads without deformation, significant deformation or breakage. The distal end or distal portion of one or more pull wires 1305 can be anchored or integrated into the distal portion of the catheter so that operation of the pull wire by the control unit can apply a force or tension to the distal portion, which force or tension can manipulate or articulate (e.g., up, down, pitch, yaw or any direction therebetween) at least the distal portion (e.g., the flexible segment) of the catheter.

[0123] The dimensions of the catheter can enable one or more electronic components to be integrated into the catheter. For example, the outer diameter of the distal tip can be approximately 4 to 4.4 millimeters (mm), and the diameter of the working channel 1303 can be approximately 2 mm, so that one or more electronic components can be embedded in the wall of the catheter. However, it should be noted that, depending on the application, the outer diameter can be in any range less than 4 mm or greater than 4.4 mm, and the diameter of the working channel can be in any range depending on the tool size or specific application.

[0124] One or more electronic components may include an imaging device, an illumination device, or a sensor. In some embodiments, the imaging device may be a video camera 1313. The imaging device may include optical elements and an image sensor for capturing image data. The image sensor may be configured to generate image data in response to the wavelength of light. Various image sensors, such as complementary metal oxide semiconductor (CMOS) or charge coupled device (CCD), may be used to capture image data. The imaging device may be a low-cost camera. In some cases, the image sensor may be provided on a circuit board. The circuit board may be an imaging printed circuit board (PCB). The PCB may include multiple electronic components for processing image signals. For example, the circuitry for a CCD sensor may include an A / D converter and an amplifier to amplify and convert the analog signal provided by the CCD sensor. Alternatively, the image sensor may be integrated with the amplifier and converter to convert the analog signal into a digital signal, eliminating the need for a circuit board. In some cases, the output of the image sensor or circuit board may be image data (digital signal) that can be further processed by the camera circuitry or the camera's processor. In some cases, the image sensor may include an array of optical sensors.

[0125] The lighting device can include one or more light sources 1311 located at the distal tip. The light source can be a light emitting diode (LED), an organic LED (OLED), quantum dots, or any other suitable light source. In some cases, the light source can be a small LED or dual-tone flash LED lighting for compact design.

[0126] The imaging device and the lighting device can be integrated into the catheter. For example, the distal portion of the catheter can include a suitable structure that matches at least one dimension of the imaging device and the lighting device. The imaging device and the lighting device can be embedded in the catheter. Figure 14 An example distal portion of a catheter with an integrated imaging device and lighting device is shown. A camera can be located at the distal portion. The distal tip can have a structure for accommodating a camera, lighting device, and / or position sensor. For example, a camera can be embedded in a cavity 1410 at the distal tip of the catheter. The cavity 1410 can be integrally formed with the distal portion of the cavity and can have a size that matches the length / width of the camera so that the camera can not move relative to the catheter. The camera can be adjacent to the working channel 1420 of the catheter to provide a near-field view of a tissue or organ. In some cases, the posture or orientation of the imaging device can be controlled by controlling the rotational movement (e.g., rolling) of the catheter.

[0127] The power of the camera can be provided by a wired cable. In some cases, the cable line can be in a harness that provides power to the camera and the lighting element or other circuit at the distal tip of the catheter. The camera and / or light source can be powered by a power supply located at the handle portion via a wire, copper wire, or any other suitable tool that travels through the length of the catheter. In some cases, real-time images or videos of tissues or organs can be transmitted wirelessly to an external user interface or display. Wireless communication can be WiFi, Bluetooth, RF communication, or other forms of communication. In some cases, the images or videos captured by the camera can be broadcast to multiple devices or systems. In some cases, the images and / or video data from the camera can be transmitted to a processor located in the handle portion via a wire, copper wire, or any other suitable tool along the length of the catheter. The image or video data can be transmitted to an external device / system via the wireless communication component in the handle portion. In some cases, the system can be designed so that the wire is not visible to the operator or the wire is not exposed to the operator.

[0128] In traditional endoscopy, illumination can be provided by a fiber optic cable that transmits light from a light source located at the proximal end of the endoscope to the distal end of the robotic endoscope. In some embodiments of the present disclosure, a small LED light can be employed and embedded in the distal portion of the catheter to reduce design complexity. In some cases, the distal portion can include a structure 1430 having dimensions that match those of the small LED light source. As shown in the illustrated example, two cavities 1430 can be integrally formed with the catheter to accommodate the two LED light sources. For example, the outer diameter of the distal tip can be approximately 4 to 4.4 millimeters (mm), and the diameter of the working channel of the catheter can be approximately 2 mm, allowing the two LED light sources to be embedded at the distal end. The outer diameter can be within any range from less than 4 mm to greater than 4.4 mm, and the diameter of the working channel can be within any range depending on the size of the tool or the specific application. Any number of light sources can be included. The internal structure of the distal portion can be designed to accommodate any number of light sources.

[0129] In some cases, each LED can be connected to a power cord that can extend to the proximal handle. In some embodiments, the LEDs can be soldered to separate power cords that are then bundled together to form a single strand. In some embodiments, the LEDs can be soldered to a pull wire that supplies power. In other embodiments, the LEDs can be crimped or directly connected to a single pair of power cords. In some cases, a protective layer such as a thin layer of biocompatible glue can be applied to the front surface of the LEDs to provide protection while allowing light to be emitted. In some cases, an additional cover 1431 can be placed at the forward end face of the distal tip, providing precise positioning of the LEDs and sufficient space for the glue. The cover 1431 can be made of a transparent material that matches the refractive index of the glue so that the illumination light is not blocked.

[0130] It should be noted that the illustrated distal end design is for illustration purposes only. Other suitable designs may be used to integrate one or more components into the distal tip. Figure 15 Another example of a distal portion 1500 of a catheter with an integrated imaging device and lighting device is shown. As shown in example 1500, the distal tip can have a structure 1501 for accommodating a camera, a structure 1503 for accommodating a lighting device and / or a position sensor. The camera can be embedded in a cavity 1501 at the distal tip of the catheter. The cavity 1501 can be integrally formed with the distal portion of the cavity and can have a size that matches the length / width of the camera so that the camera cannot move relative to the catheter. The camera can be adjacent to the working channel 1507 of the catheter to provide a near-field view of the tissue or organ. In some cases, the posture or direction of the imaging device can be controlled by controlling the rotational movement (e.g., rolling) of the catheter. As shown in example 1500, the cavity 1503 can be integrated with the catheter to accommodate an LED light source.

[0131] In one aspect, a system for controlling the motion of the tip of an articulated flexible endoscope is provided. The system includes: a memory storing computer-executable instructions; one or more processors in communication with the articulated flexible endoscope and configured to execute the computer-executable instructions to: generate commands to drive an elongated member of the articulated flexible endoscope along an anatomical path via an instrument drive mechanism (IDM); receive sensor data acquired by a position sensor disposed at a distal tip portion of the elongated member; upon determining that the distal tip portion is at a preselected position within the anatomical path, set the motion of the distal tip to zero and calculate the motion of the distal tip portion within a time window; calculate the difference between the motion of the distal tip portion and the motion of the IDM within the same time window; and detect a buckling event by comparing the difference to a threshold.

[0132] As used herein, a processor includes one or more processors, such as a single processor, or a plurality of processors of a distributed processing system. A controller or processor as described herein generally includes a tangible medium for storing instructions for implementing process steps, and the processor may include, for example, one or more of a central processing unit, a programmable array logic, a gate array logic, or a field programmable gate array. In some cases, one or more processors may be a programmable processor (e.g., a central processing unit (CPU) or a microcontroller), a digital signal processor (DSP), a field programmable gate array (FPGA), and / or one or more advanced RISC machine (ARM) processors. In some cases, one or more processors may be operably coupled to a non-temporary computer-readable medium. A non-temporary computer-readable medium may store logic, code, and / or program instructions that may be executed by one or more processor units to perform one or more steps. A non-temporary computer-readable medium may include one or more memory units (e.g., removable media or external memory, e.g., an SD card or random access memory (RAM)). One or more methods or operations disclosed herein may be implemented in a hardware component or a combination of hardware and software, e.g., an ASIC, a special-purpose computer, or a general-purpose computer.

[0133] The one or more processors may be in communication with the endoscope.The one or more processors may be located remotely from the endoscope system or on the endoscope system (eg, at a handle, at a user device used to control the endoscope).

[0134] The present invention provides embodiments including but not limited to the following:

[0135] 1. A method for controlling the movement of a tip of an articulated flexible endoscope, the method comprising:

[0136] generating commands to drive an elongated member of the articulated flexible endoscope along an anatomical path via an instrument drive mechanism (IDM);

[0137] receiving sensor data acquired by a position sensor disposed at a distal tip portion of the elongated member;

[0138] upon determining that the distal tip portion is at a preselected position within the anatomical path, setting the motion of the distal tip portion to zero and calculating the motion of the distal tip portion within a time window;

[0139] calculating a difference between the motion of the distal tip portion and the motion of the IDM within the same time window; and

[0140] Buckling events are detected by comparing the difference to a threshold value.

[0141] 2. The method of embodiment 1, wherein the preselected location is the main carina.

[0142] 3. The method according to embodiment 1 also includes determining that the distal tip portion is located at the preselected position based at least in part on the sensor data and the 3D model of the anatomical path.

[0143] 4. The method of embodiment 3, wherein the position sensor comprises an electromagnetic sensor.

[0144] 5. The method according to embodiment 1 further includes determining that the distal tip portion is located at the preselected position based at least in part on image data acquired by a camera located at the distal tip portion.

[0145] 6. The method of embodiment 1, wherein the threshold is dynamically determined based on the target anatomical region toward which the articulated flexible endoscope is moving.

[0146] 7. The method of embodiment 1, wherein the threshold is a function of the tortuosity of the anatomical path.

[0147] 8. The method according to embodiment 1, wherein the size of the time window is between 4 seconds and 8 seconds.

[0148] 9. The method of embodiment 8, wherein the size of the time window is determined based on empirical data.

[0149] 10. The method of embodiment 1, wherein the movement of the distal tip portion within the time window is an accumulation of distance traveled per time step.

[0150] 11. The method of embodiment 1 further comprises setting the insertion force applied by the IDM to zero upon determining that the distal tip portion is located at the preselected position within the anatomical path, and comparing the insertion force to a force threshold.

[0151] 12. The method of embodiment 11 further comprises generating and displaying a warning message on a user interface when it is determined that the insertion force is higher than the force threshold.

[0152] 13. The method of embodiment 1 further comprises controlling a velocity of the distal tip portion of the elongated member based on a difference between an expected velocity and a measured tip velocity.

[0153] 14. A method according to embodiment 13, wherein the measured tip velocity is calculated as the filtered time derivative of the sensor data projected in the forward direction.

[0154] 15. The method of embodiment 13, wherein the desired speed is based on an input command.

[0155] 16. The method of embodiment 14, wherein controlling the velocity of the distal tip portion of the elongated member comprises closed-loop control.

[0156] 17. The method of embodiment 16, wherein the measured tip velocity is processed through a low-pass filter to serve as a feedback signal for the closed-loop control.

[0157] 18. The method of embodiment 1, wherein the distal tip portion includes structure for accommodating an imaging device, the position sensor, and an illumination device.

[0158] 19. A method according to embodiment 1, wherein the proximal end of the slender member of the articulated flexible endoscope is connected to the IDM for applying force to one or more pull wires to articulate the distal tip portion of the slender member to insert or retract the articulated flexible endoscope.

[0159] 20. The method of embodiment 1, further comprising displaying a message on a user interface indicating the buckling event and a suggestion for taking action in response to the buckling event.

[0160] 21. A system for controlling movement of a tip of an articulated flexible endoscope, the system comprising: a memory storing computer-executable instructions; and one or more processors in communication with the articulated flexible endoscope and configured to execute the computer-executable instructions to:

[0161] generating commands to drive an elongated member of the articulated flexible endoscope along an anatomical path via an instrument drive mechanism (IDM);

[0162] receiving sensor data acquired by a position sensor disposed at a distal tip portion of the elongated member;

[0163] upon determining that the distal tip portion is at a preselected position within the anatomical path, setting the motion of the distal tip portion to zero and calculating the motion of the distal tip portion within a time window;

[0164] calculating a difference between the motion of the distal tip portion and the motion of the IDM within the same time window; and

[0165] Buckling events are detected by comparing the difference to a threshold value.

[0166] 22. The system of embodiment 21, wherein the preselected location is the main carina.

[0167] 23. The system of embodiment 21, wherein the one or more processors are further configured to determine that the distal tip portion is located at the preselected position based at least in part on the sensor data and the 3D model of the anatomical pathway.

[0168] 24. The system of embodiment 23, wherein the position sensor comprises an electromagnetic sensor.

[0169] 25. The system of embodiment 21, wherein the one or more processors are further configured to determine that the distal tip portion is located at the preselected position based at least in part on image data acquired by a camera located at the distal tip portion.

[0170] 26. The system of embodiment 21, wherein the threshold is dynamically determined based on the target anatomical region toward which the articulated flexible endoscope is moving.

[0171] 27. A system according to embodiment 21, wherein the threshold is a function of the tortuosity of the anatomical path.

[0172] 28. A system according to embodiment 21, wherein the size of the time window is between 4 seconds and 8 seconds.

[0173] 29. The system of embodiment 28, wherein the size of the time window is determined based on empirical data.

[0174] 30. The system of embodiment 21, wherein the movement of the distal tip portion within the time window is an accumulation of distance traveled per time step.

[0175] 31. The system of embodiment 21, wherein the one or more processors are further configured to set the insertion force applied by the IDM to zero upon determining that the distal tip portion is located at the preselected position within the anatomical path, and to compare the insertion force to a force threshold.

[0176] 32. The system of embodiment 31, wherein the one or more processors are further configured to generate and display a warning message on a user interface when it is determined that the insertion force is above the force threshold.

[0177] 33. The system of embodiment 21, wherein the one or more processors are further configured to control the velocity of the distal tip portion of the elongated member based on a difference between an expected velocity and a measured tip velocity.

[0178] 34. A system according to embodiment 33, wherein the measured tip velocity is calculated as the filtered time derivative of the sensor data projected in the forward direction.

[0179] 35. The system of embodiment 33, wherein the expected speed is based on an input command.

[0180] 36. A system according to embodiment 34, wherein closed-loop control is used to control the speed of the distal tip portion of the slender member.

[0181] 37. A system according to embodiment 36, wherein the measured tip velocity is processed through a low-pass filter to serve as a feedback signal for the closed-loop control.

[0182] 38. The system of embodiment 31, wherein the distal tip portion comprises a structure for accommodating an imaging device, the position sensor, and an illumination device.

[0183] 39. A method according to embodiment 31, wherein the proximal end of the slender member of the articulated flexible endoscope is connected to the IDM for applying force to one or more pull wires to articulate the distal tip portion of the slender member to insert or retract the articulated flexible endoscope.

[0184] 40. The system of embodiment 31, wherein the one or more processors are further configured to display a message on a user interface indicating the buckling event and a recommendation to take action in response to the buckling event.

[0185] Although preferred embodiments of the present invention have been shown and described herein, it will be readily understood by those skilled in the art that such embodiments are provided as examples only. Without departing from the present invention, those skilled in the art will now contemplate many variations, changes, and replacements. It should be understood that various alternatives to the embodiments of the present invention described herein may be employed in implementing the present invention. The accompanying claims are intended to define the scope of the present invention and thus cover methods and structures within the scope of these claims and their equivalents.

Claims

1. A method for controlling the movement of a tip of an articulated flexible endoscope, the method comprising: generating commands to drive an elongated member of the articulated flexible endoscope along an anatomical path via an instrument drive mechanism (IDM); receiving sensor data acquired by a position sensor disposed at a distal tip portion of the elongated member; upon determining that the distal tip portion is at a preselected position within the anatomical path, setting the motion of the distal tip portion to zero and calculating the motion of the distal tip portion within a time window; calculating a difference between the motion of the distal tip portion and the motion of the IDM within a same time window; as well as Buckling events are detected by comparing the difference to a threshold value.

2. The method of claim 1, wherein the preselected location is the main carina.

3. The method of claim 1 further comprising determining that the distal tip portion is located at the preselected position based at least in part on the sensor data and the 3D model of the anatomical pathway. The method of claim 3 , wherein the position sensor comprises an electromagnetic sensor.

5. The method of claim 1, further comprising determining that the distal tip portion is located at the preselected position based at least in part on image data acquired by a camera located at the distal tip portion.

6. The method of claim 1, wherein the threshold is dynamically determined based on a target anatomical region toward which the articulated flexible endoscope is moving. The method of claim 1 , wherein the threshold is a function of the tortuosity of the anatomical path. The method according to claim 1 , wherein the size of the time window is between 4 seconds and 8 seconds. The method of claim 8 , wherein the size of the time window is determined based on empirical data.

10. A system for controlling the movement of the tip of an articulated flexible endoscope, the system comprising: memory that stores computer-executable instructions; one or more processors in communication with the articulating flexible endoscope and configured to execute the computer-executable instructions to: generating commands to drive an elongated member of the articulated flexible endoscope along an anatomical path via an instrument drive mechanism (IDM); receiving sensor data acquired by a position sensor disposed at a distal tip portion of the elongated member; upon determining that the distal tip portion is at a preselected position within the anatomical path, setting the motion of the distal tip portion to zero and calculating the motion of the distal tip portion within a time window; calculating a difference between the motion of the distal tip portion and the motion of the IDM within a same time window; as well as Buckling events are detected by comparing the difference to a threshold value.

Images