Systems and methods for robotic bronchoscopy

Abstract

To provide an endoscopic robotic arm system and apparatus.SOLUTION: The devices and systems comprise various features improving the cost efficiency and reducing the complexity of manufacture and usage. Disposable elongate members are described herein. A user interface device may also be adapted to user behavior and may be personalized.SELECTED DRAWING: Figure 6B

Description

[Technical Field]

[0001] reference [1] This application claims priority to U.S. Provisional Patent Application No. 62 / 950,740, filed December 19, 2019, which is incorporated herein by reference in its entirety. [Background technology]

[0002] Background of the Invention [2] Early diagnosis of lung cancer is crucial. The five-year survival rate for lung cancer is approximately 18%, significantly lower than that of the three most prevalent cancers: breast cancer (90%), colorectal cancer (65%), and prostate cancer (99%). In 2018, a total of 142,000 lung cancer deaths were recorded.

[0003] [3] Overall, the typical diagnosis and surgical treatment of lung cancer can vary widely depending on the procedures, clinical protocols, and clinical settings used by healthcare providers. Inconsistent processes can not only result in high costs for patients and healthcare systems, but can also delay cancer diagnoses. Summary of the Invention [Problem to be solved by the invention]

[0004] Summary of the Invention [4] The present disclosure recognizes a need for a minimally invasive system that enables the performance of surgical or diagnostic procedures with improved reliability and cost efficiency. The present disclosure provides systems and methods that enable standardized early lung cancer diagnosis and treatment at reduced cost. The present disclosure provides accessible, more cost-effective methods and systems for early cancer diagnosis and treatment. In some embodiments of the present invention, at least a portion of the robotic bronchoscopy system is disposable. For example, the catheter portion can be designed to be disposable at low cost while maintaining surgical capabilities and functionality. Furthermore, the provided robotic bronchoscopy system is designed to access difficult-to-reach tissues, such as the bronchi and lungs, without incurring additional costs. It should be noted that the provided robotic system can be used in a variety of minimally invasive surgical procedures involving various types of tissue, including cardiac, bladder, and lung tissue. [Means for solving the problem]

[0005] [5] According to some aspects of the present disclosure, a robotic endoscopic device is provided. The device may include a disposable elongate member having a proximal end and a distal end, the proximal end being removably attached to a robotic arm. The distal end includes a plurality of pull wires integral with a wall of the elongate member. The elongate member may also be referred to as a bronchoscope or a catheter, which may be used interchangeably throughout this specification.

[0006] [6] In one aspect, a robotic endoscopic device is provided, the robotic endoscopic device including a disposable elongate member having a proximal end and a distal end, the proximal end removably attached to a robotic arm via a handle, the distal end integrated with an imaging device, a position sensor, and an illumination device, and a bending section articulated by one or more pull wires.

[0007] [7] In some embodiments, the distal end includes structure for receiving an imaging device, a position sensor, and an illumination device. In some embodiments, the imaging device, the position sensor, and the illumination device are arranged in a compact configuration. In some embodiments, the handle includes one or more components configured to process image data, provide power to the imaging device, the position sensor, and the illumination device, or establish communication with an external device.

[0008] [8] In some embodiments, the handle includes an interface configured to couple the handle to an instrument drive mechanism attached to the robotic arm. Optionally, the interface includes an electrical interface and a mechanical interface. Optionally, the mechanical interface is configured to detachably couple the handle to the instrument drive mechanism. Optionally, the device further includes an anti-buckling mechanism with an alignment feature. For example, the alignment feature is configured to assist in aligning the instrument drive mechanism with the anti-buckling mechanism. In some examples, the alignment feature includes a magnetic component, a laser, or a click button. In some examples, the anti-buckling mechanism includes a series of connected cylinders, each including a lip structure. In some examples, the lip structure of each cylinder has a retaining portion of the same diameter.

[0009] [9] In some embodiments, a robotic endoscopic system includes a robotic endoscopic device and a user interface device configured for a user to control movement of the robotic endoscopic device. Optionally, the user interface device is personalized based on past user behavior. In some examples, the user interface device is personalized using a model trained with a machine learning algorithm. Optionally, the robotic endoscopic system further includes a display configured to display image data captured by the imaging device, the image data overlaid with a virtual rendering of one or more components. In some examples, the display of the virtual rendering of the one or more components is selectively enabled or disabled by a user.

[0010]

[10] In some embodiments, both the handle and the disposable elongate member are single-use. In some embodiments, one or more pull wires are individually attached to the bending section according to a selected configuration pattern. In some embodiments, control of the articulation of the robotic endoscopic device is based at least in part on a virtual mapping algorithm. Optionally, the virtual mapping algorithm maps the selected configuration pattern to an updated configuration pattern upon a change in the state of one or more pull wires.

[0011]

[11] Additional aspects and advantages of the present disclosure will become readily apparent to those skilled in the art from the following detailed description, wherein only illustrative embodiments of the present disclosure are shown and described. As will be realized, the present disclosure is capable of other and different embodiments, and its several details are capable of modifications in various obvious respects, all without departing from the present disclosure. Accordingly, the drawings and description are to be regarded as illustrative in nature, and not as restrictive.

[0012] Incorporation by Reference

[12] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference. To the extent that the publications and patents or patent applications incorporated by reference conflict with the disclosure contained herein, the present specification is intended to supersede and / or take precedence over such conflicting matter.

[0013] BRIEF DESCRIPTION OF THE DRAWINGS

[13] The novel features of the invention are set forth with particularity in the appended claims. A better understanding of the features and advantages of the present invention will be obtained by reference to the following detailed description that sets forth illustrative embodiments, in which the principles of the invention are utilized, and the accompanying drawings (also referred to herein as "Figure" and "FIG."). [Brief explanation of the drawings]

[0014] [Figure 1]

[14] shows an exemplary workflow for standardized lung cancer diagnosis enabled by the robotic bronchoscopy system described herein. [Figure 2A]

[15] An example of a robotic bronchoscopy system, according to some embodiments of the present invention. [Figure 2B]

[16] Figures 1A-1C show different views of an exemplary robotic bronchoscopy system, according to some embodiments of the present invention. [Figure 3A]

[17] shows an example of an X-ray fluoroscopic (tomosynthesis) imaging system. [Figure 3B]

[18] shows a C-arm fluoroscopic (tomosynthesis) imaging system in different (rotational) positions during imaging of the subject. [Figure 4A]

[19] shows an example of a user interface for visualizing the optimal path and a virtual airway with catheter tip and lesion location superimposed. [Figure 4B]

[20] shows an example of a navigation view with extended information. [Figure 4C]

[21] shows an example of a navigation view with a user-selected virtual rendering. [Figure 4D]

[22] shows an example of an endoluminal view. [Figure 5]

[23] An exemplary treatment interface module is shown that allows an operator or user to interact with the bronchoscope during a surgical procedure. [Figure 6A]

[24] shows an example of a treatment control module. [Figure 6B]

[24] shows an example of a treatment control module. [Figure 7]

[25] shows an example of a robotic arm mounted on top of a robotic cart in a treatment control module. [Figure 8]

[26] Figure 1 shows an example of a robotic bronchoscope, according to some embodiments of the present invention. [Figure 9]

[27] Figure 1 shows an example of an instrument drive mechanism that provides a mechanical interface to the handle portion of a robotic bronchoscope, according to some embodiments of the present invention. [Figure 10]

[28] FIG. 1 illustrates an exemplary handle portion of a robotic bronchoscope, according to some embodiments of the present invention. [Figure 11]

[29] An exemplary steerable catheter is shown, according to some embodiments of the present invention. [Figure 12]

[30] An exemplary distal portion of a catheter with an integrated imaging and illumination device is shown. [Figure 13]

[31] Figure 3 shows an example of a compact configuration of multiple electronic elements located in the distal portion of a catheter, according to some embodiments of the present invention. [Figure 14]

[32] Figure 3 shows examples of conventional and novel configurations of pull wires attached to a control ring structure. [Figure 15]

[33] Figure 3 shows various configurations of pull wires for a robotic catheter system, according to some embodiments of the present invention. [Figure 16]

[34] Figure 1 shows an example of a guidewire with an expandable tip, according to some embodiments of the present invention. [Figure 17]

[35] Figure 3 illustrates an exemplary anti-buckling mechanism, according to some embodiments of the present invention. [Figure 18A]

[36] Figure 1 illustrates the internal structure of an exemplary anti-buckling mechanism, according to some embodiments of the present invention. [Figure 18B]

[37] An example of an assembly of an anti-buckling mechanism and a handle is shown. [Figure 18C]

[37] An example of an assembly of an anti-buckling mechanism and a handle is shown. [Figure 18D]

[38] An example of a scope handle and anti-buckling tube assembly with side connection features is shown. [Figure 18E]

[39] An example is shown in which the anti-buckling tube and scope connection assembly is mounted onto the instrument drive mechanism from above. [Figure 18F]

[40] An example of a patient-side connector and IDM is shown. [Figure 18G]

[41] Another example of an anti-buckling mechanism moved to a target position by aligning the patient connector with the IDM is shown. [Figure 18H]

[42] An example of an alignment function is shown. [Figure 18I]

[42] An example of an alignment function is shown. [Figure 19]

[43] Figure 4 illustrates an example user interface according to some embodiments of the present invention. [Figure 20]

[44] presents an example of a neural network model for generating control signals in response to individual user inputs. [Figure 21A]

[45] shows an example of a portable handle add-on module. [Figure 21B]

[46] shows various examples of robotic bronchoscopes used in conjunction with various systems, devices, and modules. [Figure 22]

[47] show an exemplary portable robotic cone beam CT. DETAILED DESCRIPTION OF THE INVENTION

[0015] Detailed Description of the Invention

[48] While various embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions may occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be used.

[0016]

[49] While the exemplary embodiments are primarily directed to bronchoscopes, those skilled in the art will understand that this is not intended to be limiting. Additionally, the devices described herein may be used for other therapeutic or diagnostic procedures in other anatomical regions of a patient's body, such as the digestive system, including but not limited to the esophagus, liver, stomach, colon, and urinary tract, or the respiratory system, including but not limited to the bronchi, lungs, and various others.

[0017]

[50] The embodiments disclosed herein can be combined in one or more of many ways to provide improved diagnosis and treatment to patients. The disclosed embodiments can be combined with existing methods and devices to provide improved treatment, such as in combination with known methods of pulmonary diagnosis, surgery, and surgery of other tissues and organs. It should be understood that any one or more of the structures and steps described herein can be combined with any one or more additional structures and steps of the methods and devices described herein, and the figures and supporting text provide a description according to the embodiments.

[0018]

[51] Although the treatment plans and definitions of diagnostic or surgical procedures described herein are presented in the context of pulmonary diagnosis or surgery, the methods and devices described herein can be used to treat any tissue of the body and any organ and blood vessel of the body, such as the brain, heart, lungs, intestines, eyes, skin, kidneys, liver, pancreas, stomach, uterus, ovaries, testes, bladder, ears, nose, mouth, bone marrow, adipose tissue, muscle, glandular and mucosal tissue, spinal cord and nerve tissue, soft tissue such as cartilage, hard biological tissue such as teeth, bone, and body lumens and passageways, such as the sinuses, ureters, colon, esophagus, pulmonary passageways, blood vessels and throat.

[0019]

[52] Whenever the term "at least," "greater than," or "greater than or equal to" appears before the first number in a series of two or more numbers, the term "at least," "greater than," or "greater than or equal to" applies to each number in the series. For example, 1, 2, or 3 or more is equivalent to 1 or more, 2 or more, or 3 or more.

[0020]

[53] Whenever the term "not exceeding," "less than," or "equal to or less than" appears before the first number in a series of two or more numbers, the term "not exceeding," "less than," or "equal to or less than" applies to each number in the series. For example, 3, 2, or 1 or less is equivalent to 3 or less, 2 or less, or 1 or less.

[0021]

[54] As used herein, a processor encompasses one or more processors, e.g., a single processor, or multiple processors, e.g., in a distributed processing system. A controller or processor as described herein generally includes a tangible medium for storing instructions for performing process steps, and a processor may include, for example, one or more of a central processing unit, programmable array logic, gate array logic, or field programmable gate array. In some cases, the one or more processors may be a programmable processor (e.g., a central processing unit (CPU) or microcontroller), a digital signal processor (DSP), a field programmable gate array (FPGA), and / or one or more Advanced RISC Machine (ARM) processors. In some cases, the one or more processors may be operably coupled to a non-transitory computer-readable medium. The non-transitory computer-readable medium may store logic, code, and / or program instructions executable by one or more processor units to perform one or more steps. The non-transitory computer-readable medium may include one or more memory units (e.g., removable media or external storage, such as an SD card or random access memory (RAM)). One or more methods or operations disclosed herein may be performed by hardware components or a combination of hardware and software, such as an ASIC, a special purpose computer, or a general purpose computer.

[0022]

[55] As used herein, the terms distal and proximal may generally refer to a location relative to the device, as opposed to an anatomical location. For example, a distal location of a bronchoscope or catheter may correspond to a proximal location of an elongate member in a patient, and a proximal location of a bronchoscope or catheter may correspond to a distal location of an elongate member in a patient.

[0023]

[56] The systems described herein include an elongate portion or member, such as a catheter. The terms "elongate member," "catheter," and "bronchoscope" are used interchangeably throughout this specification unless the context suggests otherwise. The elongate member can be placed directly within a body lumen or cavity. In some embodiments, the system may further include a support device, such as a robotic manipulator (e.g., a robotic arm), for driving, supporting, positioning, or controlling the movement and / or motion of the elongate member. Alternatively or additionally, the support device may be a handheld device or other control device that may or may not include a robotic system. In some embodiments, the system may further include peripheral devices and subsystems, such as an imaging system, that assist and / or facilitate navigation of the elongate member to a target site within a subject's body.

[0024]

[57] In some embodiments of the present disclosure, a robotic bronchoscopy system is provided for performing surgical or diagnostic procedures with improved outcomes at a low cost. For example, the robotic bronchoscopy system may include a steerable catheter that can be completely disposable. This may beneficially reduce sterilization requirements, which can be costly and difficult to operate, but sterilization or sanitization may not be effective. Furthermore, one challenge in bronchoscopy is navigating the airways to reach the upper lobes of the lungs. Optionally, the provided robotic bronchoscopy system may be designed with the ability to autonomously or semi-autonomously navigate through airways with small curvatures. Alternatively, the robotic bronchoscopy system may be navigated by an operator through a control system with visual guidance.

[0025]

[58] Typical lung cancer diagnosis and surgical treatment can vary significantly depending on the procedures, clinical protocols, and clinical settings used by healthcare providers. Inconsistent processes can result in delayed early-stage lung cancer diagnosis, high costs to the healthcare system and patients to diagnose and treat lung cancer, and high risks of clinical and procedural complications. The provided robotic bronchoscopy system can enable standardized early lung cancer diagnosis and treatment. Figure 1 shows an exemplary workflow 100 for standardized lung cancer diagnosis enabled by the robotic bronchoscopy system described herein.

[0026]

[59] As illustrated in Figure 1, preoperative imaging can be performed to identify lesions. Any suitable imaging modality, such as magnetic resonance (MR), positron emission tomography (PET), x-ray, computed tomography (CT), and ultrasound, can be used to identify lesions or regions of interest. For example, a patient suspected of having lung cancer can undergo a preoperative CT scan, and suspicious lung nodules can be identified on the CT images. The preoperative imaging process can be performed before a bronchoscopy.

[0027]

[60] The CT images can then be analyzed to generate a map for guiding the robotic bronchoscope during bronchoscopy. For example, a lesion or region of interest (ROI) can be segmented on the image. As the lungs are being imaged, a pathway or path to the lesion can be highlighted on the reconstructed image to plan a navigation path. The reconstructed image can then guide the robotic bronchoscope to the target tissue or site. In some cases, the navigation path can be pre-planned using 3D image data. For example, a catheter can be advanced toward the target site under robotic control of a robotic bronchoscopy system. The catheter can be manually steered or advanced toward the target site autonomously or semi-autonomously. In one example, catheter movement can be image-guided such that the insertion and / or steering direction can be automatically controlled.

[0028]

[61] In some cases, the lesion location on preoperative imaging may be inaccurate due to patient motion or body misalignment. In such cases, the lesion location may be confirmed prior to a surgical procedure (e.g., biopsy or treatment). The precise lesion location may be confirmed or updated using a robotic bronchoscopy system. For example, a bronchoscopy system may provide an interface to an imaging modality, such as fluoroscopy, to perform real-time in-vivo imaging of the target site and surrounding area to identify the lesion location. In an example, a C-arm or O-arm fluoroscopic imaging system may be used to generate tomosynthesis images to confirm or update the lesion location. Proceeding to a surgical procedure, such as a biopsy, various surgical tools, such as a biopsy tool, brush, or forceps, may be inserted into the working channel of the catheter to manually or automatically perform the biopsy or other surgical procedure.

[0029]

[62] A sample of the lesion or any other target tissue can then be obtained with a tool inserted through the working channel of the catheter. The system allows camera visualization to be maintained throughout the entire procedure, including while the tool is inserted through the working channel. Optionally, the tissue sample can be rapidly evaluated in situ with a rapid cytology to determine whether a repeat tissue sampling is necessary or to determine further action. Optionally, the rapid cytology process can also include a rapid analysis of the tissue sample to determine subsequent surgical treatment. For example, if the tissue sample is determined to be malignant as a result of the rapid cytology process, a manual or robotic treatment tool can be inserted through the working channel of the robotic bronchoscope to perform endobronchial treatment of lung cancer. This advantageously allows diagnosis and treatment to be performed in a single session, thereby providing targeted, painless, and rapid treatment for early-stage lung cancer.

[0030] 2A and 2B illustrate examples of robotic bronchoscopy systems 200, 230, according to some embodiments of the present invention. As shown in FIG. 2A, robotic bronchoscopy system 200 may include a steerable catheter assembly 220 and a robotic support system 210 for supporting or transporting the steerable catheter assembly. The steerable catheter assembly may be a bronchoscope. In some embodiments, the steerable catheter assembly may be a single-use robotic bronchoscope. In some embodiments, robotic bronchoscopy system 200 may include an instrument drive mechanism 213 attached to an arm of the robotic support system. The instrument drive mechanism may be provided by any suitable controller device (e.g., a handheld controller), which may or may not include a robotic system. The instrument drive mechanism may provide a mechanical and electrical interface to steerable catheter assembly 220. The mechanical interface may allow steerable catheter assembly 220 to be detachably coupled to the instrument drive mechanism. For example, the handle portion of the steerable catheter assembly may be attached to the instrument drive mechanism via quick attachment / detachment means such as a magnet, a spring-loaded level, etc. In some cases, the steerable catheter assembly may be manually coupled to or detached from the instrument drive mechanism without the use of tools.

[0031]

[64] The steerable catheter assembly 220 may include a handle portion 223, which may include components configured to process image data, provide power, or establish communication with other external devices. For example, the handle portion 223 may include circuitry and communication elements that enable electrical communication between the steerable catheter assembly 220 and the instrument drive mechanism 213 and any other external systems or devices. In another example, the handle portion 223 may include circuit elements such as a power supply for powering the endoscope's electronics (e.g., camera and LED light). In some cases, the handle portion may electrically communicate with the instrument drive mechanism 213 via an electrical interface (e.g., a printed circuit board) such that image / video data and / or sensor data can be received by the instrument drive mechanism's communications module and transmitted to other external devices / systems. Alternatively or additionally, the instrument drive mechanism 213 may provide only a mechanical interface. The handle portion may be in electrical communication with the modular wireless communication device or any other user device (e.g., a portable / handheld device or controller) to transmit sensor data and / or receive control signals, details of which are provided later in this specification.

[0032]

[65] The steerable catheter assembly 220 may include a flexible elongate member 211 coupled to a handle portion. In some embodiments, the flexible elongate member may include a shaft, a steerable tip, and a steerable section. The steerable catheter assembly may be a single-use robotic bronchoscope. In some cases, only the elongate member may be disposable. In some cases, at least a portion of the elongate member (e.g., shaft, steerable tip, etc.) may be disposable. In some cases, the entire steerable catheter assembly 220, including the handle portion and elongate member, may be disposable. The flexible elongate member and handle portion are designed to allow the entire steerable catheter assembly to be disposable at low cost. More details about the flexible elongate member and steerable catheter assembly are provided later in this specification.

[0033] In some embodiments, the provided bronchoscope system may also include a user interface. As illustrated in exemplary system 230, the bronchoscope system may include a therapy interface module 231 (user console side) and / or a therapy control module 233 (patient and robot side). The therapy interface module may enable an operator or user to interact with the bronchoscope during a surgical procedure. In some embodiments, the therapy control module 233 may be a handheld controller. The therapy control module may optionally include a dedicated user input device and one or more add-on elements removably coupled to an existing user device to improve the user input experience. For example, a physical trackball or roller may substitute for or complement the functionality of at least one virtual graphical element displayed on a graphical user interface (GUI) (e.g., a navigation arrow displayed on a touchpad) by providing similar functionality to the graphical element it replaces. Examples of user devices include, but are not limited to, mobile devices, smart phones / cell phones, tablets, personal digital assistants (PDAs), laptop or notebook computers, desktop computers, media content players, etc. User interface devices and user consoles are described in more detail later in this specification.

[0034]

[67] Figure 2B shows a different view of the bronchoscope system. The user console 231 may be mounted to the robotic support system 210. Alternatively or additionally, the user console or a portion of the user console (e.g., the treatment interface module) may be mounted on a separate mobile cart.

[0035] Robotic Intraluminal Platform

[68] In one aspect, a robotic endoluminal platform is provided. Optionally, the robotic endoluminal platform may be a bronchoscopy platform. The platform may be configured to perform one or more operations consistent with the method described in FIG. 1. FIGS. 3-7 illustrate various examples of a robotic endoluminal platform and its components or subsystems, according to some embodiments of the present invention. In some embodiments, the platform may include a robotic bronchoscopy system and one or more subsystems usable in combination with the robotic bronchoscopy system of the present disclosure.

[0036]

[69] In some embodiments, one or more subsystems may include an imaging system, such as a fluoroscopic (tomosynthesis) imaging system, for real-time imaging of a target region (e.g., including a lesion). FIG. 3A illustrates an example of a fluoroscopic (tomosynthesis) imaging system 300. For example, the fluoroscopic (tomosynthesis) imaging system may perform precise tracking or confirmation of lesion location before or during a surgical procedure, as described in FIG. 1. In some cases, the lesion location may be tracked based on position data related to the fluoroscopic (tomosynthesis) imaging system / station (e.g., a C-arm) and image data captured by the fluoroscopic (tomosynthesis) imaging system. The lesion location may be registered to the coordinate frame of the robotic bronchoscopy system. The position or movement of the fluoroscopic (tomosynthesis) imaging system can be measured using any suitable motion / position sensor 310, such as an inertial measurement unit (IMU), one or more gyroscopes, velocity sensors, accelerometers, magnetometers, position sensors (e.g., global positioning system (GPS) sensors), visual sensors (e.g., imaging devices capable of detecting visible, infrared, or ultraviolet light, such as cameras), proximity or distance sensors (ultrasonic sensors, lidar, time-of-flight cameras, or depth cameras), altitude sensors, attitude sensors (e.g., compasses), and / or magnetic field sensors (magnetometers, electromagnetic sensors, wireless sensors, etc.). One or more sensors for tracking the movement and position of the fluoroscopic (tomosynthesis) imaging station can be located on the imaging station or remotely from the imaging station, such as a wall-mounted camera 320. FIG. 3B shows a C-arm fluoroscopic (tomosynthesis) imaging system in different (rotational) positions during image capture of a subject. The various poses may be captured by one or more sensors as described above.

[0037]

[70] In some embodiments, the location of the lesion can be segmented in the image data captured by the fluoroscopic (tomosynthesis) imaging system using the signal processing unit 330. One or more processors of the signal processing unit can be configured to further overlay the treatment location (e.g., the lesion) on the real-time fluoroscopic image / video. For example, the processing unit can be configured to generate an augmentation layer including augmentation information such as the location of the treatment location or target site. Optionally, the augmentation layer can also include a graphic marker indicating a path to the target site. The augmentation layer can be a substantially transparent image layer including one or more graphic elements (e.g., a box, an arrow, etc.). The augmentation layer can be overlaid on the optical view of the optical image or video stream captured by the fluoroscopic (tomosynthesis) imaging system and / or displayed on a display device. The transparency of the augmentation layer allows a user to view the optical image with the graphic elements overlaid on top of it. Optionally, both the segmented lesion image and the optimal path for guiding the elongated member to reach the lesion may be overlaid on the real-time tomosynthesis image, allowing the operator or user to visualize the exact location of the lesion and the intended path of travel of the bronchoscope. Optionally, segmented and reconstructed images (e.g., CT images as described elsewhere) provided prior to operation of the systems described herein may be overlaid on the real-time image.

[0038]

[71] In some embodiments, one or more subsystems of the platform may include a navigation and localization subsystem. The navigation and localization subsystem may be configured to construct a virtual airway model based on preoperative images (e.g., preoperative CT images). The navigation and localization subsystem may be configured to identify segmented lesion locations in the 3D-rendered airway model, and the navigation and localization subsystem may generate an optimal path from the main bronchus to the lesion with a recommended approach angle to perform a surgical procedure (e.g., biopsy) based on the location of the lesion.

[0039]

[72] In a registration step prior to driving the bronchoscope toward the target site, the system can align a virtual view of the rendered airway to the patient's airway. Image registration can consist of a single registration step or a combination of a single registration step and real-time sensory updates of the registration information. Once registered, all airways can be aligned to the pre-procedure rendered airway. The position of the bronchoscope within the airway can be tracked and displayed as the robotic bronchoscope advances toward the target site. Optionally, the position of the bronchoscope relative to the airway can be tracked using a positioning sensor. Other types of sensors (e.g., cameras) can also be used in place of or in conjunction with the positioning sensor using sensor fusion techniques. A positioning sensor, such as an electromagnetic (EM) sensor, can be embedded in the distal tip of the catheter, and an electromagnetic field generator can be positioned next to the patient's torso during the procedure. The electromagnetic field generator can locate the EM sensor's position in 3D space or the EM sensor's position and orientation in 5D or 6D space. This may provide visual guidance to the operator when maneuvering the bronchoscope towards the target site.

[0040] 73 FIG. 4A shows an example of a user interface for visualizing a virtual airway 409 with an optimal path 403, a catheter 401 tip position, and a lesion position 405 superimposed. In this example, the catheter tip position is displayed in real time relative to the virtual airway model 409, thereby providing visual guidance. As shown in the example of FIG. 4A, the optimal path 403 may be displayed and superimposed on the virtual airway model while the robotic bronchoscope is being driven. As explained above, the virtual airway model may be constructed based on real-time fluoroscopic images / video (and imaging system position data). Optionally, a view of the real-time fluoroscopic images / video 407 may also be displayed on the graphical user interface. Optionally, the user may also be allowed real-time access to camera views or images / video 411 taken by the bronchoscope.

[0041]

[74] In some embodiments, the user interface may further include a user device that allows the user to visualize virtual renderings (e.g., of the airway) and live camera views as the device is navigated to the target during the procedure. Optionally, the virtual renderings may be superimposed on the live camera view and displayed on a display device. Optionally, the system may be integrated with or utilize immersive technology, such as immersive virtual reality (VR) and augmented reality (AR) enabled systems, to enable visualization of the virtual renderings.

[0042]

[75] For example, the user may be allowed to visualize overlays (e.g., pathways, targets, vasculature, other anatomical structures) on these views with or without an augmented reality system, thereby providing user information during the procedure. The system may also allow the user to select / control the display of overlays based on the use case or user preference.

[0043]

[76] Figure 4B shows an example of a navigation view with augmented information. As shown in Figure 4B, the navigation view 420 may include at least a virtual rendering, e.g., a live camera view 421, overlaid with augmented reality information. The virtual rendering or overlaid information may include multiple components, such as a virtual airway 423, a virtual lesion 424, and a panned virtual path to the lesion 425. The multiple virtual components may be visualized with or without the use of a virtual / augmented reality device. The navigation view may also include a directional indicator 424 indicating the navigation direction (e.g., forward, up, down, backward, left, right).

[0044] 77 The provided system may advantageously allow a user to control the display of virtual renderings based on user preferences. For example, a user may enable / disable the display of one or more selected components from a plurality of components. FIG. 4C illustrates an example of a navigation guide view with a user-selected virtual rendering. For example, a user may switch off the virtual rendering of the airway to view a live camera view overlaid with virtual renderings of a selected lesion 424 and pathway 423. FIG. 4D illustrates an example of an endoluminal view. In example 440, a virtual lumen 426 may be displayed along with virtual renderings of a proposed pathway 427 and / or vasculature 429. Similarly, a user may be provided with a directional indicator 428 within the view. In another example 441, a user may enable virtual renderings of the airway 429 and pleura 430 so that these virtual components are overlaid on the lumen view. A user may toggle any selected component on or off at any time.

[0045]

[78] In some embodiments, one or more subsystems of the platform may include one or more therapeutic subsystems, such as manual or robotic instruments (e.g., biopsy needles, biopsy forceps, biopsy brushes) and / or manual or robotic therapeutic instruments (e.g., RF ablation instruments, cryoinstruments, microwave instruments, etc.).

[0046] 79 In some embodiments, one or more subsystems of the platform may include a user console including a therapy interface module (user console side) and / or a therapy control module (patient and robot side). FIG. 5 shows an example of a user console that allows an operator or user to interact with the bronchoscope during a surgical procedure. As shown in example 510, the user console may include a therapy interface module configured to provide a user interface 511 that displays information related to the use of the bronchoscope, such as navigation guidance information, user information (e.g., control parameters), and a robotic bronchoscope camera view. The user interface may be provided on a display. The display may or may not be a touchscreen. The display may be a light-emitting diode (LED) screen, an organic light-emitting diode (OLED) screen, a liquid crystal display (LCD) screen, a plasma screen, or any other type of screen. The display may be configured to display a user interface (UI) or graphical user interface (GUI) rendered through a software application (e.g., via an application programming interface (API) executed on the system).

[0047]

[80] In some embodiments, the user console may include a therapy control interface 511 and a therapy control module 503. The therapy control interface and therapy control module may be separate, self-contained components. Alternatively or additionally, the therapy control interface and therapy control module may be integrated into a single component. For example, the therapy control module may include a user input system 503 that communicates with the therapy interface module. Alternatively, the therapy control module may be a stand-alone system.

[0048]

[81] A user console or a component of a user console (e.g., a therapy interface module) as shown in example 520 may be mounted on a robotic support system 523. Alternatively or additionally, a user console or a component of a user console (e.g., a therapy interface module) may be mounted on a separate mobile cart 513. The mobile cart 513 may include various elements such as a rechargeable power supply in electrical communication with an electrical panel providing charging ports for portable electronic devices, converters, transformers, and surge protectors for multiple AC and DC outlets as power sources for onboard equipment, including one or more computers storing application-specific software for the therapy interface module.

[0049]

[82] In some embodiments, the therapy control module 503 may include, for example, a user interface handheld device that allows a physician to easily control a robotic endoscope (e.g., a bronchoscope). In some embodiments, the user input or control device may be customized or personalized. More details about handheld user interface devices / systems are provided later in this specification. Alternatively or additionally, the therapy control module 503 need not be a handheld device. For example, the therapy control module may be integrated into a robotic support system.

[0050] 6A and 6B show an example of a system including a therapy control system. In some embodiments, the therapy control system may include or be integrated with a robotic support system 605, which includes a robotic arm 607, an instrument drive mechanism 609, a robotic control unit, and one or more peripheral devices, such as an irrigation 601 and suction 603 system. The robotic arm may initiate the positioning of a robotic bronchoscope 611 or other robotic instrument. The instrument drive mechanism may be used to control the elongated member or robotic bronchoscope with more than one degree of freedom (e.g., articulation). The irrigation and suction systems 601, 603 may reside on a robotic arm base cart or any other part of the system. The irrigation and suction systems may be connected to the working channel through connectors or luers. The irrigation system may inject fluids such as saline, and the suction system may aspirate mucus, saline, or other substances from the airway. In some embodiments, the irrigation and suction systems may be used with camera visualization.

[0051]

[84] Figure 7 shows an example of a robotic arm 710 mounted on top of a robotic cart in a treatment control system. The robotic arm 710 can automatically position a catheter assembly at an initial position (e.g., an access point) to access a target tissue. In some embodiments, the robotic arm can be passively moved by an operator. In such cases, the operator can push the arm at any position, and the arm will move compliantly. The robot can also be controlled in a compliant mode to improve human-robot interaction. For example, the robotic technology's compliant motion control can employ a collision avoidance strategy, and position and force control can be designed to reduce unnecessary energy consumption while mitigating the effects of potential collisions. In some embodiments, an instrument drive mechanism can be mounted on the robotic arm. The arm can have redundant degrees of freedom that allow the arm's elbow to be algorithmically or passively moved to a configuration convenient for the operator.

[0052] A low-cost, single-use robotic bronchoscope

[85] One aspect of the present invention provides a single-use robotic bronchoscope. The robotic bronchoscope can be the same as the steerable catheter assembly described elsewhere herein. Conventional endoscopes can be complex in design and are typically designed to be reused after procedures that require extensive cleaning, disinfection, or sterilization after each procedure. Existing endoscopes are often designed with complex structures so that the endoscope can withstand cleaning, disinfection, and sterilization processes. The provided robotic bronchoscope can be a single-use endoscope, which can beneficially reduce cross-contamination and infection between patients. In some cases, the robotic bronchoscope is delivered to a physician in a pre-sterilized package and is intended to be discarded after a single use.

[0053] 86] Figures 8-10 illustrate examples of robotic bronchoscopes according to some embodiments of the present invention. As shown in Figure 8, robotic bronchoscope 820 can include a handle portion 813 and a flexible elongate member 811. In some embodiments, flexible elongate member 811 can include a shaft, a steerable tip, and a steerable section. Robotic bronchoscope 820 can be the same as the steerable catheter assembly described in Figure 2. The robotic bronchoscope can be a single-use robotic endoscope. In some cases, the catheter alone can be disposable. In some cases, at least a portion of the catheter can be disposable. In some cases, the entire robotic bronchoscope can be detached from the instrument drive mechanism and discarded. The bronchoscope can have various levels of stiffness along the bronchoscope shaft to improve functional operation.

[0054]

[87] The robotic bronchoscope can be detachably coupled to an instrument drive mechanism 820. The instrument drive mechanism 820 can be mounted on an arm of a robotic support system or on any actuation support system such as those described elsewhere herein. The instrument drive mechanism can provide a mechanical and electrical interface to the robotic bronchoscope 820. The mechanical interface can allow the robotic bronchoscope 820 to be detachably coupled to the instrument drive mechanism. For example, a handle portion of the robotic bronchoscope can be attached to the instrument drive mechanism via quick attachment / detachment means, such as a magnet and a spring-loaded level. In some cases, the robotic bronchoscope can be manually coupled to or detached from the instrument drive mechanism without the use of tools.

[0055]

[88] Figure 9 shows an example of an instrument drive mechanism 920 that provides a mechanical interface to a handle portion 913 of a robotic bronchoscope. As shown in the example, the instrument drive mechanism 920 may include a set of motors that actuate to rotationally drive a set of pull wires on the catheter. The handle portion 913 of the catheter assembly may be attached to the instrument drive mechanism such that a pulley assembly on the handle portion 913 is driven by the set of motors. The number of pulleys may vary depending on the configuration of the pull wires. In some cases, one, two, three, four, or more pull wires may be utilized to articulate the catheter.

[0056]

[89] The handle portion can be designed to make the robotic bronchoscope low-cost and disposable. For example, conventional manual and robotic bronchoscopes may have cables at the proximal end of the bronchoscope handle. The cables often include illumination fibers, camera video cables, other sensor fibers or cables such as EM sensors, or shape-sensing fibers. Such complex cables can be costly in addition to the cost of the bronchoscope. The provided robotic bronchoscope may have an optimized design to utilize simplified structures and components while maintaining mechanical and electrical functionality. In some cases, the handle portion of the robotic bronchoscope may employ a cable-less design while providing the mechanical / electrical interface to the catheter.

[0057] 90] FIG. 10 illustrates an exemplary handle portion 1000 of a robotic bronchoscope according to some embodiments of the present invention. In some cases, the handle portion 1000 may be a housing or may include components configured to process image data, provide power, or establish communication with other external devices. In some cases, the communication may be wireless communication. For example, the wireless communication may include Wi-Fi, radio-based communication, Bluetooth, IR communication, or other types of direct communication. Such wireless communication capabilities enable robotic bronchoscope functionality in a plug-and-play manner and can be conveniently disposed of after a single use. In some cases, the handle portion may include circuit elements such as a power supply for powering electronics (e.g., a camera and LED light source) located within the robotic bronchoscope or catheter.

[0058]

[91] The handle section can be designed in conjunction with the catheter to eliminate cables or fibers. For example, the catheter section can employ a design with a single working channel that allows instruments to pass through the robotic bronchoscope, as well as low-cost electronics such as a chip-on-tip camera, an illumination source such as a light-emitting diode (LED), and an EM sensor optimally positioned depending on the mechanical structure of the catheter. This can simplify the handle section design. For example, by using LEDs for illumination, termination in the handle section can be based solely on electrical soldering or wire crimping. For example, the handle section can include a proximal board on which the camera cable, LED cable, and EM sensor cable terminate, while the proximal board connects to an interface in the handle section to establish an electrical connection to the instrument drive mechanism. As described above, the instrument drive mechanism is attached to the robot arm (robot support system) and provides the mechanical and electrical interface for the handle section. This can advantageously improve assembly and packaging efficiency and simplify the manufacturing process and costs. In some cases, the handle section, along with the catheter, can be disposed of after a single use.

[0059] Single-use steerable catheter 11 illustrates an exemplary steerable catheter 1100, according to some embodiments of the present invention. In some embodiments, the catheter may have a substantially one-piece design in which one or more components may be integral with the catheter, thereby simplifying the assembly and manufacturing process while preserving the kinematic dynamic performance of the steerable catheter. As shown in the example, the steerable catheter may include an elongate member 1101, or probing portion, that provides proximity to the tissue and / or region under examination. The elongate member 1101 may also sometimes be referred to as a catheter. The catheter 1101 may include internal structures, such as a working channel 1103, that allow for the insertion of a tool, such as those described elsewhere herein. Optionally, the working channel may have dimensions, such as a diameter of approximately 2 mm, to accommodate compatibility with standard tools.

[0060]

[93] The catheter 1101 may be constructed of a material suitable for the desired flexibility or bending stiffness. In some cases, the catheter material may be selected to be substantially flexible (e.g., capable of bending in various directions and orientations) while also maintaining structural support for the internal structure (e.g., the working channel). For example, the catheter may be made of any suitable material, such as urethane, vinyl (e.g., polyvinyl chloride), nylon (e.g., Vestamid, Grilamid), polyurethane, polyethylene, polypropylene, polycarbonate, polyester, silicone elastomer, acetate, etc. In some cases, the material may be a polymeric material, a biocompatible polymeric material, and the catheter may be sufficiently flexible to be advanced through a path with little curvature without causing pain to the subject. In some cases, the catheter may include a sheath. The sheath need not be the same length as the catheter. The sheath may be shorter than the catheter to provide the desired support. Alternatively, the catheter may be a substantially single-piece component.

[0061] 94] In some embodiments, the distal portion or tip of the catheter may be substantially flexible so that it can be steered in one or more directions (e.g., pitch, yaw). In some embodiments, the catheter may have a varying bending stiffness along its longitudinal axis. For example, the catheter may include multiple segments with different bending stiffnesses (e.g., flexible, semi-rigid, and rigid). The bending stiffness may be varied by selecting materials with different stiffness / stiffness, varying the structure in different segments, adding additional support components, or any combination of the above. In some cases, the proximal end of the catheter does not need to bend as much, and therefore the proximal portion of the catheter may be reinforced with additional mechanical structure (e.g., additional layers of material) to achieve greater bending stiffness. Such a design may provide support and stability to the catheter. In some cases, the varying bending stiffness may be achieved by using different materials during the extrusion of the catheter. The use of different materials during extrusion may advantageously result in different degrees of stiffness along the catheter shaft in the extrusion manufacturing process without additional fastening or assembly of different materials.

[0062]

[95] The distal portion of the catheter can be steered by one or more pull wires 1105. The distal portion of the catheter can be made of any suitable material, such as a copolymer, polymer, metal, or alloy, so that it can be bent by the pull wires. In some embodiments, the proximal end or proximal portion of the one or more pull wires 1105 can be operatively coupled to various mechanisms (e.g., gears, pulleys, etc.) within the handle portion of the catheter assembly. The pull wires 1105 can be metal wires, cables, or thin wires, or polymer wires, cables, or thin wires. The pull wires 1105 can also be made of natural or organic materials or fibers. The pull wires 1105 can be any type of suitable wire, cable, or thin wire capable of supporting various types of loads without deformation, significant deformation, or breakage. The distal ends or portions of one or more pull wires 1105 may be securely fixed to or integrated into the distal portion of the catheter, such that manipulation of the pull wires by the control unit may apply a force or tension to at least the distal portion (e.g., flexible section) of the catheter that may steer or articulate the distal portion (e.g., up, down, pitch, yaw, or any direction between these directions).

[0063]

[96] As discussed above, the pull wires may be made of any suitable material, such as stainless steel (e.g., SS316), a metal, an alloy, a polymer, nylon, or a biocompatible material. The pull wires may be wires, cables, or thin wires. In some embodiments, different pull wires may be made of different materials to vary the load-bearing capabilities of the pull wires. In some embodiments, different sections of the pull wire may be made of different materials to vary stiffness and / or load-bearing capacity along the pull. In some embodiments, the pull wires may be utilized to transmit electrical signals.

[0064]

[97] The catheter may have dimensions that allow one or more electronic components to be integrated into the catheter. For example, the outer diameter of the distal tip may be approximately 4 to 4.4 millimeters (mm), and the diameter of the working channel may be approximately 2 mm, so that one or more electronic components may be embedded within the wall or interstices of the catheter. However, it should be noted that, based on different applications, the outer diameter may be any range less than 4 mm or greater than 4.4 mm, and the diameter of the working channel may be any range depending on the size of the tool or the particular application.

[0065]

[98] The one or more electronic components may include an imaging device, an illumination device, or a sensor. In some embodiments, the imaging device may be a video camera 1113. The imaging device may include an optical element and an image sensor for capturing image data. The image sensor may be configured to generate image data in response to wavelengths of light. Various image sensors may be used to capture image data, such as a complementary metal-oxide semiconductor (CMOS) or a charge-coupled device (CCD). The imaging device may be an inexpensive camera. Optionally, the image sensor may be provided on a circuit board. The circuit board may be an imaging printed circuit board (PCB). The PCB may include multiple electronic elements for processing image signals. For example, circuitry for a CCD sensor may include an A / D converter and an amplifier for amplifying and converting the analog signal provided by the CCD sensor. Optionally, the image sensor may be integrated with an amplifier and converter for converting the analog signal to a digital signal, such that a circuit board is not required. Optionally, the output of the image sensor or circuit board may be image data (a digital signal), which may be further processed by the camera circuitry or a processor of the camera. In some cases, the image sensor may include an array of optical sensors.

[0066]

[99] The illumination device may include one or more light sources 1111 positioned at the distal tip. The light sources may be light emitting diodes (LEDs), organic LEDs (OLEDs), quantum dots, or any other suitable light source. In some cases, the light sources may be miniature LEDs or dual-tone flashing LED lights for compact designs.

[0067]

[0100] The imaging device and illumination device may be integrated into the catheter. For example, the distal portion of the catheter may include suitable structure that matches at least the dimensions of the imaging device and illumination device. The imaging device and illumination device may be embedded in the catheter. FIG. 12 shows an exemplary distal portion of a catheter with an integrated imaging device and illumination device. The camera may be located in the distal portion. For example, the camera may be embedded in a cavity 1210 at the distal tip of the catheter. The cavity 1210 may be integrally formed with the distal portion of the cavity and may have dimensions that match the length / width of the camera so that the camera does not move relative to the catheter. The camera may be adjacent to the working channel 1220 of the catheter to provide a near-field view of the tissue or organ. Optionally, the attitude or orientation of the imaging device may be controlled by controlling the rotational motion (e.g., roll) of the catheter.

[0068]

[0101] Power to the camera may be provided by a wired cable. Optionally, the cable wires may be located in a wire bundle that provides power to the camera as well as lighting elements or other circuitry at the distal tip of the catheter. The camera and / or light source may be powered from a power source located in the handle portion via wire, copper wiring, or any other suitable means extending along the length of the catheter. Optionally, real-time images or videos of the tissue or organ may be wirelessly transmitted to an external user interface or display. Wireless communication may be WiFi, Bluetooth, RF communication, or other forms of communication. Optionally, images or videos captured by the camera may be broadcast to multiple devices or systems. Optionally, image and / or video data from the camera may be transmitted along the length of the catheter to a processor located in the handle portion via wire, copper wiring, or any other suitable means. Image or video data may be transmitted to an external device / system via wireless communication components in the handle portion. Optionally, the system may be designed so that wires are not visible or exposed to the operator.

[0069]

[0102] In traditional endoscopy, illumination may be provided by a fiber optic cable that transmits light from a light source located at the proximal end of the endoscope to the distal end of the robotic endoscope. In some embodiments of the present disclosure, to reduce design complexity, a small LED light may be used and embedded in the distal section of the catheter. In some cases, the distal section may include a structure 1230 with dimensions that match the dimensions of the small LED light source. As shown in the illustrated example, two cavities 1230 may be integrally formed with the catheter to accommodate two LED light sources. For example, the outer diameter of the distal tip may be approximately 4 to 4.4 millimeters (mm), and the diameter of the catheter's working channel may be approximately 2 mm, allowing two LED light sources to be embedded in the distal end. The outer diameter may be any range smaller than 4 mm or larger than 4.4 mm, and the diameter of the working channel may be any range depending on the size of the tool or the specific application. Any number of light sources may be included. The internal structure of the distal section may be designed to accommodate any number of light sources.

[0070]

[0103] Optionally, each LED may be connected to a power wire that extends to the proximal handle. In some embodiments, the LEDs may be soldered to separate power wires that are later bundled together to form a single stranded wire. In some embodiments, the LEDs may be soldered to pull wires that provide power. In other embodiments, the LEDs may be crimped or connected directly to a pair of power wires. Optionally, a protective layer, such as a thin layer of biocompatible adhesive, may be applied to the front of the LEDs to provide protection while still allowing light to be emitted. Optionally, an additional cover 1231 may be placed on the front end face of the distal tip to provide accurate positioning of the LEDs and sufficient space for the adhesive. The cover 1231 may be made of a transparent material that matches the refractive index of the adhesive so that the illumination light is not obstructed.

[0071]

[0104] In some embodiments, one or more sensors may be embedded within the distal portion of the catheter. In conventional robotic bronchoscopes, sensors may be used to track tip position, and these sensors are typically located at the distal tip, resulting in a bulky tip. The provided steerable catheter may bundle one or more electronic components to provide a compact design. In some cases, an illumination source and one or more position sensors may be combined into a bundle. FIG. 13 shows an example of a compact configuration of electronic elements located in the distal portion. In some embodiments, a position sensor, such as an electromagnetic (EM) sensor, may be used to accurately track the position of the distal tip of the catheter. In some cases, one or more EM sensors 1310 may be located in the distal portion, positioned adjacent to or behind an illumination source 1320 (e.g., an LED) in a three-dimensional configuration. An electromagnetic coil located at the distal end may be used in conjunction with an electromagnetic tracking system to detect the position and orientation of the distal end of the endoscope during placement within an anatomical system. In some embodiments, the coils may be angled to provide sensitivity to electromagnetic fields along different axes, giving the disclosed guidance system the ability to measure a full six degrees of freedom (three positional and three angular degrees of freedom).

[0072]

[0105] Optionally, the EM sensor and LED light source may form a bundle 1300. The EM sensor's power cable may be bundled with the LED's wires, resulting in reduced space and complexity. Optionally, stereo registration may provide differential 5D or fused 6D measurements that allow for precise positioning and orientation sensing of the catheter distal tip. During the procedure, an electromagnetic field generator positioned next to, below, or above the patient's torso may identify the location of the EM sensor, thereby tracking the position of the catheter tip in real time.

[0073] Pull wire configuration and design

[0106] A robotic bronchoscope may include one or more pull wires for controlling the articulation of the catheter. In conventional endoscopes, the distal ends or portions of one or more pull wires may be rigidly secured or attached to a control ring, such that manipulation of the pull wires by a control unit may apply a force or tension to the control ring that can steer or articulate a particular section or portion (e.g., the distal section) of the catheter (e.g., up, down, pitch, yaw, or any direction between these directions). FIG. 14 shows examples of a conventional configuration of pull wires 1413 attached to a control ring structure 1411 and the novel configuration 1420 of the present disclosure. The control ring may be attached to the distal end of the catheter 1415. Typically, the tips of the pull wires are welded or soldered to the control ring 1411, which may also be attached to the distal tip by welding. The welding process can be costly, tedious, and complicated. Moreover, if one pull wire breaks or malfunctions, the entire steering control function may be affected.

[0074]

[0107] The provided robotic bronchoscope may include individually controlled pull wires, each of which is directly connected to the distal portion. As shown in example 1420, one or more pull wires 1423 may be attached to an integrally formed structure 1421 on the distal portion. For example, the integrally formed structure 1421 may be a groove molded with the distal tip. The groove may have dimensions or a size that match the dimensions of the distal end 1421 of the pull wire to allow for convenient crimping of the pull wire at the distal end. This may advantageously improve assembly efficiency. In some examples, the pull wire may be rigidly secured in the groove on the distal end so that the distal end of the pull wire cannot move relative to the distal portion of the catheter.

[0075]

[0108] The pull wire configuration may also provide improved reliability in steering the distal segment. For example, because each pull wire is individually connected to and individually controlled at the distal segment, articulation forces may be dynamically adjusted in response to different pull wire configurations. For example, articulation forces may be recalculated, and control signals for controlling the pull wires may be dynamically adjusted based on available pull wires in the event of a pull wire break.

[0076]

[0109] Additionally, the easy assembly of pull wires into the distal section may provide flexibility in designing the pull wire configuration. For example, the number or combination of pull wires may be dynamically selected or adjusted to meet different performance or design requirements. FIG. 15 illustrates various configurations of pull wires for a robotic catheter system. In some embodiments, integral structures (grooves) for receiving pull wires may be pre-fabricated. For example, four grooves may be integrally formed with the catheter, and one or more pull wires may be fixedly connected / fixedly crimped to one or more selected from the plurality of grooves to form different configurations 1510, 1530. As shown in the examples, any number of grooves / slots or any given subset of grooves / slots may be selected to receive or couple to pull wires at one end. Optionally, once slot / groove combinations are selected to couple to corresponding pull wires, a pull wire configuration pattern may be formed, and a mapping relationship between the selected grooves / slots and pull wires may be transmitted to a control unit. Control signals may then be generated during articulation based on the mapping relationship to achieve a desired articulation force.

[0077]

[0110] In another example, the pre-machined grooves can have various configurations. For example, a three-pull wire configuration 1520 can have three grooves spaced approximately 120° apart. Optionally, a virtual mapping algorithm can map a three-wire configuration to a four-wire configuration. The virtual mapping algorithm can also be utilized to update a new mapping relationship if one or more pull wires malfunction or break during operation. The virtual mapping algorithm maps a selected configuration pattern to an updated configuration pattern upon a change in the state of one or more pull wires. Such a one-piece design of the pull wire configuration advantageously simplifies the assembly and manufacturing process while preserving the kinematic dynamic performance of the catheter.

[0078] Guidewire with expandable tip

[0111] In some embodiments, a guidewire may be used during a bronchoscopy procedure. The guidewire is typically first inserted well beyond the tip of the bronchoscope to enter the desired airway, and then the bronchoscope may be slid over the guidewire to enter the selected passageway. Because the diameter of the guidewire is small compared to the diameter of the bronchoscope, the guidewire may not have sufficient stiffness and / or sufficient friction to firmly anchor the guidewire within the airway.

[0079]

[0112] Guidewires of the present disclosure may have an expandable outer diameter feature at their tip. FIG. 16 shows an example of a guidewire 1600 with an inflatable tip. The guidewire 1601 may be threaded through the working channel of a catheter / bronchoscope to aid in navigation through the airways within the lungs. Optionally, the guidewire may be extended beyond the tip of the catheter into the desired airway, and the catheter may then be slid over the guidewire to reach the desired location. The inflatable tip may be implemented using a variety of suitable methods. For example, an additional component 1603, such as an expandable balloon, may be positioned at or near the distal end of the guidewire. The balloon may be connected through the working channel to a balloon inflation source or pump for inflation or deflation of the balloon.

[0080]

[0113] Optionally, the guidewire may include perforations. The diameter of the deflated balloon may be equal to the diameter of the long arm (e.g., a bronchoscopy catheter). Optionally, the diameter of the deflated balloon may be slightly larger than the long arm. The guidewire may be capable of moving distally or proximally. The guidewire may be attached to an air pump for injecting and extracting air from the guidewire, thereby inflating and deflating the balloon, respectively. The balloon may remain deflated during insertion of the guidewire into the airway. While reaching the appropriate location, the balloon may be inflated by pumping air into it. Once the bronchoscope reaches the desired forward position, the balloon may be deflated by expelling air, which may allow the guidewire to move forward. In some embodiments, the inflatable tip, along with its corresponding inflation and deflation control mechanism, may be made of a foldable mesh structure using materials such as shape memory alloys (SMAs), electroactive polymers (EAPs), and ferrofluids. The fixation element may have any other shape to ensure firm fixation of the guidewire. For example, the fixation element can be a metal wire that is radially expandable or collapsible. The fixation element can be actuated by a slide actuator that linearly slides to change the position of the fixation element, in particular to deploy the fixation element or return it to a collapsed position. The sliding action of the actuator can be translated into a change in the position (state) of the fixation element (e.g., the fixation element deploys and radially expands to provide a structure that securely secures the guidewire in place, or conversely, the fixation element radially contracts and returns to a collapsed state).

[0081] Anti-buckling device

[0114] In some embodiments, catheters can be designed to be flexible. When the flexible portion of the catheter is inserted into a patient by extending the mechanism using a bronchoscope, one or more sections may bend or buckle. In such cases, an anti-buckling mechanism can be coupled to the handle of a robotic bronchoscope to support the catheter and prevent it from buckling while the bronchoscope is advanced toward the patient. Although anti-buckling mechanisms, such as telescoping mechanisms, are known, the flexible portion of the catheter may still bend or buckle. Existing anti-buckling devices can include multiple tubular elements with open ends. The diameter of the tubular elements can gradually increase. These tubular elements can be connected to each other and can be folded or expandable within each other. The diameter of the smallest tubular member is larger than the diameter of the elongate member, allowing the elongate member to move forward when the tubular member is extended. The diameter difference prevents the catheter from being retrieved when the anti-buckling device is retracted or removed. However, the catheter can still buckle in segments where the diameter of the telescoping mechanism is larger than the outer diameter of the catheter.

[0082]

[0115] The present disclosure provides an improved anti-buckling mechanism. The anti-buckling mechanism is used to prevent buckling of the insertion shaft. FIG. 17 shows an exemplary anti-buckling mechanism 1700, according to some embodiments of the present invention. The anti-buckling mechanism 1700 can be a telescoping extension device with an internal mechanism to achieve anti-buckling of the catheter during insertion and withdrawal. The anti-buckling mechanism 1700 can be removably connected at one end to the handle portion of the robotic bronchoscope and at the other end to a support surface 1701. As shown in the example, the anti-buckling tube can be attached to a bracket on the instrument drive mechanism and can be removed and disposable after the procedure using a quick-detach mechanism. The support arm can be supported by a robotic mobile cart that supports the endotracheal tube mount and provides a support surface against which the distal end of the anti-buckling tube can rest when compressed. The support arm can be controlled to rotate, translate vertically up and down, and / or can be a boom arm that extends and retracts so that it can be precisely positioned over the patient's mouth and attached to the endotracheal tube mount. The positioning of the support arm can be synchronized with the movement of the robotic arm, which can track the position of the catheter entry point.

[0083]

[0116] The anti-buckling mechanism can be designed with internal features to prevent the catheter from buckling. FIG. 18A shows an example of the internal structure of an anti-buckling mechanism, according to some embodiments of the present invention. In some cases, the anti-buckling mechanism can be a separable device that can be disposed of after a single use. The anti-buckling device can include multiple barrels 1801 with gradually decreasing barrel diameters. The barrels can be concentrically assembled or connected along an axial axis. Each barrel can consist of a thin-walled barrel 1801, a proximal end with an inner lip 1802, and a proximal closure 1803 with a threaded hole 1804 slightly larger than the catheter / sheath. The diameter of the threaded hole in all barrels can be the same so that catheter movement can be limited in a cross-sectional plane relative to the anti-buckling device. The barrel element can also include an outer stop lip 1805 (i.e., a radial protrusion slightly larger than the outer diameter of the barrel) and a stopper 1806 at the distal opening of the barrel structure. In some embodiments, the proximal closure portion 1803 and the outer lip 1805 can be a single, integral element. Alternatively, the proximal closure portion 1803 and the outer lip 1805 can be individually assembled to one another. In some embodiments, when the proximal closure portion 1803 and the outer lip 1805 are a single, integral part, the proximal closure portion 1803 and the outer lip 1805 can be in the form of a disk assembled to the proximal end of the barrel element. In some cases, the proximal closure portion 1803 and the outer lip 1805 can be integrally formed with the barrel element. The inner lip and outer stop lip can prevent the barrel from disengaging during extension of the anti-buckling device.

[0084]

[0117] A centrally located hole in the proximal disc can allow the catheter to slide smoothly along the drive axis and provide vertical compression to prevent bending or buckling of the catheter when the telescope is extended. In some embodiments, the telescope can be filled with a pressurized viscous fluid (e.g., silicone oil, buffer solution) to prevent sudden buckling events during high-force insertion. Two mounting add-ons (plates) can be provided at both the distal and proximal ends of the entire anti-buckling device, with the proximal mounting plate fastened to the robotic arm. The distal plate can be attached to a fixture fastened to the patient's bed via the add-on function. In some embodiments, the fixture can be a support post fastened to the bed, in which case no additional force is applied to the patient when the anti-buckling device is folded. In other embodiments, the fixture can be a handrail on the bed.

[0085]

[0118] Figures 18B and 18C show an example of an assembly of the anti-buckling mechanism 1810 and handle 1811. Figure 18B shows the anti-buckling mechanism connected to the handle in a retracted state, while Figure 18C shows the anti-buckling mechanism fully extended.

[0086]

[0119] In some cases, the systems and devices herein may enable a simplified assembly flow for assembling the anti-buckling mechanism and endoscope. For example, the anti-buckling mechanism and scope handle may be assembled by side-connecting the anti-buckling mechanism to the scope handle, and the assembled parts may be mounted onto the instrument drive mechanism from above as a single part. This simplified assembly advantageously allows the scope handle and anti-buckling assembly to be coupled to a robotic arm regardless of the state and current position of the instrument drive mechanism. FIG. 18D shows an example of a scope handle and anti-buckling tube assembly with side connection features. The anti-buckling tube 1827 can be detachably connected to the handle 1825 via side connection features 1821, 1823. A connector on the scope handle 1821 can be laterally attached to a corresponding connector on the anti-buckling mechanism 1823 to connect the two separate parts. In some cases, connectors can be located on both sides of the handle to connect to two connectors on the anti-buckling mechanism. Any suitable mechanism (buttons, pins, clasps, magnets) may be utilized to removably couple the anti-buckle tube to the handle. The assembled anti-buckle mechanism and handle are shown in Figure 18E.

[0087]

[0120] 18E illustrates an example in which the interface of the handle 1833 allows the user to place the anti-buckling tube 1835 and scope connection assembly onto the instrument drive mechanism 1831. Assembling the scope and anti-buckling mechanism before loading the assembly onto the instrument drive mechanism can simplify workflow.

[0088]

[0121] The anti-buckling mechanism may require a relatively linear trajectory for movement. In some cases, such a trajectory can be ensured by aligning the anti-buckling mechanism in its folded state with the patient connector. FIG. 18F shows an example of a patient connector 1841 and an IDM 1841. For example, the patient connector may be fixed to a patient mount (e.g., attached to a patient bed). The alignment workflow may include aligning the folded anti-buckling mechanism 1845 with the patient connector with alignment guidance or feedback. For example, the user may be assisted in aligning the instrument drive mechanism (IDM) with the patient connector, and feedback (e.g., visual / tactile / audible feedback) may be provided that the anti-buckling mechanism and patient connector are properly aligned. FIG. 18G shows another example of an anti-buckling mechanism moved to a target position by aligning the patient connector with the IDM.

[0089]

[0122] In some cases, the alignment process may be performed using the IDM and the anti-buckling patient connector before attaching the anti-buckling mechanism. Alternatively, the alignment process may be performed with the anti-buckling mechanism attached. The alignment features may include using click alignment, laser alignment, magnets, visual indicators, or tactile / audible feedback. FIG. 18H shows examples of alignment features 1851, 1857. In the illustrated example, mechanical alignment features, such as click buttons or magnets, may be provided on the patient connector 1853 and the IDM 1855 to provide feedback indicators regarding alignment. For example, when alignment is complete, the click buttons or magnets may trigger a tactile, audible, or visual signal indicating proper alignment. In some cases, visual indicators, such as colored dots / markers, slots, or ridges, may be provided on the IDM to aid in alignment.

[0090]

[0123] As shown in Figure 18H, a laser 1857 on the IDM can be used to aim at the patient connector to ensure linear alignment to the patient connector. As shown in Figure 181, magnets on the distal end of the anti-buckling feature and on the patient connector can aid in alignment and provide visual / tactile feedback to the user that the components are properly aligned.

[0091] User Interface

[0124] The user interface may include a variety of devices, such as a touchscreen monitor, joystick, keyboard, and other interactive devices, as shown in the example of FIG. 19 . In some embodiments, a user may be able to navigate and / or control the movement of the robotic arm and the movement of the catheter using a user input device. The user input device may have any type of user interaction component, such as a button, mouse, joystick, trackball, touchpad, pen, image acquisition device, motion capture device, microphone, touchscreen, handheld wrist gimbal, extraskeletal glove, or other user interaction system, such as a virtual reality system or an augmented reality system. In some cases, the user input device may be a touch-sensitive stylus device that makes physical contact with a touch-sensitive display screen, and the user may control the robotic system by moving the touch-sensitive stylus device across the display screen.

[0092]

[0125] In some embodiments, the therapy control module may be a handheld controller 1930. The therapy control module may include a dedicated, personalized, or customized user input device. Optionally, one or more add-on elements 1910 may be removably coupled to an existing user device 1920 to improve the user input experience of the therapy control module 1930. For example, one or more physical user input devices or add-on elements 1920 (e.g., trackballs, joysticks, or rollers) may be coupled haptically or via Bluetooth to a graphical user interface (GUI) 1910 provided on the user device. For example, the trackball, joystick, or roller 1920 may replace or complement the functionality of at least one of the virtual graphical elements (e.g., navigation arrow, slider bar 1911) displayed on the graphical user interface (GUI) by providing similar functionality to the graphical element that the trackball, joystick, or roller 1920 replaces. The add-on elements may be coupled to the GUI via physical contact on a touchscreen, via an IO port, or via wired or wireless communication such that user input received through the add-on elements can be mapped to input received by virtual graphical elements rendered on the GUI. Examples of user devices include, but are not limited to, mobile devices, smart phones / cell phones, tablets, personal digital assistants (PDAs), laptop or notebook computers, desktop computers, media content players, etc. User interface devices and user consoles are described in more detail later in this specification.

[0093]

[0126] In another example, the user input device may be a camera (e.g., an imaging sensor located on a display), and the user input may include retinal information, such as where the user is looking. User input (e.g., by pulling a trigger or pressing a button on a laparoscope handheld controller, voice command, etc.) to confirm the new alignment of the virtual component with respect to the target location. The orientation of the virtual component (e.g., the rotational orientation of the shaft) may be adjusted using a touchpad, trackball, or other suitable input to the laparoscope handheld controller or other device. User Device

[0094]

[0127] In some embodiments, a user may be allowed to personalize the user interface based on the user's personal preferences, such as handedness or the speed at which the user operates the user interface device (e.g., the speed at which the user moves a lever on a joystick to drive a robotic elongate member back and forth). Artificial intelligence methods, such as machine learning or deep learning, may be used to personalize the user interface device based on user behavior. As an example, machine learning methods may be used to learn based on user behavior, such as button usage, lever usage, frequency of button or lever usage, number of clicks, or the speed at which the user moves a lever on a joystick, to adapt and specialize. For example, the user interface may be adapted to use a combination of buttons and levers for a particular task based on the user's preference for using those buttons or levers.

[0095]

[0128] In some embodiments, the training data may include historical user interface interaction data or simulated data. Artificial intelligence algorithms may be trained to adapt to user behavior or user interactions with the interface.

[0096]

[0129] In some embodiments, the training data may include user interface interaction history data or simulated user interaction data as well as imaging and / or video data of the procedure, as described elsewhere. The user interface interaction data may be time-stamped and annotated with respect to the real-time imaging data to identify the user's specific interactions while actuating the elongate member. Having a combined training data set may allow the artificial intelligence algorithm to ascertain the user's experience level, in which case the user interface may not only adapt to the user's movements, but may also help train the user, for example, with visual or audible messages, to guide the user in actuating the elongate member.

[0097]

[0130] Various artificial intelligence models can be implemented, such as, but not limited to, neural networks. The artificial intelligence model can be a trained model or a trained machine learning algorithm. The machine learning algorithm can be any type of machine learning network, such as a support vector machine (SVM), a naive Bayes classification, a linear regression model, a quantile regression model, a logistic regression model, a random forest, a neural network, a convolutional neural network (CNN), a recurrent neural network (RNN), a gradient boosted classifier or repressor, or another supervised or unsupervised machine learning algorithm (e.g., a generative adversarial network (GAN), a cycle GAN, etc.). FIG. 20 shows an example of a neural network model for generating control signals in response to individual user inputs. Various types of neural networks can be used. The neural network can support deep learning. The neural network can be a convolutional deep neural network and / or a recurrent neural network using supervised or unsupervised training. In some embodiments, the neural network can support reinforcement learning.

[0098]

[0131] Inputs to the neural network may include user interactions and behaviors with the user interface device, such as the example shown in Figure 20. The inputs may also include time-stamped, real-time image and / or video data showing the user's manipulation and actuation of the elongate member. The neural network may extract features from the input data that indicate the user's preferences in using various aspects of the user interface of the user interface device, such as the user's preference in using buttons versus levers, the user's dominant hand, the speed at which the user moves a lever, etc.

[0099]

[0132] The output layer of the neural network may include one or more output nodes. Each output node may represent a decision based on user behavior regarding interaction with the user interface device and actuation of the elongate member. The output may output the likelihood of different actions the user can take. Based on the position of the elongate member, one or more actions may have a likelihood higher than a predetermined threshold. In some embodiments, based on the real-time image data, the position of the tip of the elongate member, and the likelihoods presented by the output of the neural network, visual and / or audible indications may be displayed on the graphical user interface to guide the user to take an action, such as stopping actuation of the elongate member, changing the actuation angle, accelerating or decelerating actuation, or using a pull wire to bend the tip of the elongate member in a particular direction. The neural network may also personalize the function of elements of the user interface device, such as personalizing the use of specific touch buttons, push buttons, or levers on a joystick or any other user interface device.

[0100]

[0133] The training dataset can be stored in local storage, such as local memory or a local server. The training dataset can also be stored in a remote storage, such as a cloud server. Training can be performed online or offline. The training dataset can be updated in real time to improve the learning and functionality of the neural network.

[0101]

[0134] In some cases, the platform may provide deep learning models with continuous training or refinement after development. The deep learning models provided by the platform may be dynamically adjusted and tailored to suit different individuals and different surgical procedures over time. The predictive models provided by the platform may be continuously refined over time (e.g., during runtime, after development). Such continuous training and refinement may be performed automatically with little or no user input or intervention.

[0102]

[0135] In some embodiments, the provided robotic endoluminal platform may use an edge intelligence paradigm in which data processing and prediction / inference are performed at the edge or edge gateway (e.g., bronchoscope, robotic system, user device), while predictive models are built, developed, and trained on the cloud / data center and executed on the user device or control device (e.g., hardware accelerator) for inference. In some cases, deep learning models may be pre-trained on the cloud and sent to the user device, control system, or edge computing system for execution. In some cases, deep learning models may undergo continuous training as new sensor data and user feedback are collected. The continuous training may be performed on the cloud or server. In some cases, sensor data may be sent to the cloud, where it is used to update the model for continuous training, and the updated model (e.g., updated model parameters) may be downloaded to the local or edge system (e.g., bronchoscopy system, robotic system, user device, bronchoscopy system software application) for execution.

[0103] Portable Handheld Add-on Module

[0136] The robotic bronchoscope is designed to work in conjunction with a robotic platform. The electronics and mechanical movement of the catheter are controlled by the robotic platform. However, at the beginning of a procedure, a physician may need to manually inspect the main airways through the bronchoscope. The present disclosure provides a handheld handle add-on module that allows a physician to manually operate the bronchoscope without using a robotic support system. For example, the handle portion of the robotic bronchoscope may have a unified interface that allows the robotic bronchoscope to be detachably connected to the instrument drive mechanism of the robotic support system, a handheld handle device, a controller or user interface device, a modular wireless communication device, and various other devices.

[0104]

[0137] An example of a portable handle add-on module design is shown below in FIG. 21A. The module may have an electrical interface that connects to a proximal board in the handle. The electrical interface may include a signal connection. Multiple matching mechanical pulleys may engage a capstan in the handle. Two or more main knobs allow for the combined motion of all pulleys that can articulate the distal tip of the robotic bronchoscope. The add-on module may be connected to a user interface through a wired connection, a wireless connection, or a combination of both. For example, a communications module such as a WiFi chip within the module may broadcast video from the bronchoscope to multiple portable displays. The portable handle add-on module may include a power source, such as a battery to provide backup power for the camera, in addition to a cable in the handle. Alternatively or additionally, cable / wired communications may be used.

[0105]

[0138] As described above, robotic bronchoscopes can be designed to interface with external devices in a plug-and-play manner. FIG. 21B illustrates various examples of robotic bronchoscopes used in conjunction with various systems, devices, and modules. In a first scenario 2110, the handle portion 2111 of the robotic bronchoscope can be connected to a handheld handle add-on module 2117 via a mechanical and electrical interface, as described above. For example, the handheld handle add-on module can provide a mechanical interface including a drive element (e.g., a motor) 2113 that actuates a set of pull wires of a catheter to rotate them. Optionally, the handheld handle add-on module 2117 can also provide an electrical interface 2115 that electrically communicates with a proximal board within the handle portion 2111 to transmit sensor data and / or control signals. In some embodiments, the same robotic bronchoscope 2121 can be detachably connected to both the handheld handle add-on module and the robotic support system, as shown in example 2120, and can be switched between the handheld handle add-on module and the robotic support system. The robotic bronchoscope may have a unified interface that allows for convenient switching between the instrument drive mechanism 2123 and the handheld handle add-on module 2117. In some embodiments, the instrument drive mechanism, the handheld handle add-on module, or both may provide only a mechanical interface. As shown in scenario 2130, a modular wireless communication device 2131 (e.g., a WiFi module) may be provided that can be removably coupled to the handle portion 2133 to expand the electrical communication capabilities of the robotic bronchoscope. For example, the modular wireless communication device 2131 (e.g., a WiFi module) may be in electrical communication with the handle portion to transmit sensor data to an external device and / or receive control signals from an external control system.This may advantageously allow the robotic bronchoscope to be used with or integrated into existing robotic systems, user devices, or surgical systems, regardless of the electrical communication capabilities of the underlying systems.

[0106] Portable robotic cone-beam CT

[0139] Conventional cone beam CT machines may have emitter and receiver panels on the same mechanical structure with a C-shape or O-shape. The connection between the emitter and receiver panel can make the cone beam CT large. This oversized design imposes limitations on use cases and occupies a large amount of space in the rather small operating room.

[0107]

[0140] This specification describes a design for decoupling the mechanical connection between the emitter and receiver panel. Figure 22 shows an example portable robotic cone beam CT. The emitter and receiver panels can be separately mounted on two separate robot arms, as shown in the example of Figure 22. In use, the two robots can move in the same coordinate system. A control algorithm can ensure that the two robots move in synchronized motion.

[0108]

[0141] Additionally, additional external sensors, i.e., IMU, EM, or image sensors, can be added to track patient movement for patient gating motion, i.e., breathing. Patient position changes can be tracked using sensors such as IMU, EM, or image sensors. The sensed signals can be used to command two robotic arms. Optionally, one or both of the robotic arms can move to track patient movement, thereby keeping the emitter and receiver substantially stationary with respect to patient movement in a region of interest (ROI) during tracking. The ROI can include a target region or location that can be determined automatically by the system or manually by a physician. Tracking can also be performed using other mechanisms, such as, but not limited to, an external camera and one or more tracking devices on the patient's body.

[0109]

[0142] It should be understood by those skilled in the art that cone-beam CT is a non-limiting example, and the designs described herein may be used with other imaging modalities, such as fluoroscopy machines, conventional CT machines, and MRI machines.

[0110]

[0143] While preferred embodiments of the present invention have been shown and described herein, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Numerous variations, changes, and substitutions will now occur to those skilled in the art without departing from the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in practicing the invention. It is intended that the following claims define the scope of the invention and that methods and structures within the scope of these claims and their equivalents be covered thereby.

Claims

1. A robotic endoscopic device, a disposable elongate member including a proximal end and a distal end, the proximal end being removably attached to a robotic arm via a handle, and the distal end being integrated with an imaging device, a position sensor, and an illumination device; a bending section that is articulated by one or more pull wires; A robotic endoscopic device comprising:

2. The robotic endoscopic device of claim 1 , wherein the distal end includes structure for receiving the imaging device, the position sensor, and the illumination device.

3. The robotic endoscopic device of claim 1 , wherein the imaging device, the position sensor, and the illumination device are arranged in a compact configuration.

4. 10. The robotic endoscopic device of claim 1, wherein the handle includes one or more components configured to process image data, provide power to the imaging device, the position sensor, and the illumination device, or establish communication with an external device.

5. The robotic endoscopic device of claim 1 , wherein the handle includes an interface configured to couple the handle to an instrument drive mechanism attached to the robotic arm.

6. The robotic endoscopic device of claim 5 , wherein the interface includes an electrical interface and a mechanical interface.

7. The robotic endoscopic device of claim 6 , wherein the mechanical interface is configured to removably couple the handle to the instrument drive mechanism.

8. The robotic endoscopic device of claim 5 , further comprising an anti-buckling mechanism with an alignment feature.

9. The robotic endoscopic device of claim 8 , wherein the alignment feature is configured to assist in aligning the instrument drive mechanism and the anti-buckling mechanism.

10. The robotic endoscopic device of claim 8 , wherein the alignment feature includes a magnetic component, a laser, or a click button.

11. The robotic endoscopic device of claim 8 , wherein the anti-buckling mechanism includes a series of connected cylinders, each including a lip structure.

12. The robotic endoscopic device of claim 11 , wherein the lip structure on each barrel has a retention portion with the same diameter.

13. A robotic endoscopic system comprising the robotic endoscopic device of claim 1 and a user interface device configured for a user to control movement of the robotic endoscopic device.

14. The robotic endoscopic system of claim 13 , wherein the user interface device is personalized based on past user behavior.

15. The robotic endoscope system of claim 14 , wherein the user interface device is personalized using a model trained with a machine learning algorithm.

16. 14. The robotic endoscopic system of claim 13, further comprising a display configured to display image data captured by the imaging device overlaid with a virtual rendering of one or more components.

17. The robotic endoscopic system of claim 16 , wherein the display of the virtual rendering of the one or more components is selectively enabled or disabled by a user.

18. The robotic endoscopic device of claim 1 , wherein both the handle and the disposable elongate member are single-use.

19. The robotic endoscopic device of claim 1 , wherein the one or more pull wires are individually attached to the bending sections according to a selected configuration pattern.

20. The robotic endoscopic device of claim 1 , wherein control of the articulation of the robotic endoscopic device is based at least in part on a virtual mapping algorithm.

21. The robotic endoscopic device of claim 20 , wherein the virtual mapping algorithm maps a selected configuration pattern to an updated configuration pattern upon a change in the state of the one or more pull wires.

Images