Sampling equipment recommendation based on target location

Abstract

Systems, devices, and methods for determining a sampling device for use in an ultrasound-guided endoscopic tissue acquisition procedure are disclosed. An exemplary system comprises an ultrasound imaging device to produce ultrasound images of an anatomical target in a real-time ultrasound field of view (FOV), a display to display the ultrasound images in the ultrasound FOV, and a processor. Based on the location of the anatomical target in the ultrasound FOV, the processor predicts, from multiple nominal sampling device trajectories, a nominal sampling device trajectory that intersects the anatomical target. The multiple nominal trajectories correspond to different hardware configurations of the multiple nominal sampling devices. The processor determines a sampling device recommendation having a hardware configuration in accordance with the predicted nominal sampling device trajectory.

Description

SAMPLING EQUIPMENT RECOMMENDATION BASED ON TARGETLOCATIONPRIORITY CLAIM

[0001] This application claims the benefit of priority to U.S. Provisional Patent Application Serial No. 63 / 581,080, filed September 7, 2023 and U.S. Provisional Patent Application Serial No. 63 / 607,643, filed December 8, 2023, the contents of which are incorporated herein by reference.FIELD OF THE DISCLOSURE

[0002] The present document relates generally to ultrasound equipment, and more particularly but without limitation, to ultrasound-guided tissue acquisition equipment including a graphical user interface to assist with tissue sampling device selection and tissue sampling procedure planning.BACKGROUND

[0003] Endoscopes have been used in a variety of clinical procedures, including, for example, illuminating, imaging, detecting and diagnosing one or more disease states, providing fluid delivery (e.g., saline or other preparations via a fluid channel) toward an anatomical region, providing a passage (e.g., via a working channel) for passing a tissue sampling or biopsy device to collect tissue samples from an anatomical target or for passing a diagnostic or therapeutic device for medical diagnosis or treatment, or providing a suction passage for collecting fluids and unwanted object (e.g., tissue or calculi structures) from an anatomical region, among other procedures. The anatomical region or targets to be intervened include gastrointestinal tract, respiratory tract and lungs, renal system organs and tissues, sinus cavities, submucosal regions, reproductive system organs, among others. Some endoscopes can be used with an energy source, such as a laser or plasma system, to provide treatment energy (e.g., laser pulses) to the anatomical target such as soft or hard tissue or calculi structures to achieve various treatment goals. For example, endoscopic lasers have been used in applications including tissue ablation, coagulation, vaporization, fragmentation, and lithotripsy to break down calculi in kidney, gallbladder,ureter, among other stone-forming regions, or to ablate large calculi into smaller fragments.

[0004] Endoscopic ultrasound (EUS) is a specialized endoscopy that combines conventional endoscopy with ultrasound imaging to obtain ultrasound images of an anatomical target or a region of interest. Such specialized endoscope, also known as an echoendoscope, includes an ultrasound transducer typically located at the distal portion of the elongate endoscope body. The ultrasound transducer emits ultrasound waves toward an anatomical target, and converts the ultrasound echoes into ultrasound images (e.g., a live image stream).

[0005] EUS has been used in diagnosing lung disease. Endobronchial ultrasound (EBUS) is a minimally invasive and highly effective procedure to diagnose lung cancer, infections, and other diseases causing enlarged lymph nodes in the chest. EBUS is typically performed with the use of a specialized bronchoscope associated with an ultrasound transducer or removable ultrasound probe delivered through a working channel and out of a port located at a distal end of the bronchoscope. EBUS has been used in a tissue sampling or biopsy procedure, known as an endobronchial ultrasound-guided transbronchial fine needle aspiration (EBUS-TBNA) procedure, where a specialized sampling device, such as a biopsy needle, may be passed down and then extended from the bronchoscope, and under the real-time guidance of EBUS, to collect tissue samples from peribronchial masses such as peribronchial nodules or tissue from peribronchial lymph nodes. The sampled tissue may be analyzed to assist in diagnosis of various diseases such as tuberculosis, sarcoid, or cancer.SUMMARY

[0006] The present inventors have recognized several technological problems to be solved with conventional EUS systems and technologies, particularly those related to EBUS-TBNA procedures. One of such technological problems is that the existing endobronchial sampling devices are generally designed for procedures involving shallow and large-diameter airways of the respiratory system, within which a clinician may freely advance, retract, or angulate a transbronchial aspiration or sampling device (e.g., a needle) to align a nominal trajectory. The sampling device may be manipulated to traverse alongthe nominal trajectory and finally to intersect an anatomical target of interest, such as a peribronchial nodule. Existing endobronchial sampling devices, however, are generally not designed for (and thus may not be ideal for) collecting tissue samples from targets located deep in the peripheral area of the lungs (e.g., primary, secondary, or tertiary bronchi, or bronchioles), which are only accessible via very distal and narrow airways. As the inner diameter (ID) of airways continually decrease toward terminal ends, the distance that a sampling device can extend into the airway is generally limited by the ID of the airway and the outer diameter (OD) of the sampling device. If a target nodule becomes clearly imaged at a location below some nominal trajectory line, the clinician may find it difficult to further advance the sampling device into the airway to align the target nodule with the nominal trajectory (e.g., to shove or force a 1.9 mm OD sampling device into a portion of the airway that has less than a 1.9 mm ID), because further advancing the sampling device may cause trauma to the airway wall or other unintended anatomical structures.

[0007] In EBUS-TBNA procedures involving targets close to shallow and large-diameter airways, a saline-filled balloon that encompasses an ultrasound transducer is typically used to achieve apposition, which refers to an appropriate level of direct contact or air-gapless contact and pressure needed for ultrasound energy to propagate from and back to the ultrasound transducer. However, a presence of airgaps between the balloon and the contact tissue may result in a high impedance mismatch at typical frequencies of the ultrasound transducer. This may cause deterioration in image quality or even a total loss of image beyond the airgap.

[0008] In EBUS-TBNA procedures involving targets located deep in the peripheral area of the lungs, generally no balloon is used; instead, the ultrasound transducer is in direct contact with the airway wall, known as “direct” apposition. Endobronchial sampling devices used in such direct apposition scenarios face some technological challenges, including aligning the target (e.g., a peribronchial nodule) with the predetermined nominal trajectory while maintaining suitable apposition between the ultrasound transducer and the airway wall so as to produce a quality ultrasound image. For example, in typical endobronchial sampling devices, the physical relationship between the side exit ramp and the ultrasound transducer is fixed. This leads to standard flexibleneedle-stylet combinations repeatedly traversing across the ultrasound field of view (FOV) displayed to a clinician along a single nominal trajectory. However, depending on patient anatomy, a clinician may have limited ability to align the target nodule precisely along this single nominal trajectory. For example, in some cases, the target nodule may appear very clearly and crisply (i.e., have good image quality) at a location above the nominal trajectory, but may disappear if the clinician retracts the sampling device (which would result in the target nodule becoming closer to and ultimately aligned with the needle trajectory) due to the airway size in the retracted position being larger than the OD of the sampling device, thereby causing apposition with the ultrasound transducer to become inadequate for imaging the target nodule.

[0009] For at least the above reasons, the present inventors, have recognized an unmet need for improved technologies that can effectively maintain a clear ultrasound image of an anatomical target during a EBUS-TBNA procedure, while aligning the target with a properly identified nominal device trajectory upon extending an EBUS into the lung periphery (e.g., primary, secondary, or tertiary bronchi, or bronchioles). The present disclosure describes systems, devices, and methods for automatically determining or recommending a sampling device for use in a medical procedure such as an EBUS-TBNA procedure based on the location of an anatomical target within a real-time ultrasound field of view (FOV) of a region of interest. An exemplary system comprises an ultrasound imaging device to generate ultrasound images of an anatomical target in a real-time ultrasound FOV, a display to display, in the realtime ultrasound FOV, the ultrasound images and graphical representations of multiple nominal sampling device trajectories that intersect anatomical targets at respective different locations, and a processor. The multiple nominal trajectories correspond to different pre-shaped hardware configurations of sampling devices and / or assistive insertion devices (e.g., stylets, guidewires, or probes) with different stiffness. The stiffness and the pre-shaped configurations of such devices dictate multiple different nominal trajectories that the devices are likely to follow through within the ultrasound FOV. Based at least in part on the target location in the ultrasound FOV, the processor can predict, from the multiple nominal sampling device trajectories, a nominal sampling device trajectory that intersects the anatomical target within the real-time ultrasound FOV, anddetermine a recommended sampling device for sampling tissue from the anatomical target, the recommended sampling device having a hardware configuration in accordance with the predicted nominal sampling device trajectory.

[0010] Example l is a system for determining a sampling device for use in a medical procedure in a patient. The system includes: an ultrasound imaging device configured to generate ultrasound images of an anatomical target in a real-time ultrasound field of view (FOV) of a region of interest; a display configured to display, in the real-time ultrasound FOV, the ultrasound images; and a processor configured to: receive an indication of a location of the anatomical target within the real-time ultrasound FOV; based at least in part on the received indication of the location of the anatomical target, predict, from multiple nominal sampling device trajectories a nominal sampling device trajectory that intersects the anatomical target within the real-time ultrasound FOV, the multiple nominal sampling device trajectories corresponding to respective hardware configurations of multiple sampling devices; and determine a recommended sampling device for sampling tissue from the anatomical target, the recommended sampling device having a hardware configuration in accordance with the predicted nominal sampling device trajectory.

[0011] In Example 2, the subject matter of Example 1 optionally includes the recommended sampling device that can include a recommended endobronchial needle for sampling tissue from a lung periphery target during an endobronchial ultrasound-guided transbronchial fine needle aspiration (EBUS- TBNA) procedure.

[0012] In Example 3, the subject matter of any one or more of Examples 1-2 optionally includes the recommended sampling device that can include a needle-stylet combination comprising a needle and a stylet insertable into the needle, wherein one or more of the stylet or the needle has a pre-formed curvature or bending angle substantially conforming to the predicted nominal sampling device trajectory.

[0013] In Example 4, the subject matter of any one or more of Examples 1-3 optionally includes the display that can be further configured to display, within the real-time ultrasound FOV, one or more of: graphical representations of the multiple nominal sampling device trajectories; the indication of thelocation of the anatomical target; an indication of the predicted nominal sampling device trajectory; or information about the recommended sampling device.

[0014] In Example 5, the subject matter of any one or more of Examples 1-4 optionally includes the multiple nominal sampling device trajectories that can each be color-coded to match respective color codes for the multiple sampling devices, wherein the predicted nominal sampling device trajectory is color-coded to match a color code for the recommended sampling device.

[0015] In Example 6, the subject matter of any one or more of Examples 1-5 optionally includes the respective hardware configurations of the multiple sampling devices that can include respective different pre-formed curvatures or bending angles, wherein the recommended sampling device is selected from the multiple sampling devices, and has a pre-formed curvature or bending angle substantially conforming to the predicted nominal sampling device trajectory.

[0016] In Example 7, the subject matter of any one or more of Examples 1-6 optionally includes the processor that can be configured to: receive, via a user interface, a user input identifying the anatomical target within the real-time ultrasound FOV; and analyze the ultrasound images within the real-time ultrasound FOV to determine the location of the anatomical target as identified by the user.

[0017] In Example 8, the subject matter of any one or more of Examples 1-7 optionally includes the processor that can be further configured to generate a recommendation to reposition one or more of the sampling device or the ultrasound imaging device to align the anatomical target with the predicted nominal trajectory of the sampling device.

[0018] In Example 9, the subject matter of any one or more of Examples 6-8 optionally includes the recommendation that can include advancing, retracing, or rotating the ultrasound imaging device by a specified amount to align the anatomical target with the predicted nominal sampling device trajectory.

[0019] In Example 10, the subject matter of any one or more of Examples 1-9 optionally includes the ultrasound imaging device that can be located at a distal end of an endobronchial sampling device configured to beinserted into a patient’s airway during an endobronchial ultrasound-guided transbronchial fine needle aspiration (EBUS-TBNA) procedure.

[0020] Example 11 is an intraluminal imaging device system that includes: an intraluminal imaging device, comprising a tubular device body having an inner lumen and a side exit port, and an ultrasound transducer configured to produce ultrasound scans of an anatomical target and generate ultrasound images in a real-time ultrasound field of view (FOV) of a region of interest; a display configured to display, in the real-time ultrasound FOV, the ultrasound images and graphical representations of multiple nominal sampling device trajectories that traverse the FOV ; and a tissue sampling device configured to operatively, under ultrasound guidance: pass through the inner lumen and exit from the side exit port of the tubular device body; and advance toward the anatomical target in accordance with a predicted nominal trajectory selected from the multiple nominal sampling device trajectories and subsequently intersect the anatomical target, wherein the multiple nominal sampling device trajectories correspond to respective hardware configurations of multiple sampling devices, wherein the tissue sampling device has a hardware configuration corresponding to the predicted nominal trajectory.

[0021] In Example 12, the subject matter of Example 11 optionally includes the tissue sampling device that can include an endobronchial needle for sampling tissue from a lung periphery target during an endobronchial ultrasound-guided transbronchial fine needle aspiration (EBUS-TBNA) procedure.

[0022] In Example 13, the subject matter of any one or more of Examples 11-12 optionally includes the tissue sampling device that can include a needle-stylet combination comprising a needle and a stylet insertable into the needle, wherein one or more of the stylet or the needle has a pre-formed curvature or bending angle substantially conforming to the predicted nominal sampling device trajectory.

[0023] In Example 14, the subject matter of any one or more of Examples 11-13 optionally includes the multiple nominal sampling device trajectories that can each be color-coded to match respective color codes for the multiple sampling devices, wherein the predicted nominal trajectory is color- coded to match a color code for the tissue sampling device.

[0024] In Example 15, the subject matter of any one or more of Examples 11-14 optionally includes the respective hardware configurations of the multiple sampling devices that can include respective different pre-formed curvatures or bending angles, wherein the tissue sampling device has a preformed curvature or bending angle substantially conforming to the predicted nominal sampling device trajectory.

[0025] In Example 16, the subject matter of any one or more of Examples 11-15 optionally include, wherein the intraluminal imaging device includes a bronchoscope configured to be inserted into a patient’s airway during an endobronchial ultrasound-guided transbronchial fine needle aspiration (EBUS-TBNA) procedure, the ultrasound transducer being located at a distal end of the bronchoscope.

[0026] Example 17 is a method for determining a sampling device for use in a medical procedure. The method includes steps of: displaying, via a graphical user interface, ultrasound images of an anatomical target in a real-time ultrasound field of view (FOV) of a region of interest; receiving, via the graphical user interface, a selection of the anatomical target within the ultrasound image; determining a location of the anatomical target within the real-time ultrasound FOV; based at least in part on the determined location of the anatomical target, predicting from multiple nominal sampling device trajectories a nominal sampling device trajectory that intersects the anatomical target within the real-time ultrasound FOV, the multiple nominal sampling device trajectories corresponding to respective hardware configurations of multiple sampling devices; and determining a recommended sampling device for sampling tissue from the anatomical target, the recommended sampling device having a hardware configuration in accordance with the predicted nominal sampling device trajectory.

[0027] In Example 18, the subject matter of Example 17 optionally includes displaying, via the graphical user interface and within the real-time ultrasound FOV, one or more of: graphical representations of the multiple nominal sampling device trajectories; the location of the anatomical target; an indication of the predicted nominal sampling device trajectory; or information about the recommended sampling device.

[0028] In Example 19, the subject matter of any one or more of Examples 17-18 optionally includes the multiple nominal sampling device trajectories that can each be color-coded to match respective color codes for the multiple sampling devices, wherein the predicted nominal sampling device trajectory is color-coded to match a color code for the recommended sampling device.

[0029] In Example 20, the subject matter of any one or more of Examples 17-19 optionally includes the respective hardware configurations of the multiple sampling devices that can include respective different pre-formed curvatures or bending angles, wherein the recommended sampling device is selected from the multiple sampling devices, and has a pre-formed curvature or bending angle substantially conforming to the predicted nominal sampling device trajectory.

[0030] In Example 21, the subject matter of any one or more of Examples 17-20 optionally includes determining the recommended sampling device includes determining a needle-stylet combination comprising a needle and a stylet insertable into the needle, wherein one or more of the stylet or the needle has a pre-formed curvature or bending angle substantially conforming to the predicted nominal sampling device trajectory.

[0031] In Example 22, the subject matter of any one or more of Examples 17-21 optionally includes generating recommendation to reposition one or more of the sampling device or the ultrasound imaging device to align the anatomical target with the predicted nominal trajectory of the sampling device.

[0032] In Example 23, the subject matter of Example 22 optionally includes the recommended repositioning that can include advancing, retracing, or rotating the ultrasound imaging device by a specified amount to align the anatomical target with the predicted nominal sampling device trajectory.

[0033] The presented techniques are described in terms of heal th -related procedures, but are not so limited. This summary is an overview of some of the teachings of the present application and not intended to be an exclusive or exhaustive treatment of the present subject matter. Further details about the present subject matter are found in the detailed description and appended claims. Other aspects of the disclosure will be apparent to persons skilled in the art upon reading and understanding the following detailed description and viewing thedrawings that form a part thereof, each of which are not to be taken in a limiting sense. The scope of the present disclosure is defined by the appended claims and their legal equivalents.BRIEF DESCRIPTION OF THE DRAWING

[0034] FIG. 1 is a schematic diagram illustrating an example of an echoendoscopy system for use in an endoscopic ultrasound (EUS) procedure.

[0035] FIG. 2 illustrates a perspective view of a distal end portion of an echoendoscope such as that illustrated in FIG. 1.

[0036] FIG. 3 illustrates an example of an endobronchial ultrasound- guided transbronchial fine needle aspiration (EBUS-TBNA) procedure, and portion of the EBUS-TBNA system being used in the procedure.

[0037] FIG. 4 is a block diagram illustrating an example of an EUS- guided tissue acquisition (EUS-TA) planning system that can automatically generate an EUS-TA plan for use in a medical procedure, such as an EBUS- TBNA procedure.

[0038] FIG. 5 is a diagram illustrating by way of example an endobronchial ultrasound (EBUS) system with an associated computing system that determines and displays on a user interface graphical representations of multiple nominal sampling device trajectories.

[0039] FIGS. 6A-6B are diagrams illustrating at least a portion of a graphical user interface to present to a user a recommended tissue sampling device for use in a tissue acquisition procedure based on a user selection of a target location from a real-time ultrasound FOV.

[0040] FIGS. 7A-7D illustrate examples of tissue sampling devices to be used in endoscopic biopsy procedure, such as a EBUS-TBNA procedure.

[0041] FIG. 8 is a flow chart illustrating an example method for generating an EUS-TA plan, including a recommended tissue sampling device, for use in a medical procedure, such as an EBUS-TBNA procedure.

[0042] FIG. 9 is a block diagram illustrating an example machine upon which any one or more of the techniques (e.g., methodologies) discussed herein may perform.DETAILED DESCRIPTION

[0043] This document describes systems, devices, and methods for automatically planning an ultrasound-guided endoscopic tissue acquisition procedure, including determining or recommending a sampling device for use in the procedure. An exemplary system comprises an ultrasound imaging device to generate ultrasound images of an anatomical target in a real-time ultrasound FOV, a display to display, in the real-time ultrasound FOV, the ultrasound images and graphical representations of multiple nominal sampling device trajectories that intersect anatomical targets at respective different locations, and a processor. The multiple nominal trajectories correspond to different pre-shaped hardware configurations of sampling devices and / or assistive insertion devices (e.g., stylets, guidewires, or probes) with different stiffness. The stiffness and the pre-shaped configurations of such devices dictate multiple different nominal trajectories that the devices are likely to follow through within the ultrasound FOV. Based at least in part on the target location in the ultrasound FOV, the processor can predict, from the multiple nominal sampling device trajectories, a nominal sampling device trajectory that intersects the anatomical target within the real-time ultrasound FOV, and determine a recommended sampling device for sampling tissue from the anatomical target, the recommended sampling device having a hardware configuration in accordance with the predicted nominal sampling device trajectory.

[0044] FIG. 1 is a schematic diagram illustrating an example of an echoendoscopy system 100 for use in an endoscopic ultrasound (EUS) procedure for diagnostic or treatment purposes, such as EUS-guided tissue acquisition. In an example, the echoendoscopy system 100 may be configured for collecting tissue samples from targets located deep in the peripheral regions of the lungs (e.g., primary, secondary, or tertiary bronchi, or bronchioles) during an EBUS- TBNA procedure. The echoendoscopy system 100 comprises an echoendoscope 120, a light source apparatus 130, a video processor 140, a first monitor 150 for displaying an optical image, an ultrasound observation apparatus 160, and a second monitor 170 for displaying an ultrasound image.

[0045] The echoendoscope 120 includes an insertion portion 111, an operation portion 112 from which the insertion portion 111 extends, and a universal cord 113 that from the operation portion 112. The insertion portion 111extends in a longitudinal direction and configured to be inserted into a living body. The universal cord 113 can be connected to the light source apparatus 130 via a scope connector 113A provided at a proximal end portion. A coiled scope cable 114 and an ultrasound signal cable 115 extend from the scope connector 113A. An electric connector portion 114A is provided at one end of the scope cable 114. The electric connector portion 114A can be connected to the video processor 140. An ultrasound connector portion 115A is provided at one end of the ultrasound signal cable 115. The ultrasound connector portion 115A can be connected to the ultrasound observation apparatus 160.

[0046] The insertion portion 111 of the echoendoscope 120 can be configured to consecutively connect a distal end portion 121, a bending portion 122, and a flexible tube portion 123 in that order from the distal end. Channel opening portions, an optical observation window, an optical illuminating window and an ultrasound transducer, or the like are arranged on one side of the distal end portion 121, as to be described further with reference to FIG. 2.

[0047] The operation portion 112 may include a bend preventing portion 124 from which the insertion portion 111 extends, a channel opening setting portion 125, an operation portion body 126 making up a grip portion, a bending operation portion 127 including two bending operation knobs 127A and 127B provided so as to superimpose on one another on one upper side of this operation portion body 126, a plurality of switches 128 that instruct the execution of various endoscope functions and a raising lever 129 for operating a raising stand. Examples of the switches 128 include an air / water feeding button, a suction button and a freeze button.

[0048] The channel opening setting portion 125 is provided on one side in the lower part of the operation portion body 126 and provided with one or more ports each configured to receive respective treatment instruments. By way of example and not limitation, two instrument ports 125A and 125B are disposed at the channel opening setting portion 125. Such instrument ports can communicate with respective two channel opening portions provided at the distal end portion 121 of the insertion portion 111 via two respective treatment instrument channels (not shown) inside the insertion portion 111. In an example, the instrument port 125A can receive a tissue acquisition tool, such as a fine needle for use in EUS-guided tissue acquisition, such as EUS-guided fine-needleaspiration (FNA) or fine-needle biopsy (FNB). In an example, the instrument port 125B can receive a cannula for use in endoscopic retrograde cholangiopancreatography (ERCP). A puncture needle handle portion Nh shown by a single-dot dashed line is fitted into the instrument port 125A.

[0049] The two instrument ports 125 A and 125B can be arranged at the channel opening setting portion 125 such that when the operator brings the right hand RH close to the channel opening setting portion 125, the instrument port closer to the right hand RH becomes the instrument port 125B and the instrument port farther from the right hand RH becomes the instrument port 125 A. More specifically, as shown by a dotted line in FIG. 1, the operator manipulates the treatment instrument inserted into each instrument port by the right hand RH while holding the operation portion body 126 by the left hand LH. The manipulation using the treatment instrument such as ERCP cannula has a higher degree of difficulty than manipulation using a treatment instrument of an EUS-FNA puncture apparatus. Thus, the instrument port 125B for a treatment instrument such as cannula requiring fine manipulation when the operator holds the operation portion body 126 by the left hand LH is arranged at the channel opening setting portion 125 so as to be located on the right side compared to the instrument port 125 A when seen from the operator.

[0050] The bending knob 127A is a vertical direction bending knob and the bending knob 127B is a horizontal direction bending knob. A bending fixing lever 127A1 for fixing the vertical direction bending state is provided on the proximal end side of the bending knob 127 A and a bending fixing lever 127B1 for fixing the horizontal direction bending state is provided on the distal end side of the bending knob 127B.

[0051] An image pickup section for acquiring an optical image inside a subject, and an illumination section and an ultrasound transducer section (see FIG. 2) for acquiring an ultrasound tomographic image inside the subject are provided at the distal end portion 121 of the echoendoscope 120. This allows the operator to insert the echoendoscope 120 into the subject and causes the monitors 150 and 170 to display an optical image and an ultrasound tomographic image inside the subject at a desired position in the subject respectively.

[0052] FIG. 2 illustrates a perspective view of the distal end portion1. The distal end portion 121 may include a metallic distal end rigid member 131 and a cylindrical synthetic resin cover member 132 in which the distal end rigid member 131 is inserted, such that the cover member 132 can partially cover the distal end rigid member 131. An ultrasound transducer section 133 is accommodated within the distal end portion 121. The ultrasound transducer section 133 may include an ultrasound transducer configured to emit ultrasound waves sideward at a predetermined angle with respect to an insertion axis of the insertion portion 111. The ultrasound transducer may have a linear array, a curvilinear, or a phased array configuration. The cylindrical synthetic resin cover member 132 provides insulation of the distal end portion 121, and allows the ultrasound transducer section 133 to be reliably fixed there within.

[0053] When the cover member 132 is attached to the distal end rigid member 131, part of the opening portion of the cylindrical cover member 132 is covered with part of the distal end rigid member 131 on which an illuminating window 141 and an optical observation window 142 are arranged. Optical light emitted from a light source, such as located at the distal end portion 121 of the echoendoscope 120 and coupled to the light source apparatus 130, can pass through the illuminating window 141 and incident on the anatomical target and surrounding environment. The optical observation window 142 allow an imaging device (e.g., a camera lens, not shown) at the distal end portion 121 of the echoendoscope 120 to view target tissue. Other part of the opening portion not covered with part of the distal end rigid member 131 forms an opening portion 144 from which a raising stand 151 protrudes when the raising stand 151 is raised.

[0054] One or more diagnostic or treatment instruments can be activated to protrude from the opening portion 144. As illustrated in FIG. 2, an instrument 240 may protrude from the opening portion 144 when the raising stand 151 is in a maximum raised position. The instrument 240 may be inserted into one of the instrument ports on the operation portion 112 of the echoendoscope 120, such as the instrument port 125 A, passed through a channel within the echoendoscope 120, and extended from the opening portion 144 at the distal end portion 121 (see FIG. 1). Depending on the devices used with the instrument 240, it may be used to achieve different diagnostic or therapeutic goals. By way of example and not limitation, the instrument 240 is a puncture device from which a needle242 protrudes. The needle 242 may be used to sample tissues from an anatomical target, such as a peripheral lung target in an EBUS-TBNA procedure. In addition or alternative to puncture devices such as the needle 242, other types of tissue sampling devices may be used and extended from the instrument 240, such as a brush, a snare, forceps, a suction device, among others. Other devices that may be used with the instrument 240 may include, for example, an object retrieval device for retrieving biological matters (e.g., soft or hard tissue samples, cancerous tissue, or calculi structures), a resection device for surgically removing tissue, a diagnostic device for performing in vivo analysis of the sampled tissue and make diagnosis, or a therapeutic device for delivering therapeutic agents (e.g., cancer treatment drugs) or energy of different modalities (e.g., ultrasound, radiofrequency, laser, or thermal energy) to the anatomical target. Examples of the EUS-guided therapeutic devices may include an ablation device, a drainage device such as a needle to tube to drain pancreatic cysts or pseudocysts, or a stricture management device to open or dilate a narrowed or obstructed portion of a duct in the pancreaticobiliary system, among others.

[0055] In some examples, the instrument 240 may house an assistive insertion device, such as a stylet, a guidewire, a catheter, a probe, or a cannula capable of being endoscopically inserted and passed through the lung peripheral during an EBUS-TBNA procedure. In some examples, the instrument 240 may house a needle-stylet combination comprising a needle and a stylet insertable into the needle to facilitate maneuvering (e.g., advancement and retraction) and positioning of the needle during the procedure. Tissue sampling devices, stylets or other assistive insertion devices, or combined needle-stylet devices used with the instrument 240 may have different stiffness. In some examples, such devices may be formed into pre-shaped configurations, such as pre-formed curvatures or bending angles. The stiffness and the pre-shaped configurations of such devices dictate nominal needle trajectories that respective devices are likely to follow through within the ultrasound FOV, as will be discussed further below such as with respect to FIGS. 4, 5 and 6A-6B.

[0056] In some examples, the echoendoscope 120 can be robotically controlled, such as by a robot arm attached thereto. The robot arm can automatically, or semi-automatically (e.g., with certain degree of user manual control or commands), via an actuator, position and navigate instrument such asthe echoendoscope 120 in an anatomical target, or position a device at a desired location with desired posture to facilitate an operation on the anatomical target (e.g., to collect tissue samples from the anatomical targe using a brush, a snare, forceps, or a suction device). In accordance with various examples discussed in this document, a controller can use artificial intelligence (Al) or machine learning (ML) technologies to determine navigation parameters and / or tool operational parameters (e.g., position, angle, posture, force, and navigation path), and generate a control signal to the actuator of the robot arm to facilitate operation of such instrument or tools in accordance with the determined navigation and operational parameters in a robot-assisted procedure.

[0057] FIG. 3 illustrate an example of endobronchial ultrasound-guided transbronchial fine needle aspiration (EBUS-TBNA) procedure, and portion of the EBUS-TBNA system being used in the procedure. EBUS-TBNA uses an intraluminal image device such as a specialized bronchoscope 320 equipped with ultrasound capabilities to image beyond the walls of the airways to detect in real time the precise locations of anatomical targets of interest, such as a lymph node. During the procedure, a clinician inserts the bronchoscope 320 into a lung periphery region 301, such as primary, secondary, or tertiary bronchi, or bronchioles. The bronchoscope 320, similar to the echoendoscope 120 as shown in FIG. 2, includes an ultrasound transducer 333 which may be located at the distal end of the bronchoscope. The ultrasound transducer 333 may have a linear array, a curvilinear, or a phased array configuration. The ultrasound transducer 333 can produce a live stream of ultrasound images of a target 302 (e.g., a peribronchial nodule) simultaneously in a real-time ultrasound field of view (FOV) 350 of the region of interest. An endobronchial sampling device 342 (e.g., a biopsy needle such as the needle 242, or a needle-stylet combination) can be inserted from a proximal port of the bronchoscope 320, passed through the entire length of the bronchoscope 320, and extended from a side exit port 344 (also referred to as a side exit ramp, an example of the opening port 144 of FIG. 2) located at the distal end of the bronchoscope 320, slightly proximal of the ultrasound transducer 333. The endobronchial sampling device 342 can then be operatively protruded from the side exit port 344 and enter the ultrasound FOV 350. Under the real-time ultrasound guidance, the endobronchial samplingdevice 342 may traverse toward and ultimately intersect the target 302 (e.g., a peribronchial nodule), and collect tissue samples therefrom.

[0058] FIG. 4 is a block diagram illustrating an example of an EUS- guided tissue acquisition (EUS-TA) planning system 400 that can automatically generate an EUS-TA plan to navigate a sampling device during a medical procedure, such as an EBUS-TBNA procedure. The EUS-TA plan may include a recommended tissue sampling device and recommended operation of the recommended device, such as navigating the recommended device along a nominal trajectory intersecting an anatomical target in an ultrasound FOV, and sample from there an adequate amount of biopsy tissue. In some examples, the EUS-TA plan may also include recommended operation and navigation of a steerable elongate instrument (such as the echoendoscope 120 or the bronchoscope 320) for passing and supporting the recommended tissue sampling device, as well as recommended positioning and navigation of the EUS probe to take ultrasound scans of the anatomical target.

[0059] The system 400 may include a processor 410, a device controller 420, an input interface 430, and a user interface 440. In some examples, the system 400 may include or be communicatively coupled to a tissue sampling device or assistive device database 406. The processor 410 may include circuit sets comprising one or more other circuits or sub-circuits that may, alone or in combination, perform the functions, methods, or techniques described herein. In an example, the processor 410 and the circuits sets therein may be implemented as a part of a microprocessor circuit, which may be a dedicated processor such as a digital signal processor, application specific integrated circuit (ASIC), microprocessor, or other type of processor for processing information including physical activity information. Alternatively, the microprocessor circuit may be a general -purpose processor that may receive and execute a set of instructions of performing the functions, methods, or techniques described herein. In an example, hardware of the circuit set may be immutably designed to carry out a specific operation (e.g., hardwired). In an example, the hardware of the circuit set may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including a computer readable medium physically modified (e.g., magnetically, electrically, moveable placement of invariant massed particles, etc.) to encode instructions of the specific operation.In connecting the physical components, the underlying electrical properties of a hardware constituent are changed, for example, from an insulator to a conductor or vice versa. The instructions enable embedded hardware (e.g., the execution units or a loading mechanism) to create members of the circuit set in hardware via the variable connections to carry out portions of the specific operation when in operation. Accordingly, the computer readable medium is communicatively coupled to the other components of the circuit set member when the device is operating. In an example, any of the physical components may be used in more than one member of more than one circuit set. For example, under operation, execution units may be used in a first circuit of a first circuit set at one point in time and reused by a second circuit in the first circuit set, or by a third circuit in a second circuit set at a different time.

[0060] The processor 410 may generate an EUS-TA plan using image data received from the input interface 430. In some embodiments, the input interface 430 may be a direct data link between the system 400 and one or more medical devices that generate at least some of the input features. For example, the input interface 430 may transmit EUS images 431, optionally along with endoscopic images 432, external image sources 433, or other information such as collected by physiological sensors during a medical procedure directly to the system 400. In some embodiments, the input interface 430 may be a part of the user interface 440 that facilitates interaction between a user and the system 400. For example, the user may manually provide input data to the system 400 via the user interface 440. In some examples, the input interface 430 may provide the system 400 with access to an electronic patient record from which one or more data features may be extracted. In any of these cases, the input interface 430 can collect one or more of sources of patient information before and during the procedure.

[0061] The EUS images 431 may include perioperative EUS images (e.g., a live stream of real-time EUS images) of the anatomical target and surrounding environment during an EUS-guided procedure. The EUS images may be produced by an ultrasound transducer, such as the ultrasound transducer associated with the echoendoscope 120, or the ultrasound transducer associated with the bronchoscope 320. Depending on the location of the target relative to the ultrasound transducer, a real-time ultrasound field of view (FOV) of thetarget may be continuously generated and presented to the user via a display 443 of the user interface 440. The endoscopic images 432 may include perioperative endoscope images or videos of the anatomical target and its surrounding environment captured by a camera associated with the echoendoscope. The external image sources 433 may include preoperative or perioperative images of the anatomical target acquired by external imaging devices other than the echoendoscope, which may include, for example, X-ray or fluoroscopy images, electrical potential map or an electrical impedance map, CT images, or MRI images, among others.

[0062] In addition to the images, the input interface 430 may receive other information including, for example, endo-therapeutic device information including, for example: size, dimension, shape, and structures of tissue sampling devices (e.g., needles, forceps, brushes, snares, knives, suction devices) or assistive insertion devices (e.g., a stylet, a guidewire, a probe, a catheter, or a cannula) to assist insertion and maneuvering of the tissue sampling device. The tissue sampling devices and / or assistive insertion devices may have pre-shaped hardware configurations, such as pre-formed curvatures or bending angles. The device information of the tissue sampling device and / or the assistive insertion devices (e.g., size, shape, and pre-shaped configuration) may be used to assist in selecting a proper tissue sampling device, and determining device operation parameters to navigate the selected sampling device to the anatomical target accurately and efficiently. The device information may be received directly from the input interface 430. Alternatively or additionally, the device information may be stored in a memory accessible to the processor 410. In an example, the input interface 430 may receive information from sensors coupled to the echoendoscope or a treatment device passing through the endoscope, or otherwise associated with the patient. In an example, a proximity sensor positioned at a distal end portion of the echoendoscope can sense information including position, direction, or proximity of a distal portion of the echoendoscope relative to an anatomical target.

[0063] The processor 410 may include an image processor 411, a target identification and localization circuit 412, a trajectory prediction circuit 413, and an EUS-TA plan generator 414. The image processor 411 may process the images received from the input interface 430, and extract image featurescharacterizing an anatomical target of interest (e.g., peribronchial nodule). The target identification and localization circuit 412 may use the extracted image features to detect a presence and location of the anatomical target within the real-time ultrasound FOV. The location of the anatomical target may be represented by a longitudinal location and a depth relative to the surface of ultrasound transducer in a coordinate system in the ultrasound of FOV. In an example, a user (e.g., a clinician) may provide, via the input unit 445 of the user interface 440, an input identifying the anatomical target, such as by highlighting or clicking on the anatomical target as appeared on the display 443, or by drawing a bounding box around the anatomical target on the display 443. The target identification and localization circuit 412 may analyze the image to determine the location of the user-selected anatomical target. In some examples, the target identification and localization circuit 412 may determine other characteristics of the anatomical target, including, for example, a size, a shape, or a structure of the anatomical target, and / or recognize a pathophysiological property, such as a lesion, an inflammation state, a stricture level, or a malignancy state (e.g., degree or area of invasion by cancer) of the anatomical target using on-site, real-time in vivo tissue diagnosis.

[0064] The trajectory prediction circuit 413 may predict a nominal sampling device trajectory (S*) when the sampling device (e.g., a biopsy needle or a needle-stylet combination) is extended out of the endoscope (such as via the side exit port 344 of the bronchoscope 320 as shown in FIG. 3) but prior to the sampling device entering the ultrasound FOV, or even before the sampling device is extended out of the endoscope. The nominal trajectory S* may be initially predicted based at least in part on the detected target location in the realtime ultrasound FOV, such that the predicted nominal trajectory S* begins at an entry point into the real-time ultrasound FOV, and intersects the anatomical target at the detected target location which is visible in the real-time ultrasound FOV. In some examples, other information, such as characteristics of the anatomical target as determined by the target identification and localization circuit 412, may also be used to predict the nominal trajectory S*. In an example, the trajectory prediction circuit 413 may predict the nominal trajectory S* further based on a posture and a puncture angle of the sampling device (e.g., a needle)upon piercing through the airway wall, or a posture and an entering angle upon entering the real-time ultrasound FOV.

[0065] In some examples, the nominal trajectory S* may be predicted additionally or alternatively based on hardware configurations (e.g., pre-formed curvatures or bending angles) of available tissue sampling devices. The predicted nominal trajectory S* thus determined corresponds to a tissue sampling device X* with a specific pre-shaped configuration, such as a pre-formed curvature or bending angle. In an example, the predicted nominal sampling device trajectory S* may be selected or otherwise determined from a plurality of nominal sampling device trajectories {Si, S2, ..., SN} for intersecting with anatomical targets at respective locations within the real-time ultrasound FOV. The plurality of nominal sampling device trajectories {Si, S2, ..., SN} each correspond to respective different hardware configurations of a plurality of nominal tissue sampling devices {Xi, X2, ..., XN}, such as pre-formed curvatures or bending angles. Graphical representations of the plurality of nominal sampling device trajectories may be overlaid upon the real-time ultrasound FOV and displayed on the display 443. Similarly, a graphical representation of the predicted nominal sampling device trajectory S* may be overlaid upon the real-time ultrasound FOV and displayed distinguishably over other trajectories on the display 443, such as in different colors. The user may confirm, reject, or modify the predicted nominal trajectory S* to ensure a high probability of intersecting the target. Examples of nominal sampling device trajectories that intersect anatomical targets at respective locations within the real-time ultrasound FOV are described below with respect to FIG. 5.

[0066] The procedure plan generator 414 may generate an EUS-TA plan based at least in part on the predicted nominal sampling device trajectory S*. As illustrated in FIG. 4, the EUS-TA plan generator 414 may include one or more of a sampling device selection unit 415, a sampling device manipulation unit 416, and an endoscope / EUS probe navigation and manipulation unit 417. The sampling device selection unit 415 may determine a tissue sampling device recommendation for use in collecting tissue samples from the anatomical target, such as a peribronchial nodule in an EBUS-TBNA procedure. The tissue sampling device recommendation may include device type, such as one of aspiration or biopsy needles, brushes, forceps, knives, snares, or suction devices.The tissue sampling device recommendation may include device size, such as 19-, 21-, 22-, or 25 -gauge (G) EBUS-TBNA needles with added flexibility and enhanced puncture capabilities for smoother tissue sampling.

[0067] In some examples, based at least on the predicted nominal sampling device trajectory S*, the tissue sampling device recommendation (X*) may be selected from a plurality of nominal sampling devices {Xi, X2, . . ., XN} of a particular type having different hardware configurations, such as tissue sampling needles having different pre-formed curvatures or bending angles. These hardware configurations correspond to different nominal sampling device trajectories {Si, S2, ..., SN}. The recommended sampling device X* may be determined as one having a unique hardware configuration corresponding to the predicted nominal trajectory S*. For example, the recommended sampling device X* has a pre-formed curvature or bending angle substantially conforming to the shape of the predicted nominal trajectory S* (e.g., within a specific margin of difference as determined from the FOV). Other data from the input interface 430 may also be used to assist in determining the tissue sampling device recommendation, including, for example: size and geometry of candidate devices (as a part of the endo-therapeutic device information); sensor data indicating spatial restrictions of an environment of the anatomical target; or size, shape, structure, or malignancy state of the anatomical targe such as determined by the target identification and localization circuit 412, among others. Examples of predicting the nominal sampling device trajectory S* and determining a recommended tissue sampling device X* corresponding to the predicted trajectory S* are discussed below with respect to FIGS. 6A-6B.

[0068] The sampling device manipulation unit 416 may determine recommended values of one or more operational parameters for navigating and manipulating the recommended tissue sampling device X* to safely and effectively collect a sufficient amount of tissue samples. Examples of the device operational parameters may include, for example, a position, a posture, a heading direction, or a rotation angle of the tissue sampling device relative to the anatomical target. Recommended values of the tool operational parameters may vary depending on the type of the tool used for tissue acquisition. For example, for endoscopic forceps, the tool operational parameters may include forceps location; orientation, angle, or orbit of the forceps toward the biopsy site;advance length or distance from the biopsy site; jaw opening states such as having one jaw open or both jaw open, among others. Propper operational parameters can assist the operating physician in precisely collecting a desired amount of tissue while minimizing complication and preserving functionality of the anatomy at the biopsy site.

[0069] The recommended values of the device operational parameters may be determined based at least on the predicted nominal sampling device trajectory S*. Characteristics of the anatomical target, such as size, shape, structure, or malignancy state of the anatomical target as identified by the target identification and localization circuit 412, may also be used to determine the recommended values of the device operational parameters. The recommended values of the device operational parameters may be displayed on the user interface during the procedure as a guidance to assist the user in adjusting the position and heading of the recommended tissue sampling device X* to better align with the predicted nominal trajectory S*.

[0070] The endoscope / EUS probe navigation and manipulation unit 417 may estimate navigation parameters of an echoendoscope 120 or the bronchoscope 320 over which the recommended sampling device X* is deployed. Examples of the endoscope / EUS probe navigation and manipulation parameters may include a position, a posture, a heading direction, or a rotation angle of the endoscope relative to the anatomical target; a speed or a force applied to the endoscope distal portion or a surgical element; a navigation path for navigating the echoendoscope to the anatomical target while avoiding injury or damage to internal organs or tissue. The endoscope / EUS probe navigation and manipulation unit 417 may additionally or alternatively estimate one or more operational parameters for positioning and navigating the EUS probe to take ultrasound scans of the anatomical target. The EUS probe operational parameters may include, for example, a position, posture, a heading direction, or a rotating angle of the EUS probe relative to the anatomical target. In some examples, the EUS probe operational parameters may be estimated through a test process involving advancing, retracing, or rotating the EUS probe to multiple different positions. For example, when the EUS probe is in a position such that (i) the target identification and localization circuit 412 does not detect the anatomical target or to properly localize the anatomical target within the present ultrasoundFOV, or (ii) the trajectory prediction circuit 413 does not properly predict a nominal trajectory S* that would intersect the anatomical target at its current location with the ultrasound FOV, or (iii) the sampling device selection unit 415 cannot recognize from the plurality of nominal tissue sampling devices {Xi, X2, . . XN} a proper sampling device X* having a hardware configuration substantially conforming to the predicted nominal trajectory S*, then a recommendation may be made and presented to the user to reposition the EUS probe, such as by advancing or retracing the EUS probe by a specified amount, or by rotating the EUS probe by a specific angle, so as to better align the anatomical target with the predicted nominal trajectory S*.

[0071] In various examples, one or more of the sampling device selection unit 415, the sampling device manipulation unit 416, or the endoscope / EUS probe navigation and manipulation unit 417 may use artificial intelligence (Al) or machine-learning (ML) technology to perform respective tasks as stated above. One or more ML models may provide the system 400 with the ability to perform tasks, without explicitly being programmed, by making inferences based on patterns found in the analysis of data. The ML model(s) explores the study and construction of algorithms (e.g., ML algorithms) that may learn from existing data and make predictions about new data. Such algorithms operate by building the ML model(s) from training data in order to make data- driven predictions or decisions expressed as outputs or assessments.

[0072] The ML model(s) may be trained using supervised learning or unsupervised learning. Supervised learning uses prior knowledge (e.g., examples that correlate inputs to outputs or outcomes) to learn the relationships between the inputs and the outputs. The goal of supervised learning is to learn a function that, given some training data, best approximates the relationship between the training inputs and outputs so that the ML model can implement the same relationships when given inputs to generate the corresponding outputs. Unsupervised learning is the training of an ML algorithm using information that is neither classified nor labeled, and allowing the algorithm to act on that information without guidance. Unsupervised learning is useful in exploratory analysis because it can automatically identify structure in data.

[0073] Common tasks for supervised learning are classification problems and regression problems. Classification problems, also referred to ascategorization problems, aim at classifying items into one of several category values. Regression algorithms aim at quantifying some items (for example, by providing a score to the value of some input). Some examples of commonly used supervised-ML algorithms are Logistic Regression (LR), Naive-Bayes, Random Forest (RF), neural networks (NN), deep neural networks (DNN), matrix factorization, and Support Vector Machines (SVM). Examples of DNN include a convolutional neural network (CNN), a recurrent neural network (RNN), a deep belief network ( DB ), or a hybrid neural network comprising two or more neural network models of different types or different model configurations.

[0074] Some common tasks for unsupervised learning include clustering, representation learning, and density estimation. Some examples of commonly used unsupervised learning algorithms are K-means clustering, principal component analysis, and autoencoders.

[0075] Another type of ML is federated learning (also known as collaborative learning) that trains an algorithm across multiple decentralized devices holding local data, without exchanging the data. This approach stands in contrast to traditional centralized machine-learning techniques where all the local datasets are uploaded to one server, as well as to more classical decentralized approaches which often assume that local data samples are identically distributed. Federated learning enables multiple actors to build a common, robust machine learning model without sharing data, thus allowing to address critical issues such as data privacy, data security, data access rights and access to heterogeneous data.

[0076] By way of example and not limitation, a trained ML model can have a neural network structure comprising an input layer, one or more hidden layers, and an output layer. Information fed to the input layer may include images data or image features derived therefrom such as produced by the image processor 411, location and / or characteristics of the anatomical target produced by the target identification and localization circuit 412, predicted sampling device trajectory S* identified by the trajectory prediction circuit 413, among others. The input information may be propagated through one or more hidden layers to the output layer that outputs one or more of a sampling device recommendation, recommended values of sampling device navigation andmanipulation parameters, or recommended positioning and navigation of the EUS probe to take ultrasound scans of the anatomical target, among others.

[0077] In some examples, a plurality of ML models can be separately trained, validated, and used (in an inference phase) in different applications. For example, a first ML model (or a first set of ML models) may be trained to determine a sampling device recommendation, a second ML model (or a second set of ML models) may be trained to determine recommended values of sampling device navigation and manipulation parameters, and a third ML model (or a third set of ML models) may be trained to provide recommended positioning and navigation of the EUS probe to take ultrasound scans of the anatomical target.

[0078] The device controller 420 can generate a control signal to one or more actuators 450, such as a motor actuating a robot arm. The one or more actuators 450 may be coupled to one or more devices, such as the recommended sampling device X* (or an assistive insertion device to facilitate insertion and maneuvering of the recommended sampling device X*), the EUS probe, or the echoendoscope 120 or the bronchoscope 320. In response to the control signal, the one or more actuators 450 can robotically adjust position, posture, direction, and navigation path of the devices coupled thereto in accordance with the sampling device operational parameters estimated by the sampling device manipulation unit 416, and / or the navigation parameters estimated by the endoscope / EUS probe navigation and manipulation unit 417.

[0079] The user interface 440 may include an output unit 442 and an input unit 445. The input unit 445 can receive input from the user or from other data sources. In an example, the input interface 430 can be included in the input unit 445. The output unit 442 may include a display 443 to display the ultrasound FOV with images of the anatomical target and tissue sampling tool recommendation. Graphical representations of the plurality of nominal sampling device trajectories {Si, S2, ..., SN} and the predicted nominal sampling device trajectory S* may be displayed on the display 443 within the real-time ultrasound FOV. Information about the recommended sampling device X*, including the type and size of the device and recommended methods of operating the device such as values of one or more operational parameters generated by the sampling device manipulation unit 416, may also be displayed on the display 443. Duringthe procedure, the display 443 may display real-time ultrasound images of the recommended sampling device being advanced toward or retracted from the target anatomy within the ultrasound FOV. Display settings can be adjusted by the user via the input unit 445. The visual indication may take the format of markers, annotations (icons, texts, or graphs), highlights, or animation, among other visual indicators. For example, markers of different shapes, colors, forms, or sizes can be display on the reconstructed or integrated image to distinguish different tissue, anatomical regions, their accessibility or criticality.

[0080] The output unit 442 may include an alert and feedback generator 444 that can generate an alert, a notification, or other formats of human- perceptible feedback to the operating physician on the status or progress of the cannulation or navigation in reference to the navigation plan. For example, an alert can be generated to indicate a risk of tissue damage. The feedback can be in one or more forms of audio feedback, visual feedback, or haptic feedback. For example, when the endoscope tip enters or comes closer to a “critical zone” (e.g., proximity sensor detecting a distance to a critical anatomy of interest shorter than a threshold distance), the critical zone can be shown in different colors to represent such distance (e.g., green zone, yellow zone, and read zone as the endoscope gets closer to the critical zone). Additionally or alternatively, haptic feedback such as touch or vibration may be generated and felt by the operating physician. In an example, the alert and feedback generator 444 can automatically adjust the vibration strength according to the distance to the critical zone. For example, a low vibration can be generated when the endoscope tip is in a green zone. If the system predicts, based on present advancing speed and direction of the endoscope, that the endoscope tip will reach the critical zone in a time lower than a predetermined threshold, then alert and feedback generator 444 can apply moderate vibration when the endoscope tip reaches a yellow zone, and apply high vibration when the endoscope tip reaches red zones to indicate a risk of tissue damage. The real-time alert and feedback in an image- guided procedure as described herein can improve the efficiency of cannulation and endoscope navigation, especially for inexperienced physicians, and can improve procedure success rate and patient outcome.

[0081] FIG. 5 is a diagram illustrating by way of example a schematic view of an EBUS sampling device 520 located in an airway enclosed by theairway walls 501, and a predicted sampling device trajectory 510 (illustrated as predicted sampling trajectory 510A, 51 OB, and 510C) displayed over a real-time ultrasound FOV 550 of an anatomical region during an EUS-TA procedure, such as an area within a patient’s lung periphery during an EBUS-TBNA procedure. The anatomical region contains one or more target nodules (e.g., a solitary pulmonary nodule) for which a biopsy sample is desired. The real-time ultrasound FOV 550 may be produced by an ultrasound transducer 533 located at a distal end of the EBUS sampling device 520. The FOV 550 may be displayed to a clinician on a monitor throughout the tissue sampling procedure.

[0082] The EBUS sampling device 520 has a side exit port 544 (also known as a needle ramp) proximal to the ultrasound transducer 533 along the length of a tubular device body of the EBUS sampling device 520. During an EUS-TA procedure such as an EBUS-TBNA procedure, a tissue sampling device (not shown here), such as the endobronchial sampling device 342 as shown in FIG. 3, may be extended from the side exit port 544, pierce through the patient’s airway wall 501, and traverse across the ultrasound FOV 550 into an anatomical target of interest (e.g., a peribronchial nodule). Depending on the location of the anatomical target (e.g., longitudinal position and depth in a coordinate system within the ultrasound of FOV 550) relative to the side exit port 544, different nominal sampling device trajectories may be created. FIG. 5 illustrates by way of example three anatomical targets 540A, 540B, and 540C with distinct locations which can be represented by respective depth DI, D2, and D3 in the coordinate system within the ultrasound of FOV 550. Accordingly, three nominal sampling device trajectories 510A, 510B, and 510C may be created between the side exit port 544 and the respective locations of the anatomical targets 540A, 540B, and 540C.

[0083] The nominal sampling device trajectories 510A, 510B, and 510C each may correspond to hardware configurations (e.g., a pre-formed curvatures or bending angles) of distinct nominal tissue sampling devices XA, XB, and Xc. In an example, the tissue sampling devices XA, XB, and Xc are needles or needle-stylet combinations (with stylet being insertable into a needle lumen) that are pre-formed with distinct curvatures or bending angles. The specific curvature profile (or radius of curvature) and the stiffness of a particular needle or needlestylet combination dictate a corresponding nominal trajectory that such needle orneedle-stylet combination is likely to follow and ultimately intersects the anatomical target within the real-time ultrasound FOV 550. In the illustrated example, the nominal trajectory 510C may be achieved by using a straight (i.e., non-bent) needle or needle-stylet combination, the nominal trajectory 51 OB may be achieved by using a first needle or needle-stylet combination with a first degree of pre-formed curvature, and the nominal trajectory 510C may be achieved by using a second needle or needle-stylet combination with a second degree of pre-formed curvature greater that the first degree of pre-formed curvature of the first needle or needle-stylet combination. Accordingly, depending at least on the target location in the ultrasound FOV 550, an appropriate nominal sampling device trajectory may be predicted, and a corresponding tissue sampling device having a specific hardware configuration conforming to the nominal sampling device trajectory may be determined.

[0084] As illustrated in FIG. 5, multiple nominal sampling device trajectories such as 510A, 51 OB, and 510C may be displayed over the real-time ultrasound FOV 550 to assist the clinician to align an anatomical target with any one of the displayed sampling device trajectories. Then, when it is time to obtain a sample, the clinician may use a specific combination of sampling needle and stylet with a specific hardware configuration (e.g., pre-shaped curvature) that uniquely corresponds to the nominal trajectory which the anatomical target is aligned with.

[0085] In some examples, color coding may be used to indicate which sampling devices (e.g., needle-stylet combinations) correspond to which nominal trajectories in the real-time ultrasound FOV 550. The multiple nominal sampling device trajectories (e.g., 510A, 510B, and 510C) may be displayed in respective color-codes, and correspond to color coded sampling devices. For example, on a computer monitor displaying the real time ultrasound image stream, the nominal trajectory 510A may be displayed in blue, the nominal trajectory 510B may be displayed in purple, and the nominal trajectory 510C may be displayed in red. Consistently therewith, a red handled stylet may be usable with a spiral laser cut needle to cause the needle to follow along the nominal trajectory line shown in red (e.g., nominal trajectory 510C). A purple handled stylet may be used to intersect a target aligned with the purple trajectory line (e.g., the nominaltrajectory 51 OB), and a blue handled stylet may be used to intersect a target aligned with the blue trajectory line (e.g., the nominal trajectory 510A).

[0086] FIGS. 6A-6B are diagrams illustrating at least a portion of a graphical user interface (GUI) 600 that interactively presents to the user a recommended tissue sampling device for use in a EUS-TA procedure, such as a EBUS-TBNA procedure, tissue acquisition procedure based on a user selection of a target location from a real-time ultrasound FOV 650. The recommended tissue sampling device may be determined by the EUS-TA plan generator 414 of the system 400. The GUI 600 can be displayed continuously or on-demand such as activated by the user during the procedure. The GUI 600 may display patient information 620, the ultrasound FOV 650, and nominal sampling device trajectories 610A, 610B, and 610C each defined between a location of the side exit port on the bronchoscope and the respective potential target locations within the ultrasound FOV 650, as similarly shown in FIG. 5. The system may enable the user to indicate where individual targets (e.g., peribronchial nodules) are located within the ultrasound FOV 650. For example, as shown in FIG. 6A, the user may select an anatomical target 640B that appears with the FOV 650, such as by clicking on the image of said target, or by drawing a bounding box around said target. Then, once the target identification and localization circuit 412 of the system 400 has determine where within the FOV 650 the target nodule is located, a dialogue box 630 may be displayed to prompt the user to confirm generating a recommendation of a sampling device based on the selected target location, such as a tissue sampling needle or needle-stylet combination that will most likely cause the needle to intersect the selected anatomical target 640B.

[0087] FIG. 6B illustrates the sampling device recommendation 660 such as generated by the EUS-TA plan generator 414 based on the user selection of the anatomical target 640B. By way of example and not limitation, the sampling device recommendation 660 may include information such as the type, size, or other characteristics of the needle and / or the stylet recommended for subsequent use in the procedure to collect tissue samples from the user selected anatomical target 640B. If the sampling device is color-coded and correspond to the color coding of the nominal trajectories, then information about the color of the recommended sampling device may be included in the sampling device recommendation 660. Location information such as depth 670 of the targetrelative to the surface of ultrasound transducer (e.g., 18 mm as an example shown here), optionally along with other characteristics of the user selected anatomical target 640B such as detected by the target identification and localization circuit 412, may be displayed on the GUI 600.

[0088] In some examples, in addition to the sampling device recommendation 660, recommended values of sampling device navigation and manipulation parameters such as produced by the sampling device manipulation unit 416, and / or recommended positioning and navigation of the EUS probe to take ultrasound scans of the anatomical target such as produced by the endoscope / EUS probe navigation and manipulation unit 417 may be displayed on the GUI 600. For example, the system may generate and display to the user a recommendation to reposition the sampling device (such as by advancing or retracting the sampling device) to best align the anatomical target with a nominal trajectory line, and / or to reposition the echoendoscope or the EUS probe, including linear translation and / or rotation, to better capture the anatomical target of interest within the ultrasound FOV 650.

[0089] FIGS. 7A-7D illustrate, by way of example and not limitation, tissue sampling devices that may be used in an endoscopic biopsy procedure, such as a EBUS-TBNA procedure. FIG. 7A illustrates a fine needle for aspiration (FNA) or for biopsy (FNB) procedures. The fine needle may vary in flexibility and size, such as 19 gauge (G), 22 G and 25 G, with 25G needle being smaller in diameter and highly flexible. Such fine needles may be used with an endoscopic ultrasound system (EUS) for ultrasonically guided FNA and FNB of submucosal and extramural lesions within the GI tract (e.g., pancreatic masses, mediastinal masses, perirectal masses and lymph nodes). The EUS guided FNA allows for the safe and efficacious sampling of solid lesions. EUS guided FNB has the potential of accruing larger tissue samples during biopsies, which may enable more precise diagnosis and rapid cytodiagnosis, a technique for on-the- spot pathological diagnosis of tissues collected during surgery.

[0090] FIG. 7B illustrates endoscopic biopsy forceps that can enter the gastrointestinal tract via a flexible endoscope to perform biopsy at the anatomical target. The forceps may come with different size to fit into a particular endoscope channel. The forceps may include a cup to facilitate capturing and collecting the biopsied tissue. The cup can be an oval cup, oralligator cup with serrated jaws to prevent slippage. A blade inside the cup can deliver sharp cutting performance to ensure clean-edged specimens. Some forceps include a swinging jaw mechanism to aid tangential biopsies. Some forceps include a needle for improved anchorage.

[0091] FIG. 7C illustrates an endoscopic biopsy brush including bristles to brush off and collect biopsy tissue. The brush can be delivered via a flexible endoscope to perform biopsy at the anatomical target. Like the biopsy forceps, the biopsy brushes may come with different size to fit into a particular endoscope channel and for easy insertion. The brush may be built on a control wire that provides increased warp resistance, improved insertion ability, and easier brushing capabilities. Some biopsy brushes include stiffer bristles and soft bristles in one brush. The stiffer bristles are to assist in creating a defect in the target tissue, and the softer bristles are to capture specimens after tissue debridement.

[0092] FIG. 7D illustrates a biopsy snare that can enter the gastrointestinal tract via a flexible endoscope and reach an anatomical target, grasp, dissect, and transect tissue therein. The snare may include one or more loops of particular shapes, e.g., circle, oval, hexagon, or diamond shaped loops. In ERCP, after placing the tip of the duodenoscope on an anatomical target, a snare can be deployed from the endoscope, opens up so that it can encircle the base of the target. Constant tension was applied to the snare loop during excision until the lesion was transected. Some snares have loops made of braided wire (such snares are therefore named “braided snares”) to prevent or reduce slippage and improve gripping force, thereby enhancing the precision in tissue transection.

[0093] FIG. 8 is a flow chart illustrating an example method 800 for generating an EUS-TA plan, including a recommended tissue sampling device, for use in a medical procedure, such as an EBUS-TBNA procedure. The method 800 may be implemented in and executed by the system 400. Although the processes of the method 800 are drawn in one flow chart, they are not required to be performed in a particular order. In various examples, some of the processes can be performed in a different order than that illustrated.

[0094] At step 810, during an EUS-TA procedure such as an EBUS- TBNA procedure, ultrasound images of an anatomical target may be displayedon a graphic al user interface. The ultrasound images may be generated using an ultrasound transducer associated with an endoscope, such as the ultrasound transducer associated with the echoendoscope 120, or the ultrasound transducer 333 associated with the bronchoscope 320. Depending on the location of the anatomical target relative to the ultrasound transducer, a real-time ultrasound field of view (FOV) of the target may be presented to the user on the graphical user interface. In addition to the ultrasound images, other images may be generate or otherwise received at step 810, including, for example, perioperative endoscope images or videos of the anatomical target and its surrounding environment captured by a camera associated with the echoendoscope, or preoperative or perioperative images of the anatomical target acquired by external imaging devices, such as X-ray or fluoroscopy images, electrical potential map or an electrical impedance map, CT images, or MRI images, among others.

[0095] Also at step 810, a plurality of nominal sampling device trajectories {Si, S2, ..., SN} intersecting with anatomical targets at respective locations within the real-time ultrasound FOV may be overlaid upon the realtime ultrasound FOV, as illustrated in FIG. 5. The plurality of nominal sampling device trajectories {Si, S2, ..., SN} each correspond to respective different hardware configurations of a plurality of nominal tissue sampling devices {Xi, X2, . . ., XN}, such as pre-formed curvatures or bending angles.

[0096] At step 820, a user selection of the anatomical target within the ultrasound image may be received. The selection can be made by highlighting or clicking on the anatomical target as appeared on the user interface, or by drawing a bounding box around the anatomical target, as illustrated in FIGS. 6A- 6B.

[0097] At step 830, a location of the user-selected anatomical target within the real-time ultrasound FOV may be determined, such as based on an analysis of the ultrasound images using the target identification and localization circuit 412. In some examples, the target location may be determined additionally or alternatively based on other characteristics of the anatomical target, including, for example, a size, a shape, or a structure of the anatomical target, and / or recognize a pathophysiological property, such as a lesion, an inflammation state, a stricture level, or a malignancy state (e.g., degree or area ofinvasion by cancer) of the anatomical target using on-site, real-time in vivo tissue diagnosis.

[0098] At step 840, based at least in part on the received indication of the location of the anatomical target, a nominal sampling device trajectory S* may be predicted from the multiple nominal sampling device trajectories {Si, S2, ..., SN}. The nominal trajectory S* begins at an entry point into the real-time ultrasound FOV, and intersects the anatomical target at the detected target location in the real-time ultrasound FOV. The prediction of the nominal sampling device trajectory S* may be based at least in part on the location of the anatomical target in the real-time ultrasound FOV, which can be detected by the target identification and localization circuit 412 using image features extracted from the ultrasound images received at step 810. Other factors taken into consideration when predicting the nominal trajectory S* may include, for example, a posture and a puncture angle of the sampling device (e.g., a needle) upon piercing through the airway wall, or a posture and an entering angle upon entering the real-time ultrasound FOV.

[0099] In some example, the multiple nominal sampling device trajectories {Si, S2, ..., SN} correspond to respective hardware configurations (e.g., pre-formed curvatures or bending angles) of multiple sampling devices {Xi, X2, . . ., XN}. The nominal trajectory S* may be selected from the multiple nominal sampling device trajectories {Si, S2, ..., SN}, and corresponds to a tissue sampling device X* with a specific pre-shaped configuration, such as a pre-formed curvature or bending angle. In some further examples, the nominal sampling device trajectory S* may be predicted using procedure data collected form a patient population. The nominal sampling device trajectory S* may represent a best fit line or curve obtained from, for example, linear or nonlinear regression analysis of needle trajectories collected under real-life or modeled circumstances from similar procedures performed on a selected patient population under similar conditions (e.g., similar types or models of bronchoscopes).

[0100] At step 850, a recommended sampling device X* may be determined for use in an EUS-TA procedure to collect tissue samples from the anatomical target, such as a peribronchial nodule in an EBUS-TBNA procedure. The recommended sampling device X* has a hardware configuration inaccordance with the predicted nominal sampling device trajectory S*. The recommendation may further include device type, such as one of aspiration or biopsy needles, brushes, forceps, knives, snares, or suction devices. The tissue sampling device recommendation may include device size, such as 19-, 21-, 22-, or 25-gauge (G) EBUS-TBNA needles with added flexibility and enhanced puncture capabilities for smoother tissue sampling.

[0101] In some examples, in addition to the recommended sampling device, recommended values of one or more device operation parameters may be determined. Examples of the device operation parameters include recommended values of one or more operational parameters for navigating and manipulating the recommended tissue sampling device X*, one or more navigation parameters of an echoendoscope (e.g., the bronchoscope 320) over which the recommended sampling device X* is deployed, or one or more operational parameters for positioning and navigating the EUS probe to take ultrasound scans of the anatomical target.

[0102] In some examples, one or more artificial intelligence (Al) or machine-learning (ML) models may be respectively trained to determine a sampling device recommendation, to determine recommended values of sampling device navigation and manipulation parameters, or to determine recommended positioning and navigation of the EUS probe to take ultrasound scans of the anatomical target, as described above with respect to FIG. 4.

[0103] FIG. 9 illustrates generally a block diagram of an example machine 900 upon which any one or more of the techniques (e.g., methodologies) discussed herein may perform. Portions of this description may apply to the computing framework of various portions of the EUS-TA planning system 400, such as one or more of components 411-417 of the processor 410.

[0104] In alternative embodiments, the machine 900 may operate as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine 900 may operate in the capacity of a server machine, a client machine, or both in server-client network environments. In an example, the machine 900 may act as a peer machine in peer-to-peer (P2P) (or other distributed) network environment. The machine 900 may be a personal computer (PC), a tablet PC, a set-top box (STB), a personal digital assistant (PDA), a mobile telephone, a web appliance, a network router, switch or bridge,or any machine capable of executing instructions (sequential or otherwise) that specify actions to be taken by that machine. Further, while only a single machine is illustrated, the term “machine” shall also be taken to include any collection of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methodologies discussed herein, such as cloud computing, software as a service (SaaS), other computer cluster configurations.

[0105] Examples, as described herein, may include, or may operate by, logic or a number of components, or mechanisms. Circuit sets are a collection of circuits implemented in tangible entities that include hardware (e.g., simple circuits, gates, logic, etc.). Circuit set membership may be flexible over time and underlying hardware variability. Circuit sets include members that may, alone or in combination, perform specified operations when operating. In an example, hardware of the circuit set may be immutably designed to carry out a specific operation (e.g., hardwired). In an example, the hardware of the circuit set may include variably connected physical components (e.g., execution units, transistors, simple circuits, etc.) including a computer readable medium physically modified (e.g., magnetically, electrically, moveable placement of invariant massed particles, etc.) to encode instructions of the specific operation. In connecting the physical components, the underlying electrical properties of a hardware constituent are changed, for example, from an insulator to a conductor or vice versa. The instructions enable embedded hardware (e.g., the execution units or a loading mechanism) to create members of the circuit set in hardware via the variable connections to carry out portions of the specific operation when in operation. Accordingly, the computer readable medium is communicatively coupled to the other components of the circuit set member when the device is operating. In an example, any of the physical components may be used in more than one member of more than one circuit set. For example, under operation, execution units may be used in a first circuit of a first circuit set at one point in time and reused by a second circuit in the first circuit set, or by a third circuit in a second circuit set at a different time.

[0106] Machine (e.g., computer system) 900 may include a hardware processor 902 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), a hardware processor core, or any combination thereof), a main memory904 and a static memory 906, some or all of which may communicate with each other via an interlink (e.g., bus) 908. The machine 900 may further include a display unit 910 (e.g., a raster display, vector display, holographic display, etc.), an alphanumeric input device 912 (e.g., a keyboard), and a user interface (UI) navigation device 914 (e.g., a mouse). In an example, the display unit 910, input device 912 and UI navigation device 914 may be a touch screen display. The machine 900 may additionally include a storage device (e.g., drive unit) 916, a signal generation device 918 (e.g., a speaker), a network interface device 920, and one or more sensors 921, such as a global positioning system (GPS) sensor, compass, accelerometer, or other sensors. The machine 900 may include an output controller 928, such as a serial (e.g., universal serial bus (USB), parallel, or other wired or wireless (e.g., infrared (IR), near field communication (NFC), etc.) connection to communicate or control one or more peripheral devices (e.g., a printer, card reader, etc.).

[0107] The storage device 916 may include a machine readable medium 922 on which is stored one or more sets of data structures or instructions 924 (e.g., software) embodying or utilized by any one or more of the techniques or functions described herein. The instructions 924 may also reside, completely or at least partially, within the main memory 904, within static memory 906, or within the hardware processor 902 during execution thereof by the machine 900. In an example, one or any combination of the hardware processor 902, the main memory 904, the static memory 906, or the storage device 916 may constitute machine readable media.

[0108] While the machine-readable medium 922 is illustrated as a single medium, the term "machine readable medium" may include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) configured to store the one or more instructions 924.

[0109] The term “machine readable medium” may include any medium that is capable of storing, encoding, or carrying instructions for execution by the machine 900 and that cause the machine 900 to perform any one or more of the techniques of the present disclosure, or that is capable of storing, encoding or carrying data structures used by or associated with such instructions. Nonlimiting machine-readable medium examples may include solid-state memories, and optical and magnetic media. In an example, a massed machine-readablemedium comprises a machine readable medium with a plurality of particles having invariant (e.g., rest) mass. Accordingly, massed machine-readable media are not transitory propagating signals. Specific examples of massed machine- readable media may include: non-volatile memory, such as semiconductor memory devices (e.g., Electrically Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EPSOM)) and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks.

[0110] The instructions 924 may further be transmitted or received over a communication network 926 using a transmission medium via the network interface device 920 utilizing any one of a number of transfer protocols (e.g., frame relay, internet protocol (IP), transmission control protocol (TCP), user datagram protocol (UDP), hypertext transfer protocol (HTTP), etc.). Example communication networks may include a local area network (LAN), a wide area network (WAN), a packet data network (e.g., the Internet), mobile telephone networks (e.g., cellular networks), Plain Old Telephone (POTS) networks, and wireless data networks (e.g., Institute of Electrical and Electronics Engineers (IEEE) 802.11 family of standards known as WiFi®, IEEE 802.16 family of standards known as WiMax®), IEEE 802.15.4 family of standards, peer-to-peer (P2P) networks, among others. In an example, the network interface device 920 may include one or more physical jacks (e.g., Ethernet, coaxial, or phonejacks) or one or more antennas to connect to the communication network 926. In an example, the network interface device 920 may include a plurality of antennas to wirelessly communicate using at least one of single-input multiple-output (SIMO), multiple-input multiple-output (MIMO), or multiple-input single-output (MISO) techniques. The term “transmission medium” shall be taken to include any intangible medium that is capable of storing, encoding or carrying instructions for execution by the machine 900, and includes digital or analog communications signals or other intangible medium to facilitate communication of such software.Additional Notes

[0111] The above detailed description includes references to the accompanying drawings, which form a part of the detailed description. The drawings show, by way of illustration, specific embodiments in which theinvention can be practiced. These embodiments are also referred to herein as “examples.” Such examples may include elements in addition to those shown or described. However, the present inventors also contemplate examples in which only those elements shown or described are provided. Moreover, the present inventors also contemplate examples using any combination or permutation of those elements shown or described (or one or more aspects thereof), either with respect to a particular example (or one or more aspects thereof), or with respect to other examples (or one or more aspects thereof) shown or described herein.

[0112] In this document, the terms “a” or “an” are used, as is common in patent documents, to include one or more than one, independent of any other instances or usages of “at least one” or “one or more.” In this document, the term “or” is used to refer to a nonexclusive or, such that “A or B” includes “A but not B,” “B but not A,” and “A and B,” unless otherwise indicated. In this document, the terms “including” and “in which” are used as the plain-English equivalents of the respective terms “comprising” and “wherein.” Also, in the following claims, the terms “including” and “comprising” are open-ended, that is, a system, device, article, composition, formulation, or process that includes elements in addition to those listed after such a term in a claim are still deemed to fall within the scope of that claim. Moreover, in the following claims, the terms “first,” “second,” and “third,” etc. are used merely as labels, and are not intended to impose numerical requirements on their objects.

[0113] The above description is intended to be illustrative, and not restrictive. For example, the above-described examples (or one or more aspects thereof) may be used in combination with each other. Other embodiments can be used, such as by one of ordinary skill in the art upon reviewing the above description. The Abstract is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. Also, in the above Detailed Description, various features may be grouped together to streamline the disclosure. This should not be interpreted as intending that an unclaimed disclosed feature is essential to any claim. Rather, inventive subject matter may lie in less than all features of a particular disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description as examples or embodiments, with each claim standing on its own as a separateembodiment, and it is contemplated that such embodiments can be combined with each other in various combinations or permutations. The scope of the invention should be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled.

Claims

What is claimed is:

1. A system for determining a sampling device for use in a medical procedure in a patient, the system comprising: an ultrasound imaging device configured to generate ultrasound images of an anatomical target in a real-time ultrasound field of view (FOV) of a region of interest; a display configured to display, in the real-time ultrasound FOV, the ultrasound images; and a processor configured to: receive an indication of a location of the anatomical target within the real-time ultrasound FOV; based at least in part on the received indication of the location of the anatomical target, predict, from multiple nominal sampling device trajectories a nominal sampling device trajectory that intersects the anatomical target within the real-time ultrasound FOV, the multiple nominal sampling device trajectories corresponding to respective hardware configurations of multiple sampling devices; and determine a recommended sampling device for sampling tissue from the anatomical target, the recommended sampling device having a hardware configuration in accordance with the predicted nominal sampling device trajectory.

2. The system of claim 1, wherein the recommended sampling device includes a recommended endobronchial needle for sampling tissue from a lung periphery target during an endobronchial ultrasound-guided transbronchial fine needle aspiration (EBUS-TBNA) procedure.

3. The system of claim 1, wherein the recommended sampling device includes a needle-stylet combination comprising a needle and a stylet insertable into the needle, wherein one or more of the stylet or the needle has a pre-formed curvature or bending angle substantially conforming to the predicted nominal sampling device trajectory.

4. The system of claim 1, wherein the display is further configured to display, within the real-time ultrasound FOV, one or more of: graphical representations of the multiple nominal sampling device trajectories; the indication of the location of the anatomical target; an indication of the predicted nominal sampling device trajectory; or information about the recommended sampling device.

5. The system of claim 1, wherein the multiple nominal sampling device trajectories are each color-coded to match respective color codes for the multiple sampling devices, wherein the predicted nominal sampling device trajectory is color-coded to match a color code for the recommended sampling device.

6. The system of claim 1, wherein the respective hardware configurations of the multiple sampling devices include respective different pre-formed curvatures or bending angles, wherein the recommended sampling device is selected from the multiple sampling devices, and has a pre-formed curvature or bending angle substantially conforming to the predicted nominal sampling device trajectory.

7. The system of claim 1, wherein the processor is configured to: receive, via a user interface, a user input identifying the anatomical target within the real-time ultrasound FOV; and analyze the ultrasound images within the real-time ultrasound FOV to determine the location of the anatomical target as identified by the user.

8. The system of claim 1, wherein the processor is further configured to generate a recommendation to reposition one or more of the sampling device or the ultrasound imaging device to align the anatomical target with the predicted nominal trajectory of the sampling device.

9. The system of claim 6, wherein the recommendation includes advancing, retracing, or rotating the ultrasound imaging device by a specified amount toalign the anatomical target with the predicted nominal sampling device trajectory.

10. The system of claim 1, wherein the ultrasound imaging device is located at a distal end of an endobronchial sampling device configured to be inserted into a patient’s airway during an endobronchial ultrasound-guided transbronchial fine needle aspiration (EBUS-TBNA) procedure.

11. An intraluminal imaging device system, comprising: an intraluminal imaging device, comprising a tubular device body having an inner lumen and a side exit port, and an ultrasound transducer configured to produce ultrasound scans of an anatomical target and generate ultrasound images in a real-time ultrasound field of view (FOV) of a region of interest; a display configured to display, in the real-time ultrasound FOV, the ultrasound images and graphical representations of multiple nominal sampling device trajectories that traverse the FOV ; and a tissue sampling device configured to operatively, under ultrasound guidance: pass through the inner lumen and exit from the side exit port of the tubular device body; and advance toward the anatomical target in accordance with a predicted nominal trajectory selected from the multiple nominal sampling device trajectories and subsequently intersect the anatomical target, wherein the multiple nominal sampling device trajectories correspond to respective hardware configurations of multiple sampling devices, wherein the tissue sampling device has a hardware configuration corresponding to the predicted nominal trajectory.

12. The intraluminal imaging device system of claim 11, wherein the tissue sampling device includes an endobronchial needle for sampling tissue from a lung periphery target during an endobronchial ultrasound-guided transbronchial fine needle aspiration (EBUS-TBNA) procedure.

13. The intraluminal imaging device system of claim 11, wherein the tissue sampling device includes a needle-stylet combination comprising a needle and a stylet insertable into the needle, wherein one or more of the stylet or the needle has a pre-formed curvature or bending angle substantially conforming to the predicted nominal sampling device trajectory.

14. The intraluminal imaging device system of claim 11, wherein the multiple nominal sampling device trajectories are each color-coded to match respective color codes for the multiple sampling devices, wherein the predicted nominal trajectory is color-coded to match a color code for the tissue sampling device.

15. The intraluminal imaging device system of claim 11, wherein the respective hardware configurations of the multiple sampling devices include respective different pre-formed curvatures or bending angles, wherein the tissue sampling device has a pre-formed curvature or bending angle substantially conforming to the predicted nominal sampling device trajectory.

16. The intraluminal imaging device system of claim 11, wherein the intraluminal imaging device includes a bronchoscope configured to be inserted into a patient’s airway during an endobronchial ultrasound-guided transbronchial fine needle aspiration (EBUS-TBNA) procedure, the ultrasound transducer being located at a distal end of the bronchoscope.

17. A method for determining a sampling device for use in a medical procedure, the method comprising: displaying, via a graphical user interface, ultrasound images of an anatomical target in a real-time ultrasound field of view (FOV) of a region of interest; receiving, via the graphical user interface, a selection of the anatomical target within the ultrasound image; determining a location of the anatomical target within the real-time ultrasound FOV;based at least in part on the determined location of the anatomical target, predicting from multiple nominal sampling device trajectories a nominal sampling device trajectory that intersects the anatomical target within the realtime ultrasound FOV, the multiple nominal sampling device trajectories corresponding to respective hardware configurations of multiple sampling devices; and determining a recommended sampling device for sampling tissue from the anatomical target, the recommended sampling device having a hardware configuration in accordance with the predicted nominal sampling device trajectory.

18. The method of claim 17, further comprising displaying, via the graphical user interface and within the real-time ultrasound FOV, one or more of: graphical representations of the multiple nominal sampling device trajectories; the location of the anatomical target; an indication of the predicted nominal sampling device trajectory; or information about the recommended sampling device.

19. The method of claim 17, wherein the multiple nominal sampling device trajectories are each color-coded to match respective color codes for the multiple sampling devices, wherein the predicted nominal sampling device trajectory is color-coded to match a color code for the recommended sampling device.

20. The method of claim 17, wherein the respective hardware configurations of the multiple sampling devices include respective different pre-formed curvatures or bending angles, wherein the recommended sampling device is selected from the multiple sampling devices, and has a pre-formed curvature or bending angle substantially conforming to the predicted nominal sampling device trajectory.

21. The method of claim 17, wherein determining the recommended sampling device includes determining a needle-stylet combination comprising aneedle and a stylet insertable into the needle, wherein one or more of the stylet or the needle has a pre-formed curvature or bending angle substantially conforming to the predicted nominal sampling device trajectory.

22. The method of claim 17, comprising generating recommendation to reposition one or more of the sampling device or the ultrasound imaging device to align the anatomical target with the predicted nominal trajectory of the sampling device.

23. The method of claim 22, wherein the recommended repositioning includes advancing, retracing, or rotating the ultrasound imaging device by a specified amount to align the anatomical target with the predicted nominal sampling device trajectory.

Images