Needle sensor derived image plane

Abstract

A system includes a flexible elongate device, a tool, and a controller. The flexible elongate device includes a working channel extending to an opening in a distal portion of the flexible elongate device and a plurality of imaging elements. A tool can extend through the working channel and the opening. The controller has one or more processors configured to determine a position associated with a portion of the tool and control at least a portion of the plurality of imaging elements to generate an image including an imaging plane including the position associated with the portion of the tool.

Description

[0001] Cross-referenced applications

[0002] This application claims priority to and the benefit of U.S. Provisional Application No. 63 / 325,082, filed on March 29, 2022, and entitled “Needle Sensor Derived Image Plane,” the entire contents of which are incorporated herein by reference. Technical Field

[0003] The present disclosure relates to systems and methods for planning and performing image-guided procedures. Background Art

[0004] Minimally invasive medical techniques are intended to reduce the amount of damaged tissue in medical procedures, thereby reducing patient recovery time, discomfort, and harmful side effects. This minimally invasive technique can be performed through natural orifices in the patient's anatomical structure or through one or more surgical incisions. Through these natural orifices or incisions, the operator can insert minimally invasive medical tools to reach the target tissue position. Minimally invasive medical tools include instruments such as treatment, diagnosis, biopsy, and surgical instruments. Medical tools can be inserted into anatomical pathways and navigated toward the area of ​​interest in the patient's anatomical structure. Optical or ultrasonic images of anatomical pathways and surrounding anatomical structures obtained preoperatively and / or intraoperatively can be used to assist navigation. Intraoperative imaging of the tool from the probe or catheter through which the tool is inserted can provide improved navigation guidance and confirmation of the tool's engagement with the target tissue.

[0005] Improved systems and methods are needed to effectively track the position of a tool relative to a catheter or probe through which the tool is inserted to improve imaging of the tool. Summary of the invention

[0006] In accordance with some examples, a system may include a flexible elongated device, a tool, and a controller. The flexible elongated device may include an open working channel extending into a distal portion of the flexible elongated device and a plurality of imaging elements. The tool can extend through the working channel and the opening. The controller may have one or more processors configured to determine a position associated with a portion of the tool and control at least a portion of the plurality of imaging elements to generate an image including an imaging plane including a position associated with the portion of the tool. In some examples, the portion of the tool may include a distal tip of the tool.

[0007] In some instances, the plurality of imaging elements may include an annular array of imaging elements disposed around an opening. The annular array may include at least two rows of imaging elements. In some instances, the plurality of imaging elements may include an array of ultrasonic transmitters configured to generate ultrasonic waves and an array of ultrasonic sensors configured to detect reflected ultrasonic waves. The ultrasonic sensors may include whispering gallery mode (WGM) resonators. In some instances, the plurality of imaging elements may include an array of transducers configured to generate and detect ultrasonic waves. In some instances, the plurality of imaging elements may include a grid array in which the plurality of imaging elements are arranged in at least two rows and at least two columns.

[0008] In some examples, the plurality of imaging elements may be configured to capture an image including an imaging plane defined within an imaging field located distally of the distal portion of the flexible, elongated device. The working channel may extend through the distal surface of the flexible, elongated device to guide a tool into the imaging field. In some examples, the plurality of imaging elements may be configured to capture an image including an imaging plane defined within an imaging field located radially outward from an outer side surface of the flexible, elongated device. The working channel may extend through a sidewall of the flexible, elongated device to guide a tool into the imaging field.

[0009] In some examples, the controller may be further configured to determine a second position associated with one of a second position of the tool or a region of interest in the target tissue.The imaging plane may include both the position and the second position.

[0010] In some examples, the flexible, elongated device may further include a balloon configured to be inflated with a fluid for acoustic transmission between the plurality of imaging elements and the tissue.

[0011] In some instances, the tool may include at least one of a biopsy needle, a therapy delivery needle, an ablation device, or a cryogenic device. At least a portion of the tool may be flexible. The tool may include a positioning sensor located at a portion of the tool. The positioning sensor may include a fiber optic sensor configured to detect wavelength changes of light in a fiber in response to pressure waves generated by a plurality of imaging elements. The positioning sensor may include a WGM resonator. The positioning sensor may include a piezoelectric sensor. The positioning sensor may include a capacitive micromachined ultrasonic transducer.

[0012] In some instances, the controller can be configured to determine the position of a portion of the tool by sequentially activating subsets of multiple imaging elements to transmit pressure waves, detecting the pressure waves from each imaging element of the subset, determining the travel time of the pressure waves from each imaging element of the subset to a positioning sensor, and triangulating the position of the portion of the tool based on the travel time of each pressure wave from the corresponding imaging element to the positioning sensor.

[0013] In some examples, the plurality of imaging elements may be arranged in an annular array, and the subset may include three imaging elements of the plurality of imaging elements. The three imaging elements may be spaced apart around the perimeter of the annular array at 120° separation.

[0014] In some instances, the controller can be configured to determine the position of a portion of the tool by activating a positioning sensor to transmit a pressure wave, activating a subset of multiple imaging elements to detect the pressure wave, determining a travel time of the pressure wave from the positioning sensor to each imaging element of the subset, and triangulating the position of the portion of the tool based on the travel time of the pressure wave from the positioning sensor to each imaging element of the subset.

[0015] In some examples, the controller can be configured to control at least a portion of the plurality of imaging elements to produce an image by determining a firing sequence of at least a portion of the plurality of imaging elements based on an imaging plane. The firing sequence can include a time delay between successive firings of the imaging elements. The time sequence delay can be determined based on a geometry of an imaging field of the image relative to the positions of the plurality of imaging elements.

[0016] In some examples, the controller may be further configured to associate imaging element sets with different image planes, determine the imaging plane as a location closest to the portion of the tool, and generate the image using the imaging element set associated with the determined imaging plane.

[0017] In accordance with some examples, a method for capturing an image performed by a controller including one or more processors may include determining a position associated with a portion of a tool. The tool may be at least partially located in a flexible, elongated device. The flexible, elongated device may include a plurality of imaging elements and a working channel extending to an opening in a distal portion of the flexible, elongated device through which the portion of the tool extends. The method may further include controlling at least a portion of the plurality of imaging elements to generate an image including an imaging plane including the position associated with the portion of the tool.

[0018] In some examples, the method may include displaying a virtual indicator that provides a visual representation of the position of the portion of the tool relative to the imaging plane. The virtual indicator may include a distance indication that provides a visual representation of the distance between the portion of the tool and the imaging plane. The virtual indicator may include a direction indication that provides a representation of the direction of the portion of the tool relative to the imaging plane. The direction indication may include displaying a color of at least a portion of the virtual indicator.

[0019] Other examples include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0020] It should be understood that the above general description and the following detailed description are exemplary and explanatory in nature, and are intended to provide an understanding of the present disclosure without limiting the scope of the present disclosure. To this end, additional aspects, features and advantages of the present disclosure will be apparent to those skilled in the art from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 An example of an imaging probe in a patient anatomy near a target is shown.

[0022] Figure 2A An example of an imaging probe according to the present disclosure is shown.

[0023] Figure 2B It shows that Figure 2A Examples of imaging arrays are shown.

[0024] Figure 3A Another example of an imaging probe according to the present disclosure is shown.

[0025] Figure 3B It shows that Figure 3A Examples of imaging arrays are shown.

[0026] Figure 3C It shows that Figure 3B An alternative example of an imaging array is shown.

[0027] Figure 3D-Figure 3H An alternative example of component arrangement in an imaging probe is shown.

[0028] Figure 4A Another example of an imaging probe according to the present disclosure is shown.

[0029] Figure 4B It shows that Figure 4A Examples of imaging arrays are shown.

[0030] Figure 5A An example of an imaging probe positioned within a catheter according to the present disclosure is shown.

[0031] Figure 5B It shows that Figure 5A Examples of imaging arrays are shown.

[0032] Fig. 6A Another example of an imaging probe positioned within a catheter according to the present disclosure is shown.

[0033] Figure 6B It shows that Fig. 6A Examples of imaging arrays are shown.

[0034] Figure 6C It shows that Fig. 6A Another example of an imaging array is shown

[0035] Fig.6D It shows that Fig. 6A Another example of an imaging array is shown.

[0036] Fig. 6E It shows that Fig. 6A Another example of an imaging array is shown.

[0037] Fig. 7A The arrangement of the imaging probe, tool and imaging plane is shown.

[0038] Figure 7B Shown based on Fig. 7A An example of an image produced by an arrangement.

[0039] Fig. 8A Another arrangement of imaging probes, tools, and imaging planes is shown.

[0040] Figure 8B Shown based on Fig. 8A An example of an image produced by an arrangement.

[0041] Fig. 9A An example of an arrangement for orienting a portion of a tool to select an imaging plane is shown.

[0042] Fig. 9B Shows that in determining Fig. 9A The orientation of the described parts is followed by examples of images generated based on the selected imaging planes.

[0043] Fig. 10A A flow chart illustrating an example of a method for selecting an imaging plane and capturing imaging.

[0044] Fig. 10B A flow chart showing another example of a method for selecting an imaging plane and capturing imaging.

[0045] Fig.11 A simplified diagram of a robotic-assisted medical system according to some examples of the present disclosure is shown.

[0046] Fig.12 A simplified diagram of a medical instrument system according to some examples of the present disclosure is shown.

[0047] By referring to the following detailed description, examples of the present disclosure and its advantages can be best understood. It should be understood that the same reference numerals are used to identify the same elements shown in one or more of the accompanying drawings, wherein the illustrations therein are for illustrating examples of the present disclosure, rather than for limiting the present disclosure. DETAILED DESCRIPTION

[0048] The technology disclosed herein can be used to enhance the working process of minimally invasive procedures using direct intraoperative visualization of tools (such as intraoperative ultrasound imaging). In some instances, imaging data from an imaging probe can be used to verify the real-time accurate placement of a therapeutic or diagnostic tool within an anatomical target in a medical procedure. Although described in the context of an ultrasound imaging probe, it is conceivable that the systems and methods described herein can be applied to other imaging modalities without departing from the scope of the present disclosure. For example, when a tool is delivered to a target via a flexible, elongated device (e.g., a catheter or an imaging probe), an imaging probe can be used to provide direct visual guidance of the tool. Such tools may include, but are not limited to, ablation devices, biopsy tools (e.g., needles), treatment delivery tools, cryogenic devices, flushing, suction, or any other suitable medical treatment tool.

[0049] Figure 1 An example of an image 100 is shown, the image 100 includes a flexible, elongated device including an imaging probe 104, the imaging probe 104 being positioned within a passage 102 of a patient's anatomy proximate to a target tissue 108 (e.g., an airway of a lung), which in some examples may be a lymph node, a suspected tumor, a nodule, or a lesion. The image 100 may be displayed to a user on a graphical display and may be based on intraoperative imaging captured from an imaging system external to the patient (e.g., fluoroscopy, computed tomography, magnetic resonance imaging, etc.), or may be based on a virtual visualization of a model of the patient's anatomy generated from preoperative or intraoperative image data. For example, the image data may be captured by a control system and used to generate a three-dimensional model of the patient's anatomy. Imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), fluoroscopy, thermal imaging, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube x-ray imaging, etc. may be used to capture the model image data. For example, a CT scan of the patient's anatomy may be performed preoperatively or intraoperatively, and the resulting image data may be used to construct a 3D model that may be displayed to a user on a display system. Image 100 may present a global view of the anatomical model of pathway 102, including a virtual representation of imaging probe 104, including real-time updates of its position and orientation as the imaging probe navigates through the patient's anatomy based on data from a sensor system (e.g., sensor system 1508 of FIG. 15 ). In the illustrated example, image 100 includes a 3D model based on a model reference frame (X M , Y M , Z M) 150, the model reference frame can be aligned with the surgical reference frame of the anatomy and / or the instrument reference frame of the imaging probe 104. For further description of the alignment between the model reference frame and the instrument reference frame or the surgical reference frame, please refer to U.S. Provisional Patent Application No. 63 / 240,471 filed on September 3, 2021 (disclosing "Ultrasound Elongate Instrument Systems And Methods"), the entire contents of which are incorporated herein by reference for all purposes.

[0050] The imaging probe 104 may include one or more cavities for delivering instruments, tools, devices, etc. In the example shown, the tool 106 including a needle has been delivered to the target tissue 108 through the cavity of the imaging probe 104. In some examples, the imaging probe 104 may include an imaging device for visualizing the tool 106, such as an ultrasound or optical imaging array, capturing images in an imaging field including an imaging plane 110. The imaging probe 104 includes a plurality of (e.g., one or more arrays) imaging elements. The imaging elements can be arranged in any size, configuration, or shape. For example, the imaging elements can be arranged along the surface to form an array with a moon, annular, circular, or rectangular pattern. In some embodiments, the imaging element includes an array of transducers (e.g., lead zirconate titanate (PZT) transducers) that generate ultrasonic waves and / or detect reflected ultrasonic waves. In some embodiments, the imaging element includes an array of ultrasonic receivers (e.g., whispering gallery mode (WGM) resonators) and an array of ultrasonic transmitters (e.g., piezoelectric arrays). For more description of individual ultrasonic receiver and ultrasonic transmitter arrays, see International Patent Publication No. WO 2021 / 119182 filed on December 9, 2020 (disclosing “Whispering Gallery Mode Resonators for Sensing Applications”), which is incorporated herein by reference in its entirety for all purposes.

[0051] The imaging probe 104 can generate images in various imaging planes based on the control of ultrasound transmission by the imaging element. The transmission imaging element can capture images with selectable imaging planes using phased array ultrasound or synthetic aperture technology. For phased array ultrasound, in order to guide the ultrasound energy to the desired direction to create the desired imaging plane, the imaging element (e.g., transducer) is excited with the desired phase or time delay. In some instances, the phase or time delay can be calculated according to the desired focus / beam pattern. The excitation sequence of the ultrasound imaging element can be set to direct the ultrasound energy to the target tissue. In some embodiments, different imaging planes that can be selected are associated with different configurations of the imaging element. For example, various combinations and / or excitation sequences of imaging elements can be associated with one or more specific imaging planes, respectively. The control of the imaging element to generate an image including the desired imaging plane can be based on the configuration and / or excitation sequence associated with the desired imaging plane. A set of specific potential imaging planes can be preset, and the relevant combination and / or excitation sequence of each preset potential imaging plane can be stored in a memory. In an instance including an imaging array having imaging elements arranged radially around a central working channel, a set of preset potential imaging planes can be established at intervals of 5° around the axis of the working channel. The physical arrangement of the imaging elements can at least partially affect the spacing and orientation between preset imaging planes. For example, an imaging array having a relatively high number of imaging elements that are densely arranged can allow for a greater number of potential imaging planes than an imaging array having a relatively small number of imaging elements that are spaced apart. As discussed further below, after determining the desired (or "optimal") imaging plane, the controller can select an appropriate preset imaging plane from a set of preset imaging planes that most closely corresponds to the desired imaging plane. The controller can then activate a combination of imaging elements associated with the selected preset imaging plane in an excitation sequence associated with the selected preset imaging plane. In some instances, one or more imaging element selection algorithms and / or excitation sequence selection algorithms can be stored in a memory, and after determining the desired imaging plane, the controller can execute the one or more algorithms to select an appropriate imaging element and / or excitation sequence to produce an image along the desired imaging plane.

[0052] Since it is beneficial to provide real-time visualization of a tool located within a target (e.g., a lesion, tumor, or nodule) to ensure accurate delivery of the tool within the target, the tool 106 can be delivered within an imaging plane 110 of the imaging probe 104 to facilitate direct visualization of the tool entering the target tissue 108. The vasculature 109 is shown visualized, for example, by Doppler processing of ultrasound data, proximate the target tissue 108 within the imaging plane 110. Identifying and avoiding the vasculature 109 in a region of interest surrounding the target tissue 108 may be particularly important in certain areas of the anatomy, such as in the mediastinum.

[0053] Figure 2A An example of an imaging probe 104 is shown, which includes a forward imaging array 114 configured to capture images in one or more imaging planes 110 taken through an imaging field disposed distal to the distal end of the imaging probe 104. To this end, the tool 106 can extend from the opening of the working channel 112 of the imaging probe 104 and enter the imaging plane 110. The imaging array 114 can be configured to provide ultrasound images or any other suitable intraoperative imaging. It should be understood that the terms "forward" or "forward-looking" as described herein include imaging arrays whose imaging fields are at least partially directed to the front (or distal) side of the distal end of the device. This may include a direct forward imaging array, which is oriented substantially parallel to the longitudinal axis of the instrument on which the imaging array is disposed; a forward angled imaging array, whose imaging field is angled relative to the longitudinal axis, and the like. Therefore, in some examples of delivering a tool using a forward imaging array, the distal tip of the tool can be oriented relative to the imaging array in a manner that causes the tool to appear as a dot, ellipse, or similar shape in the image. In order to visualize a larger portion of the tool, the distal tip of the imaging probe or portions of the imaging array may be angled, beveled, or shaped in various ways to provide one or both direct axial and lateral / transverse images. In one instance, the distal segment of the imaging probe 104 supporting the imaging array 114 may be beveled or angled. In another example, the imaging array 114 may be transverse to the longitudinal axis of the imaging probe 104, but the working channel 112 may be at an angle away from the distal tip, rather than parallel to the longitudinal axis. Thus, an angle may be formed between one or more imaging planes of the imaging array 114 and the longitudinal axis of the tool 106 and / or the working channel 112. This arrangement may reduce the likelihood that the distal tip 105 of the tool is obstructed by a more proximal portion of the tool. In one instance, the angle between the imaging plane and the tool may be approximately 2-30°, such as 4°, 8°, or 12°. Figure 2A In the example shown, the imaging probe 104 has a forward-facing imaging array 114 that can collect images from a 3D volume distal to the array.

[0054] The imaging probe 104 can be steerable to navigate the imaging probe to a deployment position near the target tissue. For example, multiple pull wires or tendons can extend along the length of the imaging probe 104 and can be manipulated to steer the distal end of the imaging probe. Optionally, the imaging probe 104 can be passively flexible and can be navigated to a deployment position by a steerable catheter or sheath having a lumen through which the imaging probe is disposed. In some instances, the imaging probe 104 may include a lumen for receiving a guide wire to navigate the imaging probe to a deployment position. Fig.11 and Fig.12Navigation of the imaging probe 104 to a deployed position is discussed further.

[0055] Figure 2B Provided Figure 2A 1. A more detailed view of an imaging array 114 of a computer system. In this example, the imaging array includes 18 imaging elements (e.g., transducers) arranged in a grid of three rows and six columns. A controller (having one or more processors and memory) in communication with the imaging array 114 can control the order in which the individual imaging elements 118 are activated, thereby generating images in multiple imaging planes. In some examples, the controller can be configured to generate a three-dimensional image data volume by aggregating image data from multiple imaging planes, similar to a CT volume constructed from multiple image slices. This process can be computationally intensive and can cause undesirable time delays during the procedure. Therefore, it may be desirable to omit such a three-dimensional volume construction and only utilize a single imaging plane that is determined to be most suitable for displaying a feature of interest. Since the distal tip 105 of the tool 106 may be the most interesting tool feature for the user to visualize, the controller can identify and select an optimal imaging plane 110, which will include the distal tip 105 or a portion of the tool 106 near the distal tip. In this regard, positioning sensor 116 may be disposed at or near distal tip 105 in a fixed, known relationship for tracking the position of the distal tip. Fig. 9A As described in more detail, portions or subsets of the imaging elements 118 may be selected and activated (or "controlled") in an order determined by the controller. Activating a subset of the imaging elements 118, in conjunction with receiving signals (e.g., pressure or sound waves) from the subset of the imaging elements using the positioning sensor 116, may allow the controller to determine the position of the positioning sensor 116 relative to the imaging array 114. Since the position of the positioning sensor 116 is fixed relative to the tool 106, the position of the distal tip 105 or other portion of the tool may also be determined. Although the examples herein describe determining the orientation of the distal tip of the tool and generating an image having an imaging plane that includes the distal tip, it should be understood that any other portion of the tool that is exposed and within the imaging field of the imaging probe may be used in place of the distal tip.

[0056] Surface enhancements may be included on the tool 106 to facilitate imaging. For example, the outer surface of the tool 106 may be coated, roughened, or include grooves or slits to affect the scattering pattern of signals (e.g., pressure or sound waves) from the imaging element. In some instances, such surface enhancements may be provided only on one or more limited portions of the tool 106. In one instance, the portion of the tool 106 near or located at the distal tip 105 may be smooth, while the portion of the tool proximal to the distal tip may include a rough surface to disperse the imaging signal. In this regard, the portion including the distal tip 105 may be more distinct in the image than its proximal portion, thereby providing visual confirmation that the distal tip 105 has been captured in the image(s). In one instance, the portion of the tool 106 near or located at the distal tip 105 may be coated with an echogenic material, while the portion of the tool proximal to the distal tip may not be coated with an echogenic material. In this regard, the portion including distal tip 105 may be more visibly distinct in the image than its proximal portion, thereby providing visual confirmation that distal tip 105 has been captured in the image(s).

[0057] As will be appreciated, where the tool 106 is rigid, it can be expected to extend distally from the opening of the working channel 112 directly along an axis 115 extending from the working channel. Thus, an alignment of the tool 106 relative to the imaging array 114 can be assumed based on the known position of the axis 115 of the working channel 112 relative to the imaging array 114. In this regard, an optimal imaging plane can be predetermined, such as an imaging plane aligned with the axis 115. However, where the tool 106 is flexible, the tool may bend away from the axis 115 as it extends from the imaging probe 104. For example, a flexible biopsy needle may bend away from the axis 115 when encountering resistance from tissue between the imaging probe 104 and the target tissue. Therefore, it should be understood that an imaging plane aligned with the working channel 112 may not always be suitable for displaying the distal tip of a flexible tool. Furthermore, when a flexible tool is bent out of plane, it may not be easy to see that the portion of the tool displayed does not include the distal tip when viewing the image from the imaging probe. Therefore, it may be desirable to utilize an alternate imaging plane that is laterally offset from the axis 115 of the working channel 112 that may provide an optimal imaging plane based on the determined position of the distal tip 105 .

[0058] After determining the position of the distal tip 105, the controller may identify another subset of imaging elements 118 and a corresponding activation order suitable for generating an imaging plane 110 that includes the distal tip 105. For example, different imaging planes 110 are associated with different positions, so the determined position of the distal tip 105 may be used to select an appropriate imaging plane 110. In some instances, the spatial resolution of the potential imaging planes of the imaging array 114 may result in the imaging array being unable to generate an imaging plane that captures the distal tip 105. That is, the potential imaging planes of the imaging array 114 may be spaced apart by some known distance (e.g., 0.25 mm), and the distal tip 105 may be determined to be located between two adjacent potential imaging planes. Therefore, the controller may determine which potential imaging plane is closest to the distal tip 105 and select that imaging plane as the optimal imaging plane 110. The controller may then activate the selected imaging element 118 to generate an image in the optimal imaging plane 110. That is, the controller may control at least a portion of the plurality of imaging elements 118 to generate an image that includes an imaging plane that includes the position of the distal tip.

[0059] Advantageously, the imaging techniques described herein can reduce the computational burden of imaging processing on multiple imaging planes by identifying the optimal imaging plane and capturing images only in that plane. This can reduce the time required to construct an image and display it to the user, thereby enhancing real-time visualization of intraoperative tools. The imaging plane can be selected to display any portion of the tool 106 of interest (e.g., not necessarily the distal tip 105), or some other feature of interest (e.g., target tissue 108).

[0060] In some embodiments, the imaging plane 110 may be selected based on a plurality of locations or features of interest. For example, an imaging plane 110 may be selected that includes the distal tip 105 of the tool 106 and a portion of the target tissue 108. In another example, an imaging plane 110 may be selected that includes the distal tip 105 of the tool 106 and another (e.g., more proximal) portion of the tool 106. Thus, an optimal imaging plane 110 may be selected that includes a plurality of locations or features of interest.

[0061] Figure 3A , Figure 4A , Figure 5A and Fig. 6A Shown with Figure 2A An example of a similar imaging probe 104, but with an alternative structural arrangement. Figure 3AAn example of an imaging probe 104 is shown that includes a forward-facing annular or ring-shaped imaging array 114 disposed about a working channel 112 and configured to capture images in one or more imaging planes 110 taken through an imaging field disposed distally of a distal end of the imaging probe 104. In this regard, a tool 106 can extend from the working channel 112 of the imaging probe 104 and into the imaging plane 110. Figure 3A The imaging probe 104 may be similar to Figure 2A The imaging probes are independently steerable. Figure 3B Shows Figure 3A 1, which includes a plurality of imaging elements 118. The imaging elements 118 are arranged in a plurality of rows or rings 120a-120d and a plurality of columns 122a-122f. In the example shown, each column 122a-122f includes eight imaging elements extending linearly across the width of the imaging array 114. It should be understood that although Figure 3A and Figure 3B As shown in FIG. 1 , each imaging element 118 may be as shown in FIG. Figure 3C Any suitable subset of imaging elements 118 may be used to determine the position of positioning sensor 116 and distal tip 105, or to generate an image in imaging plane 110 that is determined to be optimal based on the determined position of positioning sensor 116 and / or distal tip 105.

[0062] Figure 3D A front view of an example of the distal end of the imaging probe 104 is shown. The working channel 112 is positioned at an offset position from the central longitudinal axis of the imaging probe. The imaging array 114 is positioned diametrically opposite the working channel 112. In this example, the imaging array 114 is rectangular, but any suitable shape may be utilized and may include any number of imaging elements (e.g., transducers) arranged, for example, in a grid of rows and columns. An image sensor 117, such as a CCD or CMOS camera, is positioned laterally offset from the working channel 112 and the imaging array 114. A light guide 113 is located on the opposite side of the image sensor 117. The light guide 113 may include an LED, a channel to accommodate an optical fiber, or any other suitable light emitting device.

[0063] Figure 3EA front view of an example of the distal end of the imaging probe 104 is shown. The working channel 112 is positioned at an offset position from the central longitudinal axis of the imaging probe. The imaging array 114 is positioned diametrically opposite the working channel 112. In this example, the imaging array 114 is rectangular, but any suitable shape may be utilized and may include any number of imaging elements (e.g., transducers) arranged, for example, in a grid of rows and columns. An image sensor 117, such as a CCD or CMOS camera, is positioned laterally offset from the working channel 112 and the imaging array 114. A pair of light guides 113 are located on one side of the image sensor 117. The light guides 113 may include an LED, a channel to accommodate an optical fiber, or any other suitable light emitting device.

[0064] Figure 3F A front view of an example of the distal end of the imaging probe 104 is shown. The working channel 112 is positioned at an offset position from the central longitudinal axis of the imaging probe. The imaging array 114 is positioned diametrically opposite the working channel 112. In this example, the imaging array 114 is rectangular, but any suitable shape may be utilized and may include any number of imaging elements (e.g., transducers) arranged, for example, in a grid of rows and columns. An image sensor 117, such as a CCD or CMOS camera, is positioned on one side to be laterally offset from the working channel 112 and the imaging array 114, and a light guide 113 is positioned on the other side to be laterally offset from the working channel 112 and the imaging array 114. The light guide 113 may include an LED, a channel to accommodate an optical fiber, or any other suitable light emitting device.

[0065] Figure 3G A front view of an example of the distal end of the imaging probe 104 is shown. The working channel 112 is positioned along the central longitudinal axis of the imaging probe. The imaging array 114 extends in an arc around at least a portion of the working channel 112. In this example, the imaging array 114 extends through approximately 300°, but any suitable arc length may be utilized. For example, the imaging array 114 may extend through an arc of 60-350°, such as 90°, 180°, 270°, or 340°. The imaging array may include any number of imaging elements (e.g., transducers) arranged, for example, in a grid of rows and columns. An image sensor 117, such as a CCD or CMOS camera, is positioned between opposite ends of the imaging array 114. A pair of light guides 113 are positioned on opposite sides of the image sensors 117. The light guides 113 may include an LED, a channel to accommodate an optical fiber, or any other suitable light emitting device.

[0066] Figure 3HA front view of an example of the distal end of the imaging probe 104 is shown. The working channel 112 is positioned offset from the central longitudinal axis of the imaging probe. The imaging array 114 is centered at a position diametrically opposite the working channel 112 and extends in an arc. In this example, the imaging array 114 extends through approximately 340°, but any suitable arc length may be utilized. For example, the imaging array 114 may extend through an arc of 60-350°, such as 90°, 180°, 270°, or 340°. The imaging array may include any number of imaging elements (e.g., transducers) arranged, for example, in a grid of rows and columns. An image sensor 117, such as a CCD or CMOS camera, is positioned within the arc of the imaging array 114. A pair of light guides 113 are positioned on opposite sides of the image sensor 117. The light guides 113 may include an LED, a channel to accommodate an optical fiber, or any other suitable light emitting device.

[0067] Figure 4A An example of an imaging probe 104 is shown that includes a lateral imaging array 114 disposed on a side of the imaging probe and configured to capture images in one or more imaging planes 110 taken through an imaging field disposed radially outward from a side of the imaging probe 104 . Figure 4A The imaging probe 104 may include a proximal portion 111a and a distal portion 111b, and the distal portion 111b may be more flexible than the proximal portion. U.S. Patent Application No. 16 / 632,128 filed on January 17, 2020 (disclosing "Flexible Elongate Device Systems and Methods") describes a method that may be used with Figure 4A The tool 106 may extend from a working channel 112 of a distal portion 111 b of the imaging probe 104 into the imaging plane 110 . Figure 4B Shows Figure 4A 1 is a simplified diagram of an imaging array 114 of the present invention, including a plurality of imaging elements 118. The imaging elements 118 are arranged in a plurality of rows 120a-120n and a plurality of columns 122a-122n. Any suitable subset of the imaging elements 118 may be used to determine the position of the positioning sensor 116 and the distal tip 105, or to generate an image in the imaging plane 110 that is determined to be optimal based on the determined position of the positioning sensor 116 and / or the distal tip 105.

[0068] Figure 5AAn example of an imaging probe 104 is shown that includes a lateral or side-looking imaging array 114 disposed distally of a working channel 112 extending through a side port and configured to capture images in one or more imaging planes 110 taken through an imaging field disposed radially outwardly of a side of the imaging probe 104. In this regard, a tool 106 can extend from the working channel 112 and into the imaging plane 110. Figure 5A The imaging probe 104 is shown extending distally from a lumen 126 of a steerable catheter 124. The catheter 124 may be used to guide the imaging probe 104 to a deployed position within the anatomy of a patient. Figure 5B Shows Figure 5A 1 is a simplified diagram of an imaging array 114 including a plurality of imaging elements 118. The imaging elements 118 are arranged in a plurality of rows 120a-120n and a plurality of columns 122a-122n. Any suitable subset of the imaging elements 118 may be used to determine the position of the positioning sensor 116 and the distal tip 105, or to generate an image in an imaging plane 110 that is determined to be optimal based on the determined position of the positioning sensor 116 and / or the distal tip.

[0069] Fig. 6A An example of an imaging probe 104 is shown that includes a forward annular or ring-shaped imaging array 114 disposed about a working channel 112 and configured to capture images in one or more imaging planes 110 taken through an imaging field disposed distally of a distal end of the imaging probe 104. In this regard, a tool 106 can extend from the working channel 112 of the imaging probe 104 and into the imaging plane 110. Fig. 6A The imaging probe 104 may be passively steered using a catheter 124 having a lumen 126 through which the imaging probe 104 is disposed. Figure 6B Shows Fig. 6A 1 is a simplified diagram of an imaging array 114 including a plurality of imaging elements 118. The imaging elements 118 are arranged in a plurality of rows 120a-120c and a plurality of columns 122a-122f, each column including six imaging elements extending linearly across the width of the imaging array 114 in the example shown. Although shown as having an arcuate shape, it should be understood that Fig. 6A and Figure 6B Each imaging element 118 may have a square or rectangular shape, similar to Figure 3C Any suitable subset of imaging elements 118 may be used to determine the position of positioning sensor 116 and distal tip 105, or to generate an image in imaging plane 110 that is determined to be optimal based on the determined position of positioning sensor 116 and / or distal tip 105.

[0070] In some of the above examples, the imaging element 118 may be a transducer capable of transmitting and detecting sound waves. In some examples, the imaging element 118 may be a piezoelectric transmitter or resonator, respectively. In some examples, the imaging array includes a piezoelectric array configured to generate ultrasonic waves. The imaging element of the imaging array includes a WGM resonator configured to detect reflected ultrasonic waves. The WGM resonator may include a fiber resonator, a microsphere resonator, a microbubble resonator, a microbottle resonator, a microring resonator, a microdisk resonator, a microring resonator, or a resonator of some other structure. The WGM resonator measures the intensity of the reflected ultrasonic wave. For example, the incident ultrasonic echo changes the resonant frequency of the WGM resonator by adjusting the refractive index of the WGM resonator material or physically deforming the WGM resonator, thereby generating a resonant shift.

[0071] Figure 6C Shows Fig. 6A 1 is a simplified diagram of an example of an imaging array 114 including a plurality of piezoelectric transmitters 117a (labeled "T") and a plurality of WGM resonators 117b (labeled "R") or other types of acoustic sensors. Each transmitter 117a can be configured to generate and transmit acoustic waves and each resonator 117b can be configured to detect reflected acoustic waves. In the illustrated example, an outer row 120a of imaging elements includes transmitters 117a and inner rows 120b, 120c include resonators 117b. It should be understood that any of the illustrated rows 120a-120b may include either transmitters 117a or resonators 117b. For example, a single row of resonators may be disposed between two rows of transmitters.

[0072] Fig.6D Shows Fig. 6A 1 is a simplified diagram of another example of an imaging array 114 of FIG. 1 , including a plurality of piezoelectric transmitters 117 a (labeled “T”) and a plurality of WGM resonators 117 b (labeled “R”). In the illustrated example, the transmitters 117 a and resonators 117 b are interspersed with each other so that each row 120 a - 120 c of imaging elements is formed by an alternating sequence of transmitters and resonators. Although shown as a 1:1 alternating pattern, any suitable pattern or arrangement of transmitters and resonators may be utilized.

[0073] Fig. 6E Shows Fig. 6A 1 is a simplified diagram of another example of an imaging array 114, including a plurality of piezoelectric transmitters 117a (labeled "T") and a plurality of WGM resonators 117b (labeled "R"). In the illustrated example, the transmitters 117a form an imaging element layer, which is positioned proximal to (behind) a layer of resonators 117b. Alternatively, the transmitter layer may be positioned distal to the resonator layer.

[0074] Although Figure 6C-Figure 6E The example is reference Fig. 6A provided, but it should be understood that the concepts discussed in relation to these figures can be similarly applied to Figure 2A-Figure 5B Any imaging array 114 in the drawings. For example, each of the imaging elements 118 in any of these figures may represent a transducer or a transmitter 117a or a resonator 117b. In this regard, the term "imaging array" as used herein may refer to a single array formed by a plurality of transducers or to a plurality of arrays formed by a plurality of transmitters and a plurality of resonators.

[0075] and Figure 2A Like the imaging probe 104, Figure 3A-Figure 6B The flexible tool 106 inserted into the imaging probe 104 can bend away from the axis 115 and out of plane relative to one or more potential imaging planes of the imaging array 114. Thus, the position of the distal tip 105 of the tool 106 can be determined using the positioning sensor 116, and a subset of the imaging elements 118 can be selected to provide an optimal imaging plane 110 to capture the distal tip 105 or other features of interest, such as those related to the imaging array 114. Figure 2A-2B described.

[0076] It should be understood that the shape, arrangement, size, and placement of the imaging array 114 of the illustrated example may be appropriately adjusted according to any given implementation. That is, these figures should not be considered limited to the specific array shape shown, the specific number of imaging elements in the imaging array, the specific shape of each imaging element, the spacing between adjacent imaging elements, etc. These illustrations are provided merely as examples of various configurations contemplated within the scope of the present disclosure.

[0077] It should be further understood that in ultrasound imaging, it is often desirable to secure the imaging probe in place for stability and to minimize the volume of air and other gases between the imaging array and the tissue to be imaged. Accordingly, various balloons or inflatable / expandable fluid containing devices may be used to ensure fluid contact. For example, balloon 119a may be disposed near or around the distal end of imaging probe 104, and / or balloon 119b may be disposed near the distal end of catheter 124. Balloon 119 may be inflated with a coupling fluid so that the balloon expands into contact with surrounding tissue of the anatomical passageway, thereby parking or securing the imaging probe 104 in place and / or filling the space between the imaging array and the tissue to be imaged with a fluid that facilitates imaging.

[0078] In some embodiments, the imaging array can be disposed within an inflatable balloon 119a, and the inflatable balloon can be inflated with a fluid that facilitates the use of the imaging medium, thereby reducing the volume of air or other gas between the imaging array and the patient's tissue. Inflating the balloon 119a in this manner can also park the imaging probe 104, fixing it in a desired position within the passageway. The balloon 119b can be disposed on the catheter 124 to help park the catheter.

[0079] In some examples, the inflatable balloon 119 can also be used to seal an anatomical passageway to facilitate aspiration or insufflation through a device such as the imaging probe 104 or catheter 124. In this regard, the balloon 119 can be used to block the passageway so that air can be aspirated from a portion of the passageway distal to the device, thereby collapsing the passageway. As an alternative to collapsing the passageway, a fluid or imaging gel can be injected into the portion of the passageway distal to the device after the balloon is inflated, thereby filling the portion of the passageway with a medium that facilitates imaging. As described, collapsing the passageway or filling the passageway with a fluid can eliminate or reduce any air volume that may impede imaging quality.

[0080] In some instances, balloon 119 can be used to maintain the imaging array in direct contact with the tissue. For example, in the case of a lateral imaging array, the balloon on one side of the imaging probe can be inflated, thereby pushing the imaging probe laterally into contact with the tissue. In response, the imaging array on the opposite side of the imaging probe from the balloon can be forced into direct contact with the tissue. In another example, in the case of a forward imaging array, the imaging device at the distal end of the imaging probe can be driven into direct contact with the tissue to be imaged. Then, the balloon 119a extending radially around the imaging probe can be inflated to contact the surrounding tissue, thereby fixing the imaging probe in place.

[0081] In this regard, it should be appreciated that an inflatable balloon may be advantageously used with any of the forward-pointing imaging arrays or side-pointing imaging arrays discussed herein.

[0082] Additional descriptions of examples of flexible, elongated instruments similar to the imaging probes and / or inflatable balloons discussed herein are described in U.S. Provisional Patent Application No. 63 / 240,471, filed September 3, 2021, disclosing “Ultrasound Elongate Instrument Systems And Methods,” which is hereby incorporated by reference in its entirety for all purposes. Fig. 6A 1 , but balloon 119 may be similarly applied to any example of the imaging probe described herein.

[0083] Fig. 7A A perspective view of the distal end of the imaging probe 104 is shown. Fig. 7A In the example shown, the imaging probe 104 is Fig. 6A The imaging probes are basically similar, but Figures 7A-9B The concepts described apply similarly to each of the examples of imaging probes described herein. Fig. 7AThe imaging array 114 in is shown as having six potential imaging planes 110a-110f, with each column 122a-122f of imaging elements 118 corresponding to one imaging plane. However, it should be understood that various combinations of imaging elements 118 can be activated by the controller to produce images along other potential imaging planes. In this regard, a greater number of potential imaging planes are possible, but for illustration purposes, only six potential imaging planes 110a-110f are shown. The tool 106 is shown as extending directly from the working channel 112 along the axis 115, as a rigid tool might occur. Therefore, all or a majority of the tool 106, including the distal tip 105, will be visible in each of the six imaging planes 110a-110f.

[0084] Figure 7B Shown in Fig. 7A An example of an image 130 that may be captured along one of the imaging planes 110a-110f (e.g., 110a) during use of the imaging probe 104 with the tool 106 in the configuration shown. With the tool 106 positioned near the target tissue 108, the image 130 may capture a majority of the longitudinal portion of the tool 106 in addition to the target tissue 108 and the vasculature 109.

[0085] Fig. 8A Shows Fig. 7A 104, wherein the tool 106 extends from the working channel 112 and bends away from the axis 115, as may occur with a flexible tool. Thus, the distal portion of the tool 106, including the distal tip 105, is not visible in the imaging plane 110a or one or more other potential imaging planes. Figure 8B Shown in Fig. 8A 106 is an example of an image 130 that may be captured along imaging plane 110a during use of imaging probe 104 with tool 106 in the configuration shown. Although target tissue 108 and vasculature 109 are visible, image 130 may only capture a proximal longitudinal portion of tool 106. Without any additional information about the curved shape of tool 106 or the position of distal tip 105, a user may view the tool 106 as being proximal to the distal tip 105. Figure 8B105. This misconception may cause the user to continue to advance the tool 106 distally from the working channel 112 in an attempt to reach the target tissue 108. However, because the tool 106 is curved, the user may inadvertently advance the tool 106 in a direction that does not intersect the target tissue 108 and enter tissue that may have to be avoided, such as a portion of the vasculature 109 that falls outside of the imaging plane 110a. In order to provide the user with a more informative imaging plane, it is advantageous to determine the position of the distal tip 105, for example using the positioning sensor 116, and select an imaging plane (e.g., one of the imaging planes 110a-110f) that will show the distal region of the tool 106 and / or other key features of interest.

[0086] Fig. 9A Shows Fig. 8AThe imaging probe 104 is provided with the tool 106 in a similarly bent configuration. The positioning sensor 116 can be used to determine the position of the distal tip 105 relative to the imaging array 114 to select the optimal imaging plane. Many examples of positioning sensors are contemplated. For example, the positioning sensor 116 can be a piezoelectric sensor (e.g., a PZT sensor) that is in operative communication with the controller via a wire (not shown) extending along the length of the tool 106 to the controller. For example, the wire can extend along the outer surface of the tool 106, can be embedded in the wall of the tool, or more can extend within the inner cavity of the tool. The piezoelectric sensor can be configured to detect acoustic signals transmitted by one or more imaging elements. Optionally, the positioning sensor 116 can include an optical fiber extending along the length of the tool 106. A light source (e.g., a laser) can transmit light through the optical fiber to a Bragg grating near the distal tip of the fiber. Light reflected back from the distal tip through the optical fiber can be analyzed by a sensor (e.g., a reflectometer) that communicates with the controller. The signals transmitted by the imaging element 118 can cause a wavelength change of the light in the fiber when they reach the distal tip of the optical fiber. As another alternative, the positioning sensor may include a capacitive micromachined ultrasonic transducer ("CMUT") configured to receive acoustic signals from and / or transmit acoustic signals to the imaging element 118. With respect to the latter case, it should be appreciated that the term "positioning sensor" as used herein may refer to a transmitter configured to transmit a signal that can be detected by an element positioned on the imaging probe. Any other type of positioning sensor may be utilized as long as it is suitable for performing the techniques of the present disclosure. In some instances, the positioning sensor 116 may include a WGM sensor configured to measure pressure waves from multiple imaging elements 118 of the imaging array 114 to triangulate the position of the WGM sensor and, in turn, the distal tip 105 of the tool 106. The WGM sensor may be attached to an optical fiber having a known laser wavelength so that changes in wavelength can be detected and correlated to pressure waves. In one example, three or more imaging elements disposed around the outer edge of the imaging array may be activated to triangulate the position of the WGM sensor relative to the known positions of the activated imaging elements (e.g., their positions in the imaging array and the position of the imaging array on the imaging probe 104). Fig. 10B The use of positioning sensor 116 to determine the position of distal tip 105 is described.

[0087] Once the position of distal tip 105 relative to the imaging array is determined, the controller can select an imaging plane that includes distal tip 105 or is closest to distal tip 105. Fig. 9A In the example shown, the imaging plane selected by the controller may be as follows: Fig. 7A The imaging plane 110c is shown. Fig. 9B Shown in Fig. 9AAn example of an image 130 that can be captured along an imaging plane 110c during use of the imaging probe 104 with the tool 106 in the configuration shown. As shown, a majority of the length of the tool 106 can be seen in the image 130 because the imaging plane 110c crosses the working channel 112 and intersects the tool 106 along most or all of the length of the tool 106 when the tool is bent in one direction. It will be appreciated that the tool can be bent in two orthogonal directions to form more complex curvatures that cannot be captured in a single imaging plane. Therefore, the controller can select an imaging plane that captures the most distal region of the tool and omits the more proximal region of the tool.

[0088] exist Fig. 9B In the image 130 shown in FIG. 1 , the controller may determine that although a large portion of the tool 106 is visible within the plane, the distal tip 105 may be out of the plane. Therefore, the controller may generate and display a virtual indicator 107 that displays a projection of the position of the distal tip 105 relative to the imaging plane. The virtual indicator 107 may include a numerical distance indication, such as the number of millimeters between the distal tip 105 and the imaging plane being displayed. A directional indication may also be provided to communicate whether the distal tip 105 is positioned proximal to the imaging plane (e.g., outside the page) or distal to the imaging plane (e.g., within the page). The directional indication may include, for example, a symbol, an arrow, or a color indicating a direction.

[0089] Although shown as having an arcuate shape, it should be understood that Fig. 7A , Fig. 8A and Fig. 9A Each imaging element 118 may have a square or rectangular shape, similar to Figure 3C In addition, regarding Figures 7A-9B The principles described and illustrated similarly apply to each example provided herein, including Figure 2A-Figure 6B The example shown in .

[0090] Fig. 10A A flowchart of an example of a method 1000 for generating an image is provided. The method 1000 is shown to be capable of being used in conjunction with Fig. 10A A set of operations or processes performed in the same or different order as shown. In some examples of the method, one or more of the processes shown may be omitted. In addition, Fig. 10A One or more processes not explicitly shown in the method 1000 may be included before, after, between, or as part of the processes shown. In some examples, one or more processes of method 1000 may be implemented at least in part by a control system executing code stored on a non-transitory, tangible, machine-readable medium that, when executed by one or more processors (e.g., a processor of the control system), may cause the one or more processors to perform the one or more processes.

[0091] After navigating the imaging probe (e.g., imaging probe 104) to a deployment position near the target tissue, at process 1002, the position of the distal tip of the tool (e.g., tool 106, such as a needle) is determined. This may include any suitable method for determining the position of the distal tip, including, but not limited to, triangulating the position of a positioning sensor disposed on the tool, determining the position of an EM position sensor on the tool, capturing multiple images with different imaging planes, and processing the images to identify the distal tip, etc. With regard to the latter, different imaging planes may be captured sequentially using a variety or all possible combinations of imaging elements to capture images in each plane. Image processing may then be performed to identify the best image that captures the distal tip of the tool or other portion of interest. Based on the known orientation of the imaging plane of the best image, the position of the distal tip may be determined from the best image. For this, a positioning or EM sensor on the tool may not be necessary. In some instances, only a subset of all possible imaging planes may be used to determine the position of the distal tip. For example, an imaging probe with 20 possible imaging planes may only utilize a subset of 4 or 5 imaging planes. If the distal tip is not captured in one of the images in the subset of imaging planes, the position of the distal tip may be determined by interpolating between the imaging planes.

[0092] At process 1004, the controller selects imaging elements of an imaging array on an imaging probe that will be used to generate an image including the distal tip of the tool. The selected imaging elements may be selected based on the determined position of the distal tip of the tool. The selected imaging elements may include all or a subset of the imaging elements of a particular imaging array. For example, a single imaging element column (e.g., Figure 6B In order to select a column 122a including imaging elements 118a-118f, two adjacent columns of imaging elements may be selected (e.g., for an imaging plane aligned with a gap between adjacent columns), or a subset of imaging elements distributed over multiple columns and / or rows may be selected.

[0093] At process 1006, the controller may continue to generate images using the selected imaging elements. If the generated image is determined to be unsatisfactory, for example, if the tool is not displayed or only a small portion is displayed, the controller may select a different subset of imaging elements and repeat the process. The selection of a different subset may be initiated automatically based on processing of the image data or in response to user input.

[0094] Thus, the controller controls at least a portion of the plurality of imaging elements to generate an image including an imaging plane including a position associated with a portion of the tool. The position may be the distal tip of the tool, or some other portion. The imaging elements may be controlled to generate images having only a determined position of a portion of the tool. Alternatively, the imaging elements may be controlled to generate a plurality of images in different image planes, and image processing may then be performed to identify an image including an imaging plane from the plurality of images, the imaging plane including a position associated with a portion of the tool. In some embodiments, generating an image via image processing may include generating a (e.g., 3D) model of the tool based on the plurality of images in different imaging planes, identifying the position of a desired portion of the tool (e.g., the distal tip) from the model, and selecting an image including the portion of the tool based on the identified position of the distal portion of the tool.

[0095] Fig. 10B Another example of a method 1050 for generating an image is provided, which can be similar to method 1000. Method 1050 is shown as being able to be used in conjunction with Fig. 10B A set of operations or processes performed in the same or different order as shown in the method. In some examples of the method, one or more of the processes shown may be omitted. In addition, Fig. 10B One or more processes not explicitly shown in the method 1050 may be included before, after, between, or as part of the processes shown. In some examples, one or more processes of method 1050 may be implemented at least in part by a control system executing code stored on a non-transitory, tangible, machine-readable medium that, when executed by one or more processors (e.g., a processor of the control system), may cause the one or more processors to perform the one or more processes.

[0096] After navigating an imaging probe (eg, imaging probe 104 ) to a deployment position proximate to the target tissue, in process 1052 , a tool (eg, tool 106 , such as a needle) is extended from a working channel of the imaging probe.

[0097] In process 1054, a controller in communication with an imaging array (e.g., imaging array 114) of an imaging probe selects a subset of imaging elements (e.g., imaging element 118) of the imaging array for performing positioning of a positioning sensor (e.g., sensor 116) and, in turn, positioning of a distal tip of the tool (e.g., distal tip 105) through a triangulation process. In some instances, the subset of imaging elements may be a default subset of imaging elements used for all positioning processes, or the subset may be selected based on attributes or characteristics of the current procedure (e.g., the type of tool utilized, the flexibility of the tool, the distance the tool has been extended from the imaging probe, the type of tissue in which the target tissue is located, etc.). If the tool is sufficiently rigid or the tissue is sufficiently soft, it may be acceptable to select a subset of only two imaging elements to determine the position of the positioning sensor and the distal tip. On the other hand, if the tool has a large flexibility, at least three imaging elements may be required to accurately determine the position of the positioning sensor and the distal tip. Reference Fig. 9A , the controller may select a subset of imaging elements including imaging elements 118a-118c. These specific imaging elements 118a-118c may be determined by the controller as being most suitable for positioning sensor 116 and distal tip 105 based on their position outside the imaging element ( Figure 6B 120a) and are spaced approximately 120° apart. In the discussion that follows, it will be appreciated that it may be desirable to select a maximum spacing between imaging elements used for positioning in order to improve positioning accuracy. In some instances, an additional element (e.g., a transducer) may be provided on the tool that is dedicated to positioning rather than using a subset of the imaging elements of the imaging array to generate an image. For example, the positioning elements may be interspersed with the imaging elements of the imaging array or provided around the periphery of the imaging array.

[0098] At process 1056, the controller activates the selected subset of imaging elements. The selected imaging elements may be activated sequentially in an order determined by the controller or in a default order. During process 1056, the first of the selected imaging elements may be "fired" to pass through the tissue between the imaging array and the positioning sensor (e.g., along the Fig. 9A The sound waves are transmitted along the travel path 128a-128c) of the positioning sensor. After reaching the positioning sensor, the positioning sensor detects the sound waves and transmits a signal to the controller indicating that the sound waves have been detected. The controller can record the time when the imaging element is activated and the corresponding time when the positioning sensor detects the sound waves. The controller can then excite the next selected imaging element.

[0099] Alternatively, such as when the positioning sensor comprises a CMUT, the controller may instruct the positioning sensor to transmit a signal and may activate selected imaging elements to detect the signal. The controller may record the time when the signal is transmitted and the time when each corresponding imaging element or positioning element receives the signal.

[0100] Regardless of the direction in which the signal is transmitted between the imaging element and the positioning sensor, in process 1058, the controller determines the position of the positioning sensor relative to the imaging array by triangulating the travel time of the corresponding signal between the selected imaging element and the positioning sensor. When the tissue type transmitting the corresponding signal is consistent, a longer travel time indicates a longer distance between the positioning sensor and the imaging element, while a shorter travel time indicates a shorter distance. When different types of tissue are arranged along different travel paths, the controller can adjust the recorded travel time to compensate for the different signal speeds through different tissues. Using known properties about the speed at which the signal travels through a specific tissue (one or more), the controller can convert the travel time to a distance. After determining the distance along each travel path, the controller can determine the position of the positioning sensor and, in turn, the position of the distal tip. In some instances, the controller can determine the position of the distal tip using a known fixed relationship between the distal tip and the positioning sensor, and in some instances, the positioning sensor can be arranged close enough to the distal tip so that the position of the distal tip can be assumed to be the position of the positioning sensor.

[0101] Once the position of the positioning sensor and / or distal tip has been determined, the controller selects an imaging plane (e.g., Fig. 9A Imaging plane 110c). The selected imaging plane can be a potential imaging plane that is closest to the positioning sensor or the distal tip of the tool. However, the controller can consider other features of interest when selecting the imaging plane. For example, the controller can consider the orientation of the target tissue and / or the orientation of the vascular system or other tissue to be avoided. In this regard, the controller can select an imaging plane that includes the distal tip or a portion of the tool near the distal tip, as well as one or more other features of interest. For example, in some cases, the distal tip may be determined to be located between two adjacent potential imaging planes. When selecting an imaging plane, the controller can determine that one of the two adjacent potential imaging planes includes the target tissue, while the other does not include the target tissue, and thus selects the imaging plane that includes the target tissue.

[0102] At process 1062, the controller identifies imaging elements in the imaging array that are used to generate an image of the selected imaging plane. The identified imaging elements may include all imaging elements of a particular array or a subset thereof. For example, a single imaging element column (e.g., Figure 6BThe controller may identify a column 122a including imaging elements 118a-118f, two adjacent columns of imaging elements may be identified (e.g., for an imaging plane aligned with a gap between adjacent columns), or a distributed subset of imaging elements may be identified. The controller determines the order in which the identified imaging elements should be activated, and may continue to generate an image using the identified imaging elements in process 1064.

[0103] If the resulting image is determined to be unsatisfactory, for example if only a small portion of the tool or target tissue is shown, the controller may select a different subset of imaging elements. The selection of a different subset may be initiated automatically based on processing of the image data or in response to user input.

[0104] Optionally, if the generated image indicates that the tool has strayed from the target tissue or is toward tissue that should be avoided, method 1050 may further include retracting the tool, repositioning the imaging probe, and repeating processes 1052-1062. If the generated image indicates that the tool is on an acceptable path to the target tissue, processes 1054-1062 may be repeated.

[0105] In some examples, the technology of the present disclosure, such as Figure 10A-10B The techniques discussed herein can be used in medical procedures performed using an imaging probe or catheter that can be handheld or otherwise manually controlled. In other examples, these techniques can be used in medical procedures performed using, for example, Figure 11-Figure 12 A medical procedure performed by a robotic-assisted medicine system is shown. Fig.11A clinical system 10 including a robotic-assisted medical system 1100 is shown. The robotic-assisted medical system 1100 generally includes a manipulator assembly 1102 for operating a medical instrument system 1104 (e.g., including an imaging probe 104 or a catheter 124) while performing various procedures on a patient P positioned on a table T in a surgical environment 1101. The manipulator assembly 1102 can be a robotic-assisted, non-assisted, or hybrid robotic-assisted and non-assisted assembly with selected degrees of freedom of motion that can be motorized and / or robotic-assisted and selected degrees of freedom of motion that can be non-motorized and / or non-assisted. A master control assembly 1106 (which may be located inside or outside the surgical environment 1101) generally includes one or more control devices for controlling the manipulator assembly 1102. The manipulator assembly 1102 supports the medical instrument system 1104 and may include a plurality of actuators or motors that drive inputs on the medical instrument system 1104 in response to commands from a control system 1112. The actuator may include a drive system that, when coupled to the medical instrument system 1104, can advance the medical instrument system 1104 into a natural or surgically formed anatomical orifice. Other drive systems can move the distal end of the medical instrument system 1104 in multiple degrees of freedom, which may include three degrees of linear motion (e.g., linear motion along the X, Y, Z Cartesian axes) and three degrees of rotational motion (e.g., rotation about the X, Y, Z Cartesian axes). In addition, the actuator can be used to actuate an articulated end effector of the medical instrument system 1104 to grasp tissue in the jaws of a biopsy device, etc.

[0106] The robot-assisted medical system 1100 also includes a display system 1110 (which can display Figure 1 100) for displaying an image or representation of the surgical site and the medical instrument system 1104 generated by the sensor system 1108 and / or the endoscopic imaging system 1109. The display system 1110 and the main control component 1106 can be oriented so that the operator O can control the medical instrument system 1104 and the main control component 1106 through the perception of the remote presentation.

[0107] In some examples, the medical instrument system 1104 may include components for surgery, biopsy, ablation, illumination, irrigation, or suction. The medical instrument system 1104, together with the sensor system 1108, may be used to collect (i.e., measure) a set of data points corresponding to positions within an anatomical passage of a patient (e.g., patient P). In some examples, the medical instrument system 1104 may include components of an endoscopic imaging system 1109, which may include an imaging mirror assembly or an imaging instrument (e.g., an imaging probe 104) for recording concurrent or real-time images of a surgical site and providing the images to an operator or operator O through a display system 1110. The concurrent image may be, for example, a two-dimensional or three-dimensional image captured by an imaging instrument located within the surgical site. In some examples, the endoscopic imaging system components may be integrally or detachably coupled to the medical instrument system 1104. However, in some examples, a separate endoscope attached to a separate manipulator assembly may be used with the medical instrument system 1104 to image the surgical site. The endoscopic imaging system 1109 may be implemented as hardware, firmware, software, or a combination thereof interacting with or otherwise executed by one or more computer processors (which may include a processor of the control system 1112 ).

[0108] The sensor system 1108 may include a position / orientation sensor system (eg, an electromagnetic (EM) sensor system) and / or a shape sensor system for determining the position, orientation, speed, velocity, posture, and / or shape of the medical instrument system 1104 .

[0109] The robotic-assisted medical system 1100 may also include a control system 1112. The control system 1112 includes at least one memory 1116 and at least one computer processor 1114 for implementing control between the medical instrument system 1104, the main control assembly 1106, the sensor system 1108, the endoscopic imaging system 1109, the intraoperative imaging system 1118, and the display system 1110. The control system 1112 (which may include a controller in operative communication with the imaging array 114 and / or the positioning sensor 116) also includes programming instructions (e.g., a non-transitory machine-readable medium storing instructions) to implement some or all of the methods described according to the aspects disclosed herein, including instructions for providing information to the display system 1110.

[0110] The control system 1112 may further include a virtual visualization system to provide navigation assistance to the operator O when controlling the medical instrument system 1104 during an image-guided surgical procedure. Virtual navigation using the virtual visualization system may be based on reference to a preoperative or intraoperative dataset of acquired anatomical pathways. The virtual visualization system processes images of the surgical site imaged using imaging techniques such as computed tomography (CT), magnetic resonance imaging (MRI), fluoroscopy, thermal imaging, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, and / or the like.

[0111] The intraoperative imaging system 1118 can be arranged in the surgical environment 1101 near the patient P to obtain images of the anatomical structures of the patient P during the medical procedure. The intraoperative imaging system 1118 can provide real-time or near real-time images of the patient P. In some examples, the intraoperative imaging system 1118 may include an ultrasound imaging system for generating two-dimensional and / or three-dimensional images. For example, the intraoperative imaging system 1118 may be at least partially incorporated into an imaging probe (such as the imaging probe 104). In this regard, the intraoperative imaging system 1118 can be partially or completely incorporated into the medical instrument system 1104.

[0112] Fig.12 The instrument reference system (X I , Y I , Z I)250, wherein a patient P is positioned on a table T. The patient P may remain stationary in the surgical environment in the sense that all movement of the patient is limited by sedation, restraints, and / or other means. Unless the patient is asked to hold his breath to temporarily stop respiratory movement, cyclic anatomical movement of the patient P (including respiratory and cardiac movement) may continue. Within the surgical environment 1200, an elongated instrument 1214 (e.g., part of the medical instrument system 1104, such as the imaging probe 104 or the catheter 124) is coupled to an instrument holder 1206. In this example, the elongated instrument 1214 includes an elongated instrument 1202 coupled to an instrument body 1212. The instrument holder 1206 is mounted to an insertion table 1208 that is fixed within the surgical environment 1200. Optionally, the insertion table 1208 may be movable but have a known position within the surgical environment 1200 (e.g., via a tracking sensor or other tracking device). In these alternatives, the instrument reference frame 250 is fixed or otherwise known relative to the surgical reference frame. The instrument bracket 1206 can be a component of a robot-assisted manipulator assembly (e.g., the robot-assisted manipulator assembly 1102) that couples with the elongated instrument 1214 to control insertion motion (i.e., motion along axis A) and motion of the distal end 1218 of the elongated instrument 1202 in multiple directions including yaw, pitch, and roll. The instrument bracket 1206 or insertion stage 1208 can include an actuator, such as a servo motor (not shown), that controls the motion of the instrument bracket 1206 along the insertion stage 1208.

[0113] like Fig.12 As shown, the instrument body 1212 is coupled and fixed relative to the instrument bracket 1206. In some examples, the fiber optic shape sensor 1204 is fixed at a proximal point 1216 on the instrument body 1212. In some examples, the proximal point 1216 of the fiber optic shape sensor 1204 can move with the instrument body 1212, but the position of the proximal point 1216 can be known (e.g., via a tracking sensor or other tracking device). In the instrument reference system (X I , Y I , Z I )250, the shape sensor 1204 measures the shape from a proximal point 1216 to another point such as a distal end 1218 of the elongated instrument 1202.

[0114] The elongated instrument 1202 includes a channel (not shown) sized and shaped to receive a medical instrument 1210. In some examples, the medical instrument 1210 can be used for procedures such as surgery, biopsy, ablation, illumination, irrigation, or aspiration. The medical instrument 1210 can be deployed through the elongated instrument 1202 and used at a target location within the anatomical structure. In examples where the elongated instrument 1202 includes a catheter (e.g., catheter 124), the medical instrument 1210 can include, for example, an imaging probe. In examples where the elongated instrument 1202 includes an imaging probe (e.g., imaging probe 104), the medical instrument 1210 can include a biopsy instrument, a laser ablation fiber, and / or other surgical, diagnostic, or therapeutic tools. The medical instrument 1210 can be advanced from the distal end 1218 of the elongated instrument 1202 to perform a procedure and then retracted into the channel when the procedure is completed. The medical instrument 1210 can be removed from the proximal end of the elongated instrument 1202 or from another instrument port (not shown) along the elongated instrument 1202.

[0115] The elongated instrument 1202 may house cables, linkages, or other steering controls (not shown) to controllably bend the distal end 1218. In some examples, at least four cables are utilized to provide independent "up and down" steering to control the pitch of the distal end 1218, and "side to side" steering to control the yaw of the distal end 1218.

[0116] The position measurement device 1220 provides information about the position of the instrument body 1212 as it moves along the insertion axis A on the insertion stage 1208. The position measurement device 1220 may include a rotary transformer, an encoder, a potentiometer, and / or other sensor that determines the rotation and / or direction of an actuator that controls the movement of the instrument bracket 1206, thereby determining the movement of the instrument body 1212. In some examples, the insertion stage 1208 is linear, while in other examples, the insertion stage 1208 may be curved or have a combination of curved and linear segments.

[0117] The advantages of the present disclosure will be understood and include direct visualization, which provides confirmation that a tool (such as a needle) is inserted into the target tissue and avoids harm to the vascular system when the tool is bent beyond the imaging plane. In this regard, if the tip of the needle deviates from the imaging plane during insertion, a different imaging plane can be selected and a new image can be generated. In addition, the systems and methods described herein ensure that the portion of the tool visible in the image is the distal tip or at least close to the distal tip. In addition, the described technology provides reduced image processing delays. Instead of exciting all imaging elements through all imaging planes to try to locate the distal tip of the tool, the position is determined, and only the imaging elements required to produce the optimal imaging plane can be excited. It should also be appreciated that the annular imaging array used for the forward method can provide direct alignment of the imaging probe with the target tissue, and there may be less frictional resistance to the deployment of the tool compared to the lateral method (where the tool can be greatly bent to leave through the side port).

[0118] In the description, the specific details describing some examples have been set forth. In order to provide a thorough understanding of the examples, many specific details are proposed. However, it will be appreciated by those skilled in the art that some examples may be practiced without some or all of these specific details. The specific examples disclosed herein are intended to illustrate rather than to limit. Those skilled in the art may realize other elements, although not specifically described herein, but within the scope and spirit of the present disclosure.

[0119] Whenever feasible, elements described in detail with reference to one example, embodiment or application may optionally be included in other examples, embodiments or applications that are not specifically shown or described therein. For example, if an element is described in detail with reference to one example, but not described with reference to a second example, the element may still be required to be included in the second example. Therefore, in order to avoid unnecessary repetition in the following description, unless otherwise specifically stated, one or more elements shown and described in association with one example, embodiment or application may be incorporated into other examples, embodiments or aspects unless the one or more elements render the example or embodiment inoperative, or unless two or more elements provide conflicting functions.

[0120] Any changes and further modifications to the devices, instruments, methods, and any further applications of the principles of the present disclosure are generally expected by those skilled in the art associated with the present disclosure. Specifically, it is fully expected that the features, components and / or steps described in relation to one example can be combined with the features, components and / or steps described in relation to other examples of the present disclosure. In addition, the scales provided herein are for specific examples, and it is envisioned that different sizes, scales and / or ratios can be utilized to implement the concepts of the present disclosure. In order to avoid unnecessary repetitive descriptions, one or more components or actions described in accordance with an exemplary example can be appropriately utilized or omitted in other exemplary examples. For the sake of brevity, the numerous iterations of these combinations will no longer be described separately. For the sake of simplicity, in some cases, the same reference numerals are utilized throughout the accompanying drawings to refer to the same or similar parts.

[0121] Although some examples about medical procedures are provided herein, any reference to medical or surgical instruments and medical or surgical methods is non-limiting. For example, the apparatus, system and method described herein can be used for non-medical purposes, including industrial use, general robot use and sensing or manipulating non-tissue workpieces. Other example applications include cosmetic improvement, imaging of human or animal anatomical structures, collecting data from human or animal anatomical structures, and training medical or non-medical personnel. Other example applications include procedures for tissues removed from human or animal anatomical structures (not returning to human or animal anatomical structures), and performing procedures on human or animal corpses. In addition, these technologies can also be used for surgical and non-surgical medical treatment or diagnostic procedures.

[0122] The methods described herein are shown as a set of operations or processes. Not all of the processes shown may be performed in all instances of the method. In addition, one or more processes not explicitly shown or described may be included before, after, between, or as part of the instance process. In some instances, one or more processes may be performed by a control system (e.g., control system 1112), or at least partially implemented in the form of executable code stored on a non-transient, tangible, machine-readable medium, which, when run by one or more processors (e.g., processor 1114 of control system 1112), may cause the one or more processors to perform one or more processes.

[0123] Any "imaging device" described herein may include an ultrasound array, an optical imaging device, or any other suitable imaging hardware. Any described "imaging probe" may include an ultrasound probe, an optical imaging probe, or a probe incorporating any other suitable imaging modality. In addition, any "ultrasound array," "imaging array," or "imaging device" described herein may include a single imaging element (e.g., a transducer) or a plurality of such devices.

[0124] One or more elements in the examples of the present disclosure can be implemented in software to execute on a processor of a computer system such as a control processing system. When implemented in software, the elements of the examples of the present invention are essentially code segments that perform necessary tasks. The program or code segment can be stored in a processor-readable storage medium or device, which can be downloaded by a computer data signal embodied in a carrier wave on a transmission medium or a communication link. The processor-readable storage device may include any medium that can store information, including optical media, semiconductor media, and magnetic media. Processor-readable storage device examples include electronic circuits, semiconductor devices, semiconductor storage devices, read-only memories (ROMs), flash memories, erasable programmable read-only memories (EPROMs), floppy disks, CD-ROMs, optical disks, hard disks, or other storage devices. The code segment can be downloaded via a computer network (such as the Internet, intranets, etc.). Any of a variety of centralized or distributed data processing architectures can be used. Programming instructions can be implemented as multiple separate programs or subroutines, or they can be integrated into multiple other aspects of the system described herein. In one example, the control system supports wireless communication protocols such as Bluetooth, IrDA, HomeRF, IEEE 802.11, DECT, and wireless telemetry.

[0125] Note that the processes and displays presented may not be inherently related to any specific computer or other device. Various general purpose systems may be utilized with the programs according to the teachings herein, or it may prove convenient to build more specialized equipment to perform the described operations. The desired structures of various such systems will appear as elements in the claims. In addition, examples of the present invention are not described with reference to any specific programming language. It is understood that various programming languages ​​can be used to implement the teachings of the present invention as described herein.

[0126] In some cases, well-known methods, procedures, components and circuits are not described in detail so as not to unnecessarily obscure various aspects of the examples. The present disclosure describes various instruments, instrument parts and anatomical structures according to their states in three-dimensional space. As used herein, the term "position" refers to the orientation of an object or a portion of an object in three-dimensional space (e.g., three translational degrees of freedom along Cartesian x, y and z coordinates). As used herein, the term "orientation" refers to the rotational placement of an object or a portion of an object (three rotational degrees of freedom - for example, roll, pitch and yaw). As used herein, the term "pose" refers to the position of an object or a portion of an object in at least one translational degree of freedom, and the orientation of the object or a portion of an object in at least one rotational degree of freedom (up to six total degrees of freedom). As used herein, the term "shape" refers to a set of poses, positions or orientations measured along an object.

[0127] While certain exemplary embodiments of the present invention have been described and illustrated in the accompanying drawings, it should be understood that these embodiments are merely illustrative of the broad invention and not restrictive, and that the embodiments of the present invention are not limited to the specific construction and arrangements shown and described, since various other modifications may occur to those skilled in the art.

Claims

1. A system comprising: A flexible and elongated device, comprising: a working channel extending to an opening in a distal portion of the flexible elongated device; and Multiple imaging elements; a tool extendable through the working channel and the opening; and A controller comprising one or more processors configured to: At least a portion of the plurality of imaging elements are controlled to generate an image including an imaging plane including a location associated with a portion of the tool. 2 . The system of claim 1 , wherein the one or more processors are further configured to: determine the position associated with the portion of the tool prior to generating the image.

3. The system of claim 2, wherein determining the position relative to the portion of the tool comprises: capturing a plurality of images using different subsets of the plurality of imaging elements, each image in the plurality of images having a different imaging plane, and The images are processed to identify the portion of the tool within one or more of the plurality of images.

4. The system of claim 1, wherein the plurality of imaging elements comprises an annular array of imaging elements disposed about the opening, the annular array comprising at least two rows of imaging elements.

5. The system of claim 1, wherein the plurality of imaging elements comprises: an array of ultrasonic transmitters configured to generate ultrasonic waves; as well as An array of ultrasonic sensors configured to detect reflected ultrasonic waves.

6. The system of claim 5, wherein the ultrasound sensor comprises a whispering gallery mode resonator (WGM resonator).

7. The system of claim 1, wherein the plurality of imaging elements comprises an array of transducers configured to generate and detect ultrasonic waves.

8. The system of claim 1, wherein the plurality of imaging elements comprises a grid array in which the plurality of imaging elements are arranged in at least two rows and at least two columns.

9. The system of claim 1, wherein: The plurality of imaging elements are configured to capture an image including an imaging plane defined within an imaging field distal to the distal portion of the flexible elongated device; as well as The working channel extends through a distal surface of the flexible elongated device to guide the tool into the imaging field.

10. The system of claim 1, wherein: The plurality of imaging elements are configured to capture an image including an imaging plane defined within an imaging field radially outward of an outer side surface of the flexible elongated device; and The working channel extends through a side wall of the flexible elongated device to guide the tool into the imaging field.

11. The system of claim 1, wherein the portion of the tool comprises a distal tip of the tool.

12. The system of claim 2, wherein: The controller is further configured to determine a second position associated with one of a second position of the tool or a region of interest in the target tissue; and The imaging plane includes both the position and the second position.

13. The system of claim 1, wherein the flexible, elongated device further comprises a balloon configured to be inflated with a fluid to enable acoustic transmission between the plurality of imaging elements and tissue.

14. The system of claim 1, wherein the tool comprises at least one of a biopsy needle, a therapy delivery needle, an ablation device, or a cryogenic device.

15. The system of claim 1, wherein at least the portion of the tool is flexible.

16. The system of claim 2, wherein the tool includes a positioning sensor located at the portion of the tool.

17. The system of claim 16, wherein the positioning sensor comprises a fiber optic sensor configured to detect wavelength changes of light in a fiber in response to pressure waves generated by the plurality of imaging elements.

18. The system of claim 16, wherein the positioning sensor comprises a WGM resonator.

19. The system of claim 16, wherein the positioning sensor comprises a piezoelectric sensor.

20. The system of claim 16, wherein the positioning sensor comprises a capacitive micromachined ultrasound transducer.

21. The system of claim 16, wherein the controller is configured to determine the position of the portion of the tool by: sequentially activating subsets of the plurality of imaging elements to transmit pressure waves; detecting a pressure wave from each imaging element of the subset; determining a travel time of the pressure wave from each imaging element of the subset to the positioning sensor; as well as The position of the portion of the tool is triangulated based on the travel time of each pressure wave from the corresponding imaging element to the positioning sensor.

22. The system of claim 21, wherein the plurality of imaging elements are arranged in an annular array and the subset comprises three of the plurality of imaging elements, the three imaging elements being spaced 120° apart around the perimeter of the annular array.

23. The system of claim 16, wherein the controller is configured to determine the position of the portion of the tool by: activating the positioning sensor to transmit a pressure wave; activating a subset of the plurality of imaging elements to detect the pressure wave; determining a travel time of the pressure wave from the positioning sensor to each imaging element of the subset; as well as The position of the portion of the tool is triangulated based on the travel time of the pressure wave from the positioning sensor to each imaging element of the subset.

24. The system of claim 1, wherein the controller is configured to control the at least a portion of the plurality of imaging elements to generate the image by determining a firing sequence of the at least a portion of the plurality of imaging elements based on the imaging plane.

25. The system of claim 24, wherein the firing sequence includes a time delay between successive firings of imaging elements, wherein the time delay is determined based on a geometry of an imaging field of the image relative to positions of the plurality of imaging elements.

26. The system of claim 1, wherein the controller is further configured to: associating sets of imaging elements with different image planes; determining the imaging plane to be closest to the location associated with the portion of the tool; and The image is generated using a set of imaging elements associated with the determined imaging plane.

27. The system of claim 1, wherein a longitudinal axis of the working channel is oriented non-parallel to the imaging plane.

28. The system of claim 1, wherein at least a portion of the tool includes surface enhancements to alter reflections of ultrasound waves from the plurality of imaging elements.

29. A method for capturing an image performed by a controller including one or more processors, the method comprising: Controlling at least a portion of a plurality of imaging elements of a flexible, elongated device to produce an image including an imaging plane including a position associated with a portion of a tool at least partially located in a working channel of the flexible, elongated device, the working channel extending to an opening in a distal portion of the flexible, elongated device.

30. The method of claim 29, further comprising: The position relative to the portion of the tool is determined.

31. The method of claim 30, wherein determining the position relative to the portion of the tool comprises: capturing a plurality of images using different subsets of the plurality of imaging elements, each image in the plurality of images having a different imaging plane; as well as The images are processed to identify the portion of the tool within one or more of the plurality of images.

32. The method of claim 29, further comprising: A virtual indicator is displayed that provides a visual representation of the position of the portion of the tool relative to the imaging plane.

33. The method of claim 32, wherein the virtual indicator comprises a distance indication providing a visual representation of the distance between the portion of the tool and the imaging plane.

34. The method of claim 32, wherein the virtual indicator comprises a direction indication providing a representation of the orientation of the portion of the tool relative to the imaging plane.

35. The method of claim 34, wherein the directional indication comprises displaying a color of at least a portion of the virtual indicator.

Images