On-screen marker for out-of-plane needle guidance

Abstract

To provide an ultrasound imaging system including an image processor configured to receive input data for capturing ultrasound images of a region of interest.SOLUTION: An ultrasound image is taken along a first plane, and input data further includes an indication that a needle will be inserted into the region of interest. The system can capture a plurality of ultrasound images of the region of interest along the first plane. The system can determine one or more high-confidence areas of the region of interest where the needle will intersect the first plane. Each high-confidence area is based on a probability that the needle will intersect the first plane at any portion of the high-confidence area, and displays one or more on-screen markers corresponding to the one or more high-confidence areas in conjunction with the plurality of ultrasound images on the display.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Patent Application No. 16 / 440,676, filed June 13, 2019, which is incorporated by reference in its entirety.

[0002] The disclosed technology relates to ultrasound imaging systems, and more particularly to ultrasound imaging systems for depicting the position of an interventional instrument, such as a needle, within an anatomical structure. [Background technology]

[0003] Ultrasound imaging is becoming increasingly accepted as the standard of care to be used when guiding an interventional instrument, such as a needle, to a desired location within the body. One common application of this procedure is during the application of anesthesia, whereby a physician or medical technician ("operator") views an ultrasound image to aid in guiding a needle to a desired area of ​​interest. The relative position of the displayed ultrasound image to the needle can be one of two configurations: (1) "in-plane," where the longitudinal axis of the needle is within the plane of the ultrasound image (i.e., the needle appears as a linear structure), or (2) "out-of-plane," where the needle is perpendicular to the plane of the ultrasound image (i.e., the needle appears as a dot). With a mechanical needle guide, the angle of the needle relative to the ultrasound transducer (and thus the skin line) is fixed so that its trajectory is predictable. As a result, the boundary of the expected needle intersection with the plane of the ultrasound image (taking into account the mechanical variability of the needle guide) can be delineated by parallel or slightly diverging lines for in-plane applications, and by a circle or ellipse for out-of-plane applications. In certain embodiments, out-of-plane needle guidance is desirable so that the operator can view anatomical structures, such as blood vessels, relative to the needle position.

[0004] In certain embodiments, out-of-plane needle guidance, as described above, only shows the position of the needle at the point where it intersects the plane of the image. In certain embodiments, the operator may not be able to determine the position of the needle relative to the anatomical structure of interest until the needle intersects the plane, meaning that for some time after the needle is initially inserted, the operator cannot reliably predict where the needle will intersect the plane. In certain embodiments, ideally the needle will appear on the ultrasound image once it reaches the anatomical feature of interest. As an example and not by way of limitation, the operator may be able to see the needle on the ultrasound image once the tip of the needle reaches the target vessel.

[0005] To ensure better placement of the needle for out-of-plane guiding, fixed angle needle guides are used. These guides are fixed (e.g., by a bracket) to the ultrasound transducer and hold the needle at a specific position and angle relative to the ultrasound transducer, which means that the needle's position relative to the ultrasound image (and therefore relative to the anatomical feature of interest) is more precisely known, reducing the chance of operator error in needle placement. However, there are still other factors that reduce the accuracy of needle placement in out-of-plane guiding. For example, but not by way of limitation, mechanical play between the needle and the guide, the fit of the bracket on the ultrasound transducer housing, the alignment of the ultrasound transducer array within the transducer housing, the tissue density of the anatomical feature, the thickness and profile of the ultrasound image plane, and the deflection of the needle shaft traveling to the target feature can all affect the exact location of the needle as it intersects the plane of the ultrasound image.

[0006] In certain embodiments, based on a preset angle and type of needle, the ultrasound imaging system can predict where the needle will intersect the imaging plane and project an on-screen marker or other indicator onto the ultrasound image to notify the user. However, even this on-screen marker can be subject to other factors that affect the needle's trajectory. Therefore, the accuracy of the measurements may be inaccurate and may not provide a perfect indication of how likely it is that the needle will intersect the image at the indicated location. [Brief description of the drawings]

[0007] [Figure 1] 1 shows a simplified diagram of an example ultrasound imaging system with out-of-plane needle guidance. [Diagram 2] FIG. 1 shows a block diagram of an exemplary ultrasound imaging system for out-of-plane needle guidance. [Figure 3A] 1 shows an example set of on-screen depth markers indicating various depths through the plane of an ultrasound image. [Figure 3B] 1 shows an example set of on-screen depth markers indicating various depths through the plane of an ultrasound image. [Figure 4A] The relationship between the needle insertion angle and the ultrasound image plane is shown. [Figure 4B] The relationship between the needle insertion angle and the ultrasound image plane is shown. [Figure 4C] 1 shows an exemplary diagram of on-screen markers corresponding to high-confidence regions. [Figure 4D] 1 shows an exemplary diagram of on-screen markers corresponding to high-confidence regions. [Figure 4E] 1 shows an exemplary diagram of on-screen markers corresponding to high-confidence regions. [Figure 4F] 1 shows an exemplary diagram of on-screen markers corresponding to high-confidence regions. [Figure 4G] 1 shows an exemplary diagram of on-screen markers corresponding to high-confidence regions. [Figure 4H] 1 shows an exemplary diagram of on-screen markers corresponding to high-confidence regions. [Diagram 5] FIG. 2 is a diagram showing an example of an elevation profile of an ultrasound beam relative to a transducer. [Figure 6A] 1 shows the relationship between needle insertion angle and elevation angle profile of the ultrasound image plane. [Figure 6B] 1 shows the relationship between needle insertion angle and elevation angle profile of the ultrasound image plane. [Figure 7] 1 illustrates an exemplary method for displaying on-screen markers on an ultrasound imaging system, the on-screen markers corresponding to areas with high confidence that an out-of-plane needle will intersect the ultrasound image. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] As described in further detail below, the disclosed technology relates to improvements in ultrasound imaging systems, and in particular to ultrasound imaging systems configured to calculate and display on-screen markers on an ultrasound image that correspond to the probability that an out-of-plane needle insertion will intersect the plane of the ultrasound image at the location of the marker. Although the present disclosure uses the term "needle" to refer to an object being inserted from an out-of-plane direction, which may include, but is not limited to, injection needles, biopsy needles, needles for suturing tissue, and needles for withdrawing fluids (e.g., amniocentesis), other devices may also be imaged, such as robotic surgical instruments, catheters, guidewires, or other invasive medical instruments.

[0009] In certain embodiments, an ultrasound imaging system can generate and transmit multiple ultrasound beams to an area to be imaged. By way of example and not limitation, the area to be imaged can be organic tissue, a living animal, a portion of an animal or human anatomy, or any other suitable medium that can be measured via ultrasound imaging. For instructional purposes, this disclosure refers to the area to be imaged as an anatomical structure. In certain embodiments, the ultrasound imaging system can image not only the target anatomical structure, but also other objects present within the anatomical structure, such as a needle.

[0010] FIGURE 1 depicts an exemplary ultrasound imaging system implementing the disclosed techniques for imaging tissue of a patient, with the ability to perform out-of-plane needle guidance. In certain embodiments, the ultrasound imaging system 10 may be a handheld, portable, or cart-based system that uses a transducer 12 to transmit ultrasound signals into an area of ​​interest and receive corresponding echo signals to generate an image of the tissue being scanned. The transducer 12 may be a one or two dimensional linear or curved transducer, or a phased array transducer, all of which are capable of electronically selectively varying the transmit beam angle.

[0011] The ultrasound imaging system 10 can convert the characteristics of the received echo signals (e.g., their amplitude, phase, power, frequency shift, etc.) into data that can be quantified and displayed for a user as an image. The images created can be stored electronically for digital record keeping or transmitted via a wired or wireless communication link to another device or location. In certain embodiments, an operator can secure an out-of-plane needle guidance system 16 to the transducer 12, where the needle guidance system 16 can hold a needle 15 for insertion into a target area along a plane perpendicular to the plane displayed for imaging. In certain embodiments, the needle guidance system 16 can allow for a variety of needles (e.g., gauges or lengths) and insertion angles such that the needle intersects the plane of the ultrasound image at a desired depth within the target area. The ultrasound imaging system 10 can generate a composite view 22 of the target area and the needle 24 as it intersects the plane of the target area.

[0012] In certain embodiments, a fixed angle out-of-plane needle guidance system 16 may be used to perform an out-of-plane needle procedure. As an example and not by way of limitation, an out-of-plane needle procedure may be desirable when an operator is attempting to place a needle along a structure such as a blood vessel, and the out-of-plane view allows the operator to visualize the cross-section of the vessel wall and the needle. In certain embodiments, since the out-of-plane needle guidance system 16 is affixed to the ultrasound transducer 12, the ultrasound imaging system may be able to determine, based on factors such as the angle of the guidance system and the type of needle where the needle will intersect the plane of the image. The ultrasound imaging system 10 may display the needle 24 in a composite view 22, as shown in FIG. 1. As will be appreciated by those skilled in the art, there may be a variety of out-of-plane needle guidance systems available to ensure that the needle is in a fixed position relative to the ultrasound transducer.

[0013] In certain embodiments, the out-of-plane needle guidance system 16 may include a bracket that may be mounted on an ultrasound transducer. In certain embodiments, the out-of-plane needle guidance system 16 may utilize multiple tabs, each tab corresponding to a particular needle gauge or a particular depth desired by the operator. In certain embodiments, the operator may select the appropriate tab for a particular procedure, remove it from the multiple tabs, and snap the tab onto the bracket to form the guide at the desired target depth. In certain embodiments, the out-of-plane needle guide may further include a quick release system that allows the needle to be removed. In certain embodiments, once the needle is removed, the selected tab may also be removed and discarded.

[0014] In certain embodiments, the mechanical play between the needle 15 and the out-of-plane needle guidance system 16 may be up to 1.5 degrees, meaning that the angular change of the needle relative to its intended trajectory may be as much as 1.5 degrees. In certain embodiments, given this uncertainty in the needle angle, the potential magnitude of the needle's displacement from the projected point may increase the deeper the needle goes. In certain embodiments, the amount of mechanical play may be known based on the needle used and the particular tab used for that needle. As an example and not by way of limitation, it may already be known that for a particular needle, a 1 cm deep tab will allow the needle to move 0.5 degrees, and a 2.5 cm deep tab will allow the needle to move 1.0 degrees.

[0015] FIG. 2 illustrates a simplified block diagram of an ultrasound imaging system in accordance with one embodiment of the disclosed technology. In certain embodiments, the ultrasound system may be comprised of different components than those illustrated. Additionally, the ultrasound system may include parts (e.g., power supplies, etc.) that are not illustrated and are not necessary to understand how to make and use the disclosed technology. In the illustrated embodiment, the ultrasound system includes an ultrasound imaging system, as described in detail below. The ultrasound system may include a processor 40 having an internal or external memory (not shown) that includes instructions executable by the processor to operate the ultrasound imaging system. In the transmit path, the ultrasound system may include a transmit beamformer 42, a transmit gain control amplifier 44, and a transmit / receive switch 46. If the ultrasound transducer 12 is of the phased array type or is otherwise capable of electronically varying the transmit angle, the transmit beamformer 42 may be operative to generate a number of signals having relative amplitudes and phases (timing) selected to generate ultrasound beams from some or all of the transducer elements of the transducer that constructively add to a desired transmit beam direction (desired transmit beam angle). The signals from the transmit beamformer may be amplified by the transmit amplifier 44 to a sufficiently high voltage level that will cause the transducer elements to generate a desired acoustic signal in the tissue being examined. In certain embodiments, the processor 40 is connected to provide a control command, such as a digital value (e.g., 0-255), to the gain control amplifier. The value of the command may control the amount of gain provided by the transmit amplifier 44.

[0016] Other techniques for adjusting the power of the ultrasound signal may include modifying the waveform that drives the transducer elements to increase or decrease the power of the ultrasound signal. In certain embodiments, the voltage rails (+V, -V) of the amplifier that generates the drive signal may be altered to vary the power of the ultrasound signal. In certain embodiments, the drive signal may be provided to fewer or more transducer elements to vary the power of the ultrasound signal. Those skilled in the art will appreciate that these techniques are merely exemplary and that there may be many ways in which the level of acoustic power of the ultrasound signal delivered to the patient may be adjusted.

[0017] In certain embodiments, the amplified transmit signal may be fed to the transducer 12 through a transmit / receive switch 46, which disconnects or shields the sensitive receive electronics from the transmit signal when it is delivered to the transducer 12. After the signal is transmitted, the transmit / receive switch 46 can connect the receive electronics to the transducer elements to detect corresponding electronic echo signals that are generated when returning sound waves impinge on the transducer elements.

[0018] In a particular embodiment, in the receive path, the ultrasound imaging system may include a low noise amplifier 50, a time gain control amplifier 52, an analog-to-digital converter 54, a receive beamformer 56, and an image processor 58. Analog echo signals generated by the imaging transducer may be routed through the transmit / receive switch 46 to the low noise amplifier where they may be amplified. The TGC amplifier 52 may apply variable amplification to the receive signal, varying the level of amplification applied with the return time of the signal (e.g., proportional to the depth within the tissue being imaged) to counteract signal versus depth attenuation. The amplified signal may then be converted to a digital format by the analog-to-digital converter 54. The digitized echo signals may then be delayed and summed by the receive beamformer 56 before being provided to the image processor.

[0019] In certain embodiments, the number of transmit and receive beams (lines) may differ from one another. As an example and not by way of limitation, a receive beamformer may generate two or more adjacent lines per transmit beam in parallel (i.e., simultaneously), a technique sometimes known as parallel receive beamforming or multiline processing. Multiline processing may be used to increase imaging frame rates by lowering the number of transmit beams while the number of receive lines per frame (line density) can be kept constant. In certain embodiments, a higher multiline order (the number of receive line beams formed in parallel from a single transmit beam) may be used to increase the number of transmit beams, and therefore the frame rate. The number of receive lines per frame can be increased while keeping the frame rate constant. Other combinations of line density, frame rate and multi-line order are also possible. Additionally, it may be possible to transmit an unfocused beam (plane wave) and beamform all receive lines of a frame from that single transmit beam. The system may also use different combinations of line density and multi-line order for imaging tissue versus imaging an interventional instrument. While improving the frame rate in certain embodiments, using a higher multi-line order, lower line density, or unfocused transmit beam may degrade the quality of the acquired image.

[0020] Images generated by the image processor 58 from the received signals may be displayed on a display 60. Additionally, images may be stored in an image memory (not shown) for future recall and review. Several inputs 72 may be provided to allow an operator to modify various operating parameters of the ultrasound imaging system and enter data such as the patient's name or other record keeping data. Additionally, the ultrasound imaging system may include input / output (I / O) circuitry that allows the system to connect to a computer communications link (LAN, WAN, Internet, etc.) via a wired (e.g., Ethernet, USB, Thunderbolt, Firewire, etc.) or wireless (802.11, cellular, satellite, Bluetooth, etc.) communications link.

[0021] The details of the components that make up an ultrasound imaging system and how they operate can generally be considered to be known to those skilled in the art. Although the ultrasound imaging system is shown as having many separate components, devices such as ASICs, FPGAs, digital signal processors (DSPs), CPUs, or GPUs can be used to perform the functions of several of these individual components.

[0022] As discussed above, the processor 40 may be programmed to generate a composite image of the tissue being inspected and the needle being introduced into the tissue. In certain embodiments, the image processor generates an anatomical image of the tissue being inspected along with imaging parameters selected for the depth and particular type of tissue being scanned. The anatomical image generated by the image processor 58 may be stored in memory to be combined with echo data for one or more of the generated needle frames to locate the position of the interventional instrument.

[0023] 3A-3B show examples of existing on-screen depth markers for an out-of-plane needle guidance system. In the example of FIG. 3A, an ultrasound image 300 may depict an anatomical structure of interest 310, while a needle is inserted into the anatomical structure 310 from a direction perpendicular to the plane of the ultrasound image 300. In the example of FIG. 3, the ultrasound imaging system may further display a set of on-screen depth markers 321-325, each corresponding to a particular location where the needle may intersect the ultrasound plane for a given depth of the needle. As an example and not by way of limitation, depth marker 321 corresponds to a depth of 0.5 cm, depth marker 322 corresponds to 1.0 cm, depth marker 323 corresponds to 1.5 cm, depth marker 324 corresponds to 2.0 cm, and depth marker 325 corresponds to 2.5 cm. In the example of FIG. 3, depth markers 322 and 324 appear larger than depth markers 321, 323, and 325 to aid the operator in distinguishing between the series of depth markers. In certain embodiments, the depth markers may be accompanied by a numerical value indicating the depth corresponding to each depth marker. In certain embodiments, depth markers 321-325 may be automatically generated by the ultrasound imaging system based on an indication by the operator that the operator wishes to perform an out-of-plane needle insertion procedure, with each depth marker 321-325 automatically set by the ultrasound imaging system. In certain embodiments, the operator may select one or more depth markers to display and one or more depth markers not to display. As an example and not by way of limitation, if the operator knows that they are attempting an out-of-plane needle insertion at a depth of 2.0 cm, the operator manually selects only depth marker 324 to be displayed, and not depth markers 321-323 and 325. This can reduce clutter on the ultrasound image and make ultrasound image 300 clearer to the operator to avoid confusion.

[0024] FIG. 3B is used in the example of FIG. 3A, but shows an ultrasound image with visualization of the needle 24 intersecting the plane of the ultrasound image 300. In the example of FIG. 3B, continuing the example discussed above with respect to FIG. 3A, the operator may select the appropriate needle guide and needle for a depth of 1.5 cm corresponding to the depth marker 323. In certain embodiments, known methods may be used for the ultrasound imaging system to automatically detect the needle in the ultrasound image to determine that a needle is currently present in the exemplary ultrasound image 300 of FIG. 3B. In certain embodiments, when the operator perceives the needle on the ultrasound image 300, further input may be provided to the ultrasound imaging system. In certain embodiments, once the needle is detected, either automatically or by the operator, the ultrasound imaging system may remove the on-screen depth markers 321-325 or otherwise reduce their visibility on the ultrasound image 300, for example, by making one or more of the on-screen depth markers 321-325 more transparent. In the example of FIG. 3B, the needle 24 is displaced in small increments from the on-screen depth marker 323 corresponding to the depth and target of interest. In certain embodiments, the displacement may be due to a number of factors, including, but not limited to, needle deflection, mechanical play between the needle and needle guide or the needle guide and the ultrasound transducer, or tissue density. The example of FIG. 3B can highlight the potential downside to using only on-screen depth markers for the target depth, where the needle may diverge from the predicted location for any of a number of reasons.

[0025] In certain embodiments, the ultrasound imaging system may be configured to calculate the probability that an out-of-plane needle insertion will intersect the plane of the ultrasound image in a particular region. Multiple factors may affect the probability of intersection. For example, as described above, mechanical play between the needle and the guidance system may contribute to uncertainty based on the angle of the degrees of freedom that the needle has. Other factors that may affect the needle trajectory and may be used as inputs by the ultrasound imaging system may include the fit of the bracket on the transducer housing, the alignment of the transducer array within the transducer housing, potential deflection of the needle as it progresses through the tissue, the physical characteristics of the needle, the depth of needle placement, the height profile of the ultrasound beam, and tissue density. In certain embodiments, one or more of these factors may be combined by the ultrasound imaging system to determine a high-confidence region in the ultrasound image where the likelihood of the needle intersecting within the high-confidence region exceeds a threshold. In certain embodiments, the threshold may possibly be a probability of intersection exceeding 85%, 90%, 95%, 99%, or 99.9%. As an example and not by way of limitation, an ultrasound imaging system may determine a confidence region where there is a 99% probability that the needle will intersect the ultrasound image plane through a point within the confidence region.

[0026] In certain embodiments, tissue density and needle deflection can affect needle trajectory and have a compounding effect. As an example and not by way of limitation, tissue density can affect the amount of deflection a needle can have as it advances through tissue. A needle being inserted through muscle tissue can be affected by a greater resistance than a needle being inserted through fat tissue, and thus a needle passing through muscle can experience a greater amount of deflection. Additionally, in certain embodiments, shallower target depths can utilize thinner needles, and deeper target depths can accommodate thicker needles. In certain embodiments, tissue density can also affect the accuracy of ultrasound imaging of the needle. Different types of tissues may have different densities and water contents, and even tissues of the same type (e.g., muscle) can have different densities and water contents based on the patient, the location of the tissue within the patient's body, the time of day, the patient's hydration level, etc. These factors can affect the accuracy of an ultrasound imaging system's ability to accurately render an ultrasound image, as the ultrasound imaging system uses a constant speed of sound through the medium for the propagation of the ultrasound pulse through the anatomical structure. This can be noticeable in an ultrasound image depicting two different anatomical structures, such as muscle tissue immediately adjacent to a bone. In certain embodiments, the ultrasound imaging system can adjust its imaging settings to account for the tissue density of one or more of the anatomical structures featured in the ultrasound image. In certain embodiments, for example, when using a 3D or biplane transducer, the ultrasound imaging system can determine whether the tissue is different than that envisioned for imaging or whether the water content in the tissue is abnormal based on any displacement of the needle visualization from the location expected based on the tissue density. As an example and not by way of limitation, if the patient is dehydrated, the patient's tissue density may be different than expected, which may affect the final location of the needle relative to the expected target area.

[0027] In certain embodiments, the ultrasound imaging system may utilize other algorithms to detect whether a target area in an anatomical structure is within a high confidence region before and during a needle insertion procedure. As an example and not by way of limitation, an operator may indicate on the ultrasound system both the target depth of the needle and the anatomical feature that is targeted to that depth, such as a blood vessel, before any needle insertion. Once the ultrasound imaging system receives a selection of which type of needle to use, the corresponding depth, and what the target feature is, it may determine in real time or near real time whether the target feature encompasses most, if not all, of the high confidence region to ensure that the needle does not completely miss the target region. Prior to the procedure, the ultrasound imaging system may determine whether the configured needle insertion procedure will successfully place the needle in the target region, based on the needle and target region used, and notify the operator that he or she should proceed. In certain embodiments, if the ultrasound imaging system determines that the high confidence region does not overlap the target region sufficiently to ensure that the needle will reach the target region, the ultrasound imaging system may notify the operator that the operator should shift the position of the patient, the ultrasound transducer, or use a different needle and / or needle guide. In certain embodiments, once the needle insertion process begins, the position of the needle or the target area may shift, such as by the subject moving. If the ultrasound imaging system determines that the needle will miss the target area during the insertion process but before the needle intersects the ultrasound imaging plane, the ultrasound imaging system may notify the operator that the procedure may not be successful and allow the operator to reset the procedure and try again before the needle is fully inserted.

[0028] FIG. 4A illustrates the relationship between needle insertion angle and ultrasound image plane. In the diagram of FIG. 4A, needle 15 may be inserted from a position to the left of the plane of ultrasound image 300, with the needle trajectory downward toward the right. Based on the tabs used with the needle guidance system, needle 15 follows one of the marked trajectories toward the high confidence regions 421A-425A. For each possible trajectory in the example of FIG. 4A, the ultrasound imaging system may determine that the needle will shift less than 2 degrees from the projected path.

[0029] FIG. 4B provides a projection of the calculated trajectory of the needle 15 onto high confidence regions 421A-425A as it crosses the plane for the ultrasound image 300. In the example of FIG. 4B, the high confidence regions 421A-425A become larger as the depth increases due to the larger potential displacement for the needle that must move further to reach the ultrasound image plane 300. Other factors affecting the needle trajectory discussed above may also affect the size of the high confidence regions. Additionally, in the example of FIG. 4B, it can be seen that the high confidence regions 421A-425B are narrower relative to the height of each marker. In certain embodiments, this may reflect a determination that there is more mechanical play in the first direction than in the second direction. In the examples of FIGS. 4A and 4B, the high confidence regions may have a threshold of 99%. In certain embodiments, the high confidence regions may be It may be symmetric on all sides. For example, without limitation, the imaging system may determine that the needle may move by no more than 2 degrees in any direction. In certain embodiments, the high confidence region may be asymmetric. By way of example and not by way of limitation, the imaging system may determine that the needle may shift by 2 degrees in a first direction, but only 1 degree in a second direction. As another example and not by way of limitation, the high confidence level may not be centered exactly on the predicted path because the needle is more likely to shift in one direction based on the factors discussed above.

[0030] In certain embodiments, the ultrasound imaging system is configured to display one or more on-screen markers on the ultrasound image that correspond to the high confidence regions. The size, shape, opacity, and other characteristics of the on-screen markers can be based on the high confidence regions, e.g., the size, shape, or threshold of the high confidence regions. In certain embodiments, the on-screen markers may be asymmetrically shaped to indicate high confidence regions that are not symmetric in either the vertical or horizontal directions, as described above, due to factors that may affect the accuracy of imaging and needle placement. In certain embodiments, the size, shape, or other characteristics of the on-screen marks can be dynamically changed before and during the needle insertion procedure. For example, the ultrasound imaging system can continue to determine and adjust the high confidence regions during the procedure, dynamically changing the on-screen markers. As an example and not by way of limitation, as the needle approaches the target, the threshold of the high confidence region is increased, and the size, shape, or color of the corresponding on-screen marker is adjusted accordingly.

[0031] FIG. 4C shows an example diagram of an ultrasound image 300 with on-screen markers 421-425 corresponding to high confidence regions 421A-425A, respectively. Similar to FIGS. 3A and 3B, the ultrasound image 300 shows the anatomical structure 310 being imaged. In addition to the depth markers shown in the figures. 3A and 3B, the ultrasound image 300 of FIG. 4C uses on-screen markers 421-425 corresponding to high confidence regions 421A-425A, respectively. In the example of FIG. 4C, each on-screen marker may correspond to a high confidence region 421-425A with a 99% threshold.

[0032] FIG. 4D shows another exemplary view of on-screen markers for a high confidence level for needle placement. In the example of FIG. 4D, it may be desirable for the operator to remove some of the visual clutter that may be present in the example of 4C, and all five determined on-screen markers are visible in the ultrasound image 300. In the example of FIG. 4D, the operator may select a single on-screen marker to be displayed based on the selected needle and needle guide. In response to this determination, the ultrasound imaging system may prominently display only the selected on-screen marker 424, for example, by using an inverse color of the ultrasound image. This may allow the on-screen marker 424 to stand out without being overly prominent. The ultrasound imaging system may reduce the visibility of the other on-screen markers 421-423, 425. For example, the markers 421-423, 425 may be presented with a higher degree of transparency, presented in grayscale, or otherwise distinguished from the ultrasound image. In certain embodiments, the on-screen markers 421-423, 425 may be removed. In certain embodiments, the ultrasound imaging system provides information regarding one or more of the displayed on-screen markers, such as the depth, the threshold of the corresponding reliable region, the total area of ​​the corresponding reliable region, or whether the needle has entered the corresponding reliable region. In certain embodiments, the information is dynamically updated, for example, as the needle approaches and crosses (or misses) the reliable region. In the example of FIG. 4D, notation 426 is present on the selected on-screen marker 424 to indicate the depth of the on-screen marker 424. In certain embodiments, the information is automatically displayed on the ultrasound image 300. In certain embodiments, the operator may provide input specifying the information to be displayed.

[0033] In certain embodiments, the ultrasound imaging system determines multiple high-confidence regions with different thresholds for a given depth to be depicted in a corresponding on-screen marker. As an example and not by way of limitation, the ultrasound imaging system may use an on-screen marker with a first shade corresponding to the high-confidence region with a threshold of 99% which is the largest area, a second shade on-screen marker that is slightly smaller in size corresponding to the high-confidence region with a threshold of 95%, and a third shade on-screen marker that is even smaller in size corresponding to the high-confidence region with a threshold of 85%. In certain embodiments, using multiple confidence levels can indicate to the operator whether there is a particular risk of the needle missing the intended target or colliding with another anatomical feature. In certain embodiments, the ultrasound imaging system automatically determines the number of thresholds to be determined and the value of each threshold (e.g., 99%, 95%, 90%). In certain embodiments, the operator specifies, via user input, how many thresholds and what values ​​should be used for imaging. As an example and not by way of limitation, if for a particular needle insertion it is imperative that the operator avoids colliding with nearby anatomical structures, the operator may increase the required threshold (e.g., to 99.9%) so that the operator can be more confident that the operator will avoid nearby structures. With reference to FIG. 4E, for purposes of illustration and not limitation, on-screen markers 431-435 may utilize different shades to indicate different confidence levels, with the smallest marked area indicating a lower confidence level. For example, on-screen marker 435A corresponds to a high confidence area having a 99% threshold, on-screen marker 435B corresponds to a high confidence area having a 95% threshold, on-screen marker 435C corresponds to a high confidence area having a 90% threshold, and on-screen marker 435A corresponds to a high confidence area having an 85% threshold. On-screen marker 435A is the largest and brightest of markers 435A-D, and on-screen marker 435D is the smallest and darkest of markers 435A-D.As shown in the example of FIG. 4E, each smaller region reflecting a lower confidence level may not necessarily be concentric with a larger region for a higher confidence level, based on factors used by the ultrasound imaging system.

[0034] FIG. 4F shows another exemplary view of on-screen markers 431-435, similar to FIG. 4E. In the example of FIG. 4F, the operator may select a single on-screen marker 432, indicating that the operator intends to insert the needle to a depth corresponding to the on-screen marker 432. In this example, the ultrasound imaging system may modify the ultrasound image 300 to reduce the visibility of the unselected on-screen markers 431 and 433-435. Additionally, the ultrasound image 300 may add a picture-in-picture 440 of the selected on-screen marker 432 and surrounding tissue, such that the operator is provided with a close-up view of the area where the needle is expected to intersect the image. In certain embodiments, the ultrasound imaging system may choose to only reduce the visibility of the unselected on-screen markers or choose to only provide a picture-in-picture 440.

[0035] FIG. 4G illustrates another exemplary view of on-screen markers 421-425, similar to FIGS. 4C and 4D. In the example of FIG. 4G, an operator may select on-screen marker 423 as a target for an out-of-plane needle procedure. In this example, the ultrasound imaging system may then select region 450 around on-screen marker 423 and modify image processing parameters within region 450 to make it visually distinct to the operator. As an example and not by way of limitation, region 450 may be displayed with greater contrast, a different hue, or in other ways to enhance the visibility of region 450 relative to the remainder of ultrasound image 300. In certain embodiments, the screen is automatically updated based on a dynamic calculation of the high confidence region.

[0036] FIG. 4H shows another exemplary diagram of on-screen markers 441-445. is simply a loop where the on-screen markers correspond to the outline of the high confidence region. In certain embodiments, the on-screen markers 441-445 can be used to indicate to the operator where the needle is expected to appear without adding further clutter to the details of the ultrasound image 300.

[0037] In certain embodiments, the determination of the high confidence region can be based on the shape of the ultrasound beam as a function of depth. FIG. 5 shows an example ultrasound beam formed by a focused ultrasound transducer 12, forming an elevation profile 510. As shown in the example of FIG. 5, the elevation profile 510 reflects an initial wide area roughly corresponding to the aperture size of the transducer, then narrows over an initial distance, and then finally widens again. In certain embodiments, the thickness of the ultrasound image can change with depth, so the determination of the on-screen markers can also shift. It will be understood in the art that the elevation profile of the beam over depth (i.e., distance from the transducer) can be predicted from the diameter of the transducer face and the wavelength used. Because the elevation profile of the beam changes with depth in certain embodiments, the thickness of the view represented in the ultrasound image may not be consistent throughout the image.

[0038] FIG. 6A shows the relationship between the needle insertion angle and the ultrasound imaging plane of a beam with an elevation profile 510. FIG. 6B provides a projection of the calculated trajectory of the needle 15 onto the high confidence regions 611-615. Similar to FIG. 4A, in the example of FIG. 6A, the needle 15 is inserted from the left and top of FIG. 6A, and the ultrasound image plane is described by the elevation profile 510 as shown in FIG. 5. In the example of FIG. 6A, the elevation thickness 510 is plotted as a parabolic arc. In FIG. 6A, the trajectory for the needle 15 is plotted as markers 611-615 on the screen based on the same potential angle shift as shown in FIG. 4A. However, the variable thickness along the height profile 510 may affect the size of the high confidence regions 611-615. In particular, as the depth increases and the thickness of the beam increases, the shape of the high confidence regions 614 and 615 may differ compared to calculations using a beam with a constant thickness (e.g., as shown in FIGS. 4A-4B).

[0039] FIG. 7 illustrates an exemplary method 700 for displaying on-screen markers on an ultrasound imaging system, the on-screen markers corresponding to areas with high confidence that an out-of-plane needle will intersect the ultrasound image. In step 710, the ultrasound imaging system can receive instructions to capture an ultrasound image along a first plane of the region of interest. As an example and not by way of limitation, the region of interest may be an anatomical structure having a target area for needle insertion. In step 720, the ultrasound imaging system can further receive input that an out-of-plane needle insertion process will be performed for the ultrasound image along the first plane. As an example and not by way of limitation, an operator of the ultrasound imaging system can attach an out-of-plane needle guidance system to the ultrasound transducer and select and attach a tab appropriate for the needle to be used and the desired target depth. Needle insertion may be performed along a second plane orthogonal to the first plane. In step 730, the ultrasound imaging system can begin capturing ultrasound images of the region of interest along the first plane. In step 740, the ultrasound imaging system can determine one or more high confidence areas where an inserted needle may intersect the first plane. The determination may be based on the ultrasound image and information regarding the out-of-plane needle insertion. At step 750, the ultrasound imaging system may display one or more on-screen markers corresponding to one or more reliable regions along with the multiple ultrasound images on the display.

[0040] Certain embodiments may, if desired, repeat one or more of the steps disclosed in Figure 7. Although this disclosure describes and illustrates certain steps of the method of Figure 7 as being performed in a particular order, this disclosure includes any suitable steps of the method of Figure 7 performed in any suitable order. Additionally, although this disclosure describes and illustrates particular components, devices, or systems performing particular steps of the method of FIG. 7, this disclosure contemplates any suitable combination of any suitable components, devices, or systems performing any suitable steps of the method of FIG.

[0041] In certain embodiments, the present invention utilizing on-screen markers for out-of-plane needle guidance based on confidence levels may provide advantages over previous on-screen markers for out-of-plane needle guidance by providing additional feedback to the operator not only of where the needle should intersect the ultrasound image plane, but also of where the needle may intersect. In certain embodiments, this may allow the operator to adjust the positioning of the needle guide or ultrasound transducer relative to the anatomical structure prior to needle insertion to ensure the needle intersects the target structure or to ensure the needle avoids intersecting a different structure.

[0042] The subject matter and operations described herein may be implemented in digital electronic circuitry, or computer software, firmware, or hardware, or one or more combinations thereof, including the structures disclosed herein and their structural equivalents. Embodiments of the subject matter described herein may be implemented as one or more computer programs, i.e., one or more modules of computer program instructions, encoded on a computer storage medium for execution by or to control the operation of a data processing apparatus.

[0043] A computer storage medium may be or be included in a computer readable storage device, a computer readable storage substrate, a random or serial access memory array or device, or a combination of one or more of these. Additionally, a computer storage medium is not a propagating signal, but a computer storage medium can be the source or destination of computer program instructions encoded in an artificially generated propagating signal. A computer storage medium may also be or be included in one or more separate physical components or media (e.g., multiple CDs, disks, or other storage devices).

[0044] The term "processor" encompasses all kinds of apparatus, devices, and machines for processing data, including, by way of example, a programmable processor, a computer, a system on a chip, or a plurality of the foregoing, or a combination thereof. The apparatus may include special purpose logic circuitry, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). In addition to hardware, the apparatus may also include code that creates code that constitutes the execution environment of the computer program, such as processor firmware, a protocol stack, a database management system, an operating system, a cross-platform runtime environment, a virtual machine, or one or more combinations thereof. The apparatus and execution environment may implement a variety of different computing model infrastructures, such as web services, distributed computing, and grid computing infrastructures.

[0045] A computer program (also called a program, software, software application, script, or code) can be written in any form of programming language, including compiled or interpreted, declarative or procedural, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine, object, or other unit suitable for use in a computing environment. A computer program can, but need not, correspond to a file in a file system. A program can be stored in a file that holds other programs or data (e.g., one or more scripts stored in a markup language document). The program code may be stored in part of a file, in one file dedicated to the program in question, or in several coordinated files (e.g., files storing one or more modules, subprograms, or portions of code). A computer program may be deployed to be executed on one computer or on several computers that are located at one site or distributed across several sites and interconnected by a communication network.

[0046] The processes and logic flows described herein may be performed by one or more programmable processors executing one or more computer programs to perform actions by operating on input data and generating output. The processes and logic flows may also be implemented using FPGAs (Field Programmable Gate Arrays) or or may be implemented as special purpose logic circuitry, such as an ASIC (application specific integrated circuit). , the device may also be implemented.

[0047] Processors suitable for executing a computer program may include, by way of example and not limitation, both general purpose and special purpose microprocessors. Devices suitable for storing computer program instructions and data may include, by way of example and not limitation, all forms of non-volatile memory, media, and memory devices, including semiconductor memory devices, e.g., EPROM, EEPROM, and flash memory devices, magnetic disks, e.g., internal hard disks or removable disks, magneto-optical disks, and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated in, special purpose logic circuitry.

[0048] From the foregoing, it will be appreciated that, although specific embodiments of the invention have been described herein for purposes of illustration, various modifications may be made without deviating from the scope of the invention. Accordingly, the invention is not limited except as by the appended claims.

Claims

1. 1. An ultrasound imaging system comprising an image processor, The image processor includes: receiving input data for capturing an ultrasound image of a region of interest along a first plane, the input data including an indication that an interventional instrument is to be inserted into the region of interest along a second plane, the second plane being orthogonal to the first plane; sequentially acquiring the ultrasound images along the first plane; calculating, for each of a plurality of identified regions, a probability that the interventional instrument will intersect with the first plane at each of the identified regions when inserted along the second plane; determining one or more high confidence regions within the region of interest where the interventional instrument intersects the first plane based on the probability exceeding a threshold; displaying at least one of the ultrasound images and one or more on-screen markers corresponding to the one or more reliable regions; Detecting when the interventional instrument intersects with the first plane; providing an indication of a position of the interventional instrument in the first plane on at least one displayed ultrasound image; reducing visibility of the one or more on-screen markers when it is detected that the interventional instrument has intersected the first plane. It is configured as follows: Ultrasound imaging system.

2. The ultrasound imaging system of claim 1 , wherein visibility of the one or more on-screen markers is automatically reduced when it is detected that the interventional instrument has crossed the first plane.

3. The ultrasound imaging system of claim 1 , wherein visibility of the one or more on-screen markers is reduced by an operator of the ultrasound imaging system when it is detected that the interventional instrument has crossed the first plane.

4. The ultrasound imaging system of claim 1 , wherein the image processor is configured to reduce visibility of the one or more on-screen markers by increasing a transparency of the one or more on-screen markers.

5. The ultrasound imaging system of claim 1 , wherein the image processor is configured to present the one or more on-screen markers in a grayscale to reduce visibility of the one or more on-screen markers.

6. 2. The ultrasound imaging system of claim 1, wherein the determination of the one or more high confidence regions is further based on calculating potential deflection of an interventional instrument during insertion based on at least one of tissue density and physical properties of the interventional instrument.

7. the ultrasound image along the first plane includes an elevation thickness that varies as a function of depth; The ultrasound imaging system of claim 1 , wherein the determination of the one or more reliable regions is further based on an elevation thickness at each reliable region within the ultrasound image.

8. The image processor further comprises: receiving input data indicative of targets within the region of interest; determining whether the reliable region overlaps with a target; notifying an operator of the ultrasound imaging system if the reliable region overlaps a target. The ultrasound imaging system of claim 1 , configured as follows:

9. The image processor further comprises: determining that the interventional instrument is inserted into the region of interest along the second plane; determining whether the reliable region remains overlapped with a target; in response to determining that the reliable region no longer overlaps the target, notifying an operator that the reliable region no longer overlaps the target. The ultrasound imaging system of claim 8 , configured as follows:

10. The image processor further comprises: receiving a selection of at least one selected on-screen marker from the one or more on-screen markers; displaying the at least one selected on-screen marker as a picture-in-picture on a display; The ultrasound imaging system of claim 1 , configured as follows:

11. The image processor further comprises: receiving input data indicative of targets within the region of interest; determining whether at least one of the one or more reliable regions overlaps with the target before the interventional instrument is inserted; notifying an operator of the ultrasound imaging system that the one or more reliable regions do not overlap a target; The operator, Shifting the patient's position shifting a position of an ultrasound transducer of the ultrasound imaging system; Use a different intervention; and using a separate guide for the interventional instrument. The ultrasound imaging system of claim 1 , configured as follows:

12. One or more computer-readable non-transitory storage media embodying software operable when executed by one or more computing devices of an ultrasound imaging system, The software comprises: receiving input data for capturing an ultrasound image of a region of interest along a first plane, the input data including an indication that an interventional instrument is to be inserted into the region of interest along a second plane, the second plane being orthogonal to the first plane; sequentially acquiring the ultrasound images along the first plane; calculating, for each of a plurality of identified regions, a probability that the interventional instrument will intersect with the respective identified region; determining one or more high confidence regions within the region of interest where the interventional instrument intersects the first plane based on the probability exceeding a threshold; displaying at least one of the ultrasound images and one or more on-screen markers corresponding to the one or more reliable regions; Detecting when the interventional instrument intersects the first plane; providing an indication of a position of the interventional instrument in the first plane on at least one displayed ultrasound image; automatically reducing visibility of the one or more on-screen markers when it is detected that the interventional instrument has intersected the first plane; and operable to perform one or more operations including storage medium.

13. 13. The storage medium of claim 12, wherein the interventional instrument is inserted along the second plane via a guidance system, the guidance system being attached to an ultrasound transducer of the ultrasound imaging system.

14. The storage medium of claim 12 , wherein each of the one or more reliable regions corresponds to a particular depth within a region of interest from an ultrasound transducer of the ultrasound imaging system.

15. 13. The storage medium of claim 12, wherein the determination of the one or more high confidence regions is further based on calculating a potential deflection of an interventional instrument during insertion based on at least one of a density of tissue and a physical property of the interventional instrument.

16. the ultrasound image along the first plane includes an elevation thickness that varies as a function of depth; The storage medium of claim 12 , wherein the determination of the one or more reliable regions is further based on an elevation thickness at each reliable region within the ultrasound image.

17. The operation includes: receiving input data indicative of targets within the region of interest; determining whether at least one of the one or more reliable regions overlaps with the target before the interventional instrument is inserted; notifying an operator of the ultrasound imaging system that the one or more reliable regions do not overlap a target; The operator, Shifting the patient's position shifting a position of an ultrasound transducer of the ultrasound imaging system; Use a different intervention; use a different guide for the interventional instrument; and The storage medium of claim 12 , further comprising:

18. receiving input data for capturing an ultrasound image of a region of interest along a first plane, the input data including an indication that an interventional instrument is to be inserted into the region of interest along a second plane, the second plane being orthogonal to the first plane; sequentially acquiring the ultrasound images along the first plane; calculating, for each of a plurality of specific regions based on the input data and the ultrasound image, a probability that the interventional instrument will intersect with the first plane; determining one or more high confidence regions within the region of interest where the interventional instrument intersects the first plane based on the probability exceeding a threshold; displaying at least one of the ultrasound images and one or more on-screen markers corresponding to the one or more reliable regions; dynamically modifying the one or more on-screen markers as the interventional instrument approaches the first plane; Detecting when the interventional instrument intersects the first plane; providing an indication of a position of the interventional instrument in the first plane on at least one displayed ultrasound image; and A method comprising:

19. Dynamically modifying the one or more on-screen markers includes: modifying the size of the one or more on-screen markers; Changing the shape of the one or more on-screen markers; or changing the color of said one or more on-screen markers; 20. The method of claim 18, comprising:

20. 20. The method of claim 18, wherein each of the one or more reliable regions corresponds to a particular depth within the region of interest from an ultrasound transducer of an ultrasound imaging system.

Images