Magnetic tracking of ultrasound probes and systems, methods and apparatus for generating 3D visualizations thereof

Abstract

Disclosed herein is a magnetic-based tracking system for tracking an ultrasound probe to create a three-dimensional visualization. The system includes: a reference device including a reference magnet; an ultrasound probe including an ultrasound acoustic transducer or acoustic array that acquires ultrasound images and a magnetometer that detects a magnetic field generated by the reference magnet. The ultrasound probe couples a first ultrasound image with a first magnetic field strength, where the first ultrasound image is received and the first magnetic field strength is detected at a first time; and a console including a processor and a non-transitory computer readable medium having stored thereon a plurality of processor-executed logic modules that perform operations including: receiving and recording a plurality of couplings of ultrasound images and detected magnetic field strengths; and generating a 3D visualization from the ultrasound images by calibrating the ultrasound images according to corresponding detected magnetic field strengths.

Description

[0001] Priority

[0002] This application claims priority benefit of U.S. Provisional Application No. 63 / 054,622, filed July 21, 2020, the entire contents of which are incorporated herein by reference. TECHNICAL FIELD

[0003] The present application relates to the field of medical devices, and more particularly to a magnetic tracking ultrasound probe and a system, method and apparatus for generating 3D visualization thereof. SUMMARY

[0004] Briefly, the embodiments disclosed herein relate to a system, method and apparatus for magnetic tracking of an ultrasound probe, and use of the magnetic tracking ultrasound probe to generate 3D visualization of a target vessel.

[0005] In some patients, vascular cannulation can be difficult. One problem that often arises with vascular cannulation is the difficulty in visualizing the target vessel and other details important for cannulation, including vessel movement, vessel size, depth, proximity to non-intended structures, or even obstacles within the target vessel, including valves, stenosis, thrombosis, etc.

[0006] In some embodiments, a magnetic based tracking system for tracking an ultrasound probe to create a three-dimensional (3D) visualization includes a reference device comprising a reference magnet; an ultrasound probe comprising a magnetometer configured to detect a magnetic field generated by the reference magnet, wherein the ultrasound probe is configured to couple a first ultrasound image with a first magnetic field strength, and wherein the first ultrasound image is received at a first time and the first magnetic field strength is detected at the first time. The system further includes a console comprising a processor and a non-transitory computer readable medium having stored thereon a plurality of logic modules configured to perform operations including, when executed by the processor: receiving a plurality of couplings of ultrasound images and detected magnetic field strengths; recording the plurality of couplings of ultrasound images and detected magnetic field strengths; and generating the 3D visualization from the ultrasound images by aligning each of the ultrasound images according to a corresponding detected magnetic field strength.

[0007] In some embodiments, the magnetic based tracking system includes wherein the reference device is a cuff like structure that wraps around a body segment to be imaged.

[0008] In some embodiments, the magnetic-based tracking system includes wherein the reference device is a U-shaped structure and is configured to allow the body segment to be placed within the U-shaped structure for imaging.

[0009] Also disclosed is a magnetic-based tracking system for tracking an ultrasound probe to create a three-dimensional (3D) visualization, the magnetic-based tracking system comprising: a reference device comprising a magnetometer that detects a magnetic field generated by a reference magnet and creates a timestamp for a magnetic field strength reading; an ultrasound probe comprising an ultrasound acoustic transducer or acoustic array that acquires ultrasound images and creates a timestamp for each particular ultrasound image, and a reference magnet configured to generate a magnetic field. The system further comprises a console comprising a processor and a non-transitory computer readable medium having stored thereon a plurality of logic modules configured to perform operations comprising, when executed by the processor: receiving a plurality of ultrasound images and detected magnetic field strengths; joining the plurality of ultrasound images and detected magnetic field strengths by their timestamps; recording the plurality of joins of ultrasound images and detected magnetic field strengths; and generating the 3D visualization from the ultrasound images by calibrating each of the ultrasound images according to the corresponding detected magnetic field strength.

[0010] In some embodiments, the magnetic-based tracking system includes wherein the reference device is a sleeve-like structure that wraps around the body segment to be imaged.

[0011] In some embodiments, the magnetic-based tracking system includes wherein the reference device is a U-shaped structure and is configured to allow the body segment to be placed within the U-shaped structure for imaging.

[0012] Also disclosed is a method of creating a 3D image using a magnetic-based tracking system for tracking an ultrasound probe, the method comprising: configuring the reference device around a body segment to be imaged; advancing an ultrasound probe over a skin surface of the body segment to be imaged; capturing timestamped ultrasound images while detecting, by a magnetometer, timestamped magnetic field strengths of a reference magnet; determining a distance between the reference magnet and the magnetometer; and stitching the ultrasound images together using the magnetic field strength data and timestamps to create a 3D image.

[0013] In some embodiments, the method includes wherein configuring the reference device comprises the reference device being a sleeve-like structure and comprising a reference magnet.

[0014] In some embodiments, the method includes wherein advancing the ultrasound probe comprises the ultrasound probe comprising a magnetometer.

[0015] In some embodiments, the method includes: wherein configuring the reference device includes the reference device being a U-shaped structure and configured to allow placement of the body region within the U-shaped structure for imaging.

[0016] In some embodiments, the method includes where stitching the ultrasound images together includes using only the magnetic field strength data.

[0017] In some embodiments, the method includes: wherein configuring the reference device includes the reference device being a cuff-like structure and comprising a magnetometer.

[0018] In some embodiments, the method includes where advancing the ultrasound probe includes the ultrasound probe including the reference magnet.

[0019] These and other features of the concepts provided herein will become more readily apparent to those skilled in the art in view of the accompanying drawings and the following description, which disclose in more detail specific embodiments of the concepts. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Embodiments of the disclosure are illustrated by way of example and not limitation in the figures of the accompanying drawings in which like references indicate similar elements and in which:

[0021] Figure 1 shows a block diagram depicting various elements of a magnetic-based tracking system for ultrasound probes and other medical components to create 3D images, according to some embodiments;

[0022] Figure 2A According to some embodiments, the Figure 1 A top view of a magnetic-based tracking system for an ultrasound probe;

[0023] Figure 2B A system comprising a remotely controlled tourniquet, a remotely controlled heating element, and a Figure 1 A top view of a magnetic-based tracking system for an ultrasound probe;

[0024] Figure 3A According to some embodiments, the Figure 2A A side view of a magnetic-based tracking system for an ultrasound probe;

[0025] Figure 3B According to some embodiments, the Figure 2A Front view of the ultrasound probe and console with a magnetic-based tracking system;

[0026] Figure 4A side view of a magnetic-based tracking system including the ultrasound probe of FIG. 3 is shown in accordance with some embodiments;

[0027] Figure 5 is a flowchart showing an exemplary method for creating 3D images using a magnetic-based tracking system of an ultrasound probe in accordance with some embodiments; Figure 1

[0028] Figure 6 is a block diagram depicting various elements of a magnetic-based tracking system for ultrasound probes and other medical components to create 3D images in accordance with some embodiments;

[0029] Figure 7 is a top view of a magnetic-based tracking system including the ultrasound probe of FIG. 3 is shown in accordance with some embodiments; Figure 6

[0030] Figure 8 is a side view of a magnetic-based tracking system including the ultrasound probe of FIG. 3 is shown in accordance with some embodiments; Figure 7

[0031] Figure 9 is a side view of a magnetic-based tracking system including the ultrasound probe of FIG. 3 is shown in accordance with some embodiments; and Figure 8

[0032] Figure 10 is a flowchart showing an exemplary method for creating 3D images using a magnetic-based tracking system of an ultrasound probe in accordance with some embodiments. Figure 6 DETAILED DESCRIPTION

[0033] Before certain specific embodiments are disclosed in more detail, it is understood that the particular embodiments disclosed herein do not limit the scope of the concepts provided herein. It is also understood that the particular embodiments disclosed herein can have features that can be readily separated from the particular embodiments into which they are described and combined or replaced with features of any of the other various embodiments disclosed herein.

[0034] ​​​​​With respect to the terms used herein, it is also to be understood that these terms are used for the purpose of describing some particular embodiments and that the terms do not limit the scope of the concepts provided herein. Ordinal numbers (e.g., first, second, third, etc.) are generally used to distinguish or identify different features or steps in a group of features or steps and do not supply a serial or numerical limitation. For example, a “first,” “second,” and “third” feature or step need not necessarily appear in that order, and the particular embodiments including such features or steps need not necessarily be limited to the three features or steps. Labels such as “left,” “right,” “top,” “bottom,” “front,” “back,” and the like are used for convenience and are not intended to imply, for example, any particular fixed location, orientation or direction. Rather, such labels are used in connection with the orientation / fixed location relative to one another as illustrated in the drawings. Singular forms of “a,” “an” and “the” include plural references unless the context clearly dictates otherwise.

[0035] For example, a “proximal,” “proximal portion,” or “proximal end portion” of a probe disclosed herein includes a portion of the probe that is intended to be proximate to a clinician when the probe is used on a patient. Likewise, for example, a “proximal length” of a probe includes a length of the probe that is intended to be proximate to a clinician when the probe is used on a patient. For example, a “proximal end” of a probe includes an end of the probe that is intended to be proximate to a clinician when the probe is used on a patient. A proximal portion, proximal end portion, or proximal length of a probe can include a proximal end of the probe; however, a proximal portion, proximal end portion, or proximal length of a probe need not include a proximal end of the probe. That is, unless the context indicates otherwise, a proximal portion, proximal end portion, or proximal length of a probe is not a terminal portion or length of the probe.

[0036] For example, a “proximal,” “proximal portion,” or “proximal end portion” of a probe disclosed herein includes a portion of the probe that is intended to be proximate to a clinician when the probe is used on a patient. Likewise, for example, a “proximal length” of a probe includes a length of the probe that is intended to be proximate to a clinician when the probe is used on a patient. For example, a “proximal end” of a probe includes an end of the probe that is intended to be proximate to a clinician when the probe is used on a patient. A proximal portion, proximal end portion, or proximal length of a probe can include a proximal end of the probe; however, a proximal portion, proximal end portion, or proximal length of a probe need not include a proximal end of the probe. That is, unless the context indicates otherwise, a proximal portion, proximal end portion, or proximal length of a probe is not a terminal portion or length of the probe.

[0037] The term“logic” can refer to or include hardware, firmware, or software configured to perform one or more functions. As hardware, the term logic can refer to or include circuitry having data processing and / or storage functionality. Embodiments of such circuitry can include, but are not limited to or limited by, a hardware processor (e.g., a microprocessor, one or more processor cores, a digital signal processor, a programmable gate array, a microcontroller, an application-specific integrated circuit“ASIC,” etc.), semiconductor memory, or a combination element.

[0038] Additionally or alternatively, the term logic can refer to or include software such as one or more processes, one or more instances, application programming interface(s) (APIs), subroutine(s), function(s), applet(s), servlet(s), routine(s), source code, object code, shared libraries / dynamic load libraries (dlls), or even one or more instructions, etc. Such software can be stored in any type of suitable non-transitory or transitory storage medium or media (e.g., electrical, optical, acoustical, or other form of propagated signals, such as carrier waves, infrared signals, or digital signals). Embodiments of non-transitory storage media can include, but are not limited to or limited by, programmable circuitry, non-persistent storage (e.g., any type of volatile memory, such as any type of random access memory“RAM”) or persistent storage (e.g., read only memory“ROM,” power-backed RAM, flash memory, phase change memory, etc.), solid state drives, hard disk drives, optical disk drives, or portable memory devices. As firmware, the logic can be stored in persistent storage.

[0039] Referring to Figure 1 A block diagram depicting various elements of a magnetic-based tracking system 1110 for tracking an ultrasound probe 1140 and generating three-dimensional (3D) images is shown, in accordance with one example embodiment of the present application. As shown, the system 1110 generally includes a console 1120, an ultrasound probe 1140, and a reference device 800, with each of the probe 1140 and the reference device 800 configured to be communicatively coupled to the console 1120. The console 1120 is shown to include one or more processors 1122, a communication interface 1142, a display 1130, and a non-transitory computer-readable medium (“memory”) 1150. The memory 1150 is configured to store logic modules including ultrasound probe data reception logic 1152 and magnetometer data reception logic 1156. Further, the memory 1150 can include data stores such as ultrasound probe data 1154, magnetometer data 1158, and relationship data 1160.

[0040] In some embodiments, a processor 1122 including, for example, non-volatile memory such as EEPROM is included in the console 1120 for controlling system functions during operation of the system 1110, thereby serving as a control processor. The display 1130 in this embodiment can be integrated into the console 1120 and used to display information to the clinician when using the ultrasound probe 1140. In another embodiment, the display 1130 can be separate from the console 1120.

[0041] The ultrasound probe 1140 uses an ultrasonic acoustic transducer or an acoustic array to generate and receive echoes that can be converted into images. For example, in some embodiments, the ultrasound probe may include an ultrasound generating device that includes an ultrasound acoustic stack or other various forms of ultrasound generation (e.g., based on a microelectromechanical system (MEMS)). In some embodiments, within a magnetic-based tracking system, the ultrasound probe 1140 may additionally include a magnetometer for measuring magnetic field strength. In some embodiments, the ultrasound probe 1140 may include a reference magnet.

[0042] The reference device 800 is a device that, in some embodiments, contains the reference magnet 700 of the magnetic-based tracking system 1110. In some embodiments, the reference device 800 can be a cuff-like structure that contains the reference magnet 700 and is wrapped around the body section to be imaged (e.g., a body appendage, torso, midsection, chest, etc.). In other embodiments, the reference device 800 can include a U-shaped structure that contains the reference magnet 700 and is configured to allow the body appendage to be placed within the U-shaped structure for imaging. In other embodiments, the reference device 800 can include a magnetometer.

[0043] Figure 2A According to some embodiments, the Figure 1 1. A top view of a magnetic-based tracking system 1110 for an ultrasound probe 1140 is shown. The ultrasound probe 1140 is employed in conjunction with a reference device 800 to track the position of the probe 1140 as it moves around the surface of a patient. As shown, the probe 1140 is configured to move along a surface (i.e., skin) 1220 of a body appendage 1222 of the patient. The reference device 800 can be deployed within a threshold distance from the body appendage 1222. In some embodiments, the reference device 800 can include a device configured as a "U-shape" (e.g., Figure 1 ) wherein the interior is formed by the convex portion of the configuration. In this case, a body appendage 1222 (e.g., an arm) can be placed within the interior.

[0044] In some embodiments, the magnetic-based tracking system 1110 can be used in preparation for insertion of a needle and / or catheter into the vasculature. Specifically, the system 1110 employs a combination of the probe 1140 and the reference device 800 to track the positioning of the probe 1140 relative to the reference magnets 700 of the reference device 800. By tracking the positioning of the probe 1140 relative to the reference magnets 700, the system 1110 is able to correlate each ultrasound image obtained by the probe 1140 with the positioning of the probe 1140 on the body appendage 1222. By correlating the ultrasound images with the positioning on the appendage 1222, the system 1110 can then stitch together a set of ultrasound images to form a 3D visualization of the body appendage 1222. This 3D visualization gives real-time 3D ultrasound guidance and helps reduce complications typically associated with such introductions, including inadvertent arterial puncture, hematoma, pneumothorax, etc.

[0045] In some embodiments, the ultrasound probe 1140 includes a head 1180 that houses an ultrasound acoustic transducer or acoustic array 1144 for producing ultrasound pulses and for receiving echoes thereof after reflection by the patient's body when the head is placed against the patient's skin 1220, where each echo can be converted into an image. The ultrasound acoustic transducer or acoustic array 1144 can be in operable communication with the console 1120 for storing ultrasound images. In some embodiments, the ultrasound probe 1140 includes a sensor component, such as a magnetometer 1192, for detecting the position of the reference magnets 700 during an ultrasound imaging procedure, such as those described above. As will be described in further detail below, the magnetometer 1192 can be embedded within the head 1180 of the ultrasound probe 1140. The magnetometer 1192 is configured to detect a magnetic field associated with the reference magnets 700 and enable the system 1110 to track the reference magnets 700 when the reference magnets are associated with the magnetometer 1192. In the present embodiment, the magnetometer 1192 is arranged in a planar configuration in the head 1180 of the ultrasound probe 1140, although it should be understood that the magnetometer 1192 can be arranged in other configurations within the ultrasound probe 1140. The magnetometer 1192 can be present in a paired longitudinal configuration with the ultrasound acoustic transducer or acoustic array 1144. In other embodiments, the magnetometer 1192 can be present in a paired transverse configuration, a paired circular configuration, or a combination thereof, with the ultrasound acoustic transducer or acoustic array 1144.

[0046] In some embodiments, the magnetometer 1192 may include a series of magnetometers arranged in a configuration to track the reference magnet 700. In one embodiment, the magnetometer 1192 includes three orthogonal sensor coils to enable detection of magnetic fields in three spatial dimensions (not shown). An example of a 3D magnetic sensor is a 3D magnetic sensor manufactured by Honeywell Sensing and Control of Morristown, New Jersey. Further, in one embodiment, the magnetometer 1192 is configured as a Hall effect sensor, but other types of magnetic sensors may be used.

[0047] The magnetometer 1192 may be in communication with the console 1120 for storing information regarding the position of the magnetometer 1192 relative to the reference magnet 700 , as will be described in greater detail herein.

[0048] In some embodiments, as Figure 2A and Figure 2B As shown in , the reference magnet 700 generates a magnetic field that can be detected and measured using a magnetometer 1192. As the ultrasound probe 1140 moves along the surface 1220 of the body appendage 1222, the strength of the magnetic field detected by the magnetometer 1192 changes. The detected magnetic field strength is recorded together with the ultrasound image captured by the probe 1140, both of which are transmitted to the console 1120 for storage and processing, which may include generating a 3D image including multiple ultrasound images. In some embodiments, the reference magnet 700 includes an electromagnet coupled to a power supply (not shown). In such an embodiment, the strength of the magnetic field can be increased, which allows the ultrasound probe 1140 to be used at a greater distance from the reference magnet 700.

[0049] Figure 2A A body appendage 1222 is shown supported within the interior of the reference device 800. It is understood that the body appendage 1222 can be any body appendage and it is further understood that reference devices of various sizes can be deployed with the system 1110 (i.e., so as to enable the system to be used with different appendages). In some embodiments, the reference device can be specifically configured for a particular appendage (e.g., an arm, a leg) or other body part (e.g., a torso). Furthermore, some reference devices can be specifically configured for children, while other reference devices are configured for adults, with adult-sized reference devices generally having a larger size than those configured for children. In some embodiments, the body appendage 1222 can include, but is not limited to, a lower arm, an upper arm, a lower leg, a thigh, or a foot.

[0050] In some embodiments, the reference device 800 can be configured to fit a body segment, such as a torso, multiple legs, or multiple arms. In some embodiments, the reference device 800 can be reusable and made of a durable material (e.g., nylon, other polyesters, or hardened plastics). Figure 2A In some embodiments shown in , reference device 800 can be configured to surround multiple sides of a body appendage, wherein body appendage 1222 can be inserted through an open side of reference device 800 with appendage 1222 unobstructed for imaging.

[0051] As will be described in more detail below, including a reference magnet 700 and a magnetometer 1192 in system 1110 provides numerous advantages over current ultrasound systems, particularly with respect to generating visualizations based on acquired ultrasound data. In short, the reference magnet 700 generates a magnetic field that can be detected by the magnetometer 1192 in the ultrasound probe 1140. Because the reference magnet 700 remains stationary relative to the patient's appendage during use of the ultrasound probe 1140, it serves as a reference point for the ultrasound probe 1140. Consequently, the ultrasound probe 1140 can be specifically configured to associate the detected magnetic field strength with a specific ultrasound image (received as an echo). Furthermore, the ultrasound probe 1140 can be configured to continuously associate the detected magnetic field strength with acquired ultrasound images. The associated {detected magnetic field strength, ultrasound image} pairs can be provided to the console 1130, allowing the console's logic to generate a 3D visualization by stitching together the ultrasound images based on the magnetic field strength associated with each ultrasound image. In other words, the logic of the console 1130 can properly calibrate the ultrasound images based on the detected magnetic field strength associated with each ultrasound image. Specifically, the detected magnetic field strength provides an indication of the position of the patient's appendage relative to the stationary reference magnet 700, which is used to calibrate the ultrasound images.

[0052] In one particular embodiment, the reference device 800 can completely surround the body appendage 1222, for example, a sleeve-like structure that completely encloses the body appendage 1222 to be imaged. In such an embodiment, the reference device 800 can be constructed of cloth, neoprene, or a mesh material and can be attached in place by sliding the reference device 800 over the appendage 1222 to be imaged. In some embodiments, the reference device 800 can be configured to surround and secure to the body appendage 1222 to be imaged. For example, the reference device 800 can be completely wrapped around the upper arm, near the elbow, and secured to the upper arm by a first piece of temporary adhesive (e.g., double-sided tape or Velcro) (see FIG. 1 ). Figure 3A ).

[0053] In some embodiments, as Figure 2BAs shown in the middle, the reference device 800 can communicate with the remote controlled tourniquet 870 to compress the body appendage 1222 and with the remote controlled heating element 872 to cause vasodilation of the blood vessels during use.

[0054] Referring Figure 3A A side view of a magnetic based tracking system 1110 including the ultrasound probe 1140 of FIG. 2 is shown, in accordance with some embodiments. In some embodiments, as the ultrasound probe 1140 is moved over the surface of the skin 1220, the ultrasound acoustic transducer or acoustic array 1144 is capturing images while the magnetometer 1192 measures the strength of the detected magnetic field generated by the reference magnet 700 and relays the magnetic field strength information back to the console 1120 along with the captured ultrasound images. In other words, as the ultrasound probe 1140 is moved along the body appendage of the patient, the ultrasound probe 1140 transmits ultrasound pulses and receives echoes that are translated into information for a two-dimensional (2D) image of the echoes received at a particular location on the body appendage for rendering. As disclosed herein, as the ultrasound probe 1140 transmits ultrasound pulses and receives echoes, the magnetometer 1192 concurrently detects the magnetic field generated by the reference magnet 700 and records its strength, where concurrent means at least partially overlapping in time. Thus, the ultrasound probe 1140 captures information for creating a 2D ultrasound image at a particular location along the body appendage and records the strength of the magnetic field detected at that location. The ultrasound information and the magnetic field strength are coupled and transmitted to the console 1110 for storage and processing. For example, multiple ultrasound information and magnetic field strength couplings can be used to create a 3D visualization of at least a portion of the body appendage by stitching together the 2D images created from the ultrasound information based on the magnetic field strength data. Specifically, by analyzing the magnetic field strength data, the magnetic field strength data can be used to calibrate the 2D ultrasound images based on the reference point (e.g., the location of the reference magnet 700).

[0055] As Figure 3A, the ultrasound probe 1140 obtains a first ultrasound image 900 of the blood vessel 1226 using an ultrasound acoustic transducer or acoustic array 1144, while the magnetometer 1192 records a first intensity 902 of the detected magnetic field generated by the reference magnet 700. The ultrasound probe 1140 obtains a second ultrasound image 904 of the blood vessel 1226 using the ultrasound acoustic transducer or acoustic array 1144, while the magnetometer 1192 records a second intensity 906 of the detected magnetic field generated by the reference magnet 700. Further, the ultrasound probe 1140 obtains a third ultrasound image 908 of the blood vessel 1226 using the ultrasound acoustic transducer or acoustic array 1144, while the magnetometer 1192 records a third intensity 910 of the detected magnetic field generated by the reference magnet 700. The first image 900 and the first intensity 902, the second image 904 and the second intensity 906, and the third image 908 and the third intensity 910, respectively, are coupled and transmitted to the console 1120 for storage and processing. Specifically, the console 1120 may create or add a database of {ultrasound image, magnetic field strength} pairs. The database may then be used to construct a 3D image in a manner that will be described in greater detail herein.

[0056] Figure 3B Provides such Figure 1 2 shows an alternative embodiment of the reference device 800. For example, in Figure 1 2 , the reference device 800 is shown as a cuff-like device, wherein the body appendage can be received at the interior of the reference device 800. In contrast, Figure 3B The reference device 300 is shown as a wearable device (e.g., an armband) in which the reference magnet 700 is incorporated. It should be understood that Figures 1 to 5 The various embodiments of the reference devices discussed in are interchangeable in the described functionality, particularly with respect to correlating captured ultrasound images with detected magnetic field strengths.

[0057] As mentioned above, in this embodiment, the system 1110 can be configured to detect the position and movement of the ultrasound probe 1140. Specifically, the magnetometer 1192 in the probe 1140 is configured to detect the magnetic field generated by or otherwise associated with the reference magnet 700. In some embodiments, when the magnetometer 1192 includes a plurality of magnetometers, each of the magnetometers can be configured to spatially detect the magnetic field in three-dimensional space. Thus, during operation of the system 1110, the magnetic field strength data of the reference magnet 700 sensed by each of the magnetometers 1192 is forwarded to a processor (e.g., the processor 1122 ( Figure 1 )), the processor and logic (e.g., the magnetometer data receiving logic 1156 ( Figure 1)) is combined to calculate the position of the ultrasound probe 1140 relative to the reference magnet 700 and the distance between the reference magnet 700 and the ultrasound probe 1140 in real time.

[0058] Figure 4 According to some embodiments, the Figure 3A 140. In various embodiments, the reference magnet 700 within the reference device 800 can be configured at different locations relative to the magnetometer 1192 and the ultrasound probe 1140. For example, in a first embodiment, the reference magnet 700 can be positioned in line with the ultrasound probe 1140, or in a manner similar to the embodiment of FIG. Figure 4 In the second embodiment shown in FIG, the reference magnet 700 can be positioned out of line with the ultrasound probe 1140.

[0059] Reference Figure 5 , showing a method for using a Figure 1 A flow chart of an exemplary method of creating 3D images using a magnetic-based tracking system. Figure 5 Each box shown in the Figure 1 The operations performed in the method 500 for creating a 3D image using a magnetic-based tracking system are described. In one embodiment, prior to detecting the proximity of the reference magnet to the magnetometer in the probe, it is assumed that the magnetic-based tracking system includes a probe having a body and a magnetometer, and a reference magnet in a reference device that has been configured to generate a magnetic field. It is further assumed that the magnetometer is configured to detect the magnetic field generated by the reference magnet. Finally, it is assumed that the magnetic-based tracking system includes logic that is stored on a non-transitory computer-readable medium and, when executed by one or more processors, causes the operations associated with proximity detection disclosed herein to be performed.

[0060] As an initial step in method 500, a reference device is configured around a body appendage to be imaged (block 502). As will be appreciated, the reference device can completely or partially surround the body appendage to be imaged. In some embodiments, the reference device can be affixed to the body appendage and can be placed above or below the body appendage for enhanced imaging.

[0061] Advance the ultrasound probe on the patient's skin surface to image the target blood vessel (block 504). As will be appreciated, the probe may be positioned on the skin surface so that an ultrasound beam can be projected toward the target blood vessel for imaging purposes (see FIG. Figure 4 In some embodiments, the advancing step comprises advancing the ultrasound probe in any direction over the surface of the skin.

[0062] As the ultrasound probe is advanced toward the target blood vessel, an ultrasound acoustic transducer or acoustic array within the head of the probe captures images of the target blood vessel (block 506). In some embodiments, the capturing step includes capturing the ultrasound images to a database located on the console.

[0063] Meanwhile, the magnetometers of the probe detect a magnetic field strength generated by or associated with a reference magnet included in the reference device (block 508). In some embodiments, when the magnetometers include multiple magnetometers, the detecting step includes using all of the magnetometers to detect the magnetic field.

[0064] The logic determines a distance between the reference magnet and the magnetometers of the probe based at least in part on the magnetic field strength (block 510). As discussed above, the logic can be stored on the console, the probe, or an alternative electronic device. In some embodiments, when the magnetometers include multiple magnetometers, the logic can use the distance from each magnetometer to determine the distance.

[0065] Once the magnetic field strength between the reference magnet and the magnetometers of the probe is determined, the data is configured into a database that associates the distance between the reference magnet and the magnetometers with the ultrasound images recorded at that exact distance (block 512). In some embodiments, when the magnetometers include multiple magnetometers, the distance data from each magnetometer can be configured to be stored into the database.

[0066] Finally, in response to associating the magnetic field strength between the reference magnet and the magnetometers with the ultrasound images recorded at that exact distance, the logic uses the magnetic field strength associated with each particular image to stitch together multiple ultrasound images to sequentially stitch together the images (block 514). Stitching together the multiple ultrasound images provides a 3D image of the target.

[0067] Referring to Figure 6FIG. 12, shows a block diagram depicting various elements of a magnetic-based tracking system 1112 for tracking an ultrasound probe 1146 and generating three-dimensional (3D) images, in accordance with one example embodiment of the present application. As shown, the system 1112 generally includes a console 1120, an ultrasound probe 1146, and a reference device 802, with each of the probe 1146 and the reference device 802 configured to be communicatively coupled to the console 1120. The console 1120 is shown to include one or more processors 1122, a communication interface 1142, a display 1130, and a non-transitory computer readable medium ("memory") 1150. The memory 1150 is configured to store logic modules including magnetometer data reception logic 1156 and ultrasound probe data reception logic 1152. Further, the memory 1150 can include data stores such as magnetometer data 1158, ultrasound probe data 1154, and relationship data 1160. In some embodiments, the logic currently in the console 1120 will perform the association of the timestamp from the ultrasound image data with the timestamp from the magnetic field strength data. The timestamp for the ultrasound image and the timestamp for the magnetic field strength data can include in other embodiments discussed herein, including the purpose for confirming ultrasound image and magnetic field strength associations in the event of transmission errors from the ultrasound probe 1146 and the reference device 802 to the console 1120.

[0068] In some embodiments, a processor 1122 including non-volatile memory such as EEPROM or the like is included in the console 1120 for controlling system functions during operation of the system 1110, thereby serving as a control processor. The display 1130 in the present embodiment can be integrated into the console 1120 and used to display information to a clinician while using the ultrasound probe 1140. In another embodiment, the display 1130 can be separate from the console 1120.

[0069] In some embodiments, as shown in Figure 6 , the reference device 802 includes a magnetometer 1194 and a power source 1162 to power the magnetometer 1194. In some embodiments, the ultrasound probe 1146 includes an ultrasound acoustic transducer or acoustic array 1148 and a reference magnet 702 that can be configured in a head 1182 of the ultrasound probe 1146. The magnetometer 1194 as shown in Figure 6 performs in a similar manner as discussed above with respect to the magnetometer 1192 of Figures 1 to 4 . However, the magnetometer 1194 of Figure 6 is incorporated into the reference device 802, with the reference device 802 having the magnetometer 1192 detecting the reference magnet 702 strength data and communicating it to the console 1120. The reference magnet 702 as shown in Figure 6 performs in a similar manner as discussed above with respect to the reference magnet 702 ofFigures 1 to 4 in a similar manner as discussed with reference to magnet 700. However, Figure 6 magnet 702 is incorporated into ultrasound probe 1146, where ultrasound probe 1146 does not collect magnetic field strength data.

[0070] In some embodiments, magnetically-based tracking system 1112 includes reference device 802, which further includes magnetometer 1194, and ultrasound probe 1146, which includes reference magnet 702, where reference magnet 702 can be attached to ultrasound probe 1146 via a sleeve around probe handle 1200 or otherwise attached. This embodiment allows reference magnet 702 to be used with various ultrasound probes that do not include a reference magnet configured inside the head 1182 of ultrasound probe 1146.

[0071] Figure 7 A top view of magnetically-based tracking system 1112 including ultrasound probe 1146 is shown in accordance with some embodiments. Figure 6 Ultrasound probe 1146 is employed in conjunction with reference device 802 in order to track the positioning of ultrasound probe 1146 as it is moved around the skin surface of patient 1220. As shown, probe 1146 is configured to be moved along the surface (i.e., skin) 1220 of patient's body appendage 1222. Reference device 802 can be deployed within a threshold distance from body appendage 1222. In some embodiments, reference device 802 can include a hard plastic housing configured in a "U-shape" (such as shown in FIG. 12B), where the interior is formed by the convex portion of the configuration. In this case, body appendage 1222 (e.g., arm) can be placed within the interior. Figure 6

[0072] In some embodiments, magnetically-based tracking system 1112 can be used in preparation for insertion of a needle and / or catheter into the vasculature. Specifically, system 1112 employs the combination of probe 1146, reference device 802 including magnetometer 1194, to track the positioning of ultrasound probe 1146 relative to reference magnet 702 within ultrasound probe 1146. By tracking the positioning of probe 1146 relative to reference magnet 702, system 1112 is able to correlate each ultrasound image obtained by probe 1146 with the positioning of probe 1146 on body appendage 1222. By correlating the ultrasound images with the positioning on body appendage 1222, system 1112 can then stitch together a set of ultrasound images to form a 3D visualization of body appendage 1222. This 3D visualization gives real-time 3D ultrasound guidance and helps reduce complications typically associated with such introductions, including inadvertent arterial puncture, hematoma, pneumothorax, etc.

[0073] ​In some embodiments, the ultrasound probe 1146 includes a head 1182 that houses an ultrasound acoustic transducer or acoustic array 1148 that is used to produce ultrasound pulses and to receive echoes thereof after reflection by the patient's body when the head 1182 is placed against the patient's skin 1220, where each echo can be converted into an image. The ultrasound acoustic transducer or acoustic array 1148 can be in operable communication with the console 1120 for storing ultrasound images. In some embodiments, the ultrasound probe 1146 includes a reference magnet 702 for providing a location of the ultrasound probe 1146 during ultrasound imaging procedures, such as those described above. As will be described in further detail below, the reference magnet 702 can be embedded within the head 1182 of the ultrasound probe 1146. In the present embodiment, the reference magnet 702 is arranged in a planar configuration in the head 1182 of the ultrasound probe 1146, although it should be understood that the reference magnet 702 can be arranged in other configurations within the ultrasound probe 1146. The reference magnet 702 can be present in a paired longitudinal configuration, a paired transverse configuration, a paired circular configuration, or a combination thereof, with the ultrasound acoustic transducer or acoustic array 1148.

[0074] In some embodiments, the magnetometer 1194 can include a series of magnetometers arranged in a configuration to track the reference magnet 702. In one embodiment, the magnetometer 1192 includes three orthogonal sensor coils so as to enable detection of a magnetic field in three spatial dimensions (not shown). An embodiment of a 3D magnetic sensor is the HMC 1052 available from Honeywell Sensing and Control of Morristown, New Jersey. Further, in one embodiment, the magnetometer 1192 is configured as a Hall effect sensor, although other types of magnetic sensors can be employed.

[0075] The magnetometer 1194 can be in communication with the console 1120 for storing information regarding the location of the reference magnet 702 relative to the magnetometer 1194, which will be described in greater detail herein.

[0076] The magnetometer 1194 is configured to detect a magnetic field associated with the reference magnet 702 and to enable the system 1112 to track the reference magnet 702 when it is associated with the magnetometer 1194.

[0077] Referring to Figure 8 FIG. 1 1 shows a system 1100 including a reference magnet 702 according to some embodiments. Figure 7side view of the magnetic-based tracking system 1112 of the ultrasound probe 1146. In some embodiments, as the ultrasound probe 1146 is moved over the surface of the skin 1220, the ultrasound acoustic transducer or acoustic array 1148 is capturing images, while the magnetometer 1194 in the reference device 802 measures the strength of the detected magnetic field generated by the reference magnet 702 in the ultrasound probe 1146 and relays the magnetic field strength information back to the console 1120 along with the captured ultrasound images.

[0078] In other words, as the ultrasound probe 1194 is moved along the patient's body appendage, the ultrasound probe 1146 transmits ultrasound pulses and receives echoes, which are translated into information for a two-dimensional (2D) image of the echoes received at a particular location on the body appendage and time stamped. As disclosed herein, as the ultrasound probe 1146 transmits ultrasound pulses and receives echoes, the magnetometer 1194 within the reference device 802 concurrently detects the magnetic field generated by the reference magnet 702 within the ultrasound probe 1146 and records its strength and time stamps the magnetic field strength, where concurrent means at least partially overlapping in time. Thus, the ultrasound probe 1146 captures information for creating a 2D ultrasound image at a particular location along the body appendage, and the magnetometer 1194 records the strength of the magnetic field detected at that location. The ultrasound information and magnetic field strength and time stamp, including the time stamp, are coupled and transmitted to the console 1110 for storage and processing. For example, multiple ultrasound information, magnetic field strength, and time stamp couplings can be used to create a 3D visualization of at least a portion of the body appendage by stitching together the 2D images created from the ultrasound information based on the magnetic field strength data and time stamps. Specifically, by analyzing the magnetic field strength data, the magnetic field strength data and concurrent time stamps can be used to calibrate the 2D ultrasound images based on a reference point (e.g., the location of the reference magnet 702).

[0079] As Figure 8, the ultrasound probe 1146 obtains a time-stamped first ultrasound image 912 of the blood vessel 1226 using an ultrasound acoustic transducer or acoustic array 1148, while the magnetometer 1194 in the reference device 802 records a time-stamped first intensity 914 of the detected magnetic field generated by the reference magnet 702 within the ultrasound probe 1146. The ultrasound probe 1146 obtains a time-stamped second ultrasound image 916 of the blood vessel 1226 using the ultrasound acoustic transducer or acoustic array 1148, while the magnetometer 1194 records a time-stamped second intensity 918 of the detected magnetic field generated by the reference magnet 702. Further, the ultrasound probe 1146 obtains a time-stamped third ultrasound image 920 of the blood vessel 1226 using the ultrasound acoustic transducer or acoustic array 1148, while the magnetometer 1194 records a time-stamped third intensity 922 of the detected magnetic field generated by the reference magnet 702. The first image 912 and the first intensity 914, the second image 916 and the second intensity 918, and the third image 920 and the third intensity 922 are respectively concatenated and sent to the console 1120 for storage and processing. Specifically, the console 1120 can create or add a database of {(ultrasound image, time stamp)(magnetic field intensity, time stamp)} pairs. The database can then be used to construct a 3D image in a manner that will be described in more detail herein.

[0080] Figure 9 According to some embodiments, the Figure 8 146 and the reference device 802. The magnetometer 1194 within the reference device 802 can be configured at a variety of locations relative to the reference magnet 702 and the ultrasound probe 1146. For example, the magnetometer 1194 can be positioned in line with the ultrasound probe 1146 and the reference magnet 702, or the magnetometer 1194 can be positioned out of line with the ultrasound probe 1146 and the reference magnet 702 (e.g., Figure 4 ).

[0081] Reference Figure 10 , showing a method for using a Figure 6 Flowchart of an exemplary method 600 for creating a 3D image using a magnetic-based tracking system 1112 . Figure 10 Each box shown in the Figure 6operations performed in a method 1000 of creating 3D images by a magnetic-based tracking system. In one embodiment, prior to detecting proximity of a reference magnet in an ultrasound probe to a magnetometer in a reference device, it is assumed that the magnetic-based tracking system includes a probe having a body and a reference magnet configured to generate a magnetic field and a magnetometer in a reference device that has been configured to detect the magnetic field generated by the reference magnet. Finally, it is assumed that the magnetic-based tracking system includes logic stored on a non-transitory computer readable medium and that when executed by one or more processors causes performance of operations associated with proximity detection disclosed herein.

[0082] As an initial step in the method 600, a reference device is configured around a body appendage to be imaged (block 602). As understood, the reference device can completely or partially surround the body appendage to be imaged. In some embodiments, the reference device can be secured to the body appendage and can be placed above or below the body appendage for enhanced imaging.

[0083] The ultrasound probe is advanced over the skin surface of the patient to image a target blood vessel 1226 (block 604). As understood, the probe can be positioned on the skin surface such that an ultrasound beam can be projected toward the target blood vessel for imaging purposes (see FIG. 12B). In some embodiments, the advancing step includes advancing the ultrasound probe in any direction over the surface of the skin. Figure 9 ) In some embodiments, the advancing step includes advancing the ultrasound probe in any direction over the surface of the skin.

[0084] As the ultrasound probe is advanced toward the target blood vessel, an ultrasound acoustic transducer or acoustic array within the head of the probe captures an image of the target blood vessel (block 606). In some embodiments, the capturing step includes capturing the ultrasound image to a database located on a console.

[0085] Meanwhile, the magnetometer of the reference device detects a magnetic field strength generated by or associated with a reference magnet included in or on the ultrasound probe (block 608). In some embodiments, when the magnetometer includes multiple magnetometers, the detecting step includes using all of the magnetometers to detect the magnetic field.

[0086] Logic determines a distance between the reference magnet of the probe and the magnetometer of the reference device based at least in part on the magnetic field strength (block 610). As discussed above, the logic can be stored on the console, the probe, or an alternative electronic device. In some embodiments, when the magnetometer includes multiple magnetometers, the logic can use the distance from each magnetometer to determine the distance.

[0087] Once the magnetic field strength between the reference magnet of the probe and the magnetometer of the reference device is determined, the data is configured into a database that correlates the distance between the reference magnet and the magnetometer to the ultrasound image recorded at the exact distance and the timestamp of the ultrasound image and the timestamp of the magnetic field strength (block 612). In some embodiments, when the magnetometer comprises a plurality of magnetometers, the distance data from each magnetometer can be configured to be stored into the database.

[0088] Finally, in response to correlating the magnetic field strength between the reference magnet and the magnetometer to the ultrasound image recorded at the exact distance, the logic uses the magnetic field strength associated with each particular image and timestamp to stitch together a plurality of ultrasound images to sequentially stitch together the images (block 614). Stitching together the plurality of ultrasound images provides a 3D image of the target.

[0089] In addition to the embodiments described above, the inventive concept can be used in additional embodiments, such as, but not limited to, the following. For example, in one embodiment, the tracking structure of the magnetometer, the reference magnet, and the ultrasound probe are used to provide a 3D view / guidance of an anatomical structure intended for needle access (e.g., nerve block, drainage site, biopsy site, etc.).

[0090] In another embodiment, the combination of the tracking structure and needle tracking system are used to capture the trajectory of the needle in 3D. In further aspects of this embodiment, the system 1110 can record the path of the needle to a target structure (e.g., vessel center, nerve bundle, drainage site, pneumothorax, etc.). In yet another embodiment, a magnetic or electromagnetic needle guidance system is used to visually track within the 3D scan.

[0091] In another embodiment, the clinician can view the 3D scan through a virtual, augmented, or mixed reality system. In further aspects of this embodiment, the clinician can assess or track the needle, wire, or tool in real time.

[0092] In some embodiments, the tracking structure can be combined with a vessel ID method and Doppler capability to map the veins and arteries observed during the scanning process.

[0093] In another embodiment, the tracking structure can be combined with a hands-free ultrasound probe to enable real-time images at the desired insertion location and 3D pre-scan images. The hands-free probe with the combination of the tracking structure will allow the clinician to use both hands simultaneously for needle surgical device manipulation and patient skin stabilization or limb extension interaction.

[0094] In some embodiments, the remotely controlled tourniquet can be a cuff separate from the reference device 800. In some embodiments, the remotely controlled heating element can be a cuff separate from the reference device 800.

[0095] In another embodiment, the tracking structure in combination with ultrasound can be used to create a baseline image of the lung and post-surgical scans to identify changes in the lung (e.g., sliding lung behavior associated with pneumothorax or improved blood flow). To facilitate this embodiment, this combination can be combined with additional methods to identify or highlight structures of interest in the 3D scans, such as, for example, nerve bundles.

[0096] In another embodiment, the tracking structure can include radio-opaque markers according to correlation with x-ray, fluoroscopy, or a combination thereof or image combination.

[0097] While some particular embodiments have been disclosed herein, and while some details regarding the particular embodiments have been set forth, it is not the intention that the concepts provided herein be limited to these particular embodiments. Additional adaptations and / or modifications of these particular embodiments can and will be apparent to those skilled in the art to which the concepts provided herein pertain, and it is contemplated that such adaptations and / or modifications will fall within the scope of the concepts provided herein. Therefore, it is intended that the concepts provided herein be limited only to the extent required by the patent laws.

Claims

1. A magnetic-based tracking system for tracking an ultrasound probe to create three-dimensional (3D) visualization, characterized in that The system comprises: a reference device comprising a reference magnet; an ultrasound probe comprising a magnetometer, wherein the ultrasound probe is configured to capture ultrasound images, and wherein the magnetometer is configured to detect a magnetic field strength of a magnetic field generated by the reference magnet; and A console comprising a processor and a non-transitory computer-readable medium having logic stored thereon that, when executed by the processor, is configured to perform operations comprising: receiving the ultrasonic image, the magnetic field strength, and timestamps corresponding to the ultrasonic image and the magnetic field strength; pairing the ultrasound image with the magnetic field strength based on corresponding timestamps of the ultrasound image and the magnetic field strength; and The 3D visualization of a portion of a patient's body is generated from the ultrasound images by (i) calibrating each of the ultrasound images according to an ultrasound image-magnetic field strength pairing and (ii) stitching the ultrasound images together.

2. The magnetic-based tracking system of claim 1, wherein: The reference device is a cuff-like structure that is wrapped around the body region to be imaged.

3. The magnetic-based tracking system of claim 1 , wherein: The reference device is a U-shaped structure and is configured to allow placement of a body region within the U-shaped structure for imaging.

4. The magnetic-based tracking system of claim 1 , wherein: Also included is a needle tracking system configured to capture a trajectory of the needle in three dimensions, wherein the needle tracking system is configured to record a path of the needle to a target structure.

5. The magnetic-based tracking system of claim 4, wherein: The target structure is one of a vascular center, a nerve bundle, a drainage site, or a pneumothorax.

6. The magnetic-based tracking system of claim 1 , wherein: Also includes: A magnetic or electromagnetic needle guidance system configured to track the path of the needle to the target structure.

7. The magnetic-based tracking system of claim 1 , wherein: The 3D visualization is presented via a virtual, augmented or mixed reality system.

8. The magnetic-based tracking system of claim 1 , wherein: Also includes: Vascular identification system; and A Doppler system, wherein the vessel identification system and the Doppler system are configured to map one or more veins or one or more arteries.

9. The magnetic-based tracking system of claim 1 , wherein: The ultrasound probe is a hands-free ultrasound probe.

10. The magnetic-based tracking system of claim 1, wherein: Also includes: A remotely controlled hemostat is configured to compress a portion of a body region being imaged by the ultrasound probe.

11. The magnetic-based tracking system of claim 1 , wherein: Also included is a remotely controlled heating element configured to cause blood vessels to become engorged with blood.

12. A magnetic-based tracking system for tracking an ultrasound probe to create three-dimensional (3D) visualization, characterized in that The system comprises: a reference device comprising a magnetometer configured to detect a magnetic field generated by a reference magnet and to capture a timestamp for each magnetic field strength; an ultrasound probe comprising an ultrasonic acoustic transducer or an acoustic array configured to capture ultrasound images and a timestamp for each said ultrasound image, and wherein said ultrasound probe comprises a reference magnet configured to generate a magnetic field; and A console comprising a processor and a non-transitory computer-readable medium having logic stored thereon that, when executed by the processor, is configured to perform operations comprising: receiving the ultrasonic image, the magnetic field strength, and timestamps corresponding to the ultrasonic image and the magnetic field strength; pairing the ultrasound image with the magnetic field strength based on corresponding timestamps of the ultrasound image and the magnetic field strength; and The 3D visualization of a portion of a patient's body is generated from the ultrasound images by (i) calibrating each of the ultrasound images according to an ultrasound image-magnetic field strength pairing and (ii) stitching the ultrasound images together.

13. The magnetic-based tracking system of claim 12, wherein: The reference device is a cuff-like structure that is wrapped around the body region to be imaged.

14. The magnetic-based tracking system of claim 12, wherein: The reference device is a U-shaped structure and is configured to allow placement of a body region within the U-shaped structure for imaging.

15. The magnetic-based tracking system of claim 12, wherein: Also included is a needle tracking system configured to capture a trajectory of the needle in three dimensions, wherein the needle tracking system is configured to record a path of the needle to a target structure.

16. The magnetic-based tracking system of claim 15, wherein: The target structure is one of a vascular center, a nerve bundle, a drainage site, or a pneumothorax.

17. The magnetic-based tracking system of claim 12, wherein: Also included are magnetic or electromagnetic needle guidance systems configured to track the path of the needle to the target structure.

18. The magnetic-based tracking system of claim 12, wherein: The 3D visualization is presented via a virtual, augmented or mixed reality system.

19. The magnetic-based tracking system of claim 12, wherein: Also includes: Vascular identification system; and A Doppler system, wherein the vessel identification system and the Doppler system are configured to map one or more veins or one or more arteries.

20. The magnetic-based tracking system of claim 12, wherein: The ultrasound probe is a hands-free ultrasound probe.

21. The magnetic-based tracking system of claim 12, wherein: Also included is a remotely controlled hemostat configured to compress a portion of the body region being imaged by the ultrasound probe.

22. The magnetic-based tracking system of claim 12, wherein: Also included is a remotely controlled heating element configured to cause blood vessels to become engorged with blood.

Images