Target recognition and needle guidance system

Abstract

Target identification and needle guidance systems are disclosed. A console can be configured to instantiate a target identification process for identifying an anatomical target, such as a blood vessel, of a patient and a needle guidance process for guiding insertion of a needle into the anatomical target using ultrasound imaging data received by the console. The system can perform target identification based on artificial intelligence (AI) that can be trained to identify targets based on ultrasound images. An ultrasound probe can be configured to provide electrical signals corresponding to the ultrasound imaging data to the console. The ultrasound probe can include a transducer array, and optionally a magnetic sensor array, configured to convert reflected ultrasound signals from the patient and magnetic signals from the needle when magnetized, respectively, to electrical signals.

Description

[0001] Priority

[0002] This application claims priority to U.S. Patent Application No. 63 / 117,883, filed November 24, 2020, the entirety of which is incorporated by reference into this application. TECHNICAL FIELD

[0003] The present application relates to the field of medical devices, and more particularly to target identification and needle guidance systems. SUMMARY

[0004] One potential issue with ultrasound imaging during catheter insertion is identifying the proper target. For example, challenges can include distinguishing between a vein and an artery, finding a blood vessel suitable for catheter insertion, or finding a particular blood vessel. Locating the correct target can consume clinical time and resources, and errors can result in potentially dangerous complications.

[0005] A second potential issue is locating and guiding the needle tip, particularly as it approaches the target blood vessel. For example, when inserting a peripherally inserted central catheter (PICC), the needle must be guided from its peripheral insertion site through a vein or artery to the target at a large distance, and correctly tracking or estimating the needle’s position can be challenging. Moreover, the target can often be an important central blood vessel, such as the superior vena cava (SVC) or its chamber atrial junction (CAJ). It is therefore important to know the needle’s position and orientation accurately in order to aim for catheter insertion in a minimally invasive manner and with minimal risk of harm.

[0006] Other potential issues include navigation of a catheter tip (e.g., a PICC tip) during insertion, and confirmation of the tip’s location after insertion. In particular, it is important to track the location of a PICC, central venous catheter (CVC), another catheter, or another medical device in order to perform implantation in a minimally invasive and low-risk manner. It is also important to confirm that a medical device, such as a PICC, has been implanted in the correct target, such as the SVC. However, some tip tracking and confirmation methods require fluoroscopy or X-ray exposure, and can also require exposure to harmful contrast agents.

[0007] Disclosed herein is a target identification and needle guidance system. The target identification and needle guidance system includes a console including a memory and a processor, and an ultrasound probe. The console is configured to instantiate: a target identification process to identify an anatomical target of a patient by applying an artificial intelligence model to features of a candidate target in ultrasound imaging data to determine an identification score. The console is further configured to instantiate: a needle guidance process to guide insertion of a needle into the anatomical target using ultrasound imaging data received by the console. The ultrasound probe is configured to provide electrical signals corresponding to the ultrasound imaging data to the console. The ultrasound probe includes a transducer array (optionally piezoelectric) configured to convert reflected ultrasound signals from the patient into ultrasound imaging portions of the electrical signals.

[0008] In some embodiments, the artificial intelligence model includes a supervised learning model trained based on previously identified training targets.

[0009] In some embodiments, the identification score includes a probability or a confidence associated with a classification of the candidate target in the ultrasound imaging data.

[0010] In some embodiments, the artificial intelligence includes one or more of the following supervised learning methods: logistic regression; other linear classifiers; support vector machines; quadratic classifiers; kernel estimation; decision trees; neural networks; or learning vector quantization.

[0011] In some embodiments, the artificial intelligence model includes logistic regression, and the score is determined as a weighted sum of the features of the candidate target.

[0012] In some embodiments, the artificial intelligence model includes a neural network, and the score is determined based on one or more non-linear activation functions of the features of the candidate target.

[0013] In some embodiments, the supervised learning model is trained by iteratively minimizing error with respect to actual classifications of the training targets as compared to classifications predicted by a candidate model.

[0014] In some embodiments, the anatomical target includes a blood vessel.

[0015] In some embodiments, the blood vessel includes the superior vena cava.

[0016] In some embodiments, the ultrasound probe provides a first ultrasound image prior to insertion of the needle. The artificial intelligence model is based in part on a trajectory of the needle.

[0017] In some embodiments, the features of the ultrasound imaging data include one or more of: a shape of a candidate blood vessel of the candidate target in the ultrasound imaging data; a size of the candidate blood vessel; a number of branches of the candidate blood vessel; and a complexity of the branches of the candidate blood vessel.

[0018] In some embodiments, identifying the anatomical target includes one or more of: providing the user with identification information characterizing the one or more potential targets; suggesting the one or more potential targets for the user to select; and selecting a target of the one or more potential targets.

[0019] In some embodiments, identifying the anatomical target includes providing the user with identification information characterizing the one or more potential targets. The identification information includes a predicted classification of the one or more potential targets and a confidence associated with the classification.

[0020] In some embodiments, the target identification and needle guidance system further includes a display screen configured to graphically guide insertion of the needle into the anatomical target of the patient.

[0021] In some embodiments, a catheter is disposed within the needle. After insertion of the needle, the catheter is implanted into the anatomical target.

[0022] In some embodiments, the target identification and needle guidance system further includes a catheter magnetic sensor or a catheter light detector. The catheter magnetic sensor can be configured to provide electrical signals corresponding to magnetic information about the catheter to the console. The catheter light detector can be configured to provide electrical signals corresponding to optical information about the catheter to the console. The processor is further configured to instantiate a magnetic catheter tip tracking process based on the catheter magnetic information or a light catheter tip tracking process based on the catheter optical information.

[0023] In some embodiments, the target identification and needle guidance system further includes one or more electrocardiogram probes configured to provide electrical signals corresponding to electrocardiogram information to the console. The processor is further configured to instantiate a catheter tip placement confirmation process based on the electrocardiogram information.

[0024] In some embodiments, the anatomical target includes a biopsy target, a nerve to be blocked, or an abscess to be drained.

[0025] In some embodiments, the system further includes a needle magnetizer incorporated into the console, the needle magnetizer configured to magnetize the needle to obtain a magnetized needle.

[0026] In some embodiments, the needle guidance process for guiding insertion of the magnetized needle into the anatomical target uses a combination of ultrasound imaging data and magnetic field data received by the console.

[0027] In some embodiments, the ultrasound probe is configured to provide magnetic field data to the console, the ultrasound probe further comprising a magnetic sensor array configured to convert a magnetic signal from the magnetized needle into a magnetic field portion of the electrical signal.

[0028] Also disclosed herein is a method of a target identification and needle guidance system. In some embodiments, the method comprises instantiating a target identification process and a needle guidance process in a memory of a console. The target identification process can be used to identify an anatomical target of a patient by applying an artificial intelligence model to features of a candidate target in ultrasound imaging data to determine an identification score. The needle guidance process can be used to guide a needle insertion into the anatomical target using ultrasound imaging data received by the console. The method further comprises loading ultrasound imaging data into the memory, the ultrasound imaging data corresponding to electrical signals received from an ultrasound probe. The method further comprises processing, using a processor of the console, the ultrasound imaging data in accordance with the target identification process and the needle guidance process. The method further comprises guiding, in accordance with the needle guidance process, the needle insertion into the anatomical target of the patient.

[0029] Also disclosed herein is a method of a target identification and needle guidance system. In some embodiments, the method comprises obtaining a needle. The method further comprises imaging, with an ultrasound probe, an anatomical region of a patient to produce ultrasound imaging data. The method further comprises identifying, by a console, an anatomical target of the anatomical region of the patient by applying an artificial intelligence model to features of a candidate target in the ultrasound imaging data to determine an identification score. The method further comprises orienting the needle for insertion into the anatomical target of the patient while imaging the anatomical region with the ultrasound probe. The method further comprises inserting the needle into the anatomical target of the anatomical region in accordance with guidance provided by a needle guidance process instantiated by the console while processing the combination of ultrasound imaging data.

[0030] In some embodiments, a catheter is disposed within the needle. The method further comprises implanting the catheter into the anatomical target after the needle insertion. In some embodiments, the needle guidance process used to guide the magnetized needle insertion into the anatomical target uses a combination of ultrasound imaging data and magnetic field data received by the console.

[0031] In some embodiments, the ultrasound probe is configured to provide magnetic field data to the console, the ultrasound probe further comprising a magnetic sensor array configured to convert a magnetic signal from the magnetized needle into a magnetic field portion of the electrical signal.

[0032] Also disclosed herein is a target identification and needle guidance system. In some embodiments, the system includes a console configured to instantiate: a target identification process configured to identify an anatomical target of a patient, and a needle guidance process for guiding insertion of a needle into the anatomical target. In some embodiments, the system further includes an ultrasound probe configured to provide electrical signals corresponding to ultrasound imaging data to the console, the ultrasound probe including a transducer array configured to convert reflected ultrasound signals from the patient into an ultrasound imaging portion of the electrical signals.

[0033] In some embodiments, the anatomical target includes a blood vessel. In some embodiments, the blood vessel includes the superior vena cava.

[0034] In some embodiments, the system further includes a display screen configured for graphically guiding insertion of the needle into the anatomical target of the patient.

[0035] In some embodiments, a catheter is disposed within the needle, and the catheter is implanted into the anatomical target after insertion of the needle.

[0036] In some embodiments, the system further includes: a catheter magnetic sensor configured to provide electrical signals corresponding to magnetic information about the catheter to the console; or a catheter optical detector configured to provide electrical signals corresponding to optical information about the catheter to the console, wherein the console is further configured to instantiate a magnetic catheter tip tracking process based on the catheter magnetic information or a light catheter tip tracking process based on the catheter optical information.

[0037] In some embodiments, the system further includes one or more electrocardiogram probes configured to provide electrical signals corresponding to electrocardiogram information to the console, wherein the console is further configured to instantiate a catheter tip placement confirmation process based on the electrocardiogram information.

[0038] In some embodiments, the anatomical target includes a biopsy target, a nerve to be blocked, or an abscess to be drained.

[0039] In some embodiments, the system further includes a needle magnetizer incorporated into the console, the needle magnetizer configured to magnetize the needle to obtain a magnetized needle.

[0040] In some embodiments, the ultrasound probe is configured to provide magnetic field data to the console, the ultrasound probe further including a magnetic sensor array configured to convert magnetic signals from the magnetized needle into a magnetic field portion of the electrical signals.

[0041] These and other features of the concepts provided herein will become more apparent from the following description in conjunction with the accompanying drawings, in which: BRIEF DESCRIPTION OF DRAWINGS

[0042] Embodiments of the disclosure are illustrated by way of example in the drawings in which like references indicate similar elements, and in which:

[0043] Figure 1 An example ultrasound probe and needle guidance system with a patient is shown in accordance with some embodiments;

[0044] Figure 2 Insertion of a needle into a blood vessel using an ultrasound probe and needle guidance system is shown in accordance with some embodiments;

[0045] Figure 3 An example optical catheter tip tracking system and catheter is shown in accordance with some embodiments;

[0046] Figure 4A An example ultrasound probe, target identification system, and needle guidance system with a patient is shown in accordance with some embodiments;

[0047] Figure 4B An example console display for an ultrasound, needle guidance, and target identification system is shown in accordance with some embodiments;

[0048] Figure 5 A block diagram of a needle guidance and target identification system is shown in accordance with some embodiments;

[0049] Figure 6A A target identification system used in conjunction with magnetic catheter tip tracking is shown in accordance with some embodiments;

[0050] Figure 6B A target identification system used in conjunction with optical catheter tip tracking is shown in accordance with some embodiments;

[0051] Figure 7 Catheter tip confirmation is shown in accordance with some embodiments;

[0052] Figure 8 A flowchart of an example method for ultrasound imaging, target identification, and needle guidance is shown in accordance with some embodiments; and

[0053] Figure 9 A flowchart of an example method for target identification is shown in accordance with some embodiments. DETAILED DESCRIPTION

[0054] Before some particular embodiments are disclosed in more detail, it should be understood that the particular embodiments disclosed herein do not limit the scope of the concepts provided herein. It should also be understood that the particular embodiments disclosed herein can have features that are capable of being readily separated from the particular embodiments and optionally combined or replaced with features of any of the many other embodiments disclosed herein.

[0055] With respect to the terms used herein, it should also be understood that these terms are used to describe some particular embodiments and that these terms do not limit the scope of the concepts provided herein. Ordinal numbers (e.g., first, second, third, etc.) are generally used to distinguish or identify different features or steps in a group of features or steps and do not supply a serial or numerical limitation. For example, a “first,” “second,” and “third” feature or step need not be in that order and the particular embodiments including such features or steps need not be limited to three features or steps. Labels such as “left,” “right,” “top,” “bottom,” “front,” “back,” and the like are used for convenience and are not intended to imply, for example, any particular fixed location, orientation, or direction. On the contrary, such terms are used to reflect relative locations, orientations, or directions.

[0056] With respect to “proximal,” for example, a “proximal portion” or “proximal end portion” of a probe disclosed herein includes a portion of the probe that is intended to be proximate to a clinician when the probe is used on a patient. Similarly, for example, a “proximal length” of a probe includes a length of the probe that is intended to be proximate to a clinician when the probe is used on a patient. For example, a “proximal end” of a probe includes an end of the probe that is intended to be proximate to a clinician when the probe is used on a patient. A proximal portion, proximal end portion, or proximal length of a probe can include a proximal end of the probe; however, a proximal portion, proximal end portion, or proximal length of a probe need not include a proximal end of the probe. That is, a proximal portion, proximal end portion, or proximal length of a probe is not a distal portion or distal length of the probe unless the context indicates otherwise.

[0057] With respect to “proximal,” for example, a “proximal portion” or “proximal end portion” of a probe disclosed herein includes a portion of the probe that is intended to be proximate to a clinician when the probe is used on a patient. Similarly, for example, a “proximal length” of a probe includes a length of the probe that is intended to be proximate to a clinician when the probe is used on a patient. For example, a “proximal end” of a probe includes an end of the probe that is intended to be proximate to a clinician when the probe is used on a patient. A proximal portion, proximal end portion, or proximal length of a probe can include a proximal end of the probe; however, a proximal portion, proximal end portion, or proximal length of a probe need not include a proximal end of the probe. That is, a proximal portion, proximal end portion, or proximal length of a probe is not a distal portion or distal length of the probe unless the context indicates otherwise.

[0058] The term "logic" can refer to or include hardware, firmware, or software configured to perform one or more functions. As hardware, the term logic can refer to or include circuitry having data processing and / or storage functionality. Embodiments of such circuitry can include, but are not limited to or limited by, a hardware processor (e.g., a microprocessor, one or more processor cores, a digital signal processor, a programmable gate array, a microcontroller, an application-specific integrated circuit "ASIC," etc.), semiconductor memory, or a combination element.

[0059] Additionally or alternatively, the term logic can refer to or include software such as one or more processes, one or more instances, an application programming interface (API), a subroutine, a function, a applet, a server, a routine, source code, object code, a shared library / dynamic link library (dll), or even one or more instructions. Software can be stored in any type of suitable non-transitory or transitory storage medium (e.g., electrical, optical, acoustical, or other form of propagated signals, such as carrier waves, infrared signals, or digital signals). Embodiments of non-transitory storage media can include, but are not limited to or limited by, programmable circuitry; non-persistent storage, such as volatile memory (e.g., any type of random access memory "RAM"); or persistent storage, such as non-volatile memory (e.g., read-only memory "ROM," power-backed RAM, flash memory, phase change memory, etc.), a solid state drive, a hard disk drive, an optical disk drive, or a portable memory device. As firmware, logic can be stored in persistent storage.

[0060] Disclosed herein are systems, devices, and methods involving an ultrasound probe, a target identification system, and a needle guidance system. In procedures and treatments that utilize ultrasound imaging, such as catheterization, biopsy, nerve block, or drainage, one common challenge is determining the proper target. A second challenge is locating and guiding the needle tip, particularly as it approaches the target vessel. Additional challenges can include navigation of a catheter tip (e.g., a peripheral inserted central catheter (PICC) tip) during insertion of the target, and confirmation of the tip location after insertion.

[0061] Figure 1 An example ultrasound probe 106 included within a needle guidance system 100 with a patient P is shown in accordance with some embodiments. The system 100 performs ultrasound imaging of an insertion target (in this embodiment, a blood vessel system of the patient P) in conjunction with magnetic needle guidance. The needle guidance system 100 can have a variety of uses, including placing a needle in preparation for insertion of a catheter 112 or other medical component into the patient's body. In this embodiment, a clinician employs the ultrasound probe 106 and the needle guidance system 100 in performing a process of placing the catheter 112 through a skin insertion site S into the blood vessel system of the patient P. The ultrasound imaging and needle guidance functions together enable the clinician to accurately guide the needle to its intended target.

[0062] However, there remains a need for assistance in effectively and accurately identifying insertion targets. In particular, there remains a need to reduce errors in target identification that can lead to dangerous complications for patients, while also reducing the cost and clinical time required to locate, identify, and insert a needle into a target. The systems and methods disclosed below can meet these needs.

[0063] Figure 1 A needle-based device, i.e., a catheter insertion device 144, is further depicted for obtaining initial access to a patient P’s vasculature via an insertion site S to deploy a catheter 112. A needle is typically placed into the vasculature at the insertion site S prior to performing insertion of the catheter 112. The catheter 112 typically includes a proximal portion 114 that remains outside the patient after placement is complete and a distal portion 116 that resides within the patient P’s vasculature. The needle guide system 100 is used to ultimately position a distal tip 118 of the catheter 112 at a desired location within the vasculature. The proximal portion 114 of the catheter 112 also includes a Luer connector 120 that is configured to operably connect the catheter 112 with one or more other medical devices or systems.

[0064] The needle of the catheter insertion device is configured to cooperate with the needle guide system 100 to enable the needle guide system 100 to detect the position, orientation, and advancement of the needle during the ultrasound-based placement procedure. In some embodiments, other needles or medical devices can also be magnetized and used with the needle guide system 100.

[0065] The display screen 104 is integrated into the console 102 and can display information to a clinician during the placement procedure, such as ultrasound images of the target intrabody portion obtained by the ultrasound probe 106. In particular, the needle guide procedure of the needle guide system 100 can graphically guide needle insertion into the target with the aid of the display screen 104.

[0066] Figure 2 Insertion of a needle into a blood vessel using the ultrasound probe 106 and the needle guide system is shown in accordance with some embodiments. The ultrasound probe 106 includes a sensor array for detecting the position, orientation, and motion of the needle during an ultrasound imaging procedure. The sensor array includes a plurality of magnetic sensors 258 embedded on or included in the housing of the ultrasound probe 106. When the needle is magnetized and brought close to the sensor array, the magnetic sensors 258 detect a magnetic field or magnetic signal associated with the needle. The sensors 258 also convert the magnetic signal from the magnetized needle to the aforementioned magnetic field portion of the electrical signal to the console 102. (See the magnetic field B of the needle shown.) Thus, the sensor array 146 enables the needle guide system 100 to track the magnetized needle, among other things.

[0067] Although shown as magnetic sensors in this embodiment, the sensors 258 can be other types and configurations of sensors. Moreover, in some embodiments, the magnetic sensors 258 of the sensor array can be included in a component separate from the ultrasound probe 106, such as a separate handheld device. In some implementations, the magnetic sensors 258 are arranged in a ring structure around the head 146 of the ultrasound probe 106, although it will be appreciated that the magnetic sensors 258 can be arranged in other structures, such as an arched, planar, or semicircular arrangement.

[0068] Five magnetic sensors 258 are included in the sensor array in order to enable detection of the needle in three spatial dimensions (i.e., X, Y, Z coordinate space), as well as the pitch and yaw orientation of the needle itself. More generally, the number, size, type, and location of the magnetic sensors 258 of the sensor array can differ from what is expressly shown herein.

[0069] During the process of inserting the needle or associated medical device (e.g., the catheter 112 of the catheterization device 144) into the body of the patient P, the needle of magnetizable material enables the needle to be magnetized by the magnetizer 108 and subsequently tracked by the needle guidance system 100 as the needle is percutaneously inserted into the body of the patient (e.g., the body of the patient P). For example, the needle can be composed of stainless steel such as SS 304 stainless steel or some other suitable needle material that can be magnetized, and is not limited by the disclosure. The needle material can be ferromagnetic or paramagnetic. Thus, the needle can generate a magnetic field or signal that can be detected by the sensor array of the ultrasound probe 106 in order to enable the position, orientation, and motion of the needle to be tracked by the needle guidance system 100.

[0070] The needle guidance system 100 and magnetic sensors 258 are further described in US2021 / 0169585, which is incorporated by reference in its entirety into the present application.

[0071] In addition to tracking the position of the inserted needle, it is also useful to track the medical device (e.g., the tip and / or stylet of a PICC, central venous catheter (CVC), or other catheter) during insertion. For example, such tip tracking can improve the accuracy and efficiency of catheter placement, improve the treatment outcomes for patients, and reduce the risk of complications or injury. Thus, in some embodiments, the system can use passive magnetic catheter tip tracking to locate and / or track the tip of a medical device like a PICC, catheter, and / or stylet, and from this can determine where these tips are located relative to their target anatomy. This will be further discussed in the embodiments of Figure 6A In certain cases, the magnetic and electromagnetic devices used to track the medical device tips are susceptible to interference. Thus, optical tip tracking methods can also be used.

[0072] Figure 3An embodiment optical catheter tip tracking system 300 and catheter 350 are shown in accordance with some embodiments. As shown, the optical tip tracking system 300 includes a lighted stylet 310, a light detector 320, and a console 102 configured to be operatively connected to the lighted stylet 310 and the light detector 320. The optical tip tracking system 300 also includes a display screen 104, which can be a standalone screen or integrated into the console 102. The optical tip tracking system 300 can also include a medical device, such as the catheter 350 in this embodiment.

[0073] In this embodiment, the catheter 350 is a peripheral inserted central catheter (PICC). In another embodiment, the catheter 350 can be a central venous catheter (CVC). The lighted stylet 310, which includes a light source (e.g., one or more LEDs in a distal portion (e.g., a tip) of the stylet 310) can be disposed in one or more lumens of the catheter 350. Alternatively, the lighted stylet 310 can deliver light (e.g., via an optical fiber) from an external source (e.g., light within the console 102) to emit light from a distal portion (e.g., a tip) of the stylet 310.

[0074] In this embodiment, the catheter 350 is a dilution catheter, including a catheter tubing 352, a bifurcated hub 354, two extension legs 356, and two luer connectors 358. Alternatively, the catheter 350 can be a single lumen catheter, or a multi-lumen catheter having three or more lumens. In this embodiment, two lumens extend through the dilution catheter 350, and are formed by adjacent lumen portions. Each of the two extension legs 356 has an extension leg lumen fluidically connected to one of the two hub lumens. Either lumen extending through the catheter 350 can house a lighted stylet 310 disposed therein.

[0075] The system 300 can further include a light detector 320, which includes a plurality of photodetectors 122 disposed within the light detector 320 and configured to detect light emitted from the light source of the lighted stylet 310. The photodetectors 122 are arranged in an array such that light emitted from the lighted stylet 310 remains detectable by at least one of the photodetectors 122 even if the light is anatomically blocked (e.g., by a rib) from another or multiple photodetectors 122.

[0076] The optical catheter tip tracking system 300 and the lighted stylet 310 are further described in US2021 / 0154440, which is incorporated by reference in its entirety into the present application.

[0077] However, another common problem with ultrasound imaging is identifying the proper anatomical target. For example, some of the challenges clinicians face when performing ultrasound- assisted catheterization include distinguishing between veins and arteries, finding a particular target artery or vein, such as the superior vena cava (SVC), and generally finding a suitable target vessel for catheterization. Locating the correct target can be time and resource consuming for the clinician, and errors can lead to potentially dangerous complications. The disclosed systems and methods can address these issues by providing a target identification system in conjunction with an ultrasound probe and needle guidance system. Thus, the disclosed systems can simultaneously determine the location of an anatomical target and a medical instrument relative to an ultrasound probe. The disclosed embodiments can also be applied in other ultrasound-based procedures and treatments, such as for biopsy, nerve block, or drainage procedures.

[0078] Figure 4A An exemplary ultrasound probe 406 and target identification and needle guidance system 400 with a patient are shown in accordance with some embodiments. In this embodiment, the system 400 includes both target identification and needle guidance functionality. Specifically, the system 400 can perform target identification based on artificial intelligence (AI) methods, which can be trained to identify targets based on ultrasound images.

[0079] For example, the system can identify a particular blood vessel or class of blood vessels based on AI, or identify a target in ultrasound-based biopsy, nerve block, or drainage procedures. In the case of a blood vessel target, the system 400 can be trained using ultrasound data from correctly identified blood vessels, and can classify or segment blood vessels based on features such as shape (e.g., eccentricity, internal angles, or aspect ratio), size (e.g., short or long axis length, diameter, or circumference), number and complexity of branches, and the like. In one embodiment, training can proceed by iteratively minimizing the error or loss function of the predicted versus actual classification in a training set as a function of the model weights. The hyperparameters of the model can be further adjusted with respect to a validation dataset, and the model can be further evaluated against a test dataset. The trained model can then be deployed for use in a clinical setting.

[0080] In a typical use embodiment, a physician can use the ultrasound probe 406 and target identification and needle guidance system 400 while inserting a PICC into the superior vena cava of a patient P. For example, the PICC can be advanced through the right basilic vein, right axillary vein, right subclavian vein, right cephalic vein of the patient P, and into the superior vena cava. In this embodiment, the system 400 can assist the practitioner in identifying the superior vena cava in the ultrasound image, while also assisting in guiding the insertion needle toward the superior vena cava. In some embodiments, the system 400 can additionally track the location of the PICC and confirm that the PICC is properly inserted into the superior vena cava, as described below.

[0081] In one embodiment, the system 400 can assist the clinician in identifying a target by providing information characterizing potential targets, such as a predicted classification and a confidence associated with the classification, to the clinician. Alternatively, the system 400 can suggest one or more potential targets for approval by the clinician, or even automatically select a target.

[0082] The system can perform classification or segmentation of targets based on any AI or machine learning (ML) method, and is not limited by the disclosed text. In some embodiments, the AI or machine learning method includes a supervised classification method and / or an image segmentation method. For example, the system can use a logistic regression or other linear classifier, a support vector machine, a quadratic classifier, a kernel estimation, a decision tree, a neural network, a convolutional neural network, or a learning vector quantization. In typical embodiments, the system 400 uses a supervised learning method to perform image segmentation or classification of potential targets. For example, the system 400 can identify a particular target, such as positively identifying the superior vena cava or the atrioventricular junction.

[0083] Alternatively, the system 400 can provide a probability that a potential target is a target desired by the user, such as an 80% or 90% probability that a particular blood vessel is the SVC. In another embodiment, the system 400 can more broadly classify a target, such as by identifying whether a blood vessel is a vein or an artery.

[0084] The AI or machine learning method can produce a model that can be trained prior to clinical use as described above based on a training set including ultrasound images of known targets. For example, a clinician can download training data and / or a trained and adjusted model from a centralized repository. In such embodiments, the clinician can occasionally download new training data or a new training model in order to identify new targets or classes of targets, or to improve the accuracy of identification. Alternatively, the clinician can train the system 400, such as by inputting information about the features or identity of a potential target to the system 400, or correcting any false identifications of a target. In some embodiments, the system can alternatively use an unsupervised learning method to segment or classify potential targets, and is not limited by the disclosed text.

[0085] In addition to only pre-trained models (e.g., models trained prior to use), the AI or machine learning method can be utilized to generate models that can be dynamically trained, such as using the scored results of the model as training data.

[0086] By providing ultrasound-based target identification in conjunction with needle guidance, the system 400 can provide a precise, safe, and fast method of ensuring that a needle is implanted in a predetermined target. The system 400 can automate key steps of ultrasound-based therapy, such as identifying an anatomical target and advancing a needle during catheter insertion. Thus, this automation reduces the time and effort of the clinician and can improve clinical accuracy and reduce the incidence of errors and complications.

[0087] Furthermore, Figure 4A The general relationship of the system 400 to the patient P during the process of placing the catheter 412 through the skin insertion site S into the vasculature of the patient P is shown. In the above Figure 1 In embodiments of the system 400, the catheter 412 includes a proximal portion 414 that remains outside the patient after placement is complete and a distal portion 416 that resides within the vasculature. With ultrasound imaging, magnetic needle guidance, and target identification, the system 400 is employed to position the distal tip 418 of the catheter 412 at a desired location within the vasculature.

[0088] The needle is typically placed into the vasculature of a patient (e.g., the patient P) at the insertion site S prior to performing the insertion of the catheter 412. It should be appreciated that the target identification and needle guidance system 400 has a variety of uses, including placing a needle to prepare for the insertion of the catheter 412 or other medical component into the body of the patient P, such as an X-ray or ultrasound marker, a biopsy sheath, a nerve block, a drain, an ablation component, a bladder scanning component, a vena cava filter, etc. For example, a fine needle aspiration biopsy can be performed using endoscopic ultrasound by guiding the placement of a biopsy needle. In this case, the disclosed ultrasound and target identification and needle guidance system 400 can be used to image and identify a target for biopsy and guide a biopsy needle to the target. In another embodiment, ultrasound can be used to guide needle placement in a nerve block procedure. In this case, the disclosed system 400 can be used to image and identify a target nerve and guide a needle to the target. In a third embodiment, ultrasound can be used to guide a drainage procedure, e.g., to guide the placement of a drainage catheter to drain collected fluid such as from an abscess or post-surgical collected fluid. In this case, the disclosed system 400 can be used to image and identify a target abscess and guide a needle and / or catheter to the target.

[0089] The display screen 404 is integrated into the console 402 and is used to display information to the clinician, such as ultrasound images of the in-vivo portion of the target obtained by the ultrasound probe 406. In effect, the needle guidance process of the target identification and needle guidance system 400 graphically guides needle insertion into a target (e.g., a blood vessel) of a patient by means of the display screen 404. In some embodiments, the display screen 404 can be separate from the console 402. In some embodiments, the display screen 404 is an LCD device.

[0090] In this embodiment, the display 404 shows an ultrasound image obtained by the ultrasound probe 406. In particular, the image can include a target for needle insertion, such as a target blood vessel for insertion of a PICC. The system 400 can apply an Al method by applying a supervised learning model to process features of the target in the ultrasound image to identify the target, and thereby determine a classification of the target and an associated confidence score, as disclosed herein. In addition, the system can use the magnetic needle guidance in conjunction with the ultrasound to inform the clinician when the needle reaches near the target, for example by ultrasound reflections or "glint" from the needle. This will be further described in the embodiments of Figure 4B In some embodiments, the system 400 can further provide PICC tracking and PICC placement confirmation, as described in the embodiments of Figure 6A 、 Figure 6B and Figure 7 below.

[0091] Figure 4A A needle-based device, i.e., a catheter insertion device 444, is additionally depicted for obtaining initial access to the vasculature of the patient P via the insertion site S to deploy a catheter 412. The needle of the catheter insertion device 444 can cooperate with the needle guidance system 400 to enable the needle guidance system 400 to detect the position, orientation, and advancement of the needle during the ultrasound-based placement procedure. Note that the needle of the catheter insertion device 444 is merely one embodiment of a needle or medical device that can be magnetized and used with the needle guidance system 400.

[0092] The needle magnetizer 408 is configured to magnetize all or a portion of a needle, such as the needle of the catheter insertion device 444, to enable tracking of the needle during the placement procedure. The needle magnetizer 408 can include a single permanent magnet, a single electromagnet, multiple permanent magnets, multiple electromagnets, or a combination thereof within a body of the needle magnetizer 408. For example, if a single permanent magnet, the permanent magnet can be a hollow cylindrical magnet disposed within the body of the needle magnetizer. If more than one permanent magnet, the permanent magnets can be disposed within the body of the needle magnetizer 408 in a multipole arrangement. Alternatively, the permanent magnets are annular magnets disposed within the body of the needle magnetizer in a stacked arrangement. In some embodiments, the needle magnetizer 408 can also be used to magnetize a portion or all of a catheter, such as a tip of the catheter.

[0093] Figure 4B An exemplary console display 450 of an ultrasound, needle guidance, and target identification system is shown in accordance with some embodiments. In this embodiment, a clinician is inserting a needle into a blood vessel 460, as shown in an ultrasound image 465 in the console display 450. The ultrasound image 465 shows a cross-section of the depth direction of the patient tissue, which is below the ultrasound probe 406 and corresponds to the insertion site S. The needle 442 is shown in the cross-section of the ultrasound image 465, as well as the tip of the catheter 412. Figure 2The ultrasound beam 260 in the embodiment of FIG. 4B. The tissue section includes a target blood vessel 460. A depth scale 470 shows the advancing depth along the vertical dimension of the displayed tissue section. As the needle advances, the display 450 also displays the profile of the needle shaft (solid line). As described herein, the system can determine the position of the needle shaft via a magnetic or optical needle tracking method. In addition, the display 450 displays the projection of the needle’s trajectory (dashed line).

[0094] In this embodiment, the system can apply an artificial intelligence (AI) method to identify the target blood vessel 460. In typical implementations, the system uses a supervised learning method to partition or classify potential targets. For example, the system can use a logistic regression or other linear classifier, a support vector machine, a quadratic classifier, a kernel estimation, a decision tree, a neural network, a convolutional neural network, or a learning vector quantization. In some implementations, the ultrasound image 465 can be filtered (e.g., by a linear filter such as a mean filter, or a non-linear filter such as a sequential statistical filter) to improve image quality. In some implementations, a partitioning of the image 465 into smaller blocks or an active contour model can be applied. In some implementations, a pattern matching based on Markov random fields or neural networks can be applied.

[0095] In one implementation, the system 400 can classify potential targets based on an AI model. In applying the model, the system 400 can evaluate the model based on features corresponding to observable attributes of potential targets (e.g., blood vessels) in the image 465. For example, the features can include the shape (e.g., eccentricity, internal angle, or aspect ratio) of the blood vessels, the size (e.g., minor or major axis length, diameter, or circumference), the number and complexity of branches, etc. In one embodiment, the model is a regression model, and the system 400 evaluates the model as a probability function of the feature values, e.g., a weighted sum. In another embodiment, the model is an artificial neural network such as a convolutional neural network, and the system 400 evaluates the model by successively applying one or more layers of neurons on the feature values. Each neuron can include a non-linear activation function on its input. Specifically, the first layer of neurons can apply a non-linear activation function on all feature values, and subsequent layers can apply a non-linear activation function on all outputs of the previous layer. Such artificial neural networks can adjust their weighted associations according to a learning rule based on error values or another cost function. In the case of multiple hidden layers, the system 400 can identify abstract features via deep learning based on the original set of input features.

[0096] Based on AI, the system can identify the target 460, and / or can assist the clinician in identifying the target by providing the clinician with identifying information characterizing the potential target. For example, the clinician can inform the system that the desired target is the superior vena cava of the patient. In this embodiment, the target identification logic can then determine that a particular blood vessel is the superior vena cava, and can label the blood vessel accordingly to recommend to the clinician as the insertion target. Alternatively, the target identification logic can automatically assume that this identified blood vessel is the target, and can proceed to guide the clinician based on this assumption. In identifying a particular target such as the SVC, the console can receive the ultrasound image 465, and can use the image 465 as input into an AI-based model that was previously trained on ultrasound images of the identified blood vessel. The evaluation of the AI-based model can yield a probability or a confidence score that the blood vessel within the ultrasound image is the particular blood vessel requested by the user (e.g., the SVC).

[0097] In alternative embodiments, the clinician can not inform the system of a particular desired target. In this case, the target identification logic can then identify all potential targets present in the ultrasound image 465, and can label all of these potential targets to communicate to the clinician. In one embodiment, the target identification logic can rank the potential targets, for example, by relevance or importance as an insertion target, by confidence of classification, etc., and can communicate the recommendations to the clinician in the order of the ranking. In identifying all potential targets present in the ultrasound image 465, the console can receive the image 465, and can use the image 465 as input into an AI-based model trained on ultrasound images of the identified blood vessel. The console can evaluate the AI-based model for each respective potential target in the received image 465, yielding a respective probability or confidence score that each respective potential target is any one of a set of known targets.

[0098] Once potential targets have been identified and / or labeled for display, the console can instruct the display 450 to output to the user identifying information about the targets. For example, the display 450 can display a predicted classification of the target 460, such as a vessel type, location, or name, and can display a confidence associated with the classification. The predicted classification can be based on the AI model, such as by selecting the classification with the highest confidence score or the classification with a confidence score above a threshold, such as 95%. Alternatively, the system 400 can suggest one or more potential targets for approval by the clinician, or even automatically select a target. For example, if there is more than one potential target in the depth-wise portion of the patient tissue displayed on the display 450, the system 400 can indicate the potential targets to the user, such as by highlighting them on the display 450, or can provide the user with a menu of one or more suggested targets, or automatically select a target. Such suggestions can be based on the output of the AI model, such as based on having a high confidence score.

[0099] The system can use the image 465 as input to display a predicted classification of the target 460 and / or a list of potential targets based on probabilities or confidence scores from evaluating the AI-based segmentation or classification model. For example, if the image 465 contains two potential targets, and the system has been trained to identify 90 identified target vessels, the system can evaluate the probability that each of the two potential targets is a classification of the 90 known targets.

[0100] In one embodiment, the system can determine with high confidence that a first potential target is a particular vessel, such as the SVC, and can therefore identify the first potential target via the display 450. The system can identify the first potential target when the probability or confidence score from the AI-based model exceeds a threshold, such as greater than 97% confidence. In another embodiment, the system can identify that a second potential target has a 75% chance of being a right renal vein, and a 25% chance of being a right superior renal vein, and can display these two probabilities accordingly via the display 450. The system can display multiple probabilities when the scores from the AI-based model are below a threshold, such as below 97% confidence.

[0101] In some embodiments, the console can display the target by overlaying a visual indicator (e.g., a circle or square, or another icon) on the target 460 in the ultrasound image 465. In one embodiment, the display 450 can indicate, for example, whether the needle trajectory will intersect the target 460 by highlighting the intersection. In some embodiments, the system can determine the target in part based on the projected trajectory of the needle; for example, the system can identify potential targets in or near the needle's trajectory. For example, the AI ​​model can include one or more features that measure the proximity of potential targets to the needle's trajectory. In this case, the AI ​​model can be trained based on these proximity features and the geometric features of the potential target. In one embodiment, the console can determine and display the distance from the needle's current position to the target. In one embodiment, the console can also estimate and display the time until the needle will advance to the target. In various embodiments, the distance and / or time to the target can be estimated by formulas and / or AI-based methods. For example, the system can be trained to estimate the time based on a previously implanted training set.

[0102] Note that insertion can occur outside the plane (i.e., when the needle trajectory is not within the plane of the ultrasound beam 260, such as...). Figure 2 The insertion is performed either in-plane (as shown in the embodiments) or in-plane (i.e., when the needle trajectory is in the plane of the ultrasound beam). During in-plane insertion, solid lines show the outline of the actual position of the needle. However, during out-of-plane insertion, since part or all of the needle may not actually be in the plane of the ultrasound beam, solid lines show the projection of the needle outline into the plane of the beam. In both cases, the position of the needle can be determined by magnetic or optical needle tracking, as described herein, rather than directly from the ultrasound image. In the case of out-of-plane insertion, the display 450 may also display an intersection window, which is the intersection between the needle axis and the plane of the ultrasound beam.

[0103] During out-of-plane insertion, adjusting the angle of the needle up or down, or changing the distance between the needle and the probe, can alter the position of the intersection window. Shallow insertions require a flat angle, while deep insertions require a steep angle.

[0104] As the needle advances through the intersection window, a needle flash appears within the intersection window. The needle flash is a reflection of the ultrasonic energy emanating from the needle. The presence of the needle flash indicates that the needle is at the target location, as it shows that the needle tip has reached the plane of the ultrasonic beam.

[0105] In some implementation schemes, Figure 1The system 100 can include an augmented reality (AR) headset and corresponding AR anatomical representation logic. The AR anatomical representation logic can be configured to generate virtual objects that are presented in an AR environment, such as virtual reality, augmented reality, or mixed reality, where the virtual objects overlay images produced by the system 10 (e.g., ultrasound images) or a series of images (e.g., a video) associated with a real-world setting (e.g., a video including a patient or a body part of a patient, a physical structure, etc.). The virtual objects can be visible through the AR headset or visible on a display of the console 102 without the AR headset. For example, Figure 4B The display 450 can be represented as a virtual object that is visible through the AR headset or visible on the display of the console 102. Alternatively, instead of the system 100 as described herein, it is contemplated that a magnetic field imaging system can be deployed. It is contemplated that the components and functionality of the console 102 described with reference to the system 100 should be understood to apply to a magnetic field imaging system or similar system. Nonetheless, in some embodiments of the system 100, at least a portion of the functionality of the AR anatomical representation logic can be deployed within the AR headset in place of the console 102. Here, the AR headset or another component operating in cooperation with the AR headset can function as the console or perform its functionality (e.g., processing).

[0106] With respect to “augmented reality,” the term augmented reality can relate to virtual reality, augmented reality, and mixed reality, unless the context suggests otherwise. “Virtual reality” includes virtual content in a virtual setting, which can be fanciful or a simulation of a real-world setting. “Augmented reality” and “mixed reality” include virtual content in a real-world setting, such as a real depiction of a portion of a patient’s body including an anatomical element. Augmented reality includes virtual content in a real-world setting, but the virtual content is not necessarily anchored in the real-world setting. For example, the virtual content can be information that overlays the real-world setting. This information can change as the real-world setting changes due to changes in temporal or environmental conditions in the real-world setting, or the information can change as a consumer of the augmented reality moves in the real-world setting; however, the information still overlays the real-world setting. Mixed reality includes virtual content that is anchored in every dimension of the real-world setting. For example, the virtual content can be a virtual object that is anchored in the real-world setting. The virtual object can change as the real-world setting changes due to changes in temporal or environmental conditions in the real-world setting, or the virtual object can change to accommodate a perspective of a consumer of the mixed reality as the consumer moves in the real-world setting. The virtual object can also change according to any interactions with the consumer or another real-world or virtual agent. The virtual object remains anchored in the real-world setting unless the consumer of the mixed reality or some other real-world or virtual agent moves the virtual object to another location in the real-world setting. Mixed reality does not exclude the aforementioned information that overlays the real-world setting described with reference to augmented reality.

[0107] Augmented reality (AR) headsets and corresponding AR dissection representation logic are further described in U.S. Patent Application No. 17 / 397,486, filed August 9, 2021, the entirety of which is incorporated by reference into the present application.

[0108] Figure 5 A block diagram of a needle guidance and target identification system 500 is shown in accordance with some embodiments. A console button interface 540 and control buttons 142 (see Figure 1 ) included on the ultrasound probe 406 can be used by the clinician to immediately call up a desired mode to the display screen 404 to assist in the placement procedure. As shown, the ultrasound probe 406 also includes a button and memory controller 548 for control buttons and ultrasound probe operation. The button and memory controller 548 can include non-volatile memory (e.g., EEPROM). The button and memory controller 548 is in operable communication with a probe interface 550 of the console 402, which includes a piezoelectric input / output component 552 for interfacing with the probe piezoelectric array and a button and memory input / output component 554 for interfacing with the button and memory controller 548.

[0109] The console 402 houses various components of the needle guidance and target identification system 500, and the console 402 can take various forms. A processor 522 including memory 524 such as random access memory (“RAM”) and non-volatile memory (e.g., electrically erasable programmable read only memory (“EEPROM”)) is included in the console 402 for controlling system functions and executing various algorithms during operation of the needle guidance and target identification system 500.

[0110] For example, the console 402 is configured with target perception logic 560 to instantiate a target identification process for identifying an anatomical target (e.g., a blood vessel such as the SVC) of a patient. As disclosed herein, the target identification process involves processing ultrasound images by the console 402 with artificial intelligence methods to identify the anatomical target. In particular, the target perception logic 560 can apply image processing logic 562 to receive ultrasound images and locate potential targets within the images. It can then apply target identification logic 564 to identify the potential targets, for example by determining the identity of individual potential targets, or by matching one potential target to a desired target. The target identification logic 564 can use supervised learning heuristics, as described above Figure 4A and Figure 4BAs described in the embodiments. Finally, the target-aware logic 560 can apply the target recommendation logic 566 to suggest one or more targets to the user. For example, the target recommendation logic 566 can provide clinicians with information to identify and / or characterize potential targets, such as the predicted classification of the potential target and the confidence level associated with the classification. Alternatively, the target recommendation logic 566 can suggest one or more potential targets for clinician approval, or even automatically select a specific target.

[0111] Furthermore, console 402 is configured to instantiate a needle-guiding procedure for guiding needle insertion into an anatomical target. The needle-guiding procedure uses a combination of ultrasound imaging data and magnetic field data received by console 402 to guide the needle insertion into the patient's target.

[0112] The digital controller / analog interface 526 is also included in the console 402 and communicates with the processor 522 and other system components to manage the interface between the ultrasound probe 406, the needle magnetizer 408, the RFID tag reader 410, and other system components.

[0113] RFID reader 410 is configured to read at least the needle or other medical device (e.g., catheter insertion device 144, such as...) Figure 1 The passive RFID tag included with the needle guidance and target identification system 500 allows the system to customize its operation based on specific needle or medical device parameters (e.g., type, size, etc.). For example, once instantiated by the needle guidance and target identification system 500, the needle guidance process is configured to adjust needle guidance parameters based on electronically stored information read from the RFID tag used for the needle. To read such an RFID tag, an RFID tag reader is configured to transmit interrogation radio waves to the RFID tag and read the electronically stored information from the RFID tag.

[0114] The needle-guided and target recognition system 500 also includes a port 528 for connection to additional components, such as optional components 530 including printers, storage media, keyboards, etc. In some embodiments, port 528 is a Universal Serial Bus (“USB”) port, although other port types or combinations of port types may be used for this and other interface connections described herein. The console 402 includes a power connection 532 to enable an operative connection to an external power supply 534. An internal power supply 536 (e.g., a battery) may also be used with or without the external power supply 534. Power management circuitry 538 is included in the digital controller / analog interface 526 of the console 402 to regulate power usage and distribution.

[0115] In addition to tracking the location of an insertion needle, it is also useful to track a medical device (e.g., the tip and / or the trocar of a PICC, central venous catheter (CVC), or other catheter) during insertion. For example, such tip tracking can improve the accuracy and efficiency of catheter placement, improve patient outcomes, and reduce the risk of complications or injury.

[0116] Figure 6A A target identification system 600 used with magnetic catheter tip tracking is shown, in accordance with some embodiments. In some embodiments, the system 600 can use passive magnetic catheter tip tracking to locate and / or track the tips of medical devices like PICCs, catheters, and / or trocars, and from that can determine where these tips are located relative to their target anatomy. In this example, a PICC 412 is magnetically tracked by the system 600 as it is inserted into a patient P. Alternatively, another catheter or another medical instrument can be tracked. The details of the operation of the magnetic tip tracking can be similar to the function of the magnetic needle guidance, as in the example above Figure 4B In this example, a sensor 610, which can be located on the chest of the patient P, can detect the magnetic field from the PICC 412.

[0117] In this example, the sensor 610 of the tip tracking system can detect the location of the PICC 412, and can send a signal representing the location of the PICC 412 to the console 402 to be displayed on the screen 404. In conjunction with the tip tracking system, the console 402 can also apply AI to identify a target, such as a target blood vessel. The display screen 404 can display an ultrasound image of the target blood vessel 630, as well as the location of the PICC 412 and / or the insertion needle.

[0118] Initially, the tip of the PICC 412 or the tip of the trocar 635 disposed within the PICC 412 can be outside the range of the sensor 610. As the trocar tip approaches the sensor 610, the console can display the position, orientation, and depth of the trocar 635 relative to the sensor 610. The PICC 412 must be advanced slowly (e.g., 1 cm per second) and steadily in order to be accurately tracked. The clinician can continue to slowly advance the PICC 412 until the PICC 412 is inserted as far as desired.

[0119] Figure 6A ECG probes 620 and 625 are also shown. In one embodiment, this system can also be used to perform catheter tip confirmation based on ECG during or after the PICC 412 is inserted into the target blood vessel (or other target anatomy). Tip confirmation can confirm that the catheter tip has been properly inserted into the target. This will be described belowFigure 7 discussed in the embodiments above.

[0120] Fluoroscopic methods and guidance for medical devices, such as guide wires and catheters, often expose clinicians and patients to potentially harmful X-ray radiation and contrast agents. The magnetic methods for tip tracking, the optical methods for tip tracking (see Figure 6B ), and the ECG methods for tip confirmation (see Figure 7 ) disclosed in this embodiment can accurately locate and confirm medical devices, such as guide wires and catheters, without such exposure. Thus, by providing ultrasound-based target identification, needle guidance, and catheter tip tracking, the system 600 provides a safe and effective method for ensuring that the PICC 412 is properly inserted into its intended target. The combination of target identification, needle guidance, and magnetic or optical catheter tracking can reduce clinical time and effort and can improve patient treatment outcomes.

[0121] Figure 6B A target identification system used with optical catheter tip tracking is shown in accordance with some embodiments. In some cases, magnetic and electromagnetic devices for tracking medical device tips are susceptible to interference. Thus, optical tip tracking methods, such as in the embodiments above Figure 3 , can also be used in conjunction with the disclosed systems and methods. For the case of magnetic catheter tracking, the combination of target identification, needle guidance, and optical catheter tracking can reduce clinical time and effort, improving outcomes.

[0122] In this embodiment, the light detector 660 of the optical tip tracking system can detect the position of the lighted stylet 635 and send a signal representing the position of the lighted stylet 635 to the console 402 for display on the screen 404. In conjunction with the optical tip tracking system, the console 402 can also apply AI to identify a target, such as a target blood vessel. Thus, the display screen 404 displays an ultrasound image of the target blood vessel 630 and the position 640 of the lighted stylet 635.

[0123] As shown, the clinician can place the light detector 660 of the optical tip tracking system above the patient P, for example, above the patient’s chest. A sterile drape 801 can also be draped over the patient P and the light detector 660. The clinician can use the drape-piercing connector of the lighted stylet 635 to pierce the sterile drape 801. The clinician can advance the catheter 412 from the insertion site to a destination within the patient P’s vasculature while emitting light from a light source (e.g., one or more LEDs) and detecting the light with the light detector of the light detector 660. The light source of the stylet 635 can extend distally beyond the distal end of the catheter 412.

[0124] When the catheter 412 is a PICC, the PICC can be advanced through the right jugular vein, right axillary vein, right subclavian vein, right cephalic vein of the patient P with a lighted stylet 635 disposed therein into the SVC of the patient P. When the catheter 412 is a CVC, the CVC can be advanced through the right internal jugular vein, right brachiocephalic vein into the SVC with a stylet 635 disposed therein. The clinician can view the display screen 404 of the optical tip tracking system while the display screen 404 graphically tracks the distal portion of the lighted stylet 635 through the vasculature of the patient P. After determining, through the display screen 404, that the distal portion of the lighted stylet 635 is at the destination, the clinician can stop advancing the catheter 412 through the vasculature of the patient P. Specifically, the clinician can identify the destination with the aid of the target identification system. For example, the target identification system can provide the clinician with information identifying and / or characterizing potential targets, such as a predicted classification of the target vessel 630 and a confidence associated with the classification. Alternatively, the system 400 can suggest one or more potential targets for approval by the clinician, or even automatically select a target (e.g., the target vessel 630).

[0125] Figure 7 ECG-based catheter tip confirmation is shown in accordance with some embodiments. In some embodiments, the system can use changes in an ECG signal measured from an adult patient as an alternative to PICC tip placement confirmation methods such as chest X-ray and fluoroscopy. Specifically, ECG-based tip confirmation can accurately confirm proper implantation of a catheter or PICC in a target near the atrioventricular junction, providing a high confidence that the catheter or PICC is placed within 1 centimeter of the atrioventricular junction. ECG-based tip confirmation also reduces exposure of the patient P and the clinician to harmful radiation such as X-rays, and reduces clinical costs and time associated with taking such images. Furthermore, the clinician can use the ultrasound probe 406 for vascular access during insertion and then easily transition to ECG for catheter placement confirmation without requiring additional equipment. Since delays in treatment can have a negative impact on clinical care, the disclosed system potentially improves patient treatment outcomes by quickly preparing the patient for treatment. Thus, the combination of target identification, needle guidance, and catheter placement confirmation can reduce clinical time and effort and improve patient treatment outcomes.

[0126] The atrioventricular junction is the point where the superior vena cava meets the right atrium. At this junction, both blood volume and turbulent flow are high, creating a favorable location for drug delivery to the body. Thus, by helping to position the PICC tip near the atrioventricular junction, the system can favor compliance with guidelines that recommend placing the PICC tip in the lower third of the superior vena cava near the atrioventricular junction. For patients whose P-waves are manifested by a change in heart rhythm (e.g., atrial fibrillation, atrial flutter, severe tachycardia, and pacemaker-driven heart rhythms), additional confirmation methods can be required.

[0127] ECG probes 620 and 625 can be placed along the midaxillary line on the underside of the patient's right and left shoulders, respectively (see Figure 6A ), with good skin electrode contact. The system can display ECG signals, such as signals 700 and 710 in this embodiment, detected by the intravascular electrodes and body electrodes (e.g., ECG probes 620 and 625 in the embodiment described above Figure 6A ). In various embodiments, the system can display signals 700 and 710 on display 404 of console 402 or on another display. In one embodiment, display 404 can provide a view of both catheter tip tracking and ECG signals 700 and 710 simultaneously. In one embodiment, the system displays two different ECG waveforms: waveform 700 is the external rhythm establishing a baseline ECG, and waveform 710 is the intravascular waveform, showing changes in P-wave amplitude as the catheter tip approaches the atrioventricular junction.

[0128] In patients with a clear P-wave ECG signal, the amplitude of the P-wave will increase as the catheter approaches the atrioventricular junction. In one embodiment, the system can highlight the P-wave on the display signal. As the catheter is advanced into the right atrium, the P-wave in display signal 710 will decrease in amplitude and can become biphasic or inverted.

[0129] The clinician can observe the P-wave to confirm that the PICC is properly placed. Both waveforms 700 and 710 can be frozen on the display 404 reference screen to compare the P-wave amplitude over time. To document a proper catheter placement, the clinician can print or save a procedure record on the system.

[0130] Figure 8 A flowchart of an exemplary method 800 for ultrasound imaging, target identification, and needle guidance is shown in accordance with some embodiments. Figure 8 Each block in FIG. 8 represents one or more operations commonly employed by such methods 800. The method can be implemented by a target identification and needle guidance system, such as the systems described above Figure 4A and Figure 5the target identification and needle guidance system 400 in embodiments of the subject system. In one embodiment, the method can be performed by a console of such a system, such as the console 402 in embodiments of the subject system described above Figure 4A and Figure 5 the console 402 in embodiments of the subject system.

[0131] As an initial step in the method 800, a needle can be magnetized (block 810). For example, a needle magnetizer (such as the needle magnetizer 408) can be configured to magnetize all or a portion of a needle (such as a needle of a catheter insertion device) to enable tracking of the needle during a placement procedure. The needle magnetizer can include a single permanent magnet, a single electromagnet, multiple permanent magnets, multiple electromagnets, or combinations thereof within a body of the needle magnetizer.

[0132] Next, the console receives ultrasound images and magnetic field measurements associated with the needle (block 820). The console can receive ultrasound images from an ultrasound probe, such as the ultrasound probe 406 in embodiments of the subject system described above Figure 4A The console can receive magnetic field measurements from a magnetic sensor, such as the magnetic sensor 258 in embodiments of the subject system described above Figure 2 The console can receive magnetic field measurements from a magnetic sensor, such as the magnetic sensor 258 in embodiments of the subject system described above

[0133] Next, the console can process the ultrasound images and the magnetic field measurements to determine a position and orientation of the needle and to identify a target (block 830). Based on the processed magnetic field measurements, the console can display a position, orientation, and depth of the needle relative to the ultrasound probe. Processing the ultrasound images to identify a target will be further described in Figure 9

[0134] Next, the console can identify a target based on the ultrasound images (block 840). In typical embodiments, the target can be a target blood vessel for a final implantation of a medical device (such as a PICC or other catheter). However, any other type of target is possible and is not limited by the disclosure. In some embodiments, any of the AI methods disclosed herein can be used to identify a target. As one embodiment, any of the AI methods disclosed herein can be used to identify a target insertion into a blood vessel (e.g., a vein) immediately prior to insertion of a needle into a patient. In such embodiments, the method 900 of the subject system can be utilized. In alternative embodiments, any of the AI methods disclosed herein can be used to identify a target during advancement of a needle within a patient’s vasculature. For example, the ultrasound probe 406 can be used to obtain ultrasound images that are intended to track the needle, such as described above Figure 9 Figure 4A ​​As seen, the advancement along the patient P’s arm through the vasculature. Upon receiving the ultrasound image, the target perception logic 560 processing on the console 402 can analyze the ultrasound image using an AI-based model trained to recognize a particular target (e.g., the SVC). Thus, as the console 402 continues to receive ultrasound images from the probe 406, the display 404 can provide a plot indicating the confidence that the SVC is present within the ultrasound image rendered on the display 404. As one embodiment, the target perception logic 560, and in particular the target identification logic 564, can identify the SVC within the ultrasound image with at least a particular confidence score that is derived via processing of the AI-based model (e.g., by providing the ultrasound image as input to the model) and in response, generate an alert to the clinician and / or present an indicator around the SVC entry on the display 404. For example, such a presentation can include a crosshairs window around the entry to the SVC, similar to the crosshairs window shown in Figure 4B .

[0135] Finally, the console can direct the insertion of the magnetized needle into the target vessel (block 850). For example, as the clinician continues to advance the needle to the target, the console can continue to update the displayed position, orientation, and depth of the needle. The console can also continue to update the displayed ultrasound image of the patient tissue containing the target. The console can also continue to display and identify the target via any of the AI methods disclosed herein. In some embodiments, the console can also display the magnetically and / or optically tracked position of the catheter tip (e.g., PICC tip) as the clinician advances the catheter. Finally, in some embodiments, the console can receive and display ECG signals from an ECG probe, which can allow the clinician to confirm proper insertion of the catheter.

[0136] Figure 9 A flowchart illustrating an exemplary method 900 for target identification is shown, in accordance with some embodiments. Figure 9 Each block shown in the method 900 for target identification represents an operation performed in the method 900 for target identification. In various embodiments, the method 900 can be performed by a target identification and needle guidance system, such as the target identification and needle guidance system 400 in the embodiments of Figure 4A and Figure 5 above, by a console of such a system, such as the console 402 in the embodiments of Figure 4A and Figure 5 above, and / or by a target perception logic, such as the target perception logic 560 in the embodiments of Figure 5 above.

[0137] As an initial step in the method 900, the console and / or the target awareness logic can obtain an ultrasound image (block 910). In one embodiment, the console and / or the target awareness logic can receive the ultrasound image from an ultrasound probe, such as from the ultrasound probe 406 in the embodiments described above with respect to FIG. 4. Figure 4A

[0138] Next, the image processing logic of the target awareness logic can process the image with a supervised learning heuristic (block 920). The image processing logic can apply a supervised learning method to partition or classify potential targets within the ultrasound image data. For example, the image processing logic can use logistic regression or other linear classifiers, support vector machines, quadratic classifiers, kernel estimations, decision trees, neural networks, convolutional neural networks, or learning vector quantization. In the case of logistic regression or other linear classifiers, the model can determine a predicted probability as a weighted sum of the feature values. For neural networks or other deep learning processes, the model can transform the features via one or more layers of non-linear functions (e.g., artificial neurons) to determine a probability of the final prediction.

[0139] In embodiments where the target is a blood vessel, the system can be trained using ultrasound data from correctly identified blood vessels, and can classify or partition the blood vessels based on features such as shape (e.g., eccentricity, internal angles, or aspect ratio), size (e.g., short or long axis length, diameter, or circumference), number and complexity of branches, and the like. In one embodiment, for the predicted classification, the model can be trained by iteratively minimizing an error or loss function with respect to the model weights, as compared to the actual classification in the training data set. The hyperparameters of the model can be further adjusted with respect to a validation data set, and the model can be further evaluated against a test data set. The trained model can then be deployed for use in a clinical environment.

[0140] Next, the target identification logic of the target awareness logic can identify one or more insertion targets based on the processing (block 930). Based on the classification or partitioning from the AI, the target identification logic can determine the specific identity of one or more potential targets. For example, the clinician can inform the system that the desired target is the superior vena cava of the patient. In this embodiment, the target identification logic can subsequently determine that the particular blood vessel is the superior vena cava, and can label the blood vessel accordingly to recommend to the clinician as an insertion target. Alternatively, the target identification logic can automatically assume that this identified blood vessel is the target, and can proceed to guide the clinician based on this assumption.

[0141] ​In another alternative embodiment, the clinician can not inform the system of a particular desired target. In this alternative embodiment, the target identification logic can then identify all potential targets present in the ultrasound image, and can flag all of these potential targets for communication to the clinician. In one embodiment, the target identification logic can rank the potential targets, for example by relevance or importance as an insertion target, by confidence of classification, etc., and can communicate the recommendations to the clinician in the order of the ranking. In a final embodiment, the target identification logic can simply attempt to identify all blood vessels, and inform the clinician of the best estimated classification of each blood vessel, as well as any other possible classifications and the confidence of each classification.

[0142] Finally, the target recommendation logic of the target perception logic can recommend an insertion target (block 940). Based on the AI, the target recommendation logic can identify the target to the clinician, and / or can assist the clinician in identifying the target by providing information to the clinician about potential targets. For example, the console display can display a predicted classification of the target, such as a blood vessel type, location, or name, and can display a confidence associated with that classification. Alternatively, the console can suggest one or more potential targets for approval by the clinician, or even automatically select a target. For example, if there is more than one potential target present in the ultrasound image, the console can indicate a set of potential targets to the user, for example by highlighting them on the display. Alternatively, the console can provide the user with a menu of one or more suggested targets, or automatically select a target.

[0143] While some particular embodiments have been disclosed herein, and particular embodiments have been disclosed in detail, the particular embodiments are not intended to limit the scope of the concepts provided herein. Additional adaptations and / or modifications of the particular embodiments can occur to those ordinarily skilled in the art. Such adaptations and / or modifications are also within the scope of the concepts provided herein. Accordingly, while the particular embodiments have been disclosed, additional adaptations and / or modifications are intended to be within the scope of the concepts provided herein.

Claims

1. A target recognition and needle guidance system, characterized in that, include: A console, comprising a processor and a non-transitory computer-readable medium thereon storing logic configured to initiate, when executed by the processor: The target identification process is configured to (i) receive user input indicating a desired target and (ii) identify a patient's blood vessel corresponding to the desired target by applying a machine learning model to features of candidate targets in ultrasound imaging data, wherein the blood vessel identified as the desired target is associated with the highest confidence score determined by the machine learning model. A needle-guided procedure for guiding a needle to be inserted into the blood vessel identified as the desired target via the machine learning model, wherein the machine learning model determines the proximity of the needle's trajectory to the blood vessel identified as the desired target based on the needle's magnetic information, and wherein, after the needle insertion, a catheter is implanted into the blood vessel identified as the desired target. and An ultrasound probe configured to provide an electrical signal corresponding to the ultrasound imaging data to the control console, the ultrasound probe comprising: A transducer array configured to convert reflected ultrasound signals from the patient into electrical signals for ultrasound imaging, and A magnetic sensor configured to provide the console with a second electrical signal corresponding to the magnetic information of the needle.

2. The target recognition and needle guidance system according to claim 1, characterized in that, The machine learning model includes a supervised learning model trained based on previously identified training objectives.

3. The target recognition and needle guidance system according to claim 1, characterized in that, Each of the candidate targets in the ultrasound imaging data is associated with a confidence score determined by the machine learning model.

4. The target recognition and needle guidance system according to claim 1, characterized in that, The machine learning model includes one or more of the following supervised learning models: Logistic regression Other linear classifiers, Support Vector Machine Secondary classifier, Nuclear estimation, Decision tree Neural networks, and Learn vector quantization.

5. The target recognition and needle guidance system according to claim 4, characterized in that, The machine learning model includes the logistic regression, and the confidence score of the first candidate target is determined as a weighted sum of the features of the candidate target.

6. The target recognition and needle guidance system according to claim 4, characterized in that, The machine learning model includes the neural network, and wherein a confidence score for a first candidate target is determined based on one or more nonlinear activation functions of the features of the candidate target.

7. The target recognition and needle guidance system according to claim 2, characterized in that, The supervised learning model is trained by iteratively minimizing the error for the classification predicted by the candidate model, compared to the actual classification of the previously identified training target.

8. The target recognition and needle guidance system according to claim 1, characterized in that, The blood vessels include the superior vena cava.

9. The target recognition and needle guidance system according to claim 1, characterized in that, The ultrasound probe provides a first ultrasound image prior to the insertion of the needle, and the machine learning model is based in part on the trajectory of the needle.

10. The target recognition and needle guidance system according to claim 1, characterized in that, The candidate target is characterized by one or more of the following features: The shape of the candidate blood vessel of the candidate target in the ultrasound imaging data. The size of the candidate blood vessel, The number of branches of the candidate blood vessels, and The complexity of the branching of the candidate blood vessels.

11. The target recognition and needle guidance system according to claim 1, characterized in that: Identifying the blood vessel identified as the desired target includes providing the user with identification information characterizing one or more candidate targets, wherein the identification information includes a predicted classification of one or more potential targets and a confidence level associated with the predicted classification.

12. The target recognition and needle guidance system according to claim 1, characterized in that, It also includes a display screen configured to graphically guide the needle insertion into the blood vessel identified as the desired target.

13. The target recognition and needle guidance system according to claim 1, characterized in that, Also includes: One or more electrocardiogram (ECG) probes configured to provide electrical signals corresponding to ECG information to the console. When executed by the processor, the logic is further configured to initiate a catheter tip placement confirmation process based on the electrocardiogram information.

14. The target recognition and needle guidance system according to claim 1, characterized in that, It also includes a needle magnetizer incorporated into the console, the needle magnetizer being configured to magnetize the needle.

15. The target recognition and needle guidance system according to claim 14, characterized in that, The needle guidance process for guiding the magnetized needle into the blood vessel identified as the desired target uses a combination of ultrasound imaging data and magnetic field data received by the console.

16. The target recognition and needle guidance system according to claim 15, characterized in that, The ultrasound probe is configured to provide the magnetic field data to the console, and the ultrasound probe also includes a magnetic sensor array configured to convert the magnetic signal from the magnetized needle into the magnetic field portion of the second electrical signal.

17. A target recognition and needle guidance system, characterized in that, include: A console, comprising a processor and a non-transitory computer-readable medium thereon storing logic configured to initiate, when executed by the processor: The target identification process is configured to (i) receive user input indicating a desired target and (ii) identify a patient's blood vessel corresponding to the desired target by applying a machine learning model to features of candidate targets in ultrasound imaging data, wherein the blood vessel identified as the desired target is associated with the highest confidence score determined by the machine learning model. A needle-guided procedure for guiding a needle to be inserted into the blood vessel identified as the desired target via the machine learning model, and wherein, after the needle insertion, a catheter is implanted into the blood vessel identified as the desired target, wherein the machine learning model determines the proximity of the needle trajectory to the blood vessel identified as the desired target based on optical information of the catheter. and An ultrasound probe configured to provide an electrical signal corresponding to the ultrasound imaging data to the control console, the ultrasound probe comprising: A transducer array configured as an ultrasound imaging component that converts reflected ultrasound signals from the patient into electrical signals; and A catheter photodetector is configured to provide the console with a second electrical signal corresponding to the optical information of the catheter.

Images