Systems and methods for artificial intelligence and ultrasound guided intervention

Abstract

Systems and methods are provided for determining safe pathways of an interventional device into a target structure of a subject while avoiding critical structures. The systems and methods include segmenting anatomical landmarks, injecting a fluid injectant to highlight the various structures in imaging data, and to confirm or update a safe pathway to the target. The systems and methods herein also include performing a nerve block.

Description

Client Ref. MIT25673JLSYSTEMS AND METHODS FOR ARTIFICIAL INTELLIGENCE AND ULTRASOUND GUIDED INTERVENTIONPRIORITY STATEMENT

[0001] The present application is based on, claims priority to, and incorporates herein by reference in its entirety for all purposes, US Provisional Application Serial No.63 / 788,402, filed April 14, 2025.STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

[0002] This invention was made with government support under Grant No. FA8702-15-D-0001 awarded by the U.S. Air Force and Grant Nos. 2022-318 and MTEC-21-06-MPAI-061 awarded by the U.S. Army Medical Research and Development Command. The government has certain rights in the invention.BACKGROUND

[0003] Guiding an intervention device into a patient can be a difficult task for nonexperts or in trauma applications. Multiple attempts at penetration may result in extreme discomfort to the patient, loss of valuable time during emergency situations, or in further trauma. Furthermore, central veins and arteries are often in close proximity to each other. Associated nerve pathways may also be found in close proximity to a vessel, such as the femoral nerve located nearby the femoral artery, puncture of which may cause significant pain or loss of function for a patient. Thus, delivering or guiding an interventional device can be very complicated and difficult.

[0004] Therefore, there is a need for less cumbersome and more accurate systems and methods of guiding interventional device, particularly when controlled by a nonexpert.SUMMARY OF THE DISCLOSURE

[0005] The present disclosure addresses the aforementioned drawbacks by providing systems and methods for guiding an interventional device with increased accuracy, even when the individual overseeing the intervention is not an expert. The systems and methods provide image analysis for the segmentation of targets (e.g., vessels of interest, connective tissue, etc.) and critical structures (e.g., nerves, etc.) from imageClient Ref. MIT25673JLdata. The image analysis provides guidance for insertion of an interventional device into a subject and may be accomplished by a non-expert based upon the guidance and / or control provided. Moreover, the systems and methods described herein determine safe pathways for the interventional device in reaching the target while avoiding critical structures. A second confirmation and / or updating step of the safe pathway is provided by first targeting for injecting a fluid injectate based on the spread of the fluid injectate in the image data.

[0006] In some aspects, the present disclosure relates to a system for guiding an interventional device in an interventional procedure of a subject. The system includes an ultrasound probe and a guide system coupled to the ultrasound probe and configured to guide the interventional device into a field of view (FOV) of the ultrasound probe, wherein the interventional device includes a needle. The system further includes a non-transitory memory having instructions stored thereon and a processor configured to access the non-transitory memory and execute the instructions. The processor is caused to access image data acquired from the subject using the ultrasound probe, wherein the image data include at least one image of a target. The processor is further caused to locate anatomical landmarks in the image data, segment the anatomical landmarks including the target, determine, from the image data, an aimpoint location on the target for injecting a fluid injectant, determine, from the image data, a pathway for the interventional device to reach the target without impinging on at least one critical structure, and control an insertion of the needle and injection of the fluid injectant into the target.

[0007] In some aspects, the disclosure relates to a system for performing a nerve block. The system includes an ultrasound probe and a guide system coupled to the ultrasound probe and configured to guide a needle into a field of view [FOV] of the ultrasound probe. The system further includes a non-transitory memory having instructions stored thereon and a processor configured to access the non-transitory memory and execute the instructions. The processor is caused to access image data acquired from a subject using the ultrasound probe, wherein the image data include at least one image of a target and at least one critical structure to avoid, wherein the at least one critical structure includes a nerve. The processor is further caused to locate anatomical landmarks in the image data, segment the anatomical landmarks including the at least one of the target or the nerve, determine, from the image data, an aimpoint location on the target for injecting at least one of an anesthetic solution or an analgesicClient Ref. MIT25673JLsolution for providing a nerve block, determine, from the image data, a pathway for the needle to reach the target without impinging on the at least one critical structure, and control insertion of the needle and injection of at least one of the anesthetic solution or the analgesic solution into the target.

[0008] In some aspects, the disclosure relates to a method of controlling a robotically controlled system to determine a safe pathway for guiding an interventional device in an interventional procedure of a subject. The method includes causing a processor to carry out steps including accessing image data acquired from the subject using an ultrasound probe, wherein the image data include at least one image of a target. The method further includes locating anatomical landmarks in the image data, segmenting the anatomical landmarks including the target, determining, from the image data, an aimpoint location on the target, determining, from the image data, a pathway for the interventional device to reach the target without impinging on the at least one critical structure, wherein the interventional device includes a needle, and controlling an insertion of the needle and injection of the fluid injectant into the target.

[0009] In some aspects, the disclosure relates to a method of performing a nerve block. The method includes causing a processor to carry out steps including accessing image data acquired from a subject using an ultrasound probe, wherein the image data include at least one image of a target and at least one critical structure to avoid, wherein the at least one critical structure includes a nerve. The method further includes locating anatomical landmarks in the image data, segmenting the anatomical landmarks including the at least one of the target or the nerve, determining, from the image data, a location of the target for injecting at least one of an anesthetic solution or an analgesic solution for providing a nerve block, determining, from the image data, a pathway for a needle to reach the target without impinging on the at least one critical structure, and controlling insertion of the needle and injection of at least one of the anesthetic solution or the analgesic solution into the target.

[0010] The foregoing and other aspects and advantages of the present disclosure will appear from the following description. In the description, reference is made to the accompanying drawings that form a part hereof, and in which there is shown by way of illustration a preferred embodiment. This embodiment does not necessarily represent the full scope of the invention, however, and reference is therefore made to the claims and herein for interpreting the scope of the invention. Like reference numerals will beClient Ref. MIT25673JLused to refer to like parts from Figure to Figure in the following description.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG. 1 is a schematic diagram of a non-limiting example ultrasound system that can implement the systems and methods described in the present disclosure.

[0012] FIG. 2 is a schematic diagram of a non-limiting example configuration for guiding needle insertion into a structure of interest using an ultrasound-guided device, as described in the present disclosure.

[0013] FIG. 3A is a flowchart of non-limiting example steps for a method of operating a system for guiding an interventional device.

[0014] FIG. 3B is a flowchart of non-limiting example steps for a method of operating a system for guiding an interventional device.

[0015] FIG. 4A is an example user interface displaying segmented anatomical landmarks, according

[0016] FIG. 4B is a series of ultrasound images during a nerve block procedure, according to aspects of the present disclosure.

[0017] FIG. 5A is an example method for a needle insertion and tracking in a nerve block procedure, according to aspects of the present disclosure.

[0018] FIG. 5B is an example method for needle tracking in a nerve block procedure according to FIG. 5A.

[0019] FIG. 5C is an example method for needle tracking in a nerve block procedure according to FIG. 5A.

[0020] FIG. 5D is an example method of a segmentation approach of FIG. 5C.

[0021] FIG. 6A shows results of the segmentation approach of FIG. 5B.

[0022] FIG. 6B shows results of the segmentation approach of FIG. 5C.

[0023] FIG. 7A is a flowchart of non-limiting example steps for determining an insertion position offset.

[0024] FIG. 7B is a series of images of a user interface demonstrating the offset determination method of FIG. 7A.

[0025] FIG. 8 is a block diagram of an example system that can implement a vessel of interest image processing system for generating images of a vessel of interest or otherwise measuring or predicting a location for a vessel of interest using a hybrid machine learning and mechanistic model.Client Ref. MIT25673JL

[0026] FIG. 9 is a block diagram of example hardware components of the system of FIG. 8.

[0027] FIG. 10A is a perspective view of a non-limiting example interventional device guide coupled to an ultrasound probe.

[0028] FIG. 10B is a side view of the interventional device guide of FIG. 10A.

[0029] FIG. 10C is a side view of the base and ultrasound probe fixture for the interventional device guide of FIG. 10B.

[0030] FIG. 10D is a cross-section of a non-limiting example cartridge compatible with the injection assembly of FIG. 10B.

[0031] FIG. 10E is a cross-section of a non-limiting example interventional device coupled to an ultrasound probe.

[0032] FIG. 11A is a perspective view of a non-limiting example interventional device guide integrated with an ultrasound probe.

[0033] FIG. 11B is an exploded view of the integrated interventional device guide and ultrasound probe of FIG. 11A.

[0034] FIG. 12 is a plot of precision-recall curves of the femoral nerve region landmark detection model tested on the full test set including both in vivo and ex vivo data. The model achieved area under the precision-recall curve values of 0.93 for the artery, 0.89 for the vein, and 0.80 for the fascia, with an overall precision-recall of 0.88 across all classes.

[0035] FIG. 13 show plots of precision-recall curves of the femoral landmark detection model tested on subsets of the test set: in vivo data (left) and ex vivo data (right). The overall area under the precision-recall curve was 0.92 for in vivo and 0.65 for ex vivo.

[0036] FIG. 14 is a plot of the relative frequency of Dice similarity coefficients between the fascia iliaca segmentation model output and the ground truth masks on the test set. The ideal Dice coefficient is 1.0. Results show that 91% of the model’s output segmentation masks were greater than 0.90, with false negatives at 0.0 primarily due to the model's poor performance on segmenting the FI when it is compressed.

[0037] FIG. 15 is a custom software application displaying algorithm outputs-fascia iliaca detection (yellow box) and segmentation (yellow outline), femoral artery detection (red), and femoral vein detection (blue)-on the ultrasound image feed in real time in transverse view of the porcine anatomy.

[0038] FIG. 16 is a plot of the times for the image processing algorithms to guideClient Ref. MIT25673JLthe operator to a suitable target aimpoint for hypothetical needle insertion. The average time was 40.5 seconds across 60 trials on eight in vivo porcine models.

[0039] FIG. 17 is an example ultrasound image of the sciatic nerve with a bounding box detection [top] and a plot of precision-recall curve of the sciatic nerve detection model on in vivo test data with an AUPRC of 0.69 (bottom).DETAILED DESCRIPTION

[0040] FIG. 1 illustrates an example of an ultrasound system 100 that can implement the methods described in the present disclosure. The ultrasound system 100 includes a transducer array 102 that includes a plurality of separately driven transducer elements 104. The transducer array 102 can include any suitable ultrasound transducer array, including linear arrays, curved arrays, phased arrays, and so on. Similarly, the transducer array 102 can include a ID transducer, a 1.5D transducer, a 1.75D transducer, a 2D transducer, a 3D transducer, and so on.

[0041] When energized by a transmitter 106, a given transducer element 104 produces a burst of ultrasonic energy. The ultrasonic energy reflected back to the transducer array 102 (e.g., an echo) from the object or subject under study is converted to an electrical signal (e.g., an echo signal) by each transducer element 104 and can be applied separately to a receiver 108 through a set of switches 110. The transmitter 106, receiver 108, and switches 110 are operated under the control of a controller 112, which may include one or more processors. As one example, the controller 112 can include a computer system.

[0042] The transmitter 106 can be programmed to transmit unfocused or focused ultrasound waves. In some configurations, the transmitter 106 can also be programmed to transmit diverged waves, spherical waves, cylindrical waves, plane waves, or combinations thereof. Furthermore, the transmitter 106 can be programmed to transmit spatially or temporally encoded pulses.

[0043] The receiver 108 can be programmed to implement a suitable detection sequence for the imaging task at hand. In some embodiments, the detection sequence can include one or more of line-by-line scanning, compounding plane wave imaging, synthetic aperture imaging, and compounding diverging beam imaging.

[0044] In some configurations, the transmitter 106 and the receiver 108 can be programmed to implement a high frame rate. For instance, a frame rate associated withClient Ref. MIT25673JLan acquisition pulse repetition frequency ("PRF") of at least 100 Hz can be implemented. In some configurations, the ultrasound system 100 can sample and store at least one hundred ensembles of echo signals in the temporal direction.

[0045] The controller 112 can be programmed to implement an imaging sequence using the techniques described in the present disclosure, or as otherwise known in the art. In some embodiments, the controller 112 receives user inputs defining various factors used in the design of the imaging sequence.

[0046] A scan can be performed by setting the switches 110 to their transmit position, thereby directing the transmitter 106 to be turned on momentarily to energize transducer elements 104 during a single transmission event according to the implemented imaging sequence. The switches 110 can then be set to their receive position and the subsequent echo signals produced by the transducer elements 104 in response to one or more detected echoes are measured and applied to the receiver 108. The separate echo signals from the transducer elements 104 can be combined in the receiver 108 to produce a single echo signal.

[0047] The echo signals are communicated to a processing unit 114, which may be implemented by a hardware processor and memory, to process echo signals or images generated from echo signals. As an example, the processing unit 114 can guide an interventional device to target a target while avoiding at least one critical structure using the methods described in the present disclosure. Images produced from the echo signals by the processing unit 114 can be displayed on a display system 116.

[0048] In some configurations, a non-limiting example method may be deployed on an imaging system, such as a commercially available imaging system, to provide for a portable ultrasound system with interventional device guidance. The method may locate a vessel of interest, such as a vein or an artery as a user or medic moves an ultrasound probe. The system and method may provide real-time guidance to the user to position the ultrasound probe to the optimal needle insertion point. The probe may include one or more of a fixed needle guide device, an adjustable mechanical needle guide, a displayed-image needle guide, and the like. An adjustable guide may include adjustable angle and / or depth. The system may guide, or communicate placement or adjustments for the guide for the needle. The system may also regulate the needle insertion distance based upon the depth computed for the vessel of interest. The user may then insert a needle through the mechanical guide attached to the probe or displayed guide projectedClient Ref. MIT25673JLfrom the probe in order to ensure proper insertion. During needle insertion, the system may proceed to track the target blood vessel and the needle until the vessel is penetrated. A graphical user interface may be used to allow the medic to specify the desired blood vessel and to provide feedback to the medic throughout the process.

[0049] For the purposes of this disclosure and accompanying claims, the term "real time” or related terms are used to refer to and defined a real-time performance of a system, which is understood as performance that is subject to operational deadlines from a given event to a system’s response to that event. For example, a real-time extraction of data and / or displaying of such data based on acquired ultrasound data may be one triggered and / or executed simultaneously with and without interruption of a signalacquisition procedure.

[0050] In some configurations, the system may automate all ultrasound image interpretation and insertion computations, while a medic or a user may implement steps that require dexterity, such as moving the probe and inserting the needle. Division of labor in this manner may avoid using a dexterous robot arm and may result in a small system that incorporates any needed medical expertise.

[0051] Referring to FIG. 2, a diagram is shown depicting a non-limiting example embodiment of an interventional device guide injection assembly 200 for guiding needle insertion into a target in the femoral triangle. An ultrasound probe 210 is used to acquire an image 220 of a region ofinterest that includes a portion of the fascia iliaca 230, femoral vessel 240 and other objects of interest such as femoral nerve 250. The locations of the femoral iliaca 230, femoral vessel 240, and femoral nerve 250 may be annotated on the image 220. A mechanical needle guide 260 may be included to guide a needle 270 to penetrate the target or structure of interest, such as femoral vein 240 as shown. In some configurations, visual needle guide 265 may be included where a penetration guide image 266 is projected onto the surface of a subject to guide a needle 270 to penetrate the target or structure of interest, such as to fascia iliaca 230 as shown. Penetration guide image 266 may reflect the actual size or depth of the target of interest for penetration when projected onto the subject, or may provide other indicators such as measurements or a point target for penetration, and the like.

[0052] The target or structure of interest may include a femoral artery, femoral vein, jugular vein, peripheral veins, subclavian vein, cephalic vein, basilic vein, airway, lumen, luminal organ, body cavity, fluid filled anatomic space, location requiring biopsy,Client Ref. MIT25673JLbreast, kidney, lymph node, spinal canal, perineural location, peritoneal space, pleural space, and / or other vessels or non-vessel structures.

[0053] Referring to FIG. 3A, non-limiting example steps of an overview method 300 of operating a system for interventional device penetration for a nerve block are shown. At step 310, imaging data is accessed. This may be achieved by performing an imaging acquisition. Using the imaging data, anatomical landmarks may be detected at step 320 (e.g., anatomical landmarks in the femoral triangle). The nerve region landmarks may be detected by segmenting anatomical structure in the imaging data, as will be described in further detail. An aimpoint location may then be calculated at step 330 for an interventional device (e.g., a needle). Calculating the aimpoint location may be based upon the determined location for a target in the region of interest and calculating a depth and a pathway for the interventional device from the surface of a subject to the target without the interventional device penetrating the nerve or other off-target (e.g., blood vessels, organs, etc.). At step 340, an insertion point on the surface of the subject may be determined for a user based on the calculated aimpoint location of the target of structure of interest. The insertion point may be identified by illuminating a portion of the surface of a subject, adjusting a mechanical needle guide to the appropriate settings for the user, indicating proper probe placement on an integrated or separate display, and the like. As used herein, the aimpoint location refers to the needle point target on the structure or interest, while the insertion point refers to the injection point of the needle on the surface of the subject. The needle or interventional device enters the subject at the insertion point location and travels along the needle pathway to reach the aimpoint location on the target or structure of interest.

[0054] Depth of the needle penetration may also be controlled by a setting or a height of the mechanical guide. The assembly may be guided on the surface of subject to align the interventional device with the insertion point for reaching the aimpoint location on the target or structure of interest at step 350. At step 360, the interventional device, such as a needle, may be inserted and tracked by acquiring images of the target and the interventional device as the interventional device is inserted into the subject and displaying the tracked images for the user. At step 370, once the interventional device has reached the target, an analgesia agent may be delivered through the interventional device and successful injection confirmed through image acquisition of the target. As will be described in further detail with reference to FIG 3B below, the system may confirm theClient Ref. MIT25673JLinterventional device and its trajectory are aligned with the pathway towards the aimpoint location after injection at the insertion point. If the interventional device and / or its trajectory are not aligned with the pathway, then the system may adjust the interventional device, by retracting and reinserting it based on the adjustment.

[0055] Any ultrasound probe may be used in accordance with the present disclosure, including ID, 2D, linear, phased array, and the like. In some configurations, an image is displayed for a user of the structure of interest with tracking information for the needle overlaid on the image. In some configurations, no image is displayed for a user and instead only the insertion point may be identified by illuminating a portion of the surface of a subject. In some configurations, no image is displayed and the user is only informed of the probe reaching the proper location whereby a mechanical needle guide is automatically adjusted to the appropriate settings, such as angle and / or depth to target a vessel of interest. The user may be informed of the probe reaching the proper location by any appropriate means, such as light indicator, a vibration of the probe, and the like.

[0056] In some configurations, identification of placement of the ultrasound transducer at a target surface location may be performed automatically by the system. Image data may be used for identifying anatomy, such as a femoral triangle, jugular region, and the like, and may be accessed by the system to provide automatic identification for where the ultrasound transducer has been placed. In some configurations, a user may specify the structure of interest to be targeted, such as whether to target an artery, a vein, fascia, or other point relative to an anatomical structure. In a non-limiting example combination of the configurations, the location of the ultrasound transducer on the subject may be automatically determined along with the anatomy being imaged, with the user specifying the vessel of interest to target in the automatically identified anatomy. A minimum of user input may be used in order to mitigate the time burden on a user.

[0057] Segmenting the vessels of interest may be based on machine learning of morphological and spatial information in the ultrasound images. In some configurations, a neural network may be deployed for machine learning and may learn features at multiple spatial and temporal scales. Structure of interest may be distinguished based on shape and / or appearance of their walls, shape and / or appearance of surrounding tissues, and the like. In a non-limiting example, stiffer walls and a circular shape may be used to distinguish an artery in an image, whereas an ellipsoidal shape may be used to identify aClient Ref. MIT25673JLvein. Real-time segmentation may be enabled by a temporally trained routine without a need for conventional post-hoc processing.

[0058] Temporal information may be used with segmenting the structures of interest. For example, vessel appearances and shape may change with movement of the anatomy over time, such as changes with heartbeat, or differences in appearance between hypotensive and normal-tensile situations. Machine learning routines may be trained with data from multiple time periods with differences in anatomy being reflected over the different periods of time. With a temporally trained machine learning routine, vessel segmentation may be performed in a robust manner over time for a subject without misclassification and without a need to find a specific time frame or a specific probe position to identify vessels of interest.

[0059] In some configurations, to prevent any potential misclassifications conflicting information checks may be included in the system. A conflicting information check may include taking into consideration the general configuration of the anatomy at the location of the probe. In a non-limiting example, if the system initially identifies two arteries at a location of the probe, but the general anatomy at the location of the probe indicates that an artery and a vein should be returned as results instead, then the system will automatically correct to properly identify an artery and a vein instead of the mistaken two arteries to prevent a misclassification.

[0060] Identifying an insertion point for a user may also include where the system automatically takes into account the orientation of the probe on a body. A conventional ultrasound probe includes markings on the probe to indicate the right vs left side of probe, which allows a user to position the probe such that the mark is oriented the right of the patient, for example. The probe orientation may be also be determined from an analysis of the acquired ultrasound images, or monitoring of the orientation of the markings, such as by an external camera. In some configurations, the needle guide attachment may be configured fit into the markings on the probe to ensure that the device is consistent with the orientation of the probe.

[0061] In some configurations, the method includes guiding a user in placement of the ultrasound probe on the subject. A target for penetration may be identified, such as by machine learning in accordance with the present disclosure, and localized. A user may then be guided in which direction to move the ultrasound probe for placement over an identified target. Once the ultrasound probe has reached the target location, a signal mayClient Ref. MIT25673JLindicate for the user to stop moving the probe. Guidance may be provided by the signal, such as the light on the probe, in a non-limiting example. Needle placement and penetration may proceed after the location of the target has been reached.

[0062] Referring to FIG. 3B, a method 380 setting forth a non-limiting example for operating a system for guiding an interventional device is described. It is noted that the interventional device may be used interchangeably with the needle herein. Method 380 is a more detailed example of method 300. Ultrasound imaging data is acquired, and a probe location is determined at step 381. In a non-limiting example, the ultrasound imaging data is acquired in real time. In a non-limiting example, the ultrasound imaging data include a region of interest with one or more anatomical landmarks. The anatomical landmarks may include, but are not limited to, an artery, a vein, a femoral artery, a femoral vein, a jugular vein, a peripheral vein, a subclavian vein, an airway, a lumen, a luminal organ, a body cavity, a fluid filled anatomic space, a location requiring biopsy, a breast, a kidney, a lymph node, a spinal canal, a perineural location, a peritoneal space, a pleural space a muscle, or connective tissue. Specifically, the connective tissue may include fascia. For example, the fascia may include the fascia iliaca. The ultrasound imaging data may also include at least one critical structure to avoid, for example bones, unintended blood vessels, non-target organs, or nerves.

[0063] In a non-limiting example, the image data includes a plurality of views of the target, and wherein the processor is configured to assess the plurality of views to identify at least one of the critical structures to avoid in the subject and identify a location on the subject where the interventional device reaches at least one of the target from an insertion point location without penetrating the at least one critical structure. The target may include any of the anatomical landmarks mentioned above. However, the target need not be an anatomical landmark and may also be a location between two or more anatomical landmarks, a location between an anatomical landmark and a critical structure, a predefine offset from an anatomical landmark or critical structure, or the like.

[0064] An image quality may be determined at step 382, and the image depth and gain may be adjusted at step 383. In a non-limiting example, adjusting image depth and gain may be automatic. For example, the process may start by initializing an image. In this non-limiting example, gain for the ultrasound system may be automatically controlled based on depth. In this case, image initialization may be performed for a selected depth. For example, in one non-limited application, the depth for imageClient Ref. MIT25673JLinitialization may be 6 cm. Regardless of the particular mechanism for initialization or, if depth, the particular depth is provided to the user. In one non-limiting example, the cue can communicate to the user to move caudally until bifurcation is detected. Then a calibration is activated. In one non-limiting example, the calibration can be turned on while cuing the user to move cranially. When the vessel(s) are detected, the process finds the deepest vessel and calculates the buffer, for example, to the image bottom. In one non-limiting example, if an artery, the buffer may be set to 1.25 cm, else the buffer may be set to 0.75 cm. Adjustments may be made, for example, by rounding up to the nearest integer. The data may then be saved and sorted. For example, non-zero depths may be saved in an array. Then, the array is sorted, such that a threshold can be calculated based thereon. In one non-limiting example, the threshold may be at a selected percentile, such as the 75thpercentile. Then the image depth can be updated, for example, to the calculated depth. The data can be cleared and the process repeated for the next set of detected vessels.

[0065] Thus, an automated gain control based on depth may be configured to balance too much gain that results in washout and artifacts, with too low of gain that results in a lack of signal. A machine learning or Al routine may be used to determine optimal image depth and gain such that the vessels of interest are well visualized. Since spatial resolution is poorer outside the ultrasound focal zone, the Al may automatically adjust the image depth so that the vessels are as close as possible to the center of the focal zone, while also ensuring the vessels are not cut off at the bottom of the image. The image gain optimization may be performed with histogram analysis of pixel intensities. The gain is adjusted to reach a dynamic range of intensities determined from well-gained training images.

[0066] Automatic gain control may start at a maximum depth, and the vessel detection model may be run. If a vessel is found, gain may be swept and an optimal gain may be found based on the optimal depth calculated for the vessel centroid. The ultrasound probe may then be reset to the optimal depth setting with the optimal gain, or gain may be swept if not at an optimal setting.

[0067] At step 384, the anatomical landmarks may be located in the imaging data. Detecting anatomical landmarks may include a You-Only-Look-Once [YOLO detection model utilizing bounding boxes to detect the anatomical landmarks. In a non-limiting example, a YOLO detection model may detect and apply bounding boxes to a fascia iliacaClient Ref. MIT25673JL402, an artery 404, and a vein 406 as illustrated in FIG. 4A.

[0068] At step 385, the anatomical landmarks located in step 384 are segmented. As shown in the non-limiting example of FIG. 4A, the fascia iliaca 402 may be segmented into a target subregion 408. The target subregion 408 may be identified by detecting a lateral percentage of the fascia iliaca 402 and / or an exclusion zone lateral to an estimated nerve circle. In FIG. 3D, the target subregion 408 was detected by a combination of 10% or 20% lateral percentage of the fascia and the exclusion zone lateral to the estimated nerve circle.

[0069] The aimpoint location may be determined at step 386, and the probe subsequently guided on the surface of the subject to align interventional device with an insertion point. In the example in FIG. 4A, the aimpoint location 410 may be determined within the target subregion 408 by computing the centroid of the target subregion 408, centroid of a cropped area of the target subregion 408, a centroid of the medial 10% of the target subregion 408, or a corner closest to the femoral artery 404. Determining an aimpoint location for the fascia iliaca 402 may depend on its shape and the surrounding anatomy.

[0070] At step 387, a safe pathway for the interventional device to reach the target without impinging on the at least one critical structure may be determined. This may involve determining an insertion angle for an interventional device using the insertion point location, the aimpoint location, the location of the at least one critical structure, or any combination thereof.

[0071] A signal may be provided to a user to proceed with needle insertion at step 388. Such a signal may be provided on a graphical user interface, or a light in the probe, or the like. In a non-limiting example, a needle is included in the interventional device. The needle may be tracked during its insertion and the needle pathway confirmed at step 389. As will be described in more detail with reference to FIGS. 5A-5D below, the processor may adjust the needle if it and / or its trajectory are not confirmed to be aligned the pathway between the insertion point and the aimpoint location. The needle may be retracted from the subject and reinjected based on a required adjustment to align the needle with the pathway between the insertion point and the aimpoint location. For example, the needle may be observed on a graphical user interface providing real time image data, such as images 412, 414, and 416 of FIG. 4B. In image 412, the tip of a needle 418 can be seen adjacent to the fascia iliaca 402 and a femoral vessel 420. If the needle isClient Ref. MIT25673JLnot confirmed to be inserted in the target aimpoint location, an adjustment signal may be provided to adjust the needle as described above. Alternatively, if the needle is confirmed to be inserted in the target, a signal may be provided to a user thereafter to inject a fluid injectant at step 390. As used herein, "injectant” refers to an injectable substance. For example, the fluid injectant may be one of a saline solution for increased visibility in the ultrasound imaging data, an anesthetic solution, or an analgesic solution for a nerve block.

[0072] At step 391, the injection location and spread may be confirmed. In a nonlimiting example, the spread of the injected fluid injectant may be observed on a graphical user interface for visual confirmation. For example, a halo effect may be seen around a nerve when the fluid injectant is injected into the fascia iliaca. In images 414 and 416 of FIG. 4B, the fluid injectant has been injected, highlighting the femoral nerve 422. Other confirmation includes a change in pressure as the fluid is injected into anatomical structures, as well as confirmation of the needle tip location relative to the nerve. After injection, the safe pathway of the interventional device is confirmed or updated based on the spread of the fluid injectant and the locations of at least one of the target or at least one critical structure. In a non-limiting example, the system may segment the fluid injectant and the femoral nerve to confirm injection.

[0073] At step 392, a signal may be provided to signal needle adjustment. For example, the injection into the target may be halted and the needle of the interventional device may be adjusted to target the structure of interest while avoiding the at least on critical structure. For example, the needle may be adjusted by actively steering the needle after insertion to avoid the at least one critical structure. Alternatively, no adjustment is needed and injection of the target is continued or resumed.

[0074] It is noted that the systems and methods described may be used to detect vessel, nerves, and fascia anywhere on the body and are not limited to a femoral nerve block. For example, a nerve block procedure as described may be performed on in the upper extremity (e.g., shoulder, arm, wrist, or hand) and lower extremity (e.g., hip, thigh, knee, ankle, foot).

[0075] Referring now to FIGS. 5A-5C, the system may track the needle in step 389 based on segmentation. At step 502, the system may begin segmenting the needle shaft and tip in acquired ultrasound images. At step 504, the needle insertion is stopped, and the system may apply either a VibNet-based segmentation approach as discussed with reference to FIG. 5B or a dead reckoning approach as described in FIGS. 5C and 5D. AfterClient Ref. MIT25673JLstopping the segmentation of the needle shaft and tip in step 514, an error offset line for the needle path is calculated and displayed [step 516). In a non-limiting example, if the error offset line is greater than a predetermined threshold value, the device may retract and or reposition the needle to repeat the insertion and tracking steps 388 and 389. If the error offset lines is below a predetermined threshold value, then the process may continue to the fluid injectant injection step 390. In a non-limiting example, the threshold value is 1 mm. Alternatively, the threshold value may be 0.9 mm, 0.8 mm, 0.7 mm, 0.6 mm, 0.5 mm, 0.4 mm, 0.3 mm, 0.2 mm, 0.1 mm, or any value therebetween.

[0076] Needle shafts and tips can be difficult to see in B-mode ultrasound images. The VibNet-based approach of FIG. 5B overcomes this difficulty by segmenting the needle as it vibrates. At step 506, the needle is vibrated at a known frequency. In a non-limiting example, the frequency may be between 1-10 Hz. The frequency may be between 2-8 Hz. The frequency may be 4 Hz. At step 508, the needle shaft and tip are segmented by utilizing frequency signatures to extract a segmentation mask of the vibrating needle before ceasing the vibration at step 510. FIG. 6A illustrates the VibNet-based approach. An advantage of this approach is its ability to localize out of plane needles.

[0077] Referring now to FIGS. 5C and 5D, the Dead Reckoning approach may improve the situation where a needle tip is not visible in B-mode ultrasound images inside hyperechoic structures like fascia. The dead reckoning estimation of the needle tip at step 512 includes measuring the needle length via a microcontroller driving a motor responsible for needle insertion. This step identifies the true needle tip position. Subsequently or concurrently, the needle shaft may be segmented using a YOLO detection model in step 520. At step 522, an angled rectangle may be fit to the needle shaft segmentation and overlayed thereon, resulting in a determination of the needle angle. At step 524, the needle length from step 518 is projected onto the angled rectangles major axis. Next, the system calculates proximal and distal points of the needle bevel at step 526 and displays the merged needle segmentation at step 528. The images of FIG.6B illustrate the Dead Reckoning segmentation approach.

[0078] FIG 7A shows a non-limiting method 700 accounting for the geometric constraints of an articulating interventional device relative to the ultrasound probe. In a non-limiting example, where the region or structures of interest cannot be accessed because of their superficial position located below the midline of the probe, a method for offsetting the interventional device insertion is described. Method 700 may continueClient Ref. MIT25673JLafter locating the anatomical landmarks in the ultrasound imaging data at step 384. Specifically, at step 702, the depth of the target relative to the surface of the subject is determined. Based on the determined depth, it may be determined whether an offset insertion position is needed to reach the superficial structures of interest that are inaccessible due to the interventional device’s geometric constraints relative to the ultrasound probe (step 704). At step 706, the offset insertion position may be determined, and the probe may be guided to the offset insertion position. In a nonlimiting example, the insertion offset may be 8 mm or 17 mm to the left of the image center. These offset distances are non-limiting and may be based on the specific device geometry. In another example, the insertion offset may be to the right of the image center. It is noted that steps 385-392 may continue after the method of 700.

[0079] FIG. 7B shows an example user interface 710 showing the detection of the femoral artery by boundary box 712 and femoral vein by boundary box 714, segmentation of the fascia by box 716, and a needle insertion with an 8 mm offset.

[0080] In some configurations, an integrated sheath and guidewire and deployment mechanism may be used. Using a shuttle, a sheath, needle, and guidewire may be selectively deployed into a subject as desired.

[0081] In some configurations, a safe method of cartridge-based guidewire and sheath insertion may be used that prevents sharps from being exposed outside of the system when the needle is not being inserted. The guidewire, sheath, or the system itself may be used without the needle tip ever being exposed as the needle is always fully enclosed in the cartridge when not being deployed. This provides for patient and operator inadvertent stick safety, reduces the likelihood of infection, and provides for increased speed of deployment.

[0082] Referring to FIG. 8, an example of a system 800 for generating and implementing a hybrid machine learning and mechanistic model in accordance with some embodiments of the systems and methods described in the present disclosure is shown. As shown in FIG. 8, a computing device 850 can receive one or more types of data (e.g., ultrasound, multiparametric MRI data, vessel of interest image data, and the like) from image source 802. In some embodiments, computing device 850 can execute atleast a portion of an anatomical landmark image processing system 804 to generate images of anatomical landmarks from data received from the image source 802.

[0083] Additionally or alternatively, in some embodiments, the computing deviceClient Ref. MIT25673JL850 can communicate information about data received from the image source 802 to a server 852 over a communication network 854, which can execute at least a portion of the anatomical landmark image processing system 804 to generate images of an anatomical landmark from data received from the image source 802. In such embodiments, the server 852 can return information to the computing device 850 (and / or any other suitable computing device) indicative of an output of the anatomical landmark image processing system 804 to generate images of anatomical landmarks from data received from the image source 802.

[0084] In some embodiments, computing device 850 and / or server 852 can be any suitable computing device or combination of devices, such as a desktop computer, a laptop computer, a smartphone, a tablet computer, a wearable computer, a server computer, a virtual machine being executed by a physical computing device, and so on. The computing device 850 and / or server 852 can also reconstruct images from the data.

[0085] In some embodiments, image source 802 can be any suitable source of image data (e.g., measurement data, images reconstructed from measurement data), such as an ultrasound system, another computing device (e.g., a server storing image data), and so on. In some embodiments, image source 802 can be local to computing device 850. For example, image source 802 can be incorporated with computing device 850 (e.g., computing device 850 can be configured as part of a device for capturing, scanning, and / or storing images). As another example, image source 802 can be connected to computing device 850 by a cable, a direct wireless link, and so on. Additionally or alternatively, in some embodiments, image source 802 can be located locally and / or remotely from computing device 850, and can communicate data to computing device 850 (and / or server 852) via a communication network (e.g., communication network 854).

[0086] In some embodiments, communication network 854 can be any suitable communication network or combination of communication networks. For example, communication network 854 can include a Wi-Fi network (which can include one or more wireless routers, one or more switches, etc.), a peer-to-peer network (e.g., a Bluetooth network), a cellular network (e.g., a 3G network, a 4G network, etc., complying with any suitable standard, such as CDMA, GSM, LTE, LTE Advanced, WiMAX, etc.), a wired network, and so on. In some embodiments, communication network 854 can be a local area network, a wide area network, a public network (e.g., the Internet), a private or semiClient Ref. MIT25673JLprivate network (e.g., a corporate or university intranet), any other suitable type of network, or any suitable combination of networks. Communications links shown in FIG.8 can each be any suitable communications link or combination of communications links, such as wired links, fiber optic links, Wi-Fi links, Bluetooth links, cellular links, and so on.

[0087] Referring now to FIG. 9, an example of hardware 900 that can be used to implement image source 802, computing device 850, and server 854 in accordance with some embodiments of the systems and methods described in the present disclosure is shown. As shown in FIG. 9, in some embodiments, computing device 850 can include a processor 902, a display 904, one or more inputs 906, one or more communication systems 908, and / or memory 910. In some embodiments, processor 902 can be any suitable hardware processor or combination of processors, such as a central processing unit ("CPU”), a graphics processing unit ("GPU”), and so on. In some embodiments, display 904 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 906 can include any suitable input devices and / or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.

[0088] In some embodiments, communications systems 908 can include any suitable hardware, firmware, and / or software for communicating information over communication network 854 and / or any other suitable communication networks. For example, communications systems 908 can include one or more transceivers, one or more communication chips and / or chip sets, and so on. In a more particular example, communications systems 908 can include hardware, firmware and / or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0089] In some embodiments, memory 910 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 902 to present content using display 904, to communicate with server 852 via communications system(s) 908, and so on. Memory 910 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 910 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 910 can have encoded thereon, or otherwise stored therein, a computer program for controllingClient Ref. MIT25673JLoperation of computing device 850. In such embodiments, processor 902 can execute at least a portion of the computer program to present content (e.g., images, user interfaces, graphics, tables), receive content from server 852, transmit information to server 852, and so on.

[0090] In some embodiments, server 852 can include a processor 912, a display 914, one or more inputs 916, one or more communications systems 918, and / or memory 920. In some embodiments, processor 912 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, display 914 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, and so on. In some embodiments, inputs 916 can include any suitable input devices and / or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, and so on.

[0091] In some embodiments, communications systems 918 can include any suitable hardware, firmware, and / or software for communicating information over communication network 854 and / or any other suitable communication networks. For example, communications systems 918 can include one or more transceivers, one or more communication chips and / or chip sets, and so on. In a more particular example, communications systems 918 can include hardware, firmware and / or software that can be used to establish a Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0092] In some embodiments, memory 920 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 912 to present content using display 914, to communicate with one or more computing devices 850, and so on. Memory 920 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 920 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 920 can have encoded thereon a server program for controlling operation of server 852. In such embodiments, processor 912 can execute at least a portion of the server program to transmit information and / or content (e.g., data, images, a user interface) to one or more computing devices 850, receive information and / or content from one or more computing devices 850, receive instructions from one or more devices (e.g., a personal computer, a laptopClient Ref. MIT25673JLcomputer, a tablet computer, a smartphone), and so on.

[0093] In some embodiments, image source 802 can include a processor 922, one or more image acquisition systems 924, one or more communications systems 926, and / or memory 928. In some embodiments, processor 922 can be any suitable hardware processor or combination of processors, such as a CPU, a GPU, and so on. In some embodiments, the one or more image acquisition systems 924 are generally configured to acquire data, images, or both, and can include an RF transmission and reception subsystem of an MRI system. Additionally or alternatively, in some embodiments, one or more image acquisition systems 924 can include any suitable hardware, firmware, and / or software for coupling to and / or controlling operations of an MRI system or an RF subsystem of an MRI system. In some embodiments, one or more portions of the one or more image acquisition systems 924 can be removable and / or replaceable.

[0094] Note that, although not shown, image source 802 can include any suitable inputs and / or outputs. For example, image source 802 can include input devices and / or sensors that can be used to receive user input, such as a keyboard, a mouse, a touchscreen, a microphone, a trackpad, a trackball, and so on. As another example, image source 802 can include any suitable display devices, such as a computer monitor, a touchscreen, a television, etc., one or more speakers, and so on.

[0095] In some embodiments, communications systems 926 can include any suitable hardware, firmware, and / or software for communicating information to computing device 850 (and, in some embodiments, over communication network 854 and / or any other suitable communication networks). For example, communications systems 926 can include one or more transceivers, one or more communication chips and / or chip sets, and so on. In a more particular example, communications systems 926 can include hardware, firmware and / or software that can be used to establish a wired connection using any suitable port and / or communication standard (e.g., VGA, DVI video, USB, RS-232, etc.), Wi-Fi connection, a Bluetooth connection, a cellular connection, an Ethernet connection, and so on.

[0096] In some embodiments, memory 928 can include any suitable storage device or devices that can be used to store instructions, values, data, or the like, that can be used, for example, by processor 922 to control the one or more image acquisition systems 924, and / or receive data from the one or more image acquisition systems 924; to images from data; present content (e.g., images, a user interface) using a display;Client Ref. MIT25673JLcommunicate with one or more computing devices 850; and so on. Memory 928 can include any suitable volatile memory, non-volatile memory, storage, or any suitable combination thereof. For example, memory 928 can include RAM, ROM, EEPROM, one or more flash drives, one or more hard disks, one or more solid state drives, one or more optical drives, and so on. In some embodiments, memory 928 can have encoded thereon, or otherwise stored therein, a program for controlling operation of image source 802. In such embodiments, processor 922 can execute at least a portion of the program to generate images, transmit information and / or content (e.g., data, images) to one or more computing devices 850, receive information and / or content from one or more computing devices 850, receive instructions from one or more devices (e.g., a personal computer, a laptop computer, a tablet computer, a smartphone, etc.), and so on.

[0097] In some embodiments, any suitable computer readable media can be used for storing instructions for performing the functions and / or processes described herein. For example, in some embodiments, computer readable media can be transitory or non-transitory. For example, non-transitory computer readable media can include media such as magnetic media (e.g., hard disks, floppy disks), optical media (e.g., compact discs, digital video discs, Blu-ray discs), semiconductor media (e.g., random access memory ("RAM”), flash memory, electrically programmable read only memory ("EPROM”), electrically erasable programmable read only memory ("EEPROM”)), any suitable media that is not fleeting or devoid of any semblance of permanence during transmission, and / or any suitable tangible media. As another example, transitory computer readable media can include signals on networks, in wires, conductors, optical fibers, circuits, or any suitable media that is fleeting and devoid of any semblance of permanence during transmission, and / or any suitable intangible media.

[0098] Referring to FIG. 10A, a perspective view of a non-limiting example interventional device guide injection assembly 1000 coupled to an ultrasound probe 1010 is shown. Base 1040 is shown with ultrasound handle fixture 1030 that provides detachable coupling to ultrasound probe 1010. The injection assembly 1000 may be attached to any ultrasound device, such as by being strapped onto an ultrasound probe 1010 using the ultrasound handle fixture 1030. Base 1040 may include a mechanical support resting on the skin in order to minimize kick-back and improve needle insertion accuracy.

[0099] Referring to FIG. 10B, a side view of the interventional device guideClient Ref. MIT25673JLinjection assembly 1000 of FIG. 10A is shown. In a non-limiting example, base 1040 contains a motor to set the angle at which the interventional device, which may be a needle, will be inserted. The base 1040 may also contain a second drive motor to drive the interventional device to the desired depth. The motor may be controlled to vary the needle insertion speed at different insertion depths, e.g., the needle may be inserted relatively slowly through the skin to minimize kick-back and improve accuracy, and then inserted faster subsequently. In some configurations, the drive motor function may be replaced or augmented by a spring or any suitable method of storing mechanical energy, and an additional motor or other suitable method of mechanical actuation to enable injection into a subject. Cartridge 1020 is detachably coupled to base 1040 and may be configured for the intervention being performed. In non-limiting examples, cartridge 1020 may include configurations to treat indications requiring vascular access, alleviate tension pneumothorax, establish an airway, perform cricothyrotomy, access an organ or body cavity, take a tissue sample, access image guided tumor ablation or other image guided targeted cancer therapy, such as radiofrequency ablation, ethanol ablation, cryoablation, electroporation and the like, or percutaneous minimally invasive surgery, such as ligament release and the like. Non-limiting example cartridge configurations are listed in Table 1 below.

[0100] Table 1 - Non-limiting example cartridge configurationsIntervention Cartridge Capability Generation1 Needle onlyFemoral 2 Needle with dilator and / or guide wire (or Artery / Vein similarly functioning guide)3 REBOA, clotting agent, other intervention Vascular1 Needle onlyInternal Jugular 2 Needle with dilator and / or guide wire (or Vein similarly functioning guide)3 REBOA, clotting agent, other intervention PeripheralNerve Needle with nerve blocking agent Regional 1Block (analgesia or anesthesia)SpinalClient Ref. MIT25673JLC r icothyrotomy, 1 Needle only Tracheostomy 2 Breathing tube (or similar 3 Breathing tube + forced air methods ofAirestablishingairway access)Tension 1 Needle only Pneumothorax 2 Chest tube Abdomen Ascites 1 Needle only2 CatheterBladder 1 Needle and Catheter Pregnant uterus 1 Needle only amniocentesisSoft Focal 1 Needle onlytissue lesion / tumorbiopsyImage Focal anatomy / 1 Needle with dilator and / or guide wire (or Guided lesion / tumor similarly functioning guide) and an ablation Tumor deviceAblation

[0101] FIG. 10C is a side view of the base and ultrasound probe fixture for the interventional device guide of FIG. 10B. Base 1040 includes a drive motor 1045 to set an insertion angle and / or depth for an interventional device held by cartridge slot 1025 coupled by cartridge coupling 1022. Advancement motor 1047 may be included to advance an interventional device with activation by advancement control 1055, which in a non-limiting example is a button. Electrical interface connector 1052 may provide communication to an ultrasound imaging system or separate display system. User guidance signal 1050 provides feedback to a user and may take the form or any display intending to direct the user in gross and / or precise placement of the device. In a nonlimiting example, user guidance signal 1050 includes an arrangement of LEDs. In some configurations, user guidance signal 1050 maybe coupled to the cartridge 1020 and may be specific to the particular indication being treated.Client Ref. MIT25673JL

[0102] In a non-limiting example, the drive motor 1045 and advancement 1047 may be automatically controlled by the processor to set an insertion angle and / or depth for the interventional device and to advance the interventional device, respectively.

[0103] FIG. 10D is a cross-section of a non-limiting example cartridge 1020 compatible with the injection assembly 1000 of FIG. 7B. Lead screw 1060 may provide for actuation of base coupling 770 to couple the non-limiting example cartridge 1020 to base 1040 in FIG. 10B. Needle carriage 1065 is shown as a non-limiting example of a needle cartridge application.

[0104] FIG. 10E shows a cross-section of a non-limiting example cartridge 1020 including an ampule 1080 with fluid injectant and needle 1082, coupled to an ultrasound probe 1010 via the base 1040 and ultrasound handle fixture 1030. In a non-limiting example, the fluid injectant includes, but is not limited to, saline solution, an anesthesia solution, or an analgesic solution. In a non-limiting example, the ampule may be absent from the cartridge 1020, and the needle 1082 may be preloaded with the fluid injectant instead.

[0105] In another non-limiting example, an injection motor may be included (not shown) to inject the injectant. The injection motor may controlled automatically by processor to inject the injectant.

[0106] Referring to FIG. 11A, a perspective view of a non-limiting example interventional device guide integrated with an ultrasound probe is shown. Integrated interventional device guide 1100 is shown being placed on a subject 1110. The integrated interventional device guide 1100 may include functionality similar to injection assembly 1000 described above with integration with an ultrasound probe. The integrated interventional device guide 1100 may be ultrasound guided, and may employ machine learning or artificial intelligence for identifying a target for penetration and guiding penetration of the target, in accordance with the present disclosure. The integrated ultrasound transducer may provide for excitation, for reading a source, for processing ultrasound signals, and the like. Integrated interventional device guide 1100, may include onboard artificial intelligence algorithms, motors, associated drive circuitry, other electronics / mechanics, and the like fit within a housing 1105 for the integrated device guide 1100. A cartridge, such as described herein, may be detachably coupled to integrated interventional device guide 1100. In some configurations, the integrated interventional device guide 1100 may be robotically controlled.Client Ref. MIT25673JL

[0107] Referring to FIG. 11B is an exploded view of the integrated interventional device guide 800 and ultrasound probe of FIG. 11A is shown. Circuit boards 1120 may provide for ultrasound guidance from ultrasound transducers 1140, and may employ machine learning or artificial intelligence for identifying a target for penetration and guiding penetration of the target, in accordance with the present disclosure. Battery 1130 may provide power for the integrated device. One battery cell is shown in FIG. 11B, but it is to be appreciated that any number of battery cells may be used, such as two cells for extended life, or any other form of power supply. Drivetrain 1150 may provide for independent needle or interventional device insertion. Needle and interventional device 1170 maybe inserted into a subject with motors 860.

[0108] The following example are provided for illustrative purposes and are not intended to be limiting.

[0109] The present disclosure has described one or more embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention.

[0110] Example

[0111] This example evaluated object detection and segmentation algorithms for femoral landmarks trained on both in vivo and ex vivo porcine data, including the femoral artery (FA), femoral vein (FV), and fascia iliaca (FI), which is a fibrous layer around the femoral nerve. The detection model was developed for the FA, FV, and FI, while the segmentation model was developed only for the FI for aimpoint identification. In a standard ultrasound-guided femoral block, a clinician identifies the FI and an aimpoint in the FI, as the local anesthesia is injected to surround the nerve. The needle must not puncture the nerve itself, as that can lead to permanent nerve damage. The FI is also more visible in the ultrasound image than the femoral nerve. Prospective testing of aimpoint identification and guidance was completed through in vivo and ex vivo porcine experiments. A preliminary sciatic nerve detection model was also trained and tested on a limited in vivo porcine dataset, showing that the Al framework can be adapted to additional nerve sites.

[0112] Femoral Landmark Detection

[0113] FIG. 12 shows the precision-recall (PR) curve from five-fold cross-validation of the femoral landmark detection algorithm on the combined test set of in vivoClient Ref. MIT25673JLand ex vivo images. The average area under the precision-recall curve (AUPRC) was 0.88 across all classes, 0.93 for the FA, 0.89 for the FV, and 0.80 for the FI. The FI tends to deform more than the FA and FV, making consistent identification of its anatomical features more challenging, leading to lower model performance.

[0114] The model’s all-class average AUPRC was 0.92 for in vivo test data only, while the average AUPRC for ex vivo test data was 0.65, as shown in FIG. 13. This marked difference in performance between in vivo and ex vivo test data is likely a result of the training dataset being heavily imbalanced in favor of in vivo data. Additionally, the ex vivo tissue degrades post-mortem as time passes, so image quality tends to be less clear.

[0115] Fascia iliaca segmentation

[0116] The FI segmentation model achieved mean and median Dice similarity coefficients of 0.91 and 0.96, respectively, across all test set images relative to ground truth FI masks. FIG. 14 presents the distribution of Dice coefficients, indicating that 91% of the model’s output segmentation masks matched with the ground truth mask at greater than 0.90. The false negatives observed at the far left of FIG. 14 are largely a result of the model’s poor performance on the FI when it is compressed, distorting the image, which reduces the area of the FI. Overall, however, the high level of performance achieved by this model enables reliable estimation of an aimpoint for needle insertion into the FI adjacent to the femoral nerve.

[0117] Prospective testing on aimpoint identification

[0118] The femoral algorithms were integrated into a custom AI-GUIDE Nerve Block software application to interpret ultrasound images in real time and display Al model outputs for user guidance (see FIG. 15). The operational workflow of the device emulates the clinical workflow of a nerve block. The operator first places the device in the target region on the patient, then scans the region by moving the device mediolaterally as instructed by the LCD display on the device. When the Al algorithm detects the target landmarks, the app guides the operator to align the device over the aimpoint for hypothetical needle insertion.

[0119] A total of 11 ex vivo porcine models were used during the first phase of prospective testing. Eight operators of varying experience levels, drawn from a multidisciplinary group of engineers, animal technicians, and clinicians, utilized the device to reach an aimpoint location determined by the Al models. A summary of ex vivo testing is shown in Error! Reference source not found.. The time to identify needleClient Ref. MIT25673JLaimpoint decreased as device updates were implemented across four rounds. Initial rounds of testing were more exploratory, resulting in longer scanning times and fewer aimpoint identification successes. As more training data were added to the Al algorithms after each round, performance of the system improved. During Round 1 of testing, a total of nine aimpoint identification attempts were made with only two successes. The third ex vivo model in Round 1 had poor tissue quality, thus no attempts were made. Round 2 had 28 successful aimpoint identifications out of 38 attempts (73.7%), Round 3 had 65 successes out of 67 attempts (97.0%), and Round 4 had 43 successes out of 46 attempts (93.5%). The high success rates of needle aimpoint identification in Rounds 3 and 4 were accompanied by average attempt times of 47.7 ± 46.2 seconds and 96.1 ± 100.4 seconds, respectively. Round 4 of testing included two ex vivo models, one of which had poor tissue quality resulting in longer scanning times per attempt. The results of the final two rounds of testing showed that the system could consistently identify needle aimpoints and was ready for in vivo testing.

[0120] Table 2. Summary of attempt success rates and timing for the image processing algorithms to guide the operator to a suitable needle insertion aimpoint in the ex vivo models.Round 1 Round 2 Round 3 Round 4 Number of Ex Vivo Porcine Models 3 3 3 2 Number of Operators 5 6 2 2 Attempts 9 38 67 46 Successful Aimpoint Identifications 2 28 65 43 Success Rate 22.2% 73.7% 97.0% 93.5% Mean Time per Attempt (s) 433.3 193.2 47.7 96.1 Standard Deviation Time per Attempt (s) 158.1 167.1 46.2 100.4

[0121] For in vivo prospective testing, a total of 60 trials were attempted by two operators across eight live pigs. Successful aimpoint identification was achieved in 59 trials (98.3%). FIG. 16 presents the time-to-aimpoint identification that was recorded for each trial. This data excludes four trials in which the software application froze intermittently, one trial where timing data was not collected, and one trial where the animal died mid-trial. The maximum trial time was capped at 3 minutes (180 seconds). If the device was not successfully aligned over a calculated aimpoint within this time limit,Client Ref. MIT25673JLthe trial was stopped and a time of 180 seconds was recorded. The maximum time was reached in only one of the 60 trials.

[0122] The time to identify the target location in the fascial space varied across the eight test pigs, with an average time of 40.5 seconds and a standard deviation of 33.5 seconds. Under normovolemic conditions, the average time to identification was 45.7 seconds (SD = 37.9) across 35 trials. Under hypovolemic conditions, the average time was 33.2 seconds (SD = 27.0) across 25 trials, indicating that the algorithm is robust to variations in blood volume and shock conditions.

[0123] While most of the aimpoints were identified based on the centroid calculated by the FI segmentation model, the user had the option to switch the aimpoint calculation to the detection model bounding box if the segmentation algorithm could not reliably identify the FI or showed inconsistent segmentation. There were 10 trials in which the bounding box centroid was chosen as the aimpoint over the segmentation model aimpoint during testing, with an average time to aimpoint identification of 52.5 seconds (SD = 33.4). The majority of these cases occurred when the fascia was very shallow, so the target was near the edge of the image and the model would not segment well with some landmark features out of view.

[0124] Preliminary sciatic nerve detection

[0125] While the primary aim of Al model development was for femoral nerve block, investigated Al models for sciatic nerve (SN) detection were separately using a small amount of in vivo porcine data. A sciatic nerve block involves needle insertion and analgesia injection into the gluteal, subgluteal, or popliteal area around the SN. FIG. 17 displays an example ultrasound image with the SN detected. The average four-fold cross-validation AUPRC of the SN detection model was 0.69, also shown in FIG. 17. The lower performance of the SN model is related to a number of factors, including the inherent difficulty identifying the SN, label noise, and limited training data with the SN in view. In humans, it is noted that it is a more difficult block to achieve, due to its depth and lack of nearby structures, whereas the femoral block is near the more obvious FA and FV landmarks. Porcine structures are generally similar to human structures in this area. Although the SN detection model exhibited less robust performance, it shows that the algorithm approach can be adapted for multiple nerve block sites.

[0126] Algorithm training details

[0127] A training-validation-testing split of 70:10:20 was used to train andClient Ref. MIT25673JLevaluate the performance of a You-Only-Look-Once version 8 nano (Y0L0v8n) object detection model in PyTorch for femoral landmark detection. The percentage splits were done at a subject / pig level to prevent training data from contaminating the validation or test sets. A hyperparameter sweep was performed across the batch size, learning rate, optimizer, weight decay, image augmentation settings (reflection about the vertical axis, translation, mosaicking, rotation, color jitter], optimizer, and image size. Image resizing was done with padding of the smaller dimension to preserve the aspect ratio of the original ultrasound image. The learning rate was linearly increased across the first five epochs, reaching a maximum learning rate shown in Table 3, then decayed following a cosine annealing schedule. The final settings for these parameters, shown in bold in Table 3, were then used in a five-fold cross validation training of the model. Each fold was set to train for 100 epochs on a single NVIDIA H100 GPU, but training was halted if performance on the validation set did not improve for 10 epochs.

[0128] Table 3. Tested hyperparameters, with final hyperparameters in bold, used for the five-fold cross-validation training of the modelLearning Rate WeightBatch Size Optimizer Image Size (max) Decay16, 64, Adam, AdamW,IE-2, IE-3, IE-4 IE-3, IE-4 320, 640 256 SGD

[0129] A Y0L0v8n segmentation model was trained for FI segmentation using a similar procedure as outlined for femoral detection training, with a key difference of increasingthe image size from 320x320 to 640x640. We observed worse performance on the validation sets when the image resolution was coarser than 640x640.

[0130] The optimal hyperparameters found for the femoral detection model were also used to train the sciatic nerve detection model. With the smaller dataset and four total subjects, four-fold cross validation was performed such that each subject served as the test set for one given iteration with the remaining data used for training and validation.

[0131] Evaluation metrics

[0132] To evaluate the detection models, precision-recall curves were generated at an intersection over union (I0U) threshold of 0.5 between the model output bounding boxes and the ground truth bounding boxes. To evaluate the segmentation, the DiceClient Ref. MIT25673JLcoefficient was calculated as a measure of overlap between the model segmentation mask output and the ground truth mask of the fascia iliaca, with 1.0 representing maximum overlap.

[0133] Software design and workflow

[0134] A custom C++ software application (app) was developed on the Terason uSmart 3200T Plus to run image processing and deep learning algorithms in real time (30 Hz) on the ultrasound imaging feed. The Terason software development kit (SDK) was used as a starting point and implemented on a Windows 10 tablet with a NVIDIA Quadro P3000 1280 CUDA core GPU. The SDK allows independent software apps to communicate with the Terason ultrasound app, uSmart 3200T. The SDK enables passing of images from the uSmart 3200T app to the custom AI-GUIDE Nerve Block app and allows setting and receiving imaging parameters such as depth and gain. The SDK uses COM Automation, Active Object Registration, and the Windows MFC Library. Our custom AI-GUIDE software app for Nerve Block was designed to determine where anatomical landmarks (including arteries, veins, and fascia) were located in the ultrasound image supplied via the SDK. Using this information, the app guides the device operator to position the device over the correct aimpoint.

[0135] Multi-threading was employed to integrate the detection and segmentation algorithms into the AI-GUIDE Nerve Block app for real-time image interpretation and guidance. NVIDIA TensorRT and Open Neural Network Exchange (ONNX) were used to optimize the trained PyTorch models to deliver low-latency and high-throughput on the NVIDIA GPU. The total time for the algorithm chain to process a frame ranges between 30-50 ms. This workflow can support real-time Al output overlay on ultrasound imaging rates of 20-30 Hz. The main thread sends ultrasound images to the algorithm thread, which then returns the generated array of bounding boxes and segmentation masks detected in the images. Each of the detections returned is reviewed to make sure it is valid to be passed back to the main thread. Only the object with the highest probability in each class is kept. For example, if the FA and FV class detections overlap by greater than 75%, then the higher probability detection is kept and the next highest probability detection of the other vessel is used if available and below the overlap threshold. Temporal information is also used by the app to determine an estimated location of landmarks based on detections from recent frames if no detections are found on the current frame.

[0136] WorkflowClient Ref. MIT25673JL

[0137] The first step in the image processing workflow is to confirm contact between the device and skin and to optimize the depth of the ultrasound image to ensure good visualization of the landmarks using previously developed algorithms. In general, image quality can suffer when deeper imaging depths are used when not necessary, so choosing an appropriate depth for the subject-specific anatomy helps maximize device utility.

[0138] The initial depth for calibration was set to be 6 cm, and the femoral vessels were used as the landmarks for which to optimize the depth. Once one or more vessels are detected, the depth calibration process calculates a buffer of approximately 1 cm relative to the image bottom using the deepest vessel bounding box. Adjustments are made by rounding up to the nearest integer. Once the depth is optimally set, the detection and segmentation algorithms calculate the insertion aimpoint, then provide guidance to the operator to align the device over the aimpoint. Once that position is found, a final algorithm checks to ensure the aimpoint in the FI is sufficiently far away from the edge of the femoral artery (FA) for safe needle insertion (1 mm by default), instructing the operator to adjust craniocaudal positioning as necessary. When the safe distance is confirmed, the device displays a green box.

[0139] AI-GUIDE Device Integration

[0140] AI-GUIDE is an ultrasound-based percutaneous intervention device that was originally prototyped for vascular access was adapted for nerve block applications as shown in FIG. 7E. In the present non-example, it combines a housing for a commercial ultrasound probe, a display screen for user guidance, an arm that contains a needle cartridge and custom electronics to control the needle angle, and a button designed to insert the needle when the algorithm locates a suitable aimpoint. The device interfaces with the Terason uSmart 3200T Plus commercial ultrasound system running on a Windows 10 tablet. All prospective testing was done using the handheld device running the AI-GUIDE Nerve Block software.

[0141] Testing methods

[0142] The ex vivo model was used for four rounds of testing (N=ll ex vivo subjects) with the device, making software updates in between testing rounds based on feedback from the testers. Testing rounds allowed each user to scan using the device for up to 10 minutes (600 seconds) or until an aimpoint was identified and the device was aligned over it. If the aimpoint alignment was achieved, the time to do so was collected.Client Ref. MIT25673JLOtherwise, 10 minutes was recorded as the trial time. During the first two rounds of testing, the device was operated by users with different levels of experience (engineers, clinicians, and animal technicians to obtain feedback from various perspectives. During the last two rounds of testing, we down-selected to the top two performing users, based on quicker aimpoint targeting rates, to conduct a more focused assessment of device performance prior to moving into in vivo porcine studies. The number of attempts for each ex vivo tissue model was limited by the image quality and tissue deterioration due to edema buildup.

[0143] For the second round of testing, the device was tested in a live swine model Live animal subjects were maintained under a surgical plane of anesthesia and analgesia throughout the studies. For one of protocols, the swine were modelling an acute kidney injury and then were monitored for 24 hours followed by humane euthanasia. The device testing occurred right before euthanasia, when the subject was normotensive. The second animal protocol was evaluating automated resuscitation controller performance during hemorrhagic shock in tandem hemorrhage-resuscitation events. This allowed for evaluating performance of the AI-GUIDE Nerve Block device in both normotensive and hypotensive conditions, before and after the second hemorrhage, respectively. The maximum trial time was capped at 3 minutes (180 seconds). If the device was not successfully aligned over a calculated aimpoint within this time limit, the trial was stopped and a time of 180 seconds was recorded.

[0144] A total of eight animals were tested on the right and left femoral nerves, with multiple tests for each, totaling 60 trials. Porcine results were collected from animals under both normovolemic and hypovolemic conditions, as both conditions are likely in battlefield scenarios.

[0145] During testing the operator could choose their approach depending on the ability of the device to segment the fascia. If the fascia was not segmenting quickly, the device operator could override the segmentation model and use the bounding box instead.

[0146] In some implementations, devices or systems disclosed herein can be utilized or installed using methods embodying aspects of the invention. Correspondingly, description herein of particular features or capabilities of a device or system is generally intended to include disclosure of a method of using such features for intended purposes and of implementing such capabilities. Similarly, express discussion of any method ofClient Ref. MIT25673JLusing a particular device or system, unless otherwise indicated or limited, is intended to inherently include disclosure, as embodiments of the invention, of the utilized features and implemented capabilities of such device or system.

[0147] As used herein, unless otherwise limited or defined, "or" indicates a nonexclusive list of components or operations that can be present in any variety of combinations, rather than an exclusive list of components that can be present only as alternatives to each other. For example, a list of "A, B, or C" indicates options of: A; B; C; A and B; A and C; B and C; and A, B, and C. Correspondingly, the term "or" as used herein is intended to indicate exclusive alternatives only when preceded by terms of exclusivity, such as "only one of," or "exactly one of." For example, a list of "only one of A, B, or C" indicates options of: A, but not B and C; B, but not A and C; and C, but not A and B. In contrast, a list preceded by "one or more" (and variations thereon) and including "or" to separate listed elements indicates options of one or more of any or all of the listed elements. For example, the phrases "one or more of A, B, or C" and "at least one of A, B, or C" indicate options of: one or more A; one or more B; one or more C; one or more A and one or more B; one or more B and one or more C; one or more A and one or more C; and one or more A, one or more B, and one or more C. Similarly, a list preceded by "a plurality of’ (and variations thereon) and including "or” to separate listed elements indicates options of one or more of each of multiple of the listed elements. For example, the phrases "a plurality of A, B, or C” and "two or more of A, B, or C” indicate options of: one or more A and one or more B; one or more B and one or more C; one or more A and one or more C; and one or more A, one or more B, and one or more C.

[0148] The present invention has been described in terms of one or more preferred embodiments, and it should be appreciated that many equivalents, alternatives, variations, and modifications, aside from those expressly stated, are possible and within the scope of the invention.

Claims

Client Ref. MIT25673JLCLAIMS1. A system for guiding an interventional device in an interventional procedure of a subject, comprising:an ultrasound probe;a guide system coupled to the ultrasound probe and configured to guide the interventional device into a field of view (FOVj of the ultrasound probe, wherein the interventional device includes a needle;a non-transitory memory having instructions stored thereon;a processor configured to access the non-transitory memory and execute the instructions, wherein the processor is caused to:access image data acquired from the subject using the ultrasound probe, wherein the image data include at least one image of a target;locate anatomical landmarks in the image data,segment the anatomical landmarks including the target; determine, from the image data, an aimpoint location on the target for injecting a fluid inj ectant;determine, from the image data, a pathway for the interventional device to reach the target without impinging on at least one critical structure; and control an insertion of the needle and injection of the fluid injectant into the target.

2. The system of claim 1, wherein the processor is further caused to confirm the injection of the fluid injectant into the target.

3. The system of claim 2, wherein confirming the injection of the fluid injectant includes acquiring ultrasound images during the injection and segmenting at least one of the injectant or the target in the ultrasound images.

4. The system of claim 2, where the processor is further caused to confirm and / or update the pathway for the interventional device to reach at least one of the target without impinging on the at least one critical structure based on a spread of the fluid injectant in the target.Client Ref. MIT25673JL5. The system of claim 1, wherein the processor is further caused to track a location of the interventional device during the insertion and determine an error offset line of the interventional device.

6. The system of claim 5, wherein the processor segments the interventional device during the insertion.

7. The system of claim 5, wherein the processor is further caused to determine an adjustment to a trajectory of the interventional device.

8. The system of claim 7, wherein the interventional device is retracted and reinserted based on the adjustment.

9. The system of claim 1, wherein the processor is further caused to determine an insertion angle for the interventional device using at least one of the aimpoint location, a location of the target, or a location of the at least one critical structure.

10. The system of claim 1, wherein the target is one of an artery, a vein, a femoral artery, a femoral vein, a jugular vein, a peripheral vein, a subclavian vein, an airway, a lumen, a luminal organ, a body cavity, a fluid filled anatomic space, a location requiring biopsy, a breast, a kidney, a lymph node, a spinal canal, a perineural location, a peritoneal space, a pleural space, a muscle, connective tissue, a location between two or more thereof, a location between one thereof and a critical structure, or a predefined offset relative to one thereof.

11. The system of claim 10, wherein the connective tissue includes fascia.

12. The system of claim 1, wherein the processor is configured to receive a plurality of images of the target of the subject acquired in real time to access the image data.Client Ref. MIT25673JL13. The system of claim 12, wherein the plurality of images include a plurality of views of the, and wherein the processor is configured to assess the plurality of views to identify at least one of the critical structures to avoid in the subject and identify an insertion point location on the subject where the interventional device, when injected, reaches the target or aimpoint location from the insertion point location without penetrating the at least one critical structure in the subject.

14. The system of claim 13, wherein the at least one critical structure includes at least one of a bone, a lung, heart, an unintended blood vessel, a non-target organ, or a nerve.

15. The system of claim 1, wherein the guide system includes a removable cartridge coupled to a base of the guide system, wherein the cartridge contains the interventional device.

16. The system of claim 1, wherein the fluid injectant includes at least one of a saline solution, an anesthetic solution, or an analgesic solution.

17. The system of claim 1, wherein the processor is further configured to track the needle during the insertion by acquiring ultrasound images.

18. The system of claim 17, wherein the processor is further configured to segment the needle in the ultrasound images.

19. The system of claim 1, wherein the processer is further caused to determine, from the image data, a depth of the target within the subject;determine a need for an offset insertion position for the interventional device based on the depth relative to the ultrasound probe; anddetermine an offset insertion position distance based on a position of the target in an imaging frame of the image data.Client Ref. MIT25673JL20. The system of claim 19, wherein the offset insertion position is 8 mm or 17 mm.

21. A system for performing a nerve block, comprising:an ultrasound probe;a guide system coupled to the ultrasound probe and configured to guide a needle into a field of view (FOV) of the ultrasound probe;a non-transitory memory having instructions stored thereon;a processor configured to access the non-transitory memory and execute the instructions, wherein the processor is caused to:access image data acquired from a subject using the ultrasound probe, wherein the image data include at least one image of a target and at least one critical structure to avoid, wherein the at least one critical structure includes a nerve;locate anatomical landmarks in the image data;segment the anatomical landmarks including the at least one of the target or the nerve;determine, from the image data, an aimpoint location on the target for injecting at least one of an anesthetic solution or an analgesic solution for providing a nerve block;determine, from the image data, a pathway for the needle to reach the target without impinging on the at least one critical structure; andcontrol an insertion of the needle and injection of at least one of the anesthetic solution or the analgesic solution into the target.

22. The system of claim 21, wherein the processor is further caused to confirm the injection of the at least one of the anesthetic solution or the analgesic solution into the target.

23. The system of claim 22, wherein confirming the injection of the at least one of the anesthetic solution or the analgesic solution includes acquiring ultrasound images during the injection and segmenting at least one of the injectant or the target in the ultrasound images.Client Ref. MIT25673JL24. The system of claim 22, wherein the processor is further caused to confirm and / or update the pathway for the needle to reach the target without impinging on the at least one critical structure based on a spread of at least one of the anesthetic solution or the analgesic solution in the target.

25. The system of claim 21, wherein the processor is further caused to track a location of the needle during the insertion and determine an error offset line of the needle.

26. The system of claim 25, wherein the processor segments the needle during the insertion.

27. The system of claim 25, wherein the processor is further caused to determine an adjustment to a trajectory of the needle.

28. The system of claim 27, wherein the needle is retracted and reinserted based on the adjustment.

29. The system of claim 21, wherein the target includes connective tissue.

30. The system of claim 29, wherein the connective tissue includes fascia.

31. The system of claim 21, wherein the nerve is a femoral nerve.

32. The system of claim 21, wherein the processer is further caused to determine, from the image data, a depth of the target within the subject;determine a need for an offset insertion position for the needle based on the depth relative to the ultrasound probe; anddetermine an offset insertion position distance based on a position of the target or the critical structure to avoid in an imaging frame of the image data.

33. The system of claim 32, wherein the offset insertion position is 8 mm or 17 mm.Client Ref. MIT25673JL34. A method of controlling a robotically controlled system to determine a safe pathway for guiding an interventional device in an interventional procedure of a subject, the method including causing a processor to carry out steps comprising:accessing image data acquired from the subject using an ultrasound probe, wherein the image data include at least one image of a target;locating anatomical landmarks in the image data,segmenting the anatomical landmarks including the target; determining, from the image data, an aimpoint location on the target; determining, from the image data, a pathway for the interventional device to reach the target without impinging on at least one critical structure, wherein the interventional device includes a needle; andcontrolling an insertion of the needle and injection of a fluid injectant into the target.

35. The method of claim 34, wherein the processor further carries out a step of confirming the injection of the fluid injectant in the target.

36. The method of claim 35, wherein confirming the injection of the fluid injectant includes acquiring ultrasound images during the injection and segmenting at least one of the injectant or the target in the ultrasound images.

37. The method of claim 35, wherein the processor further carries out a step of confirming and / or updating the pathway for the interventional device to reach at least one of the aimpoint location or the target without impinging on the at least one critical structure based on a spread of the fluid injectant in the target.

38. The method of claim 34, further tracking a location of the interventional device during the insertion and determine an error offset line of the interventional device.

39. The method of claim 38, further segmenting the interventional device during the insertion.Client Ref. MIT25673JL40. The method of claim 38, further determining an adjustment to a trajectory of the interventional device.

41. The method of claim 40, wherein the interventional device is retracted and reinserted based on the adjustment.

42. The method of claim 34, wherein the processor further carries out the steps of determining an insertion angle for the interventional device using at least one of the aimpoint location, a location of the target, or a location of the at least one critical structure.

43. The method of claim 34, wherein the target is one of an artery, a vein, a femoral artery, a femoral vein, a jugular vein, a peripheral vein, a subclavian vein, an airway, a lumen, a luminal organ, a body cavity, a fluid filled anatomic space, a location requiring biopsy, a breast, a kidney, a lymph node, a spinal canal, a perineural location, a peritoneal space ,a pleural space a muscle, connective tissue, a location between two or more thereof, a location between one thereof and a critical structure, or a predefined offset relative to one thereof.

44. The method of claim 43, wherein the connective tissue includes fascia.

45. The method of claim 44, wherein the image data includes a plurality of views of the target, and wherein the processor is configured to assess the plurality of views to identify at least one of the critical structure in the subject and identify an insertion point location on the subject where the interventional device, when injected, reaches at least the target or aimpoint location from the insertion point location without penetrating the critical structure in the subject.

46. The method of claim 45, wherein the critical structure includes at least one of a bone, an unintended blood vessel, a non-target organ, or a nerve.Client Ref. MIT25673JL47. The method of claim 34, wherein the processor further carries out the steps of determining, from the image data, a depth of the target within the subject;determining a need for an offset insertion position for the interventional device based on the depth relative to the ultrasound probe; anddetermining an offset insertion position distance relative to the target.

48. The method of claim 47, wherein the offset insertion position is 8 mm or 17 mm.

49. The method of claim 34, wherein the processor further carries out the step of adjusting a position of the needle after insertion to avoid the at least one critical structure.

50. A method of performing a nerve block, the method including causing a processor to carry out steps comprising:accessing image data acquired from a subject using an ultrasound probe, wherein the image data include at least one image of a target and at least one critical structure to avoid, wherein the at least one critical structure includes a nerve;locating anatomical landmarks in the image data;segmenting the anatomical landmarks including the at least one of the target or the nerve;determining, from the image data, a location of the target for injecting at least one of an anesthetic solution or an analgesic solution for providing a nerve block;determining, from the image data, a pathway for a needle to reach the target without impinging on the at least one critical structure; andcontrolling insertion of the needle and injection of at least one of the anesthetic solution or the analgesic solution into the target.

51. The method of claim 50, wherein the processor further carries out the step of confirming the injection of at least one of the anesthetic solution or the analgesic solution in the target.Client Ref. MIT25673JL52. The method of claim 51, wherein confirming the injection of the at least one of the anesthetic solution or the analgesic solution includes acquiring ultrasound images during the injection and segmenting at least one of the injectant or the target in the ultrasound images.

53. The method of claim 50, wherein the processor further carries out a step of confirming and / or updating the pathway for the needle to reach the target without impinging on the at least one critical structure based on a spread of at least one of the anesthetic solution or the analgesic solution in the target.

54. The method of claim 50, further tracking a location of the needle during the insertion and determining an error offset line of the needle.

55. The method of claim 54, further segmenting the needle during the insertion.

56. The method of claim 54, further determining an adjustment to a trajectory of the needle.

57. The method of claim 56, wherein the needle is retracted and reinserted based on the adjustment.

58. The method of claim 50, wherein the processer further carries out the steps of determining, from the image data, a depth of the target within the subject;determining a need for an offset insertion position for the needle based on the depth relative to the ultrasound probe; anddetermining an offset insertion position distance based on a position of the target or the critical structure to avoid in an imaging frame of the image data.

59. The method of claim 58, wherein the offset insertion position is 8mm or 17 mm.

Images