Ultrasound imaging based determination of elevational VS. azimuthal transducer motion using blind sweep protocol

Abstract

A system includes a processor configured for communication with an ultrasound probe. The processor is configured to control the ultrasound probe to obtain ultrasound image frames while the ultrasound probe is moved during a sweep of a blind sweep protocol on a patient. The processor is also configured to determine, using the ultrasound image frames, whether the movement of the probe during the sweep is perpendicular to an imaging plane of the ultrasound image frames, and provide, to a display in communication with the processor, an output representative of whether the movement of the probe during the sweep is perpendicular.

Description

Attorney Docket No.: 2024PF00291ULTRASOUND IMAGING BASED DETERMINATION OF ELEVATIONAL VS. AZIMUTHAL TRANSDUCER MOTION USING BLIND SWEEP PROTOCOLFIELD

[0001] The subject matter described herein relates to devices, systems, and methods for using ultrasound data to distinguish between elevational motion and azimuthal motion of the ultrasound probe during a blind abdominal imaging sweep to verify that the ultrasound probe is being held in the correct orientation during the sweep.BACKGROUND

[0002] Ultrasound imaging is often used for diagnostic purposes in an office or hospital setting, but may also be used in resource-constrained care settings (e.g., homes, accident sites, ambulances, mobile health facilities, etc.) by emergency personnel, home health nurses, midwives, etc., who may lack ultrasound expertise. To facilitate ultrasound image acquisition by untrained or minimally trained users, a “blind sweep” protocol is often employed, in which the user follows pre-determined probe paths (e.g., sweeping out a pattern on the patient’s abdomen) during imaging.

[0003] Ultrasound imaging is a vital component of high-quality obstetric care. For example, detection of high-risk pregnancies through ultrasound can allow for appropriate referral for delivery care in highly resourced centers with providers trained to handle any associated risks and complications. However, in rural and under-resourced communities, the scarcity of ultrasound imaging results in a considerable gap in the healthcare of pregnant mothers. Antepartum complications - including preterm labor, preterm premature rupture of the membranes, intrauterine growth restriction (IUGR), intrauterine fetal demise, gestational diabetes, and preeclampsia - develop in over 25% of pregnancies. While ultrasound can be very useful in identifying complications, access to ultrasound may be out of reach in low- resource settings. Access and skill to perform the ultrasound in low-resource setting is therefore deficient in many parts of the world.

[0004] Novice user compliance with protocol requirements (e.g., moving the transducer along its elevational axis when imaging a blind sweep grid) is essential to ensuring sufficient coverage of the uterus. If the user sweeps along the transducer’s azimuthal axis, adjacent image frames overlap while gaps in coverage between sweeps occur, resulting in missedAttorney Docket No.: 2024PF00291 anatomical detections and incorrect clinical recommendations. To mitigate this frequent error mode, there is a need to detect non-compliant motion.

[0005] The information included in this Background section of the specification, including any references cited herein and any description or discussion thereof, is included for technical reference purposes only and is not to be regarded as subject matter by which the scope of the disclosure is to be bound.SUMMARY

[0006] Disclosed is an ultrasound probe orientation verification system that, during or following a blind sweep protocol (sweeps at particular locations along body without trying to find specific anatomy), detects whether the ultrasound probe is being correctly moved along the transducer’s elevational axis, or incorrectly moved along the transducer’s azimuthal axis. A benefit of the ultrasound blind sweep anatomy detection system is to improve the quality of care by using information obtained during the blind sweep protocol to identify whether the probe orientation is compliant with the protocol. The user may be instructed to repeat or correct non-compliant sweeps.

[0007] A system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions. One general aspect includes a system with a processor configured for communication with an ultrasound probe, where the processor is configured to: control the ultrasound probe to obtain a plurality of ultrasound image frames while the ultrasound probe is moved during a sweep of a blind sweep protocol on a patient; determine, using the plurality of ultrasound image frames, whether the movement of the probe during the sweep is perpendicular to an imaging plane of the plurality of ultrasound image frames; and provide, to a display in communication with the processor, an output representative of the determination of whether the movement of the probe during the sweep is perpendicular. Other aspects include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0008] Implementations may include one or more of the following features. In some aspects, to determine whether the movement of the probe during the sweep is perpendicular,Attorney Docket No.: 2024PF00291 the processor is configured to determine a motion vector using a first frame and a second frame of the plurality of ultrasound image frames. In some aspects, to determine the motion vector, the processor is configured to: with the first frame and the second frame, generating an inter-frame cross-correlation map; identifying a pixel of the inter-frame cross-correlation map that has the largest cross-correlation value; and determining the motion vector by linking a center pixel of the inter-frame cross-correlation map to the pixel of the inter-frame crosscorrelation map that has the largest cross-correlation value. In some aspects, the first frame and the second frame are separated by an offset. In some aspects, to determine whether the movement of the probe during the sweep is perpendicular, the processor is configured to determine whether a norm of the motion vector is greater than a threshold value. In some aspects, the processor is configured to determine an angle of the ultrasound probe based on a value of the norm. In some aspects, to determine whether the movement of the probe during the sweep is perpendicular, the processor is configured to: provide at least one of the plurality of ultrasound image frames or the motion vector to a machine learning model trained to determine whether the movement of the probe during the sweep is perpendicular; and generate, as an output of the machine learning model, the determination of whether the movement of the probe during the sweep is perpendicular. In some aspects, the output includes user guidance to at least one of rotate or translate the ultrasound probe. The output includes user guidance to repeat the sweep. In some aspects, when the movement of the probe during the sweep is not perpendicular, the processor is configured to discard the plurality of ultrasound image frames such that the plurality of ultrasound image frames is not provided to a machine learning model trained to detect at least one of fetal anatomy or maternal anatomy. In some aspects, when the movement of the probe during the sweep is perpendicular, the processor is configured to: provide the plurality of ultrasound image frames to the machine learning model trained to detect at least one of fetal anatomy or maternal anatomy; and output, to the display, a visual representation associated with an output of the machine learning model. In some aspects, the processor is configured to classify the sweep as: adherent to the blind sweep protocol when the movement of the probe during the sweep is perpendicular; or not adherent to the blind sweep protocol when the movement of the probe during the sweep is not perpendicular; and where the output is representative of the classification of the sweep as adherent or not adherent. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.Attorney Docket No.: 2024PF00291

[0009] One general aspect includes a system with a processor configured for communication with an ultrasound probe, where the processor is configured to: control the ultrasound probe to obtain a plurality of ultrasound image frames while the ultrasound probe is moved on a patient; determine at least one motion vector using the plurality of ultrasound image frames; determine whether the plurality of ultrasound image frames depict a complete view of anatomy; and when the plurality of ultrasound image frames does not depict the complete view of the anatomy: determine, using the at least one motion vector, whether the ultrasound probe is moving in a direction to obtain the complete view; provide, to a display in communication with the processor, an output representative of the determination of whether the ultrasound probe is moving the direction to obtain the complete view. When the ultrasound probe is not moving in the direction to obtain the complete view, the output may include user guidance to at least one of rotate or translate the ultrasound probe into the direction to obtain the complete view. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0010] Implementations may include one or more of the following features. In some aspects, the processor is configured to determine the direction to move the ultrasound probe to obtain the complete view; where, to determine whether the user is moving the ultrasound probe in the direction to obtain the complete view, the processor is configured to determine if the at least one motion vector matches the direction to obtain the complete view. In some aspects, to determine the direction to move the ultrasound probe to determine the complete view, the processor is configured to: provide at least one of the plurality of ultrasound image frames or the at least one motion vector to a machine learning model trained to determine the direction to move the ultrasound probe to obtain the complete view; and generate, as an output of the machine learning model, the direction to move the ultrasound probe to obtain the complete view. In some aspects, the processor is configured to: output at least one selector field to the display; and receive a user selection representative of the complete view via the at least one selector field. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0011] One general aspect includes a system with a processor configured for communication with an ultrasound probe, where the processor is configured to: control the ultrasound probe to obtain a plurality of ultrasound image frames of an interventional device while the ultrasound probe is moved on a patient; determine at least one motion vector using the plurality of ultrasound image frames; determine whether the plurality of ultrasound imageAttorney Docket No.: 2024PF00291 frames depict the interventional device in a desired view plane; and when the plurality of ultrasound image frames do not depict the interventional device in the desired view plane: determine, using the at least one motion vector, whether the ultrasound probe is moving in a direction to obtain the desired view plane; provide, to a display in communication with the processor, an output representative of the determination of whether the ultrasound probe is moving the direction to obtain the desired view plane. When the ultrasound probe is not moving in the direction to obtain the desired view plane, the output may include user guidance to at least one of rotate or translate the ultrasound probe into the direction to obtain the desired view plane. Other examples of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0012] Implementations may include one or more of the following features. In some aspects, the processor is configured to determine the direction to obtain the desired view plane; where, to determine whether the user is moving the ultrasound probe in the direction to obtain the desired view plane, the processor is configured to determine if the at least one motion vector matches the direction to obtain the desired view plane. In some aspects, to determine the direction to obtain the desired view plane, the processor is configured to: provide at least one of the plurality of ultrasound image frames or the at least one motion vector to a machine learning model trained to determine the direction to obtain the desired view plane; and generate, as an output of the machine learning model, the direction to obtain the desired view plane. In some aspects, the processor is configured to: output at least one selector field to the display; and receive a user selection representative of the desired view plane via the at least one selector field. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0013] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. A more extensive presentation of features, details, utilities, and advantages of the ultrasound blind sweep uterine imaging extent verification system, as defined in the claims, is provided in the following written description of various aspects of the disclosure and illustrated in the accompanying drawings.Attorney Docket No.: 2024PF00291BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Illustrative aspects of the present disclosure will be described with reference to the accompanying drawings, of which:

[0015] Figure l is a schematic, diagrammatic representation of an ultrasound imaging system, according to aspects of the present disclosure.

[0016] Figure l is a schematic diagram of a processor circuit, according to aspects of the present disclosure.

[0017] Figure 3 is a schematic, diagrammatic representation of a patient on whose abdomen the ultrasound blind sweep protocol will be followed, according to aspects of the present disclosure.

[0018] Figure 4 is a perspective view of a transducer array, according to aspects of the present disclosure.

[0019] Figure 5 is a schematic, diagrammatic side view of an ultrasound imaging plane 500, according to aspects of the present disclosure.

[0020] Figure 6 is a schematic, diagrammatic top view of an ultrasound imaging plane, according to aspects of the present disclosure.

[0021] Figure 7 is a schematic, diagrammatic, top view of adherent probe motion for vertical sweeps, according to aspects of the present disclosure.

[0022] Figure 8 is a schematic, diagrammatic, top view of adherent probe motion for horizontal sweeps, according to aspects of the present disclosure.

[0023] Figure 9A is a schematic, diagrammatic side view of adherent motion of an ultrasound probe over the abdomen of a patient, according to aspects of the present disclosure.

[0024] Figure 9B is a schematic, diagrammatic side view of non-adherent motion of an ultrasound probe over the abdomen of a patient. According to aspects of the present disclosure.

[0025] Figure 10A is a schematic, diagrammatic top view of adherent motion of an ultrasound probe transducer array for a vertical sweep, according to aspects of the present disclosure.

[0026] Figure 10B is a schematic, diagrammatic top view of non-adherent motion of an ultrasound probe transducer array for a vertical sweep, according to aspects of the present disclosure.Attorney Docket No.: 2024PF00291

[0027] Figure 11A is a schematic, diagrammatic top view of adherent motion of an ultrasound probe transducer array for a vertical sweep, according to aspects of the present disclosure.

[0028] Figure 11B is a schematic, diagrammatic top view of adherent motion of an ultrasound probe transducer array for a vertical sweep, according to aspects of the present disclosure.

[0029] Figure 12A is a schematic, diagrammatic top view of adherent motion of an ultrasound probe transducer array for a horizontal sweep, according to aspects of the present disclosure.

[0030] Figure 12B is a schematic, diagrammatic top view of non-adherent motion of an ultrasound probe transducer array for a horizontal sweep, according to aspects of the present disclosure.

[0031] Figure 13 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound probe orientation verification system, according to aspects of the present disclosure.

[0032] Figure 14 is a schematic, diagrammatic representation, in block diagram form, of at least a portion of an example ultrasound probe orientation verification system, according to aspects of the present disclosure.

[0033] Figure 15 is a schematic, diagrammatic representation, in block diagram form, of at least a portion of an example ultrasound probe orientation verification system, according to aspects of the present disclosure.

[0034] Figure 16 is a schematic, diagrammatic representation, in flow diagram form, of an example ultrasound probe orientation verification method, according to aspects of the present disclosure.

[0035] Figure 17 is a schematic, diagrammatic representation of an anatomy detectionbased motion determination, according to aspects of the present disclosure.

[0036] Figure 18 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound probe orientation verification system, according to aspects of the present disclosure.

[0037] Figure 19 is an example inter-frame cross-correlation map, according to aspects of the present disclosure.

[0038] Figure 20 is a screen display showing graphical instructions and text instructions 2020 to the user about how to correct a sweep, according to aspects of the present disclosure.Attorney Docket No.: 2024PF00291

[0039] Figure 21 is a schematic, diagrammatic top view of a transducer array moving at a non-adherent angle, according to aspects of the present disclosure.

[0040] Figure 22 is an example binning or bucketing method for a probe angle 0, according to aspects of the present disclosure.

[0041] Figure 23 is a screen display showing graphical instructions and text instructions to the user about how to correct a sweep, according to aspects of the present disclosure.

[0042] Figure 24 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound probe orientation verification system that provides real-time guidance for ultrasound probe motion, according to aspects of the present disclosure.

[0043] Figure 25 is an example screen display showing partially imaged anatomy and user instructions, according to aspects of the present disclosure.

[0044] Figure 26 is an example screen display showing partially imaged anatomy and user instructions, according to aspects of the present disclosure.

[0045] Figure 27 is an example screen display showing completely imaged anatomy and user instructions, according to aspects of the present disclosure.

[0046] Figure 28 is an example screen display showing partially imaged anatomy and user instructions, according to aspects of the present disclosure.

[0047] Figure 29 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound probe orientation verification system that provides real-time guidance for ultrasound probe motion to image an interventional device, according to aspects of the present disclosure.

[0048] Figure 30 is an example screen display showing an interventional device in a live ultrasound image, according to aspects of the present disclosure.

[0049] Figure 31 is an example screen display showing an interventional device in a live ultrasound image, according to aspects of the present disclosure.

[0050] Figure 32 is a schematic, diagrammatic representation, in block diagram form, of an inference mode of an example machine learning-based motion detection process, according to aspects of the present disclosure.

[0051] Figure 33 is a schematic, diagrammatic view, in block diagram form, of a training system for a machine learning model, according to aspects of the present disclosure.Attorney Docket No.: 2024PF00291

[0052] Figure 34 is a schematic, diagrammatic representation, in block diagram form, of an inference mode of an example machine learning-based motion detection process, according to aspects of the present disclosure.

[0053] Figure 35 is a schematic, diagrammatic view, in block diagram form, of a training system for a machine learning model, according to aspects of the present disclosure.

[0054] Figure 36 is a schematic diagram of a deep learning network configuration, according to aspects of the present disclosure.Attorney Docket No.: 2024PF00291DETAILED DESCRIPTION

[0055] Deep learning-assisted blind sweep imaging enables obstetric screening to be performed by novice users. Novice user compliance with protocol requirements (e.g., moving the transducer along its elevational axis when imaging a blind sweep grid) is essential to ensuring sufficient coverage of the uterus for proper diagnostics. If the user sweeps along the transducer’s azimuthal axis, adjacent image frames overlap, while gaps in coverage between sweeps occur, resulting in missed anatomical detections and incorrect clinical recommendations. To mitigate this frequent error mode, there is a need to detect non- compliant motion due to incorrect probe orientation. The proposed method enables the detection and classification of azimuthal or elevational image motion using image crosscorrelation. Non-compliant motion may be flagged for quality control purposes, or user motion in response to active guidance may be verified and used as input for continuous active guidance updates.

[0056] The disclosure is particularly advantageous for minimally trained users in the obstetrics domain. To simplify the imaging workflow for this cohort of users, the users are instructed to perform imaging sweeps along a pre-determined grid on the mother’s abdomen (blind sweeps). This protocol is in contrast to the free hand “guided” sweeps that would be performed by a trained sonographer, during which the sonographer will move the transducer as needed to localize relevant anatomical structures of interest. To ensure data quality, checks must be put in place to identify novice user non-compliance with the imaging protocol. With respect to probe orientation, during the blind sweeps, the protocol requires the long axis of the transducer array to be oriented perpendicular to the direction of motion of the probe. This ensures maximal coverage of the anatomy lateral to the sweep direction, since it minimizes the possibility of un-scanned space in between neighboring sweeps. On the other hand, if the array’s long axis is oriented in the direction of probe motion, it can result in many gaps in the overall scan.

[0057] During anticipated use, the user may not follow the sweep protocol in the manner that the product was intended. This type of probe motion is non-adherent to the protocol, which has implications for image quality and other downstream processes. For example, the blind sweep protocol requires the user to sweep along the elevational axis of the probe (out- of-plane image direction). If the user instead sweeps along the azimuthal axis (in-plane image direction), adjacent image frames can overlap significantly, minimizing potential imaging coverage. Reduced imaging coverage increases the probability of missed anatomicalAttorney Docket No.: 2024PF00291 detections and, as a result, incorrect clinical decision making. The methods described herein are intended to identify this mode of non-adherence, to provide quality control feedback to novice users. This method may also have applications for active user guidance, by detecting the magnitude and direction of motion. Furthermore, while this method can be used with mobile ultrasound imaging platforms (e.g., Lumify from Philips), it may be applicable to other ultrasound platforms and general imaging modalities, including but not limited to optical (visible, infrared, ultraviolet), radiographic (fluoroscopy, biplanar videoradiography, etc.), optical coherence tomography, etc.

[0058] Important elements of the invention include 1) a processing controller that receives pre-scan-converted images as input, and convolves a cross-correlation algorithm across image frames. The results of this computation produce a classification of whether the motion measured between the input frames is adherent or non-adherent. The results are presented to the user via 2) a user interface (UI). In alternate embodiments, active guidance may be provided via the UI and / or other methods (sound, haptic) to direct the user to move the probe in a particular direction and with a particular orientation.

[0059] Aspect 1 : An overall approach to estimating “in-plane” vs “out-of-plane” motion, subsequent to the sweep acquisition: the processing controller receives the pre-scan- converted images, and convolves a cross-correlation function across the image stack. Image frames are cross-correlated (e.g., with an offset of 3), to avoid measurement jitter due to angular image compounding. In one implementation, 3 consecutive angularly offset frame transmits are compounded together to improve the signal-to-noise ratio. This number can change depending on the scanner and hence, the offset used in the disclosed method can also change accordingly. In alternative aspects, this offset may be modified or eliminated.

[0060] A motion vector is defined for each inter-frame cross-correlation map from the center of the image to the coordinates of the maximum cross-correlation value (e.g., the lag). If the norm of the motion vector is > 1, the inter-frame image motion is classified as in-plane (since in the case of in-plane motion, a finite motion vector will usually be detected in between the frames. On the other hand, in case of out-of-plane motion, this motion vector will be negligible since the image content changes significantly between the frames). When image motion is in-plane (e.g., within and parallel to the imaging plane), the transducer is moving along its azimuthal axis, and therefore it is not adhering to the blind sweep protocol. If >70% of inter-frame classifications are in-plane, then the entire blind sweep cineloop may be classified as non-adherent. An error notification may then be relayed to the user via the UI, directing them to repeat the sweep and providing instructions on how to correct the error.Attorney Docket No.: 2024PF00291The thresholds specified herein are intended as examples; they may be modified to achieve varying levels of sensitivity, including for the classification of varying levels of obliqueness (intermediate angles between pure azimuthal and elevational movement planes).

[0061] Aspect 2: a method to suggest a change in the direction of probe motion, to capture specific anatomic views. In another scenario, it may be desirable to provide active directional guidance to a user. For example, if an Al model such as those used by several ultrasound applications for maternal assessment detects an incomplete anatomy (e.g., partial heart view instead of 4-chamber heart view), a complete anatomical view may be obtained by translating (or rotating) the probe in a specified direction. In this aspect, the processing controller determines the required direction of probe translation based on the location of the anatomical detection within prior image frames, and relays this information to the user via the UI, sound, and / or haptic feedback. The direction of the probe motion vector computed in aspect 1 is then used to determine if the user is moving the probe in the correct direction to obtain a full anatomical view of the structure of interest. Other sub-aspects here include UI elements where the user can specify their objective (e.g., which anatomy they would like to visualize and in which view (e.g., A4C view of fetal heart)) and real-time updates on the UI for the output of this aspect (e.g., directional graphics and / or text indicating to the user how they should change their maneuvering of the probe to achieve the intended objective).

[0062] Aspect 3: a method for adequate interventional device guidance. In another scenario, the motion detector may be used for active directional guidance for alternative applications (and potentially even other imaging modalities), such as guidewire placement in interventional guidance systems. For example, depending on the specifics of the interventional application, the user might want to visualize the interventional device (such as guidewire, catheter or needle) in a certain plane (e.g., transverse cross-section or longitudinal view) (this can be an input option on the UI, e.g., a dropdown list or other such list selector). Based on the changing appearance of the device in the live ultrasound images and the estimate of probe orientation (using the method described in embodiment 1), a specific suggestion can be made to the user on how to change the direction of probe manipulation to achieve the intended objective.

[0063] Aspect 4: an Al-enabled processing controller. In another scenario, the functionalities described in previous aspects may also be performed with relevant Al models. For example, the series of cross-correlation-derived motion vectors may be used as input to a long short-term memory (LSTM) model that classifies the axis of motion or the direction to move the probe in the active guidance scenarios. In another implementation, an Al modelAttorney Docket No.: 2024PF00291(e.g., LSTM, temporal convolutional neural network (CNN), or attention-based model) may take the ultrasound images directly as input, and provide the same outputs described above.

[0064] The present disclosure aids substantially in the capture of high-quality ultrasoundbased diagnoses by minimally trained users, by automatically detecting whether the probe is being held and moved correctly to achieve complete coverage of the anatomy of interest. Implemented on a processor in communication with an ultrasound probe, the ultrasound probe orientation verification system disclosed herein provides practical improvements in the diagnosis of medical conditions for patients in underserved areas. This improved imaging methodology transforms a process that is heavily reliant on professional experience into one that is accurate and repeatable even for minimally trained personnel, without the normally routine need to train clinicians such as emergency department personnel to recognize whether a blind sweep protocol has been performed correctly. This unconventional approach improves the functioning of the ultrasound imaging system, by providing reliable, repeatable imaging and diagnosis in hospital, office, vehicle, field, and home settings, as well as referral recommendations for patients suspected to have, for example, a pregnancy complication.

[0065] The ultrasound probe orientation verification system may be implemented as a process at least partially viewable on a display, and operated by a control process executing on a processor that may accept user inputs from a keyboard, mouse, touchscreen interface, and that is in communication with one or more sensor probes. In that regard, the control process performs certain specific operations in response to different inputs or selections made at different times. Certain structures, functions, and operations of the processor, display, sensors, and user input systems provide novel features or aspects of the present disclosure.

[0066] These descriptions are provided for exemplary purposes only, and should not be considered to limit the scope of the ultrasound probe orientation verification system. Certain features may be added, removed, or modified without departing from the spirit of the claimed subject matter.

[0067] For the purposes of promoting an understanding of the principles of the present disclosure, reference will now be made to the aspects illustrated in the drawings, and specific language will be used to describe the same. It is nevertheless understood that no limitation to the scope of the disclosure is intended. Any alterations and further modifications to the described devices, systems, and methods, and any further application of the principles of the present disclosure are fully contemplated and included within the present disclosure as would normally occur to one skilled in the art to which the disclosure relates. In particular, it is fully contemplated that the features, components, and / or steps described with respect to oneAttorney Docket No.: 2024PF00291 aspect may be combined with the features, components, and / or steps described with respect to other aspects of the present disclosure. For the sake of brevity, however, the numerous iterations of these combinations will not be described separately.

[0068] Figure l is a schematic, diagrammatic representation of an ultrasound imaging system 100, according to aspects of the present disclosure. The ultrasound imaging system 100 may for example be used to acquire ultrasound video sweeps, which can then be analyzed by a human clinician or an artificial intelligence to diagnose medical conditions.

[0069] The ultrasound imaging system 100 is used for scanning an area or volume of a subject’s body. A subject may include a patient of an ultrasound imaging procedure, or any other person, or any suitable living or non-living organism or structure. The ultrasound imaging system 100 includes an ultrasound imaging probe 110 in communication with a host 130 over a communication interface or link 120. The probe 110 may include a transducer array 112, a beamformer 114, a processor circuit 116, and a communication interface 118. The host 130 may include a display 132, a processor circuit 134, a communication interface 136, and a memory 138 storing subject information.

[0070] In some aspects, the probe 110 is an external ultrasound imaging device including a housing 111 configured for handheld operation by a user. The transducer array 112 can be configured to obtain ultrasound data while the user grasps the housing 111 of the probe 110 such that the transducer array 112 is positioned adjacent to or in contact with a subject’s skin. The probe 110 is configured to obtain ultrasound data of anatomy within the subject’s body while the probe 110 is positioned outside of the subject’s body. In some aspects, the probe 110 can be an external ultrasound probe, a transthoracic probe, and / or a curved array probe.

[0071] In other aspects, the probe 110 can be an internal ultrasound imaging device and may comprise a housing 111 configured to be positioned within a subject’s body. In some aspects, the probe 110 may be a curved array probe. Probe 110 may be of any suitable form for any suitable ultrasound imaging application including both external and internal ultrasound imaging.

[0072] For an ultrasound imaging device, the transducer array 112 emits ultrasound signals towards an anatomical object 105 of a subject and receives echo signals reflected from the object 105 back to the transducer array 112. The ultrasound transducer array 112 can include any suitable number of acoustic elements, including one or more acoustic elements and / or a plurality of acoustic elements. In some instances, the transducer array 112 includes a single acoustic element. In some instances, the transducer array 112 may include an array of acoustic elements with any number of acoustic elements in any suitableAttorney Docket No.: 2024PF00291 configuration. For example, the transducer array 112 can include between 1 acoustic element and 10000 acoustic elements, including values such as 2 acoustic elements, 4 acoustic elements, 36 acoustic elements, 64 acoustic elements, 128 acoustic elements, 500 acoustic elements, 812 acoustic elements, 1000 acoustic elements, 3000 acoustic elements, 8000 acoustic elements, and / or other values both larger and smaller. In some instances, the transducer array 112 may include an array of acoustic elements with any number of acoustic elements in any suitable configuration, such as a linear array, a planar array, a curved array, a curvilinear array, a circumferential array, an annular array, a phased array, a matrix array, a one-dimensional (ID) array, a 1.x dimensional array (e.g., a 1.5D array), or a two- dimensional (2D) array. The array of acoustic elements (e.g., one or more rows, one or more columns, and / or one or more orientations) can be uniformly or independently controlled and activated. The transducer array 112 can be configured to obtain one-dimensional, two- dimensional, and / or three-dimensional images of a subject’s anatomy. In some aspects, the transducer array 112 may include a piezoelectric micromachined ultrasound transducer (PMUT), capacitive micromachined ultrasonic transducer (CMUT), single crystal, lead zirconate titanate (PZT), PZT composite, other suitable transducer types, and / or combinations thereof.

[0073] The object 105 may include any anatomy or anatomical feature, such an abdomen of a pregnant patient, one or multiple fetuses inside the abdomen of the pregnant patient, etc.

[0074] The beamformer 114 is coupled to the transducer array 112. The beamformer 114 controls the transducer array 112, for example, for transmission of the ultrasound signals and reception of the ultrasound echo signals. In some aspects, the beamformer 114 may apply a time-delay to signals sent to individual acoustic transducers within an array 112 such that an acoustic signal is steered in any suitable direction propagating away from the probe 110. The beamformer 114 may further provide image signals to the processor circuit 116 based on the response of the received ultrasound echo signals. The beamformer 114 may include multiple stages of beamforming. The beamforming can reduce the number of signal lines for coupling to the processor circuit 116. In some aspects, the transducer array 112 in combination with the beamformer 114 may be referred to as an ultrasound imaging component.

[0075] The processor 116 is coupled to the beamformer 114. The processor 116 may also be described as a processor circuit, which can include other components in communication with the processor 116, such as a memory, beamformer 114, communication interface 118, and / or other suitable components. The processor 116 may include a central processing unit (CPU), a graphical processing unit (GPU), a digital signal processor (DSP), an applicationAttorney Docket No.: 2024PF00291 specific integrated circuit (ASIC), a controller, a field programmable gate array (FPGA) device, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 116 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The processor 116 is configured to process the beamformed image signals. For example, the processor 116 may perform filtering and / or quadrature demodulation to condition the image signals. The processor 116 and / or 134 can be configured to control the array 112 to obtain ultrasound data associated with the object 105.

[0076] The communication interface 118 is coupled to the processor 116. The communication interface 118 may include one or more transmitters, one or more receivers, one or more transceivers, and / or circuitry for transmitting and / or receiving communication signals. The communication interface 118 can include hardware components and / or software components implementing a particular communication protocol suitable for transporting signals over the communication link 120 to the host 130. The communication interface 118 can be referred to as a communication device or a communication interface module.

[0077] The communication link 120 may be any suitable communication link. For example, the communication link 120 may be a wired link, such as a universal serial bus (USB) link or an Ethernet link. Alternatively, the communication link 120 may be a wireless link, such as an ultra-wideband (UWB) link, an Institute of Electrical and Electronics Engineers (IEEE) 802.11 WiFi link, or a Bluetooth link.

[0078] At the host 130, the communication interface 136 may receive the image signals. The communication interface 136 may be substantially similar to the communication interface 118. The host 130 may be any suitable computing and display device, such as a workstation, a personal computer (PC), a laptop, a tablet, or a mobile phone.

[0079] The processor 134 is coupled to the communication interface 136. The processor 134 may also be described as a processor circuit, which can include other components in communication with the processor 134, such as the memory 138, the communication interface 136, an optional speaker 139, haptic feedback device 135, and / or other suitable components. Although the haptic feedback device 135 is shown as being inside the host 130, in other aspects it may, instead or in addition, be located inside the probe 110, such that haptic feedback is felt by the user’s hand gripping the probe 110.

[0080] The processor 134 may be implemented as a combination of software components and hardware components. The processor 134 may include a central processing unit (CPU), aAttorney Docket No.: 2024PF00291 graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a controller, an FPGA device, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 134 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The processor 134 can be configured to generate image data from the image signals received from the probe 110. The processor 134 can apply advanced signal processing and / or image processing techniques to the image signals. In some aspects, the processor 134 can form a three-dimensional (3D) volume image from the image data. In some aspects, the processor 134 can perform real-time processing on the image data to provide a streaming video of ultrasound images of the object 105. In some aspects, the host 130 includes a beamformer. For example, the processor 134 can be part of and / or otherwise in communication with such a beamformer. The beamformer in the in the host 130 can be a system beamformer or a main beamformer (providing one or more subsequent stages of beamforming), while the beamformer 114 is a probe beamformer or micro-beamformer (providing one or more initial stages of beamforming).

[0081] The memory 138 is coupled to the processor 134. The memory 138 may be any suitable storage device, such as a cache memory (e.g., a cache memory of the processor 134), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory, solid state memory device, hard disk drives, solid state drives, other forms of volatile and non-volatile memory, or a combination of different types of memory.

[0082] The memory 138 can be configured to store subject information, measurements, data, or files relating to a subject’s medical history, history of procedures performed, anatomical or biological features, characteristics, or medical conditions associated with a subject, computer readable instructions, such as code, software, or other application, as well as any other suitable information or data. The memory 138 may be located within the host 130. Subject information may include measurements, data, files, other forms of medical history, such as but not limited to ultrasound images, ultrasound videos, and / or any imaging information relating to the subject’s anatomy. The subject information may include parameters related to an imaging procedure such as an anatomical scan window, a probe orientation, and / or the subject position during an imaging procedure. The memory 138 canAttorney Docket No.: 2024PF00291 also be configured to store information related to the training and implementation of machine learning algorithms (e.g., neural networks) and / or information related to implementing image recognition algorithms for detecting / segmenting anatomy, image quantification algorithms, and / or image acquisition guidance algorithms, including those described herein.

[0083] The display 132 is coupled to the processor circuit 134. The display 132 may be a monitor or any suitable display. The display 132 is configured to display the ultrasound images, image videos, and / or any imaging information of the object 105.

[0084] The ultrasound imaging system 100 may be used to assist a sonographer in performing an ultrasound scan. The scan may be performed in a point-of-care setting. In some instances, the host 130 is a console or movable cart. In some instances, the host 130 may be a mobile device, such as a tablet, a mobile phone, or portable computer. During an imaging procedure, the ultrasound system can acquire an ultrasound image of a particular region of interest within a subject’s anatomy. The ultrasound imaging system 100 may then analyze the ultrasound image to identify various parameters associated with the acquisition of the image such as the scan window, the probe orientation, the subject position, and / or other parameters. The ultrasound imaging system 100 may then store the image and these associated parameters in the memory 138. At a subsequent imaging procedure, the ultrasound imaging system 100 may retrieve the previously acquired ultrasound image and associated parameters for display to a user which may be used to guide the user of the ultrasound imaging system 100 to use the same or similar parameters in the subsequent imaging procedure, as will be described in more detail hereafter.

[0085] In some aspects, the processor 134 may utilize deep learning-based prediction networks to identify parameters of an ultrasound image, including an anatomical scan window, probe orientation, subject position, identify and location of anatomical features, and / or other parameters. In some aspects, the processor 134 may receive metrics or perform various calculations relating to the region of interest imaged or the subject’s physiological state during an imaging procedure. These metrics and / or calculations may also be displayed to the sonographer or other user via the display 132.

[0086] In some aspects, the host 130 may also include a speaker 180. The speaker 180 may for example be used to provide advisory tones, beeps, or other auditory feedback to the user.

[0087] Before continuing, it should be noted that the examples described above are provided for purposes of illustration, and are not intended to be limiting. Other devices and / or device configurations may be utilized to carry out the operations described herein.Attorney Docket No.: 2024PF00291

[0088] Figure l is a schematic diagram of a processor circuit 250, according to aspects of the present disclosure. The processor circuit 250 may be implemented in the ultrasound imaging system 100, or other devices or workstations (e.g., third-party workstations, network routers, etc.), or on a cloud processor or other remote processing unit, as necessary to implement the method. As shown, the processor circuit 250 may include a processor 260, a memory 264, and a communication module 268. These elements may be in direct or indirect communication with each other, for example via one or more buses.

[0089] The processor 260 may include a central processing unit (CPU), a digital signal processor (DSP), a controller, or any combination of general-purpose computing devices, reduced instruction set computing (RISC) devices, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other related logic devices, including mechanical and quantum computers. The processor 260 may also comprise another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 260 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0090] The memory 264 may include a cache memory (e.g., a cache memory of the processor 260), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory, solid state memory device, hard disk drives, other forms of volatile and nonvolatile memory, or a combination of different types of memory. In an aspect, the memory 264 includes a non-transitory computer-readable medium. The memory 264 may store instructions 266. The instructions 266 may include instructions that, when executed by the processor 260, cause the processor 260 to perform the operations described herein. Instructions 266 may also be referred to as code. The terms “instructions” and “code” should be interpreted broadly to include any type of computer-readable statement(s). For example, the terms “instructions” and “code” may refer to one or more programs, routines, subroutines, functions, procedures, etc. “Instructions” and “code” may include a single computer-readable statement or many computer-readable statements.

[0091] The communication module 268 can include any electronic circuitry and / or logic circuitry to facilitate direct or indirect communication of data between the processor circuit 250, and other processors or devices. In that regard, the communication module 268 can beAttorney Docket No.: 2024PF00291 an input / output (I / O) device. In some instances, the communication module 268 facilitates direct or indirect communication between various elements of the processor circuit 250 and / or the ultrasound imaging system 100. The communication module 268 may communicate within the processor circuit 250 through numerous methods or protocols. Serial communication protocols may include but are not limited to United States Serial Protocol Interface (US SPI), Inter-Integrated Circuit (I2C), Recommended Standard 232 (RS- 232), RS-485, Controller Area Network (CAN), Ethernet, Aeronautical Radio, Incorporated 429 (ARINC 429), MODBUS, Military Standard 1553 (MIL-STD-1553), or any other suitable method or protocol. Parallel protocols include but are not limited to Industry Standard Architecture (ISA), Advanced Technology Attachment (ATA), Small Computer System Interface (SCSI), Peripheral Component Interconnect (PCI), Institute of Electrical and Electronics Engineers 488 (IEEE-488), IEEE-1284, and other suitable protocols. Where appropriate, serial and parallel communications may be bridged by a Universal Asynchronous Receiver Transmitter (UART), Universal Synchronous Receiver Transmitter (USART), or other appropriate subsystem.

[0092] External communication (including but not limited to software updates, firmware updates, model sharing between the processor and central server, or readings from the ultrasound imaging system 100) may be accomplished using any suitable wireless or wired communication technology, such as a cable interface such as a universal serial bus (USB), micro USB, Lightning, or FireWire interface, Bluetooth, Wi-Fi, ZigBee, Li-Fi, or cellular data connections such as 2G / GSM (global system for mobiles) , 3G / UMTS (universal mobile telecommunications system), 4G, long term evolution (LTE), WiMax, or 5G. For example, a Bluetooth Low Energy (BLE) radio can be used to establish connectivity with a cloud service, for transmission of data, and for receipt of software patches. The controller may be configured to communicate with a remote server, or a local device such as a laptop, tablet, or handheld device, or may include a display capable of showing status variables and other information. Information may also be transferred on physical media such as a USB flash drive or memory stick.

[0093] Figure 3 is a schematic, diagrammatic representation of a patient 300 on whose abdomen the ultrasound blind sweep protocol will be followed, according to aspects of the present disclosure. Visible on the abdomen 310 of the patient 300 is a desired sweep pattern 320 intended to capture images of desired features of the patient’s anatomy. The sweep pattern 320 includes multiple vertical sweep lines 330 and multiple horizontal sweep lines 340. Each sweep line 330, 340 represents a desired path for one imaging sweep of theAttorney Docket No.: 2024PF00291 abdomen 310. In the example shown in Figure 3, the sweep pattern includes three vertical sweep lines 330 labeled L (patient’s left), M (patient’s middle), and R (patient’s right), all in an upward direction with respect to the patient, and three horizontal sweep lines 340 labeled Cl (bottom), C2 (middle), and C3 (top), all in a right-to-left direction with respect to the patient. However, it is understood that a sweep pattern 320 may include more or fewer sweep lines 330, including vertical sweep lines 330, horizontal sweep lines 340, or combinations thereof, in any combination of upward, downward, left, or right directions based on the patient’s fundal height. For example, if the patient’s belly is bigger in size, more sweeps may be needed. Furthermore, a sweep pattern 320 may cover other portions of the patient’s body, including but not limited to the head, neck, spine, limbs, etc. Types of blind sweep protocol include but are not limited to obstetric sweep imaging (OSI), volume sweep imaging (VSI), 6-Stage, Fetal Age Machine Learning Initiative (FAMLI), and otherwise.

[0094] These sweep patterns represent desired probe motion information, including desired positions, a desired velocity or velocities, and / or a desired orientation of the ultrasound probe while the ultrasound probe is obtaining a plurality of ultrasound image frames during the sweep. It is noted that the desired sweep patterns or blind sweep protocols stored in a memory of the processor may include only vertical sweeps, only horizontal sweeps, may include a grid (e.g., 3x3, 5x5, etc.) of vertical and horizontal sweeps, and may also include associated parameters such as desired probe motion (e.g., positions, velocities, and / or orientations) stored in the memory(e.g., blind sweep protocol 320). Depending on the implementation, sweeps may include curved, diagonal, and other types of sweeps. The protocols and their associated parameters can for example be based on standards established by authorities in the field (physician organizations, sonographer organizations, etc.), published in scholarly journals / textbooks, etc.

[0095] Figure 4 is a perspective view of a transducer array 400, according to aspects of the present disclosure. The transducer array 400 includes a plurality of transducer elements 412. The array 400 includes a short axis, elevational axis, out-of-plane axis (e.g., perpendicular to the imaging plane), or Y-axis 430. The transducer array 400 also includes a long axis, azimuthal axis, in-plane axis (e.g., parallel to the imaging plane), or X-axis 420. The transducer array also has an axial direction (e.g., into the viewing plane), depth axis, or Z-axis 440. It is noted that the number of transducer elements 412 is relatively greater along the long / azimuthal / X-axis than along the short / elevational / Y-axis. During a blind sweep protocol, it may be desirable to move the transducer array 400 in the elevational directionAttorney Docket No.: 2024PF00291430, and undesirable to move the transducer array 400 in the azimuthal direction 420, for reasons that are described in detail below.

[0096] Figure 5 is a schematic, diagrammatic side view of an ultrasound imaging plane 500, according to aspects of the present disclosure. In the example shown in Figure 5, the imaging plane 500 is a fan shape of points within a given depth from the ultrasound probe 520. Visible is the direction 510 that the probe 520 is pointed, and thus the average direction of travel for the emitted ultrasound waves. Also visible are the azimuthal / long axis 420 and the axial / depth axis 440.

[0097] Figure 6 is a schematic, diagrammatic top view of an ultrasound imaging plane 500, according to aspects of the present disclosure. Visible are the azimuthal / long axis 420 and the elevational / short axis 430. In the example shown in Figure 6, the imaging plane is substantially thinner along the elevational / short axis 430 than along the azimuthal / long axis 420.

[0098] Figure 7 is a schematic, diagrammatic, top view of adherent probe motion 710 for vertical sweeps (e.g., right (R), middle (M), and left (L) sweeps), according to aspects of the present disclosure. Visible are the long / azimuthal axis 420 and the short / elevational axis 430 of the transducer array 400. Adherent motion 710 (e.g., motion that adheres to the blind sweep protocol) is in the direction of the short / elevational axis, which is aligned with the cranial -caudal direction (e.g., the patient’s head-foot direction) , and may therefore be referred to as “out of plane motion”, or motion that is substantially perpendicular to the imaging plane, and thus perpendicular to the long / azimuthal axis 420 of the imaging array 400.

[0099] Figure 8 is a schematic, diagrammatic, top view of adherent probe motion 810 for horizontal sweeps (e.g., Cl, C2, and C3 sweeps), according to aspects of the present disclosure. Visible are the long / azimuthal axis 420 and the short / elevational axis 430 of the transducer array 400. Adherent motion 810 (e.g., motion that adheres to the blind sweep protocol) is once again in the direction of the short / elevational axis, which is aligned with the lateral (e.g., left / right) axis of the patient, and may therefore be referred to as “out of plane motion”, or motion that is substantially perpendicular to the imaging plane, and thus perpendicular to the long / azimuthal axis 420 of the imaging array 400.

[0100] Figure 9A is a schematic, diagrammatic side view of adherent motion 710 of an ultrasound probe 520 over the abdomen 920 of a patient 300, according to aspects of the present disclosure. For a vertical sweep (e.g., an R, M, or L sweep), the adherent motion 710Attorney Docket No.: 2024PF00291 is toward the head 910 of the patient 300 along the head-foot axis, with the probe 520 aligned such that its short / elevational axis is aligned with the direction of motion.

[0101] Figure 9B is a schematic, diagrammatic side view of non-adherent motion 900 of an ultrasound probe 520 over the abdomen 920 of a patient 300. According to aspects of the present disclosure. For a vertical sweep (e.g., an R, M, or L sweep), the non-adherent motion 710 may for example be toward the head 910 of the patient 300, but with the probe 520 incorrectly aligned such that its long / azimuthal axis is aligned with the direction of motion. In other words, the probe 520 is oriented 90 degrees from the orientation seen in Figure 9A.

[0102] Figure 10A is a schematic, diagrammatic top view of adherent motion 710 of an ultrasound probe transducer array 400 for a vertical sweep, according to aspects of the present disclosure. Orientation of the transducer array 400 in Figure 10A is the same as in Figures 7, 9A, and 11 A, such that the short / elevational axis 430 is aligned with the direction of motion and the long / azimuthal axis 420 is perpendicular to the direction of motion. Thus, the area 1000 that is imaged by the sweep has the same width as the transducer array 400 along the long / azimuthal axis 420, and a length equal to the length of the sweep.

[0103] Figure 10B is a schematic, diagrammatic top view of non-adherent motion 900 of an ultrasound probe transducer array 400 for a vertical sweep, according to aspects of the present disclosure. Orientation of the transducer array 400 in Figure 10B is the same as in Figures 9B, and 1 IB, such that the short / elevational axis 430 is perpendicular to the direction of motion and the long / azimuthal axis 420 is aligned with the direction of motion. Thus, the area 1000 that is imaged by the sweep has the same width as the transducer array 400 along the short / elevational axis 430, and a length equal to the length of the sweep. As compared with Figure 10A, this results in significant areas 1010 that are not imaged, even though the blind sweep protocol calls for them to be part of the imaged area 1000.

[0104] Figure 11A is a schematic, diagrammatic top view of adherent motion of an ultrasound probe transducer array for a vertical sweep, according to aspects of the present disclosure. Orientation of the transducer array in Figure 11 A is the same as in Figures 7, 9A, and 10A, such that the short / elevational axis 430 is aligned with the direction of motion and the long / azimuthal axis 420 is perpendicular to the direction of motion. Thus, a series of image planes 500 are captured of the imaged area 1000, with each plane 500 stacking cleanly against the plane before it, in the direction of motion. One aspect of out-of-plane motion is the azimuthal / long axis direction does not overlap in adjacent image frames.

[0105] Figure 11B is a schematic, diagrammatic top view of adherent motion of an ultrasound probe transducer array for a vertical sweep, according to aspects of the presentAttorney Docket No.: 2024PF00291 disclosure. Orientation of the transducer array in Figure 1 IB is the same as in Figures 9B, and 10B, such that the short / elevational axis 430 is perpendicular to the direction of motion and the long / azimuthal axis 420 is aligned with the direction of motion. Thus, a series of image planes 500 are captured of the imaged area 1000, with each plane 500 largely overlapping with the plane before it along the (azimuthal) direction of motion. This results in large areas 1010 that are not imaged, even though the blind sweep protocol calls for these areas 1010 to be part of the imaged area 1000. Anatomy located in the unimaged areas 1010 will not be part of the captured images, and thus cannot contribute to measurements, diagnoses, clinical assessments, etc.

[0106] In some aspects, movement that is substantially perpendicular to the imaging plane (e.g., more than 45 degrees from the imaging plane) may be deemed adherent, whereas movement that is substantially parallel to the imaging plane (e.g., less than 45 degrees from the imaging plane) may be deemed non-adherent. In other aspects, movement that is more than 60 degrees from the imaging plane may be deemed adherent, and all other movement may be deemed non-adherent.

[0107] Figure 12A is a schematic, diagrammatic top view of adherent motion 810 of an ultrasound probe transducer array 400 for a horizontal sweep (e.g., sweep Cl, C2, or C3), according to aspects of the present disclosure. Orientation of the transducer array 400 in Figure 12A is the same as in Figure 8, such that the short / elevational axis 430 is aligned with the direction of motion (e.g., the patient’s left-right axis), and the long / azimuthal axis 420 is perpendicular to the direction of motion. Thus, the area 1200 that is imaged by the sweep has the same width as the transducer array 400 along the long / azimuthal axis 420, and a length equal to the length of the sweep.

[0108] Figure 12B is a schematic, diagrammatic top view of non-adherent motion 1220 of an ultrasound probe transducer array 400 for a horizontal sweep (e.g., sweep Cl, C2, or C3), according to aspects of the present disclosure. Orientation of the transducer array 400 in Figure 12B is such that the short / elevational axis 430 is perpendicular to the direction of motion (e.g., the patient’s left-right axis), and the long / azimuthal axis 420 is aligned with the direction of motion. Thus, the area 1200 that is imaged by the sweep is smaller than in Figure 12A, with substantial areas 1210 that are not imaged, even though the blind sweep protocol calls for them to be part of the imaged area 1200.

[0109] Figure 13 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound probe orientation verification system 1300, according to aspects of the present disclosure. An ultrasound probe 110 operated by aAttorney Docket No.: 2024PF00291 novice user 1310 performs a blind sweep protocol 320 on the body of a patient 300 and sends ultrasound imaging data to a host 130 such as a tablet, smartphone, ultrasound cart, etc. In step 1315, the host 130 generates ultrasound images using the ultrasound image data obtained by the ultrasound probe 110. The host 130 can control the ultrasound probe 110 to obtain the ultrasound image data (e.g., the host 130 establishes communication with the ultrasound probe 110, the host 130 sends control signals to start and / or stop acquisition of ultrasound image data, the host 130 sends power signals to power the ultrasound probe 110, etc.).

[0110] In step 1320, the host 130 uses the ultrasound images generated in step 1315 and / or the ultrasound image data used to generate the ultrasound images to determine if the probe orientation is / was correct during the sweep, as described in more detail below. In step 1330, the host generates a visual representation on a display, showing an indication of whether the motion / orientation of the probe is adherent or non-adherent to the blind sweep protocol. In step 1360 the host generates audio guidance (e.g., via a speaker), indicating whether the motion / orientation of the probe is adherent or non-adherent to the blind sweep protocol. In step 1365 the host generates haptic feedback (e.g., via a haptic feedback motor), indicating whether the motion / orientation of the probe is adherent or non-adherent to the blind sweep protocol.

[0111] In step 1370, if the motion / orientation is non-adherent, the system instructs the user to repeat the incomplete sweeps. In step 1390, if the motion / orientation is adherent, then the blind sweep protocol is complete.

[0112] Flow diagrams and block diagrams are provided herein for exemplary purposes; a person of ordinary skill in the art will recognize myriad variations that nonetheless fall within the scope of the present disclosure. For example, any of the steps described herein may optionally include an output to a user of information relevant to the step, and may thus represent an improvement in the user interface over existing art by providing information not otherwise available. Similarly, block diagrams may show a particular arrangement of components, modules, services, steps, processes, or layers, resulting in a particular data flow. It is understood that some aspects of the systems disclosed herein may include additional components, that some components shown may be absent from some aspects, and that the arrangement of components may be different than shown, resulting in different data flows while still performing the methods described herein. The logic of flow diagrams may be shown as sequential. However, similar logic could be parallel, massively parallel, object oriented, real-time, event-driven, cellular automaton, or otherwise, while accomplishing the same or similar functions. In order to perform the methods described herein, a processor mayAttorney Docket No.: 2024PF00291 divide each of the steps described herein into a plurality of machine instructions, and may execute these instructions at the rate of several hundred, several thousand, several million, or several billion per second, in a single processor or across a plurality of processors. Such rapid execution may be necessary in order to execute the method in real time or near-real time as described herein. For example, to provide an assessment of whether the probe motion is adherent, the system may need to perform object recognition in real time, as the ultrasound images are acquired, at the rate of 60 cycles per second, 120 cycles per second, or other rate typical for ultrasound image acquisition.

[0113] Figure 14 is a schematic, diagrammatic representation, in block diagram form, of at least a portion of an example ultrasound probe orientation verification system 1400, according to aspects of the present disclosure. A set of ultrasound image sequences 14110, also known as cineloops or cine scans (e.g., one cine scan per sweep of the blind sweep protocol) are received (whether one at a time, simultaneously, or in groups) by a probe orientation determination module 1420. Each cine scan can include a plurality of ultrasound image frames (e.g., 100-1000 ultrasound image frames, and / or other values both larger and smaller). The probe orientation determination module 1420 can be a deep learning network (e.g., convolutional neural network or CNN) trained to detect maternal anatomy and / or fetal anatomy.

[0114] If the probe orientation determination module 1420 determines that the probe motion is incorrect (e.g., in-plane motion along the long axis of the transducer array rather than out-of-plane motion along the short axis of the transducer array), the processor (e.g., processor 134 of Fig. 1, processor 116 of Fig. 1, and / or other processors) provides, to a display (e.g., display 132 of Fig. 1 and / or other displays) in communication therewith, an output 1330 that is representative of the determination that the sweep is non-adherent. The visual representation 1330 can be or include text and / or graphical guidance 1430 to the user to repeat the sweep and / or to correct the probe orientation.

[0115] In this situation, ultrasound image frames from the completed sweeps do not get sent to an object detector when the probe orientation results in in-plane motion (not adherent), so there will be no anatomy detections or anatomy detection-based determinations or outputs when there is in-plane motion and thus a non-adherent sweep.

[0116] Figure 15 is a schematic, diagrammatic representation, in block diagram form, of at least a portion of an example ultrasound probe orientation verification system 1500, according to aspects of the present disclosure. A set of ultrasound image sequences 141010, also known as cineloops or cine scans (e.g., one cine scan per sweep of the blind sweepAttorney Docket No.: 2024PF00291 protocol) are received (whether one at a time, simultaneously, or in groups) by a probe orientation determination unit 1510. If the probe motion is determined to be adherent, (e.g., out-of-plane motion along the short axis of the transducer array rather than in-plane motion along the long axis of the transducer array), the processor (e.g., processor 134 of Fig. 1, processor 116 of Fig. 1, and / or other processors) provides, to a display (e.g., display 132 of Fig. 1 and / or other displays) in communication therewith, an output 1330 that is representative of the determination that the sweep is adherent. The visual representation 1330 can be or include text and / or graphical guidance 1520 to the user to conduct the next sweep, or an indication that image acquisition for all sweeps is complete.

[0117] If the probe motion is determined to be adherent (e.g., out-of-plane), the cine scans 1410 are also received by an object detector 1530 (which may for example be a “you only look once” (YOLO) model). Each cine scan can include a plurality of ultrasound image frames (e.g., 100-1000 ultrasound image frames, and / or other values both larger and smaller). The object detector 15300 may for example be a machine learning (ML) neural network as described below, although other types of object detectors may be used instead or in addition, including classical image recognition algorithms. The object detector 153 can be a deep learning network (e.g., convolutional neural network or CNN) trained to detect maternal anatomy and / or fetal anatomy. An output of the object detector may for example include anatomy detections 1535, which can be used for anatomy detection-based determination of whether the sweep(s) is / are adherent, based on motion of the detected anatomy.

[0118] The processor (e.g., processor 134 of Fig. 1, processor 116 of Fig. 1, and / or other processors) then provides, to a display (e.g., display 132 of Fig. 1 and / or other displays) in communication therewith, an output 1330 that is representative of the determination of whether or not the sweep is adherent. The visual representation 1330 can be or include text or graphics 1550 indicative of the anatomy detection-based determination. This may for example include drawings, ultrasound image frames, cineloops, or generated graphics, either with or without text or symbols as annotations, including graphics, images, text, and / or other visual representations, whether described herein or otherwise.

[0119] Figure 16 is a schematic, diagrammatic representation, in flow diagram form, of an example ultrasound probe orientation verification method 1600, according to aspects of the present disclosure. It is understood that the steps of method 1600 may be performed in a different order than shown in Figure 16, additional steps can be provided before, during, and after the steps, and / or some of the steps described can be replaced or eliminated in other embodiments. One or more of steps of the method 1600 can be carried by one or moreAttorney Docket No.: 2024PF00291 devices and / or systems described herein, such as components of the system 100 and / or processor circuit 250. The transducer array emits ultrasound energy, receives ultrasound echoes, and generates electrical signals (ultrasound image data) representative of ultrasound echoes.

[0120] In step 1610, the method 1600 includes controlling the ultrasound probe to obtain ultrasound image data. Execution then proceeds to step 1620.

[0121] In step 1620, the method 1600 includes generating ultrasound image frames based on the obtained ultrasound image data. Execution then proceeds to step 1630.

[0122] In step 1630, the method 1600 includes determining, based on the ultrasound image frames, whether the probe orientation during sweep is adherent (out-of-plane motion) or non-adherent (in-plane motion). Step 1630 can be performed after the sweep is complete. If the motion is adherent, execution then proceeds to step 1650. If the motion is nonadherent, execution then proceeds to step 1640.

[0123] In step 1640, the method 1600 includes generating and providing outputs (display, audio, haptic, etc.) to repeat a sweep and / or to correct the probe orientation. Execution then returns to step 1610.

[0124] In step 1650, the method 1600 includes providing the ultrasound image frames as an input to an object detector (e.g., a neural network). Execution then proceeds to step 1660.

[0125] In step 1660, the method 1600 includes providing, as output of the object detector (e.g., a neural network), anatomy detections identified in the ultrasound image frames. The anatomy detections may for example include the identify of an anatomical object (e.g., fetal heart), as well as a bounding box. Execution then proceeds to step 1670.

[0126] In step 1670, the method 1600 includes performing, based on the anatomy detections, a determination as to whether the motion is adherent or non-adherent (e.g., based on the motion of the imaged anatomy for a given sweep). Execution then proceeds to step 1680.

[0127] In step 1680, the method 1600 includes generating and providing outputs (display, audio, haptic, etc.) based on the anatomy detection-based determination, indicating whether the motion is adherent or non-adherent. The method 1600 is now complete.

[0128] Figure 17 is a schematic, diagrammatic representation of an anatomy detectionbased motion determination 1700, according to aspects of the present disclosure. Ultrasound image frames 1710 from a sweep or cineloop are received by a cross-correlation calculation 1720 that determines the motion of detected anatomy between successive frames (or pairs of image frames offset by a certain number of frames), represented by the motion vectorsAttorney Docket No.: 2024PF00291(arrows). The motion between successive frames (or two frames of an offset pair) is then used to determine whether the motion is adherent 1730 (e.g., out of plane) or non-adherent 1740 (e.g., in-plane) to the blind sweep protocol. The system then generates user feedback 1750, which may for example include displayed, audio, or haptic feedback.

[0129] Depending on the implementation, the ultrasound image frames 1710 can be prescan-conversion image frames, scan-converted (post-scan-conversion) image frames, or B- mode images. Using pre-scan-conversion image frames can be advantageous because it may be computationally easier than using pos-scan-conversion or B-mode image frames.

[0130] Figure 18 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound probe orientation verification system 1800, according to aspects of the present disclosure. In the example shown in Figure 18, two image frames, 1710A and 1710D, offset from one another by an offset 1810 of three frames, are used to generate an inter-frame cross-correlation map 1820, from which a motion vector 1830 is extracted. The value of the offset 1810 can vary (e.g., one frame, two frames, three frames, or more). A test 1840 is then performed to determine whether the norm of the motion vector is greater than a threshold value (e.g., 1.0). If yes, execution proceeds to step 1860. If no, execution proceeds to step 1850.

[0131] In step 1850, the motion is adherent (e.g., out of plane). Execution then proceeds to step 1870.

[0132] In step 1860, the motion is non-adherent (e.g., in-plane). Execution then proceeds to step 1870.

[0133] In step 1870, a determination is made as to whether the percentage of image frames with an in-plane classification is greater than a given threshold (e.g., 70%). If yes, execution proceeds to step 1880. If no, execution proceeds to step 1890.

[0134] In step 1880, the sweep is non-adherent.

[0135] In step 1890, the sweep is adherent.

[0136] Steps 1820-1860 can then be repeated for additional pairs of image frames (e.g., 1710B and 1710E, 1710C and 1710F, etc.) until the end of the sweep is reached.

[0137] An offset 1810 between images may for example be used in the case of spatial compounding. For example if an image plane in frame A has beam steering angle +a, an image plane for frame B has 0 beam steering angle / no beam steering, an image plane for frame C has image plane beam steering angle -a, image plane in frame D has beam steering angle +a and so on. In this example, three images with different angles (e.g., frames A, B, C, with beam steering angles +a, 0, and -a), are combined with spatial / angular compounding.Attorney Docket No.: 2024PF00291Spatial / angular compounding can improve SNR / image quality. The spatially compounded image frame (not individual image frames A, B, C, etc.) can be displayed. Using the offset 1810 ensures that only frames with the same beam steering angle are compared, such that the motion vector is attributable to changes resulting from movement of probe (not to changes resulting from beam steering angles).

[0138] An offset 1810 may also be used for noise reduction (e.g., to account for variability in the speed of the sweep), and to compensate for a high frame rate, wherein consecutive image frames may not have enough change in image content to yield a usable motion vector). An offset 1810 can help ensure that there is enough time between the compared frames for the motion to be detectable (e.g., for the motion vector to be non-zero). Nevertheless, in some aspects, the offset can be 1, such that successive image frames are compared. It is noted that an offset of zero would result in each frame being compared to itself, which would result in a zero motion vector.

[0139] Figure 19 is an example inter-frame cross-correlation map 1900, according to aspects of the present disclosure. The inter-frame cross-correlation map 1900 includes pixels 1910 that correspond to the pixels of the two images being compared, and the brightness or hue of each pixel being representative of the degree of cross-correlation between the same pixel in the two compared images. To generate the inter-frame cross-correlation map 1900, two m x n images (note: 2nd image may be flipped) may be convolved with the fast Fourier transform (FFT) method. The output is then an m x n inter-frame cross-correlation map with a cross-correlation value for each pixel. In the example shown in Figure 19, lighter colors indicate larger cross-correlation values, and the coordinate or pixel 1930 with the maximum cross-correlation value is white.

[0140] A motion vector 1830 is defined as the vector from the center 1920 of the interframe cross-correlation map 1900 to the pixel 1930 having the largest cross-correlation value. The direction of the motion vector 1830 is representative of the azimuthal motion of the probe over the detected anatomy, and the length of the motion vector 1830 is proportional to the speed of the probe in the azimuthal plane in between the capture times of the two compared frames.

[0141] The norm of the motion vector can be the magnitude of the motion vector {sqrt((x2-xl)A2 + (y2-yl)A2), where (xl,yl) and (x2,y2) are the points defining the motion vector}. Generally, if there is a large norm, then the motion between images is in- plane / azimuthal and thus non-adherent to the blind sweep protocol. The norm of the motion vector can thus be a key measurement for classifying in-plane vs. out-of-plane motion. InAttorney Docket No.: 2024PF00291 other aspects, the motion vector may be used to determine the direction of motion, and provide corresponding directional feedback to the user. In some aspects, an Al model such as a neural network may receive as input a series of motion vectors, norms of motion vectors, or ultrasound images to perform the same tasks.

[0142] Figure 20 is a screen display 2000 showing graphical instructions 2010 and text instructions 2020 to the user about how to correct a sweep, according to aspects of the present disclosure. In the example shown in Figure 20, the graphical instructions 2010 and the text instructions 2020 are directing the user to rotate the probe by 90 degrees in a shown direction, e.g., to bring the motion of the probe from a non-adherent in-plane motion to an adherent out- of-plane motion. The user is also instructed to repeat the sweep, and a start button 2030 is provided so the user can indicate when the repeated sweep will begin.

[0143] Figure 21 is a schematic, diagrammatic top view of a transducer array 400 moving at a non-adherent angle, according to aspects of the present disclosure. Instead of being held perpendicular to the direction or motion (e.g., at an angle 0 of 90 degrees), the transducer array 400 is held at a nonzero, non-90-degree angle 0. For values of 0 that are close to 90 degrees, this may not make a significant difference in the scanned area or the anatomies detected within it. However, for values of 0 that exceed a certain threshold (e.g., 35 degrees), these differences may be unacceptable. Therefore, it may be desirable to measure or estimate the value of 0 for a sweep, to determine whether it is necessary for the user to repeat the sweep.

[0144] Figure 22 is an example binning or bucketing method 2200 for a probe angle 0, according to aspects of the present disclosure.

[0145] In step 2210, the variable x is set to the norm of the motion vector.

[0146] In step 2220, if x is between 0.0 and 0.35, then the value of 0 is estimated to be 35 degrees or less. In some implementations, this may be considered an adherent motion.

[0147] In step 2230, if x is between 0.35 and 0.65, then the value of 0 is estimated to be approximately 55 degrees.

[0148] In step 2240, if x is between 0.65 and 1.0, then the value of 0 is estimated to be 0.75 degrees or greater.

[0149] The value of 0 may then be used to determine the orientation of the probe during a sweep, and to provide appropriate instructions to the user as needed. The values shown in Figure 22 are exemplary; other values both higher and lower may be used instead or in addition.Attorney Docket No.: 2024PF00291

[0150] Figure 23 is a screen display 2300 showing graphical instructions 2310 and text instructions 2320 to the user about how to correct a sweep, according to aspects of the present disclosure. In the example shown in Figure 23, the graphical instructions 2310 and the text instructions 2320 are directing the user to rotate the probe by 30 degrees (e.g., the value of 0) in the shown direction, e.g., to bring the motion of the probe from a non-adherent in-plane motion to an adherent out-of-plane motion. The user is also instructed to repeat the sweep, and a start button 2330 is provided so the user can indicate when the repeated sweep will begin.

[0151] Figure 24 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound probe orientation verification system 2400 that provides real-time guidance for ultrasound probe motion, according to aspects of the present disclosure. Live ultrasound image frames 1710 can be obtained by an ultrasound probe, while the ultrasound imaging is ongoing. Live ultrasound image frames 1710 are received by the object detector 1530, which produces anatomy detections 1535, which provide either a complete anatomical view 2420 or an incomplete anatomical view 2430. In the case of an incomplete anatomical view 2430, both the locations 2410 of the anatomy detections in the image frames and the incomplete anatomical view 2430 are used in a determination step 2440, to determine a direction to translate and / or rotate the probe for a complete anatomical view. This determination 2440 is then used to provide user guidance 2450 to translate and / or rotate the probe in the determined direction. The determination 2440 is also passed to step 2460.

[0152] The live ultrasound image frames 1710 are also used to generate inter-frame cross-correlation maps 1900, from which motion vectors 1940 can be extracted. In a matching step 2460, it is determined whether the user is moving the probe in the determined direction. If Yes, then user guidance 2470 is provided, affirming that the user is moving the probe in the determined direction. If No, then user guidance 2480 is provided, prompting the user to change the probe motion to be in the determined direction.

[0153] In the case that an out-of-plane motion is needed to obtain a complete view plane, the role of the motion vector method may be to check that the user is not moving in the inplane direction, while the object detector may be the primary method to determine when the view plane becomes complete. In the case that the user is directed to rotate the probe to obtain a complete view plane, an inertial measurement unit (IMU) could also be used toAttorney Docket No.: 2024PF00291 detect said rotation. It is also noted that a rotation may include a roll and / or pitch change as well.

[0154] The process shown in Figure 24 can be repeated (e.g., continuously, every frame, every n frames, etc.) during image acquisition of the live ultrasound image frames 1710. Aspects related to complete vs. incomplete anatomical views can be found for example in U.S. Provisional Application No. 63 / 324,939, filed 29 March 2022, titled “Completeness Of View Of Anatomy In Ultrasound Imaging And Associated Systems, Devices, And Methods”, U.S. Publication No. 2022 / 0096053, titled “Systems And Methods For Guided Ultrasound Data Acquisition”, and U.S. Provisional Application No. , filed > , titled“Verification Of Uterine Ultrasound Imaging Extent Using Blind Sweep Protocol” (Atty Dkt. No. 2024PF00110 / 44755.2421PV01), which are incorporated by reference as though fully set forth herein.

[0155] Aspects related to determining and indicating the direction to move (e.g., translate / rotate) the ultrasound probe can be found for example in U.S. Publication No. 2021 / 0369249, titled “Deep Learning-Based Ultrasound Imaging Guidance And Associated Devices, Systems, And Methods”, U.S. Publication No. 2021 / 0000446, titled “Ultrasound Imaging Plane Alignment Guidance For Neural Networks And Associated Devices, Systems, And Methods”, U.S. Publication No. 2023 / 0094631, titled “Ultrasound Imaging Guidance And Associated Devices, Systems, And Methods”, U.S. Publication No. 2022 / 0370034, titled “Automatic Closed-Loop Ultrasound Plane Steering For Target Localization In Ultrasound Imaging And Associated Devices, Systems, And Methods”, and U.S. Provisional Application No. 63 / 540,740, filed September 27, 2023, and titled “Ultrasound Imaging with Ultrasound Probe Guidance In Blind Sweep Protocol”, which are incorporated by reference as though fully set forth herein.

[0156] Figure 25 is an example screen display 2500 showing partially imaged anatomy 2540 and user instructions, according to aspects of the present disclosure. In the example shown in Figure 25, an anatomy selector 2510 allows the user to select which anatomy the anatomy detector should be looking for (e.g., fetal heart), and a view selector 2520 allows the user to select which view is desired (e.g., apical 4-chamber view or A4C). An ultrasound image 2530 includes partially imaged anatomy 2540 of the fetal heart. Text instructions 2550 and graphical instructions 2560 instruct the user to rotate the probe by 30 degrees and translate the probe 2 centimeters to the left, in order to obtain a complete image of the selected anatomy (e.g., the fetal heart). Depending on the implementation, the screen display 2500 may update in real time as the position and orientation of the probe are adjusted.Attorney Docket No.: 2024PF00291

[0157] Figure 26 is an example screen display 2600 showing partially imaged anatomy 2540 and user instructions 2550, according to aspects of the present disclosure. Visible are the anatomy selector 2510 and view selector 2520. The ultrasound image 2530 includes partially imaged anatomy 2540 of the fetal heart. Text instructions 2550 instruct the user to continue the translation and rotation of the probe to bring the selected anatomy (e.g., the fetal heart) into the center of the image 2530 at the desired orientation.

[0158] Figure 27 is an example screen display 2700 showing completely imaged anatomy 2740 and user instructions 2550, according to aspects of the present disclosure. Visible are the anatomy selector 2510 and view selector 2520. The ultrasound image 2530 includes fully imaged anatomy 2740 of the fetal heart. Text instructions 2550 instruct the user to stop the translation and rotation of the probe, as the selected anatomy (e.g., the fetal heart) is now in the center of the image 2530 at the desired orientation.

[0159] Figure 28 is an example screen display 2800 showing partially imaged anatomy 2540 and user instructions 2550, according to aspects of the present disclosure. Visible are the anatomy selector 2510 and view selector 2520. The ultrasound image 2530 includes partially imaged anatomy 2540 of the fetal heart. In the example shown in Figure 28, the user has not correctly followed the instructions of Figure 25. Thus, text instructions 2550 instruct the user to continue the translation of the probe, and to rotate the probe in the opposite direction from the current rotation, to bring the selected anatomy (e.g., the fetal heart) into the center of the image 2530 at the desired orientation.

[0160] Figure 29 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound probe orientation verification system 2900 that provides real-time guidance for ultrasound probe motion to image an interventional device, according to aspects of the present disclosure. Live ultrasound image frames 1710 are received by a block 2910, which determines whether the interventional device (e.g., a catheter, guidewire, needle, etc.) is in the desired view plane 2920 or a non-desired view plane 2930. Block 2910 can be performed using non-AI based imaging processing / processing (e.g., using brightness of pixels) or neural -network based object detection and / or segmentation. In the case of a non-desired view plane 2930, both the changing appearance 2915 of the detected interventional device in the image frames, and the non-desired view plane 2930 are used in a determination step 2940, to determine a direction to translate and / or rotate the probe to bring the interventional device into the desired view plane. Aspects related to obtaining a desired imaging plane are described for example in U.S. Publication No. 2021 / 0000446, titled “Ultrasound Imaging Plane Alignment Guidance ForAttorney Docket No.: 2024PF00291Neural Networks And Associated Devices, Systems, And Methods” and U.S. Publication No. 2022 / 0370034, titled “Automatic Closed-Loop Ultrasound Plane Steering For Target Localization In Ultrasound Imaging And Associated Devices, Systems, And Methods”, which are incorporated by reference as though fully set forth herein. This determination 2940 is then used to provide user guidance 2950 to translate and / or rotate the probe in the determined direction.

[0161] The live ultrasound image frames 1710 are also used to generate inter-frame cross-correlation maps 1900, from which motion vectors 1940 can be extracted and passed to the determination step 2940.

[0162] In the case that an out-of-plane motion is needed to obtain the desired view plane, the role of the motion vector method may be to check that the user is not moving in the inplane direction, while the object detector may be the primary method to determine when the view plane becomes complete. In the case that the user is directed to rotate the probe to obtain the desired view plane, an inertial measurement unit (IMU) could also be used to detect said rotation. It is also noted that a rotation may include a roll and / or pitch change as well.

[0163] The process shown in Figure 29 can be repeated (e.g., continuously, every frame, every n frames, etc.) during image acquisition of the live ultrasound image frames 1710.

[0164] Figure 30 is an example screen display 3000 showing an interventional device 3040 in a live ultrasound image 2350, according to aspects of the present disclosure. A view selector 3010 allows the user to select the desired view plane for the interventional device 3040. Text instructions 2550 and graphical instructions 2560 inform the user how to bring the interventional device 3040 from the current view plane (in this case, a lateral or transverse cross-section) into the desired view plane (in this case, a longitudinal cross-section). In the example shown in Figure 30, the user is instructed to rotate the probe by 90 degrees in a shown direction, and to translate the probe by 2 cm in a shown direction.

[0165] Figure 31 is an example screen display 3100 showing an interventional device 3040 in a live ultrasound image 2350, according to aspects of the present disclosure. The screen display 3100 is identical to the screen display 3000 of Figure 30, except that the user has followed the instructions 2550, 2560, thus bringing the interventional device 3040 into the desired orientation in the ultrasound image 2530 (e.g., a longitudinal cross section).

[0166] Figure 32 is a schematic, diagrammatic representation, in block diagram form, of an inference mode of an example machine learning-based motion detection process 3200, according to aspects of the present disclosure. In the example shown in Figure 32, ultrasoundAttorney Docket No.: 2024PF00291 image frames 1710 from a sweep or cineloop are turned into inter-frame cross-correlation maps 1900 from which motion vectors 1940 are extracted, as described above in Figure 19. The motion vectors are then fed to a trained machine learning model 3210, which produces as an output a determination of out-of-plane motion 3220 (e.g., adherent to the blind sweep protocol) or in-plane motion 3230 (e.g., non-adherent to the blind sweep protocol), which are then used to provide user feedback 3240 (e.g., display, audio, or haptic feedback).

[0167] In some implementations, instead of receiving the motion vectors 1940, the machine learning model 3210 receives the ultrasound images 1710 directly, and outputs its determinations 3220, 3230 based on these.

[0168] Figure 33 is a schematic, diagrammatic view, in block diagram form, of a training system 3300 for a machine learning model 3330, according to aspects of the present disclosure. Training system 3300 trains the machine learning model 3330 to accurately predict in-plane motion or out-of-plane motion 3340. In some aspects, machine learning model 3330 is trained by providing historic information from completed ultrasound sweep procedures. Training machine learning model 3330 aims to improve the performance of the model. Training system 3300 includes training data 3310, machine learning model 3330, and model objectives / functions 3350.

[0169] Training data 3310 may include data for a plurality of patient records 3320. Each patient record may contain a plurality of images or views, treatment outcomes, and / or a plurality of user annotations. In some aspects, patient record 3320 may include ultrasound sweep images 3380, motion vectors 3382, and user annotation 3390 of in-plane or out-of- plane motion. It should be appreciated that not all the patient records 3320 depicted in Fig. 33 needed to train the machine learning model 3330. Furthermore, additional medical records from a patient’s medical history may be included in the training data 3310.

[0170] Machine learning model 3330 may receive the training data 3310. From the training data 3310, machine learning model 3330 may output a prediction of the likelihood of in-plane or out-of-plane motion 3340. For example, the output may be a number between 0 and 1.

[0171] Using model objectives / functions 3350 training system 3300 compares the prediction 3340 with associated ground truth predictions from the training data 3310. For example, in some embodiments, a user annotation of an ultrasound sweep may note out-of- plane motion. The annotation identifying whether the motion is in-plane or out-of-plane may act as the ground truth label. If an image or view is available but lacks an annotation in associated medical records, then ground truth labels can be generated by having a physicianAttorney Docket No.: 2024PF00291 review the image. Alternatively, a separate machine learning model trained for object detection and / or image segmentation be used to analyze sweep images and / or views for evidence of in-plane or out-of-plane motion.

[0172] In some embodiments, model objectives / functions 3350 may be used to compare predicted vessel measurements from a machine learning model 3330 with user annotated images 3390, which represent the ground truth labels.

[0173] Model objectives / functions 3350 may include objectives / functions which penalize to a greater extent predicted likelihoods which are further from the ground truth labels.Model objectives / functions 3350 may also penalize predictions which are close to the ground truth labels but which are based on the incorrect measurements. In some aspects, model objectives / functions 3350 may include objectives / functions which penalize deviations of predicted motion from user-annotated motion 3390. In some aspects, the model objectives / functions 3350 may be a mean squared error or mean absolute error. The error can be a numerical representation of the difference between the ground truth / user annotation 3390 and the output 3340 of the predictive network 3330.

[0174] Comparisons from the model objectives / functions 3350 may update parameters 3360 of the machine learning model 3330. In some instances, updating may be accomplished using gradient of the objective functions and backpropagation to update the parameters of the machine learning model.

[0175] Figure 34 is a schematic, diagrammatic representation, in block diagram form, of an inference mode of an example machine learning-based motion detection process 3400, according to aspects of the present disclosure. In the example shown in Figure 32, ultrasound image frames 1710 from a sweep or cineloop are turned into inter-frame cross-correlation maps 1900 from which motion vectors 1940 are extracted, as described above in Figure 19. The motion vectors are then fed to a trained machine learning model 3410, which produces as an output a direction 3420 to rotate and / or translate the ultrasound probe to achieve a complete anatomical view and / or a view in the desired view plane, which is used to generate user feedback 3430 (e.g., display, audio, or haptic feedback).

[0176] In some implementations, instead of receiving the motion vectors 1940, the machine learning model 3410 receives the ultrasound images 1710 directly, and outputs its directions 3420 based on these.

[0177] Figure 35 is a schematic, diagrammatic view, in block diagram form, of a training system 3500 for a machine learning model 3530, according to aspects of the present disclosure. Training system 3500 trains the machine learning model 3530 to accuratelyAttorney Docket No.: 2024PF00291 predict the direction 3540 to rotate / translate the ultrasound probe. In some aspects, machine learning model 3530 is trained by providing historic information from completed ultrasound sweep procedures. Training machine learning model 3530 aims to improve the performance of the model. Training system 3500 includes training data 3510, machine learning model 3530, and model objectives / functions 3550.

[0178] Training data 3510 may include data for a plurality of patient records 3520. Each patient record may contain a plurality of images or views, treatment outcomes, and / or a plurality of user annotations. In some aspects, patient record 3520 may include ultrasound sweep images 3580, motion vectors 3582, and user annotation 3590 of the direction to rotate and / or translate the ultrasound probe in order to achieve the desired coverage and image plane. It should be appreciated that not all the patient records 3520 depicted in Fig. 35 needed to train the machine learning model 3530. Furthermore, additional medical records from a patient’s medical history may be included in the training data 3510.

[0179] Machine learning model 3530 may receive the training data 3510. From the training data 3510, machine learning model 3530 may output a direction of desired motion 3340. For example, the output may be a magnitude between -10 centimeters and 10 centimeters for a desired translation, and an angle between 0 and 360 degrees for a desired rotation.

[0180] Using model objectives / functions 3550 training system 3500 compares the prediction 3540 with associated ground truth predictions from the training data 3510. For example, in some embodiments, a user annotation of an ultrasound sweep may note a need to rotate and / or translate the ultrasound probe. The annotation identifying the direction and magnitude of the desired rotation or translation may act as the ground truth label. If an image or view is available but lacks an annotation in associated medical records, then ground truth labels can be generated by having a physician review the image. Alternatively, a separate machine learning model trained for object detection and / or image segmentation be used to analyze sweep images and / or views for evidence of a need to rotate or translate the probe.

[0181] In some aspects, model objectives / functions 3550 may be used to compare predicted vessel measurements from a machine learning model 3530 with user annotated images 3590, which represent the ground truth labels.

[0182] Model objectives / functions 3550 may include objectives / functions which penalize to a greater extent predicted angles or lengths which are further from the ground truth labels. Model objectives / functions 3550 may also penalize predictions which are close to the ground truth labels but which are based on incorrect measurements. In some aspects, modelAttorney Docket No.: 2024PF00291 objectives / functions 3550 may include objectives / functions which penalize deviations of predicted desired motion from user-annotated desired motion 3590. In some aspects, the model objectives / functions 3550 may be a mean squared error or mean absolute error. The error can be a numerical representation of the difference between the ground truth / user annotation 3590 and the output 3540 of the predictive network 3530.

[0183] Comparisons from the model objectives / functions 3550 may update parameters 3560 of the machine learning model 3530. In some instances, updating may be accomplished using gradient of the objective functions and backpropagation to update the parameters of the machine learning model.

[0184] Figure 36 is a schematic diagram of a deep learning network configuration, according to aspects of the present disclosure. The configuration 3600 can be implemented by a deep learning network. The configuration 3600 includes a deep learning network 3610, which may include one or more CNNs 3612. The CNN 3612 is one example of a type of predictive model and / or machine learning model, which may be used in the anatomy detector 1530, machine learning model 3210, artificial intelligence model 3410, and otherwise. For simplicity of illustration and discussion, Fig. 36 illustrates one CNN 3612. However, any suitable number of CNNs 3612 (e.g., about 2, 3 or more) may be included. The configuration 3600 can be trained for identification of various anatomy (organs, tissue, bone) and / or other features (natural and / or man-made) within a patient anatomy, for detection of motion between image frames, for determining a desired direction of probe movement, and otherwise. The configuration 3600 can be further trained for segmenting human anatomy, diagnosis of medical conditions or any number of other diagnostic or medical tasks.

[0185] The CNN 3612 may include a set of N convolutional layers 3620 followed by a set of K fully connected layers 3630, where N and K may be any positive integers. The convolutional layers 3620 are shown as 3620(1) to 3620(N). The fully connected layers 3630 are shown as 3630(1) to 3630(K). Each convolutional layer 3620 may include a set of filters 3622 configured to extract features from an input 3602 (e.g., x-ray images or other data). The values N and K and the size of the filters 3622 may vary depending on the use of the CNN. In some instances, the convolutional layers 3620(1) to 3620(N) and the fully connected layers 3630(1) to 3630(K-l) may be interspersed with rectified non-linear (ReLU) or leaky ReLU or other activation functions and / or batch normalization layers. The fully connected layers 3630 may gradually shrink the high-dimensional output to a lower dimension or the dimension of the predicted result 3640 (e.g., location for an object detection landmark, or location and dimension for an object detection box, number of classes for a classification output, etc.).Attorney Docket No.: 2024PF00291The fully connected layers 3630 may also be referred to as a classifier. In some aspects, the fully convolutional layers 3620 may additionally be referred to as representation or encodings or features.

[0186] When the prediction result 3640 takes the form of classification output, it may indicate a confidence score (e.g., a probability) for each class 3642 based on the input image 3602. The classes 3642 are shown as 3642a, 3642b, . . ., 3642c. For example, when the CNN 3612 is trained for classification of present anatomical features, the classes 3462 may indicate a first anatomical feature class 3642a, a second anatomical feature class 3642b, a third anatomical feature class 3642c, a fourth anatomical feature class 3642d, or any other suitable class. A class 3642 indicating a high confidence score indicates that the input image 3602 or a section or pixel of the image 3602 is likely to include an anatomical object / feature of the class 3642. Conversely, a class 3642 indicating a low confidence score indicates that the input image 3602 or a section or pixel of the image 3602 is unlikely to include an anatomical object / feature of the class 3642.

[0187] The CNN 3612 can also output a feature vector 3650 at the output of the last convolutional layer 3620(N), though any of the layers in the CNN are feature vectors. A feature vector 3650 or encodings or representations may encode some representation of objects detected from the input medical image 3602 or other data. These representations can be decoded using a reversed CNN where the fully connected layers expand the lowdimensional representation to a higher dimension and transposed convolutional layers can expand feature layers up to the size of the original input, where pixel-wise outputs (e.g., binary segmentation map, multi-class segmentation map) can be generated.

[0188] The deep learning network 3610 may implement or include any suitable type of learning network. For example, in some aspects, and as described in relation to Fig. 36, the deep learning network 3610 could include a convolutional neural network 3612. In addition, the deep learning network 3610 may additionally or alternatively be or include a multi-class classification network, an encoder-decoder type network, a fully connected deep learning network, or any suitable network or means of identifying features within an image.

[0189] In some aspects, when the deep learning network 3610 includes a fully- connected neural network, the fully connected neural network may transform the data not related to a medical image or it may transform data derived from an image (e.g., detected objects) generated by another network, program, or human annotator. Data for detected objects may be concatenated into a layer (e.g., as an additional channel. For simple numeric features like age, weight, and other data - these can be concatenated into fully connectedAttorney Docket No.: 2024PF00291 layers. For example, the fully connected neural network may transform information about a patient, such as age, weight, or other low-dimensional data.

[0190] In some aspects, when the deep learning network 3610 includes an encoderdecoder network, the network may include two components. One component may be a constricting component or encoder, in which a large image, such as the image 3602, may be convolved by several convolutional layers 3620 such that the size of the image 3602 changes in relation to the depth of the network layer. For instance, the CNN 3612 may be the encoder. The image 3602 may then be represented in a low dimensional space, or a flattened space. From this flattened space, an additional component or decoder may expand the flattened space to the original size of the image 3602. For instance, the reverse of the CNN 3612 may be the decoder. In some aspects, the encoder-decoder network may reconstruct the input image 3602. In some aspects, the encoder-decoder network may segment the image 3602 into patches. In some aspects of the present disclosure, the deep learning network 3610 may include a multi-class classification network. In that aspect, the multi-class classification network may include an encoder path. For example, the image 3602 may be a high dimensional image. The image 3602 may then be processed with the convolutional layers 3620 such that the size is reduced. The resulting low dimensional representation of the image 302 may be used to generate the feature vector 3650 shown in Fig. 36. The low dimensional representation of the image 3602 may additionally be used by the fully connected layers 3630 to regress and output one or more classes 3642. In some regards, the fully connected layers 3630 may process the output of the convolutional layers 3620. The fully connected layers 3630 may additionally be referred to as task layers or regression layers, among other terms.

[0191] Any suitable combination or variations of the deep learning network 3610 described is fully contemplated. For example, the deep learning network may include fully convolutional networks or layers or fully connected networks or layers or a combination of the two. In addition, the deep learning network may include a multi-class classification network, an encoder-decoder network, or any combination of networks. The process of training the deep learning network 3610 includes adjusting its parameters, or more particularly the weights and biases, which control the operation of activation functions in the neural network. In supervised learning, the training process automatically adjusts the weights and the biases, such that when presented with the input data, the neural network accurately provides the corresponding expected output data. In order to do this, the value of the loss functions, or errors, are computed based on a difference between predicted output data and the expected output data. The value of the loss function may be computed using functionsAttorney Docket No.: 2024PF00291 such as the negative log-likelihood loss, the mean squared error, or the Huber loss, or the cross-entropy loss. During training, the value of the loss function is typically minimized. Various methods are known for solving the loss minimization problem such as gradient descent, Quasi-Newton methods, and so forth. Various algorithms have been developed to implement these methods and their variants including but not limited to Stochastic Gradient Descent “SGD”, batch gradient descent, mini -batch gradient descent, Gauss-Newton, Levenberg Marquardt, Momentum, Adam, Nadam, Adagrad, Adadelta, RMSProp, and Adamax “optimizers”. These algorithms compute the derivative of the loss function with respect to the model parameters using the chain rule. This process is called backpropagation since derivatives are computed starting at the last layer or output layer, moving toward the first layer or input layer. These derivatives inform the algorithm how the model parameters must be adjusted in order to minimize the error function. The training process is performed iteratively by making adjustments to the weights and biases in each iteration. Training is terminated when the error, or difference between the predicted output data and the expected output data, is within an acceptable range for the training data, or for some validation data. Subsequently the neural network may be deployed, and the trained neural network makes predictions on new input data using the trained values of its parameters. If the training process was successful, the trained neural network accurately predicts the expected output data from the new input data.

[0192] As will be readily appreciated by those having ordinary skill in the art after becoming familiar with the teachings herein, the ultrasound probe orientation verification system advantageously permits untrained and minimally trained users to perform an ultrasound blind sweep protocol to gather anatomical images of high quality. This may result in higher accuracy and higher clinician trust in the results, while potentially improving health outcomes and / or decreasing the total cost of care. Potential benefits include detection of pregnancy complications via blind sweeps performed by novice ultrasound users. The solution can be a quick initial check scan for a center with high volume ultrasound turnover to triage patients for a more detailed obstetric scan, and can provide or support referral of the subject diagnosed with complications for further diagnosis and management to a tertiary care center. Early detection of pregnancy complications may be extremely helpful for follow-up and monitoring of the pregnancy.

[0193] Use of the present disclosure can be detected if an acquired sweep is detected to have insufficient coverage or data, and a user interface (UI) shows visualizations to prompt the user to repeat the sweep when the probe orientation is incorrect. Similarly, while a sweepAttorney Docket No.: 2024PF00291 is being acquired, visualizations on UI or audio, provided based on image coverage in realtime as guidance, are indicative of the ultrasound probe orientation verification system being used. Additionally, if the system disclosed herein is implemented in an ultrasound imaging system, sweeps that fail to cover the entirety of the uterus may result in “incorrect orientation” feedback through UI or audio either after or during the acquisition, as mentioned above, or if a competitor provides user feedback related to the direction of probe motion for the purposes of quality control, active guidance, or any other application, without the use of any external sensors (e.g., IMU, EM / optical tracking, etc.).

[0194] The system disclosed herein can be associated with portable Al-assisted obstetric screening platform for novice ultrasound users that leverages, e.g., the Philips Lumify ultrasound probe. However, the system and methods described herein may be applied to any ultrasound probe and application in which a region of interest is being imaged and in which a determination of complete coverage should be made based on anatomy detections or similar input described in the above aspects.

[0195] The systems, methods, and devices described herein may be applicable in point of care and handheld ultrasound use cases. The disclosed technology can be used for any handheld imaging applications, including but not limited to obstetrics and echocardiography. The technology could be deployed on handheld mobile ultrasound devices, and on portable or cart-based ultrasound systems. The ultrasound probe orientation verification system can be used in a variety of settings including emergency departments, ambulances, accident sites, and homes. The applications could also be expanded to other settings. The disclosed technology increases the value proposition of ultrasound applications in the obstetrics context, especially for use by minimally trained users.

[0196] Accordingly, the logical operations making up the aspects of the technology described herein are referred to variously as operations, steps, objects, layers, elements, components, algorithms, or modules. Furthermore, it should be understood that these may occur or be performed or arranged in any order, unless explicitly claimed otherwise or a specific order is inherently necessitated by the claim language.

[0197] All directional references e.g., upper, lower, inner, outer, upward, downward, left, right, lateral, front, back, top, bottom, above, below, vertical, horizontal, clockwise, counterclockwise, proximal, and distal are only used for identification purposes to aid the reader’s understanding of the claimed subject matter, and do not create limitations, particularly as to the position, orientation, or use of the ultrasound blind sweep multiple pregnancy detection system. Connection references, e.g., attached, coupled, connected,Attorney Docket No.: 2024PF00291 joined, or “in communication with” are to be construed broadly and may include intermediate members between a collection of elements and relative movement between elements unless otherwise indicated. As such, connection references do not necessarily imply that two elements are directly connected and in fixed relation to each other. The term “or” shall be interpreted to mean “and / or” rather than “exclusive or.” The word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. Unless otherwise noted in the claims, stated values shall be interpreted as illustrative only and shall not be taken to be limiting.

[0198] The above specification, examples and data provide a complete description of the structure and use of exemplary aspects of the ultrasound probe orientation verification system as defined in the claims. Although various aspects of the claimed subject matter have been described above with a certain degree of particularity, or with reference to one or more individual aspects, those skilled in the art could make numerous alterations to the disclosed aspects without departing from the spirit or scope of the claimed subject matter.

[0199] Still other aspects are contemplated. It is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative only of particular aspects and not limiting. Changes in detail or structure may be made without departing from the basic elements of the subject matter as defined in the following claims.

Claims

Attorney Docket No.: 2024PF00291CLAIMSWhat is claimed is:

1. A system, comprising: a processor configured for communication with an ultrasound probe, wherein the processor is configured to: control the ultrasound probe to obtain a plurality of ultrasound image frames while the ultrasound probe is moved during a sweep of a blind sweep protocol on a patient; determine, using the plurality of ultrasound image frames, whether the movement of the probe during the sweep is perpendicular to an imaging plane of the plurality of ultrasound image frames; and provide, to a display in communication with the processor, an output representative of the determination of whether the movement of the probe during the sweep is perpendicular.

2. The system of claim 1, wherein, to determine whether the movement of the probe during the sweep is perpendicular, the processor is configured to determine a motion vector using a first frame and a second frame of the plurality of ultrasound image frames.

3. The system of claim 2, wherein, to determine the motion vector, the processor is configured to: with the first frame and the second frame, generating an inter-frame cross-correlation map; identifying a pixel of the inter-frame cross-correlation map that has the largest crosscorrelation value; and determining the motion vector by linking a center pixel of the inter-frame crosscorrelation map to the pixel of the inter-frame cross-correlation map that has the largest cross-correlation value.

4. The system of claim 2, wherein the first frame and the second frame are separated by an offset.Attorney Docket No.: 2024PF002915. The system of claim 2, wherein, to determine whether the movement of the probe during the sweep is perpendicular, the processor is configured to determine whether a norm of the motion vector is greater than a threshold value.

6. The system of claim 5, wherein the processor is configured to determine an angle of the ultrasound probe based on a value of the norm.

7. The system of claim 2, wherein, to determine whether the movement of the probe during the sweep is perpendicular, the processor is configured to: provide at least one of the plurality of ultrasound image frames or the motion vector to a machine learning model trained to determine whether the movement of the probe during the sweep is perpendicular; and generate, as an output of the machine learning model, the determination of whether the movement of the probe during the sweep is perpendicular.

8. The system of claim 1, wherein the output includes user guidance to at least one of rotate or translate the ultrasound probe.

9. The system of claim 1, wherein the output includes user guidance to repeat the sweep.

10. The system of claim 1, wherein, when the movement of the probe during the sweep is not perpendicular, the processor is configured to discard the plurality of ultrasound image frames such that the plurality of ultrasound image frames is not provided to a machine learning model trained to detect at least one of fetal anatomy or maternal anatomy.

11. The system of claim 10, wherein, when the movement of the probe during the sweep is perpendicular, the processor is configured to: provide the plurality of ultrasound image frames to the machine learning model trained to detect at least one of fetal anatomy or maternal anatomy; and output, to the display, a visual representation associated with an output of the machine learning model.

12. The system of claim 1,Attorney Docket No.: 2024PF00291 wherein the processor is configured to classify the sweep as: adherent to the blind sweep protocol when the movement of the probe during the sweep is perpendicular; or not adherent to the blind sweep protocol when the movement of the probe during the sweep is not perpendicular; and wherein the output is representative of the classification of the sweep as adherent or not adherent.

13. A system, comprising: a processor configured for communication with an ultrasound probe, wherein the processor is configured to: control the ultrasound probe to obtain a plurality of ultrasound image frames while the ultrasound probe is moved on a patient; determine at least one motion vector using the plurality of ultrasound image frames; determine whether the plurality of ultrasound image frames depict a complete view of anatomy; and when the plurality of ultrasound image frames does not depict the complete view of the anatomy: determine, using the at least one motion vector, whether the ultrasound probe is moving in a direction to obtain the complete view; and provide, to a display in communication with the processor, an output representative of the determination of whether the ultrasound probe is moving the direction to obtain the complete view, wherein, when the ultrasound probe is not moving in the direction to obtain the complete view, the output comprises user guidance to at least one of rotate or translate the ultrasound probe into the direction to obtain the complete view.

14. The system of claim 13, wherein the processor is configured to determine the direction to move the ultrasound probe to obtain the complete view; wherein, to determine whether the user is moving the ultrasound probe in the direction to obtain the complete view, the processor is configured to determine if the at least one motion vector matches the direction to obtain the complete view.Attorney Docket No.: 2024PF0029115. The system of claim 14, wherein, to determine the direction to move the ultrasound probe to determine the complete view, the processor is configured to: provide at least one of the plurality of ultrasound image frames or the at least one motion vector to a machine learning model trained to determine the direction to move the ultrasound probe to obtain the complete view; and generate, as an output of the machine learning model, the direction to move the ultrasound probe to obtain the complete view.

16. The system of claim 13, wherein the processor is configured to: output at least one selector field to the display; and receive a user selection representative of the complete view via the at least one selector field.

17. A system, comprising: a processor configured for communication with an ultrasound probe, wherein the processor is configured to: control the ultrasound probe to obtain a plurality of ultrasound image frames of an interventional device while the ultrasound probe is moved on a patient; determine at least one motion vector using the plurality of ultrasound image frames; determine whether the plurality of ultrasound image frames depict the interventional device in a desired view plane; and when the plurality of ultrasound image frames do not depict the interventional device in the desired view plane: determine, using the at least one motion vector, whether the ultrasound probe is moving in a direction to obtain the desired view plane; and provide, to a display in communication with the processor, an output representative of the determination of whether the ultrasound probe is moving the direction to obtain the desired view plane wherein, when the ultrasound probe is not moving in the direction to obtain the desired view plane, the output comprises user guidance to at least one of rotate or translate the ultrasound probe into the direction to obtain the desired view plane.Attorney Docket No.: 2024PF0029118. The system of claim 17, wherein the processor is configured to determine the direction to obtain the desired view plane; wherein, to determine whether the user is moving the ultrasound probe in the direction to obtain the desired view plane, the processor is configured to determine if the at least one motion vector matches the direction to obtain the desired view plane.

19. The system of claim 18, wherein, to determine the direction to obtain the desired view plane, the processor is configured to: provide at least one of the plurality of ultrasound image frames or the at least one motion vector to a machine learning model trained to determine the direction to obtain the desired view plane; and generate, as an output of the machine learning model, the direction to obtain the desired view plane.

20. The system of claim 17, wherein the processor is configured to: output at least one selector field to the display; and receive a user selection representative of the desired view plane via the at least one selector field.

Images