Low-lying placenta and / or placenta location in ultrasound imaging with blind sweep protocol

Abstract

A system includes a processor configured for communication with an ultrasound probe. The processor is configured to control the ultrasound probe to obtain a plurality of ultrasound image frames during a blind sweep protocol on a pregnant patient, and provide the plurality of ultrasound image frames as an input to a deep learning network trained to detect anatomy of the pregnant patient. The processor is further configured to generate, as a first output of the deep learning network, a detection of a placenta within the plurality of ultrasound image frames, determine, using the detection of the placenta within the plurality of ultrasound image frames, whether the placenta is low-lying, and output, to a display in communication with the processor, an output representative of the determination of whether the placenta is low-lying.

Description

LOW-LYING PLACENTA AND / OR PLACENTA LOCATION IN ULTRASOUND IMAGING WITH BLIND SWEEP PROTOCOLTECHNICAL FIELD

[0001] The subject matter described herein relates to devices, systems, and methods for using ultrasound data from a blind abdominal imaging sweeps to detect an anatomical condition such as a low-lying placenta (e.g., placenta previa) and / or a placenta location.BACKGROUND

[0002] Ultrasound imaging is often used for diagnostic purposes in an office or hospital setting, but may also be used in resource-constrained care settings (e.g., homes, accident sites, ambulances, mobile health facilities, etc.) by emergency personnel, home health nurses, midwives, etc., who may lack ultrasound expertise. To facilitate ultrasound image acquisition by untrained or minimally trained users, a “blind sweep” protocol is often employed, in which the user follows pre-determined probe paths (e.g., sweeping out a pattern on the patient’s abdomen) during imaging.

[0003] Ultrasound imaging is a vital component of high-quality obstetric care. For example, detection of pregnancy complications through ultrasound can allow for appropriate referral for delivery care in highly resourced centers with providers trained to handle the complications. However, in rural and under-resourced communities, the scarcity of ultrasound imaging results in a considerable gap in the healthcare of pregnant mothers.

[0004] The information included in this Background section of the specification, including any references cited herein and any description or discussion thereof, is included for technical reference purposes only and is not to be regarded as subject matter by which the scope of the disclosure is to be bound.SUMMARY

[0005] Disclosed is an ultrasound sweep anatomy detection system that, following a blind sweep protocol (sweeps at particular locations along body without trying to find specific anatomy), detects anatomical features in the captured images and uses them to identify pregnant patients who may need to be referred for evaluation by human experts. For example, the system may detect the placenta, determine whether the placenta is low-lying (e.g., potentially blocking the cervix, as in placenta previa), and / or determine the location of the placenta within the uterus. A pregnant patient can referred to an obstetrician trained to deal with low-lying placenta based on the output of the system. A deep learning network, such as a convolutional neural network, is used to detect anatomy, such as the placenta, amniotic fluid, etc. A benefit of the ultrasound sweep anatomy detection system is to improve the quality of care by using information obtained during the blind sweep protocol to identify medical conditions (e.g., low-lying placenta, placenta previa, etc.) that may require expert care.

[0006] A system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions. One general aspect includes a system that includes a processor configured for communication with an ultrasound probe. The processor is configured to: control the ultrasound probe to obtain a plurality of ultrasound image frames during a blind sweep protocol on a pregnant patient; provide the plurality of ultrasound image frames as an input to a deep learning network trained to detect anatomy of the pregnant patient; generate, as a first output of the deep learning network, a detection of a placenta within the plurality of ultrasound image frames; determine, using the detection of the placenta within the plurality of ultrasound image frames, whether the placenta is low-lying; and output, to a display in communication with the processor, an output representative of the determination of whether the placenta is low-lying. Other aspects include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0007] Implementations may include one or more of the following features. In some aspects, the processor is configured to generate, as a second output of the deep learningnetwork, a detection of a cervix within the plurality of ultrasound image frames, and the processor is further configured to determine whether the placenta is low-lying based on the detection of the cervix within the plurality of ultrasound image frames. In some aspects, the processor is configured to generate a spatial-likelihood map for the placenta and the cervix, and the processor is further configured to determine whether the placenta is low-lying based on the spatial-likelihood map. In some aspects, the output representative of the determination may include the spatial-likelihood map. In some aspects, the processor is configured to define a low-lying placenta zone in the spatial-likelihood map, where the processor is further configured to determine whether the placenta is low-lying based on the low-lying placenta zone in the spatial -likelihood map. In some aspects, the processor is configured to determine if overlap between the placenta and cervix is present in the low-lying placenta zone in the spatial-likelihood map, and when the overlap is present, the processor is configured to determine that the placenta is low-lying. In some aspects, the processor is configured to calculate a distance between the placenta and the cervix based on the spatial likelihood map, where, when the overlap is not present, the processor is configured to determine if the distance is less than a threshold distance, where, when the distance is less than the threshold distance, the processor is configured to determine that the placenta is low-lying, and where, when the distance is greater than the threshold distance, the processor is configured determine that the placenta is not low-lying. In some aspects, the processor is configured to: determine, using the detection of the placenta within the plurality of ultrasound image frames, a location of the placenta within a uterus; and output, to the display in communication with the processor, a visual representation of the location of the placenta, where the determination of the location of the placenta is distinct from the determination of whether the placenta is low- lying. In some aspects, the location of the placenta within the uterus may include at least one of anterior, posterior, left lateral, or right lateral. In some aspects, the system may include the ultrasound probe. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0008] One general aspect includes a system that includes a processor configured for communication with an ultrasound probe. The processor is configured to: control the ultrasound probe to obtain a plurality of ultrasound image frames during a blind sweep protocol on a pregnant patient; provide the plurality of ultrasound image frames as an input to a deep learning network trained to detect anatomy of the pregnant patient; generate, as a first output of the deep learning network, a detection of a placenta within the plurality of ultrasound image frames; determine, using the detection of the placenta within the plurality ofultrasound image frames, a location of the placenta within a uterus; and provide, to a display in communication with the processor, a visual representation of the location of the placenta. Other aspects include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0009] Implementations may include one or more of the following features. In some aspects, to determine the location of the placenta within the uterus, the processor is configured to: divide a field of view of the ultrasound probe into a plurality of regions; and plot, on the plurality of regions, the detection of the placenta within the plurality of ultrasound image frames. In some aspects, the location of the placenta may include: one of anterior depth position or posterior depth position; and one of left lateral position or right lateral position. In some aspects, the plurality of regions is four quadrants corresponding respectively to anterior, posterior, left lateral, and right lateral. In some aspects, the detection of the placenta may include a bounding box, where, to plot the detection of the placenta on the plurality of regions, the processor is configured to plot a location of the bounding box for each ultrasound image frame in which there is the detection of the placenta, where the processor is configured to determine a density of the plotted locations of the bounding boxes on the plurality of regions, where the processor is further configured to determine a region of the plurality of regions as the location of the placenta based on the density of the plotted locations for the region being greater than the density of the plotted locations for the other regions of the plurality of regions. In some aspects, To divide the field of view into the plurality of regions, the processor is configured to: determine a depth midline dividing the field of view into two depth regions; and determine a lateral midline dividing the field of view into two lateral regions. In some aspects, the depth midline is based on a depth setting of the ultrasound probe. In some aspects, the processor is configured to generate, as a second output of the deep learning network, a detection of amniotic fluid within the plurality of ultrasound image frames, and where the depth midline is based on the detection of the amniotic fluid. In some aspects, the processor is configured to: determine, using the detection of the placenta within the plurality of ultrasound image frames, whether the placenta is low-lying; and provide, to the display in communication with the processor, an output representative of the determination of whether the placenta is low-lying, where the determination of whether the placenta is low-lying is distinct from the determination of the location of the placenta.

[0010] One general aspect includes a system. The system includes a processor configured for communication with an ultrasound probe, where the processor is configured to: receive a plurality of ultrasound image frames obtained by the ultrasound probe during a first sweep of a blind sweep protocol on a patient body; in at least one image frame of the plurality of ultrasound image frames, detecting a first anatomical object; based on the detected first anatomical object and the plurality of ultrasound image frames, determining a location of the first anatomical object; and provide an output representative of the location of the first anatomical object. The output is provided in real time or near-real time after the ultrasound probe obtains the plurality of ultrasound image frames during the blind sweep protocol.

[0011] Implementations may include one or more of the following features. Determining the location of the first anatomical object may include: for each image frame in the at least one image frame, calculating a respective center point of a respective bounding box of the detected first anatomical object; determining an average center of the respective center points, and determining a location of the average center. Calculating the location of the first anatomical object may include: receiving a second plurality of ultrasound image frames obtained by the ultrasound probe during a second sweep of a blind sweep protocol on a patient body; in at least one image frame of the second plurality of ultrasound image frames, detecting the first anatomical object; determining whether the first anatomical object detected in the at least one image frame of the sweep and the first anatomical object detected in the at least one image frame of the second sweep are in a same location; and based on the same location, determining a location of the first anatomical object. The first anatomical object is a placenta. The location of the first anatomical object may include one of anterior or posterior and / or one of left lateral or right lateral. The processor is further configured to: determine, based on the location of the first anatomical object, whether the placenta is low-lying; and if the placenta is low-lying, reporting to a user that the placenta is low-lying. The processor is further configured to: in at least one image frame of the plurality of ultrasound image frames, detect a second anatomical object; and based on the detected second anatomical object and the plurality of ultrasound image frames, revise the location of the first anatomical object. The second anatomical object is amniotic fluid or a cervix. The processor is further configured to: determine, based on the location of the first anatomical object, whether the placenta is low-lying; and if the placenta is low-lying, reporting to a user that the placenta is low-lying. The image frames of the plurality of ultrasound image frames can be 2D or 3D image frames.

[0012] In some aspects, the system may include the ultrasound probe. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0013] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. A more extensive presentation of features, details, utilities, and advantages of the ultrasound sweep anatomy detection system, as defined in the claims, is provided in the following written description of various aspects of the disclosure and illustrated in the accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Illustrative aspects of the present disclosure will be described with reference to the accompanying drawings, of which:

[0015] Figure l is a schematic, diagrammatic representation of an ultrasound imaging system, according to aspects of the present disclosure.

[0016] Figure l is a schematic diagram of a processor circuit, according to aspects of the present disclosure.

[0017] Figure 3A is a schematic, diagrammatic, cross-sectional view of a fetus surrounded by amniotic fluid within a uterus accessible through a cervix of a patient, according to aspects of the present disclosure.

[0018] Figure 3B is a schematic, diagrammatic, cross-sectional view of a fetus surrounded by amniotic fluid within a uterus accessible through a cervix of a patient, according to aspects of the present disclosure.

[0019] Figure 3C is a schematic, diagrammatic, cross-sectional view of a fetus surrounded by amniotic fluid within a uterus accessible through a cervix of a patient, according to aspects of the present disclosure.

[0020] Figure 3D is a schematic, diagrammatic, cross-sectional view of a fetus surrounded by amniotic fluid within a uterus accessible through a cervix of a patient, according to aspects of the present disclosure.

[0021] Figure 4 is a schematic, diagrammatic representation of a patient, according to aspects of the present disclosure.

[0022] Figure 5 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound sweep anatomy detection system, according to aspects of the present disclosure.

[0023] Figure 6 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound sweep anatomy detection method to triage and / or monitor a low-lying placenta, according to aspects of the present disclosure.

[0024] Figure 7A is a schematic, diagrammatic overview, in block diagram form, of a training mode for an untrained neural network, according to aspects of the present disclosure.

[0025] Figure 7B is a schematic, diagrammatic overview, in block diagram form, of an inference mode or clinical usage mode for the trained neural network, according to aspects of the present disclosure.

[0026] Figure 8 is a schematic, diagrammatic illustration, in block diagram form, of the detection of anatomy (e.g., the placenta, amniotic fluid, cervix, etc.), according to aspects of the present disclosure.

[0027] Figure 9 is an ultrasound image frame that has been annotated with bounding boxes for detected anatomy, according to aspects of the present disclosure.

[0028] Figure 10 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example placenta location determination method, for an aspect using only placenta detection and not automated detection of low-lying placenta, according to aspects of the present disclosure.

[0029] Figure 11A is a schematic, diagrammatic representation of an ultrasound image, along with a depth setting, according to aspects of the present disclosure.

[0030] Figure 11B is a schematic, diagrammatic representation of an ultrasound image, along with a sector width setting, according to aspects of the present disclosure.

[0031] Figure 11C is a schematic, diagrammatic representation of an ultrasound image that has been divided into four quadrants, according to aspects of the present disclosure.

[0032] Figure 12 is a schematic, diagrammatic representation of an ultrasound image that has been divided into the four quadrants defined in Figure 11C, according to aspects of the present disclosure.

[0033] Figure 13A is an ultrasound image that includes a placenta detection bounding box with a centroid, according to aspects of the present disclosure.

[0034] Figure 13B is an ultrasound image that includes a placenta detection bounding box 910 with a centroid 920, according to aspects of the present disclosure.

[0035] Figure 14A is a graphical representation of the locations of the placenta bounding box centroids across all horizontal sweeps (Cl, C2, and C3), according to aspects of the present disclosure.

[0036] Figure 14B is a graphical representation of the locations of the placenta bounding box centroids across all horizontal sweeps (Cl, C2, and C3), according to aspects of the present disclosure.

[0037] Figure 15A is a set of four graphical representations of the locations of the placenta bounding box centroids across multiple vertical sweeps (R0, M, L0, LI), according to aspects of the present disclosure.

[0038] Figure 15B is a set of four graphical representations of the locations of the placenta bounding box centroids across multiple vertical sweeps (R0, M, L0, LI), according to aspects of the present disclosure.

[0039] Figure 16 is an ultrasound image frame that has been annotated with bounding boxes for detected anatomy, according to aspects of the present disclosure.

[0040] Figure 17 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example placenta location determination method, for an aspect using placenta location detection based on detections of both the placenta and the amniotic fluid, according to aspects of the present disclosure.

[0041] Figure 18 is a set of three graphical representations of the locations of the placenta bounding box centroids and the amniotic fluid bounding box centroids across horizontal sweeps Cl, C2, and C3, according to aspects of the present disclosure.

[0042] Figure 19 is a set of three graphical representations of the locations of the placenta bounding box centroids and the amniotic fluid bounding box centroids across vertical sweeps R, M, and L, according to aspects of the present disclosure.

[0043] Figure 20 is a schematic, diagrammatic view of a scanning process, according to aspects of the present disclosure.

[0044] Figure 21A is a graphical representation of a horizontal scan or horizontal sweep, according to aspects of the present disclosure.

[0045] Figure 21B is a graphical representation of a horizontal scan or horizontal sweep, according to aspects of the present disclosure.

[0046] Figure 22A is a graphical representation of a vertical scan or vertical sweep, according to aspects of the present disclosure.

[0047] Figure 22B is a graphical representation of a vertical scan or vertical sweep, according to aspects of the present disclosure.

[0048] Figure 23A is a graphical representation of a set of horizontal scans Cl, C2, and C3, marked with the suspected low-lying region, according to aspects of the present disclosure.

[0049] Figure 23B is a graphical representation of a set of vertical scans R, M, and L, marked with the suspected low-lying region, according to aspects of the present disclosure.

[0050] Figure 23C is a graphical representation of a set of combined vertical and horizontal scans, marked with the suspected low-lying region, according to aspects of the present disclosure.

[0051] Figure 24A is a graphical representation of a high confidence detection region 2350 rendered as a heat map or spatial likelihood map, according to aspects of the present disclosure.

[0052] Figure 24B is a graphical representation of a high confidence detection region rendered as a heat map or spatial likelihood map, according to aspects of the present disclosure.

[0053] Figure 25A is a graphical representation of a high confidence detection region rendered as a heat map or spatial likelihood map, according to aspects of the present disclosure.

[0054] Figure 25B is a graphical representation of a high confidence detection region rendered as a heat map or spatial likelihood map, according to aspects of the present disclosure.

[0055] Figure 26A is a graphical representation of a spatial likelihood map of placenta location, according to aspects of the present disclosure.

[0056] Figure 26B is a graphical representation of a spatial likelihood map of placenta location, according to aspects of the present disclosure.

[0057] Figure 27A is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example placenta location determination method, for an aspect using placenta location detection based on detections of both the placenta and the cervix, according to aspects of the present disclosure.

[0058] Figure 27B is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example placenta location determination method, for an aspect using placenta location detection based on detections the placenta, according to aspects of the present disclosure.

[0059] Figure 28 is a schematic, diagrammatic representation of an example output screen display or the ultrasound sweep anatomy detection system, according to aspects of the present disclosure.DETAILED DESCRIPTION

[0060] In accordance with at least one aspect of the present disclosure, an ultrasound sweep anatomy detection system is provided which can identify anatomy of interest such as a low-lying placenta - a potential pregnancy complication - based on a blind sweep protocol with an ultrasound imaging probe. The ultrasound sweep anatomy detection system presents a novel approach to imaging quality control by ensuring that features such as the placenta, amniotic fluid, and / or cervix are detected, and using the detected locations of these features to, for example, determine whether the placenta is low-lying and thus potentially in need of expert care.

[0061] Ultrasound imaging is a vital component of high-quality obstetric care. In rural and under-resourced communities, the scarcity of ultrasound imaging results in a considerable gap in healthcare for pregnant mothers. Increased detection of pregnancy complications through ultrasound can allow for appropriate referral for delivery care in more- resourced centers with more highly trained providers. The present disclosure seeks to overcome this barrier to ultrasound access in a locally sustainable and resource-conscious way, using standardized blind-sweep scanning protocols combined with artificial intelligence, obviating the need for an interpreting provider (e.g., a radiologist or obstetrician) and an experienced sonographer in the remote location.

[0062] Monitoring the intrauterine structures is essential for normal fetal development and perinatal outcome. The placenta is often overlooked during obstetric routine evaluations, hampering the early detection of abnormalities. Women with low-lying placenta may be at increased risk of maternal, fetal and postnatal adverse outcomes. The present disclosure provides a computer assisted simple triaging (CAST) setting for ultrasound systems, to assist novice users (e.g., midwives with minimal training) with an algorithm for obstetrical ultrasound screening for low-lying placenta that pre-selects appropriate referral cases for experts / trained providers from community health, for delivery care in more resourced centers. Automated CAST can help rural health providers like midwives with minimal training / non- expertise to screen out benign cases, to avoid additional steps in the workflow, and to increase exam capabilities in community medicine.

[0063] The present disclosure leverages deep learning techniques for identifying / detecting the placenta and automatically diagnosing potential low-lying placenta using statistical, heuristic, machine learning, and / or image-processing techniques. The approach involves a placenta detection stage (e.g., using a Yolo model); a localization stage,for detecting the placenta positioning; and a representation stage, for depicting the probable location of the placenta within the intrauterine region by heat maps to identify potential low- lying placenta.

[0064] The anatomical data is acquired from, e.g., a 3x3 or 5x5 blind sweep protocol, which covers most of the uterus of the mother. From these images of these sweeps (cineloops or cine scans), the present disclosure provides a method to localize the placenta and then rule-in low-lying and previa cases, which may require special care from a more experienced obstetrician. By screening out benign cases, the system can avoid additional steps in the workflow , thus saving time and effort while freeing up experienced healthcare professionals, especially in poorer regions with clinician shortages.

[0065] The present disclosure provides a method to detect placenta location and potential low-lying placenta in blind sweeps using, e.g., low cost ultrasound devices, such as Philips Lumify. This method is designed to work on lot-to-moderate quality ultrasound scans captured by inexperienced / amateur sonographers (e.g., midwives with a week or less of training). Detection of pregnancy complications through ultrasound can allow for appropriate referral for delivery care in better-resourced centers with better-trained providers, and may result in lower overall costs and / or better health outcomes.

[0066] The components in the ultrasound sweep anatomy detection system block structure include:

[0067] Placenta Detector: a deep learning network that takes the cine scans as input and outputs detections (e.g., bounding boxes) for the placenta.

[0068] Placenta Localizer: from the detected placenta bounding boxes, find the distribution of position of placenta bounding boxes in the entire cine loop from each sweep.

[0069] Spatial Location Map Generator: the detected placenta (as well as potentially other features such as the amniotic fluid, cervix, etc.) from multiple cine scans and multiple frames within a cine scan are collated to generate a spatial location map that will retain the positional and temporal relationship of the placenta with respect to the mother’s abdomen. These spatial maps may then be used to rule in or out a potential low-lying placenta, or other related conditions such as placenta previa.

[0070] Aspects of the present disclosure can include features described in U.S.Provisional Application No. 63 / 540,740, filed September 27, 2023, titled “Ultrasound Imaging With Ultrasound Probe Guidance In Blind Sweep Protocol” and / or U.S. Provisional Application No. 63 / 540,755, filed September 27, 2023, titled “Ultrasound Imaging withFollow Up Sweep Guidance After Blind Sweep Procedure”, which are incorporated by reference as though fully set forth herein.

[0071] The present disclosure may improve quality of delivery and reduce mother / child mortality and complications. To sidestep barriers to visiting a hospital - cost, transportation, and distance- one solution is encouraging community medicine with easy tools like that can be handled by non-expertise people and easily available, for early identification of high-risk pregnancies and timely treatment to improve the quality of delivery. This can also address a shortage of doctors / sonographers / expertise, and limited access to health assistance and infrastructure (such as costly ultrasound devices). Lack of specialists and experienced ultrasound operators can prevent the timely detection of pathologies. By screening out benign cases, normal additional steps in the workflow can be avoided which would save time, effort, and free up experienced healthcare professionals for more urgent cases.

[0072] Literature shows that pregnant women often do not take initiative to go to a hospital on the recommendation of local experts. The visualization report generated by the present disclosure may motivate and make the risks and benefits more understandable not only to novice operators, but to naive mothers and family members, encouraging them to get follow up scans. This may result in a reduction of child and mother mortality and complications, and an increase in the quality of delivery.

[0073] The present disclosure aids substantially in the capture of high-quality ultrasound images by minimally trained users, by detecting the placenta and other related anatomy, assessing the images for low-lying placenta or other pregnancy complications, and requiring that the user repeat any sweeps that do not capture the required anatomy. Implemented on a processor in communication with an ultrasound probe, the ultrasound sweep anatomy detection system disclosed herein provides practical improvements in the quality of care available to patients in underserved areas. This improved imaging methodology transforms a process that is heavily reliant on professional experience into one that is accurate and repeatable even for minimally trained personnel, without the normally routine need to train clinicians such as emergency department personnel to recognize low-lying placenta and other prenatal conditions. This unconventional approach improves the functioning of the ultrasound imaging system, by providing reliable, repeatable imaging in hospital, office, vehicle, field, and home settings, as well as referral recommendations for patients suspected to have a pregnancy complication.

[0074] The ultrasound sweep anatomy detection system may be implemented as a process at least partially viewable on a display, and operated by a control process executing on aprocessor that accepts user inputs from a keyboard, mouse, or touchscreen interface, and that is in communication with one or more sensor probes. In that regard, the control process performs certain specific operations in response to different inputs or selections made at different times. Certain structures, functions, and operations of the processor, display, sensors, and user input systems are known in the art, while others are recited herein to enable novel features or aspects of the present disclosure with particularity.

[0075] These descriptions are provided for exemplary purposes only, and should not be considered to limit the scope of the ultrasound sweep anatomy detection system. Certain features may be added, removed, or modified without departing from the spirit of the claimed subject matter.

[0076] For the purposes of promoting an understanding of the principles of the present disclosure, reference will now be made to the aspects illustrated in the drawings, and specific language will be used to describe the same. It is nevertheless understood that no limitation to the scope of the disclosure is intended. Any alterations and further modifications to the described devices, systems, and methods, and any further application of the principles of the present disclosure are fully contemplated and included within the present disclosure as would normally occur to one skilled in the art to which the disclosure relates. In particular, it is fully contemplated that the features, components, and / or steps described with respect to one aspect may be combined with the features, components, and / or steps described with respect to other aspects of the present disclosure. For the sake of brevity, however, the numerous iterations of these combinations will not be described separately.

[0077] Figure l is a schematic, diagrammatic representation of an ultrasound imaging system 100, according to aspects of the present disclosure. The ultrasound imaging system 100 may for example be used to acquire ultrasound video sweeps, which can then be analyzed by a human clinician or an artificial intelligence to diagnose medical conditions.

[0078] The ultrasound imaging system 100 is used for scanning an area or volume of a subject’s body. A subject may include a patient of an ultrasound imaging procedure, or any other person, or any suitable living or non-living organism or structure. The ultrasound imaging system 100 includes an ultrasound imaging probe 110 in communication with a host 130 over a communication interface or link 120. The probe 110 may include a transducer array 112, a beamformer 114, a processor circuit 116, and a communication interface 118. The host 130 may include a display 132, a processor circuit 134, a communication interface 136, and a memory 138 storing subject information.

[0079] In some aspects, the probe 110 is an external ultrasound imaging device including a housing 111 configured for handheld operation by a user. The transducer array 112 can be configured to obtain ultrasound data while the user grasps the housing 111 of the probe 110 such that the transducer array 112 is positioned adjacent to or in contact with a subject’s skin. The probe 110 is configured to obtain ultrasound data of anatomy within the subject’s body while the probe 110 is positioned outside of the subject’s body for general imaging, such as for abdomen imaging, liver imaging, etc. In some aspects, the probe 110 can be an external ultrasound probe, a transthoracic probe, and / or a curved array probe.

[0080] In other aspects, the probe 110 can be an internal ultrasound imaging device and may comprise a housing 111 configured to be positioned within a lumen of a subject’s body for general imaging, such as for abdomen imaging, liver imaging, etc.. In some aspects, the probe 110 may be a curved array probe. Probe 110 may be of any suitable form for any suitable ultrasound imaging application including both external and internal ultrasound imaging.

[0081] In some aspects, aspects of the present disclosure can be implemented with medical images of subjects obtained using any suitable medical imaging device and / or modality. Examples of medical images and medical imaging devices include x-ray images (angiographic images, fluoroscopic images, images with or without contrast) obtained by an x-ray imaging device, computed tomography (CT) images obtained by a CT imaging device, positron emission tomography-computed tomography (PET-CT) images obtained by a PET- CT imaging device, magnetic resonance images (MRI) obtained by an MRI device, singlephoton emission computed tomography (SPECT) images obtained by a SPECT imaging device, optical coherence tomography (OCT) images obtained by an OCT imaging device, and intravascular photoacoustic (IVPA) images obtained by an IVPA imaging device. The medical imaging device can obtain the medical images while positioned outside the subject body, spaced from the subject body, adjacent to the subject body, in contact with the subject body, and / or inside the subject body.

[0082] For an ultrasound imaging device, the transducer array 112 emits ultrasound signals towards an anatomical object 105 of a subject and receives echo signals reflected from the object 105 back to the transducer array 112. The ultrasound transducer array 112 can include any suitable number of acoustic elements, including one or more acoustic elements and / or a plurality of acoustic elements. In some instances, the transducer array 112 includes a single acoustic element. In some instances, the transducer array 112 may include an array of acoustic elements with any number of acoustic elements in any suitableconfiguration. For example, the transducer array 112 can include between 1 acoustic element and 10000 acoustic elements, including values such as 2 acoustic elements, 4 acoustic elements, 36 acoustic elements, 64 acoustic elements, 128 acoustic elements, 500 acoustic elements, 812 acoustic elements, 1000 acoustic elements, 3000 acoustic elements, 8000 acoustic elements, and / or other values both larger and smaller. In some instances, the transducer array 112 may include an array of acoustic elements with any number of acoustic elements in any suitable configuration, such as a linear array, a planar array, a curved array, a curvilinear array, a circumferential array, an annular array, a phased array, a matrix array, a one-dimensional (ID) array, a 1.x dimensional array (e.g., a 1.5D array), or a two- dimensional (2D) array. The array of acoustic elements (e.g., one or more rows, one or more columns, and / or one or more orientations) can be uniformly or independently controlled and activated. The transducer array 112 can be configured to obtain one-dimensional, two- dimensional, and / or three-dimensional images of a subject’s anatomy. In some aspects, the transducer array 112 may include a piezoelectric micromachined ultrasound transducer (PMUT), capacitive micromachined ultrasonic transducer (CMUT), single crystal, lead zirconate titanate (PZT), PZT composite, other suitable transducer types, and / or combinations thereof.

[0083] The object 105 may include any anatomy or anatomical feature, such as a kidney, liver, and / or any other anatomy of a subject. The present disclosure can be implemented in the context of any number of anatomical locations and tissue types, including without limitation, organs including the liver, kidneys, gall bladder, pancreas, lungs; ducts; intestines; nervous system structures including the brain, dural sac, spinal cord and peripheral nerves; the urinary tract; as well as valves within the blood vessels, blood, abdominal organs, and / or other systems of the body. In some aspects, the object 105 may include malignancies such as tumors, cysts, lesions, hemorrhages, or blood pools within any part of human anatomy. The anatomy may be a blood vessel, as an artery or a vein of a subject’s vascular system, including cardiac vasculature, peripheral vasculature, neural vasculature, renal vasculature, and / or any other suitable lumen inside the body. In addition to natural structures, the present disclosure can be implemented in the context of man-made structures such as, but without limitation, heart valves, stents, shunts, filters, implants and other devices.

[0084] The beamformer 114 is coupled to the transducer array 112. The beamformer 114 controls the transducer array 112, for example, for transmission of the ultrasound signals and reception of the ultrasound echo signals. In some aspects, the beamformer 114 may apply a time-delay to signals sent to individual acoustic transducers within an array in the transducer112 such that an acoustic signal is steered in any suitable direction propagating away from the probe 110. The beamformer 114 may further provide image signals to the processor circuit 116 based on the response of the received ultrasound echo signals. The beamformer 114 may include multiple stages of beamforming. The beamforming can reduce the number of signal lines for coupling to the processor circuit 116. In some aspects, the transducer array 112 in combination with the beamformer 114 may be referred to as an ultrasound imaging component.

[0085] The processor 116 is coupled to the beamformer 114. The processor 116 may also be described as a processor circuit, which can include other components in communication with the processor 116, such as a memory, beamformer 114, communication interface 118, and / or other suitable components. The processor 116 may include a central processing unit (CPU), a graphical processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a controller, a field programmable gate array (FPGA) device, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 116 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The processor 116 is configured to process the beamformed image signals. For example, the processor 116 may perform filtering and / or quadrature demodulation to condition the image signals. The processor 116 and / or 134 can be configured to control the array 112 to obtain ultrasound data associated with the object 105.

[0086] The communication interface 118 is coupled to the processor 116. The communication interface 118 may include one or more transmitters, one or more receivers, one or more transceivers, and / or circuitry for transmitting and / or receiving communication signals. The communication interface 118 can include hardware components and / or software components implementing a particular communication protocol suitable for transporting signals over the communication link 120 to the host 130. The communication interface 118 can be referred to as a communication device or a communication interface module.

[0087] The communication link 120 may be any suitable communication link. For example, the communication link 120 may be a wired link, such as a universal serial bus (USB) link or an Ethernet link. Alternatively, the communication link 120 may be a wireless link, such as an ultra-wideband (UWB) link, an Institute of Electrical and Electronics Engineers (IEEE) 802.11 WiFi link, or a Bluetooth link.

[0088] At the host 130, the communication interface 136 may receive the image signals. The communication interface 136 may be substantially similar to the communication interface 118. The host 130 may be any suitable computing and display device, such as a workstation, a personal computer (PC), a laptop, a tablet, or a mobile phone.

[0089] The processor 134 is coupled to the communication interface 136. The processor 134 may also be described as a processor circuit, which can include other components in communication with the processor 134, such as the memory 138, the communication interface 136, an optional speaker 139, and / or other suitable components. The processor 134 may be implemented as a combination of software components and hardware components. The processor 134 may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a controller, an FPGA device, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 134 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The processor 134 can be configured to generate image data from the image signals received from the probe 110. The processor 134 can apply advanced signal processing and / or image processing techniques to the image signals. In some aspects, the processor 134 can form a three-dimensional (3D) volume image from the image data. In some aspects, the processor 134 can perform real-time processing on the image data to provide a streaming video of ultrasound images of the object 105. In some aspects, the host 130 includes a beamformer. For example, the processor 134 can be part of and / or otherwise in communication with such a beamformer. The beamformer in the in the host 130 can be a system beamformer or a main beamformer (providing one or more subsequent stages of beamforming), while the beamformer 114 is a probe beamformer or micro-beamformer (providing one or more initial stages of beamforming).

[0090] The memory 138 is coupled to the processor 134. The memory 138 may be any suitable storage device, such as a cache memory (e.g., a cache memory of the processor 134), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory, solid state memory device, hard disk drives, solid state drives, other forms of volatile and non-volatile memory, or a combination of different types of memory.

[0091] The memory 138 can be configured to store subject information, measurements, data, or files relating to a subject’s medical history, history of procedures performed, anatomical or biological features, characteristics, or medical conditions associated with a subject, computer readable instructions, such as code, software, or other application, as well as any other suitable information or data. The memory 138 may be located within the host 130. Subject information may include measurements, data, files, other forms of medical history, such as but not limited to ultrasound images, ultrasound videos, and / or any imaging information relating to the subject’s anatomy. The subject information may include parameters related to an imaging procedure such as an anatomical scan window, a probe orientation, and / or the subject position during an imaging procedure. The memory 138 can also be configured to store information related to the training and implementation of machine learning algorithms (e.g., neural networks) and / or information related to implementing image recognition algorithms for detecting / segmenting anatomy, image quantification algorithms, and / or image acquisition guidance algorithms, including those described herein.

[0092] The display 132 is coupled to the processor circuit 134. The display 132 may be a monitor or any suitable display. The display 132 is configured to display the ultrasound images, image videos, and / or any imaging information of the object 105.

[0093] The ultrasound imaging system 100 may be used to assist a sonographer in performing an ultrasound scan. The scan may be performed in a point-of-care setting. In some instances, the host 130 is a console or movable cart. In some instances, the host 130 may be a mobile device, such as a tablet, a mobile phone, or portable computer. During an imaging procedure, the ultrasound system can acquire an ultrasound image of a particular region of interest within a subject’s anatomy. The ultrasound imaging system 100 may then analyze the ultrasound image to identify various parameters associated with the acquisition of the image such as the scan window, the probe orientation, the subject position, and / or other parameters. The ultrasound imaging system 100 may then store the image and these associated parameters in the memory 138. At a subsequent imaging procedure, the ultrasound imaging system 100 may retrieve the previously acquired ultrasound image and associated parameters for display to a user which may be used to guide the user of the ultrasound imaging system 100 to use the same or similar parameters in the subsequent imaging procedure, as will be described in more detail hereafter.

[0094] In some aspects, the processor 134 may utilize deep learning-based prediction networks to identify parameters of an ultrasound image, including an anatomical scan window, probe orientation, subject position, identify and location of anatomical features,and / or other parameters. In some aspects, the processor 134 may receive metrics or perform various calculations relating to the region of interest imaged or the subject’s physiological state during an imaging procedure. These metrics and / or calculations may also be displayed to the sonographer or other user via the display 132.

[0095] In some aspects, the host 130 may also include a speaker 180. The speaker 180 may for example be used to provide advisory tones, beeps, or other auditory feedback to the user.

[0096] Before continuing, it should be noted that the examples described above are provided for purposes of illustration, and are not intended to be limiting. Other devices and / or device configurations may be utilized to carry out the operations described herein.

[0097] Figure l is a schematic diagram of a processor circuit 250, according to aspects of the present disclosure. The processor circuit 250 may be implemented in the ultrasound imaging system 100, or other devices or workstations (e.g., third-party workstations, network routers, etc.), or on a cloud processor or other remote processing unit, as necessary to implement the method. As shown, the processor circuit 250 may include a processor 260, a memory 264, and a communication module 268. These elements may be in direct or indirect communication with each other, for example via one or more buses.

[0098] The processor 260 may include a central processing unit (CPU), a digital signal processor (DSP), a controller, or any combination of general-purpose computing devices, reduced instruction set computing (RISC) devices, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other related logic devices, including mechanical and quantum computers. The processor 260 may also comprise another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 260 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0099] The memory 264 may include a cache memory (e.g., a cache memory of the processor 260), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory, solid state memory device, hard disk drives, other forms of volatile and nonvolatile memory, or a combination of different types of memory. In an aspect, the memory 264 includes a non-transitory computer-readable medium. The memory 264 may storeinstructions 266. The instructions 266 may include instructions that, when executed by the processor 260, cause the processor 260 to perform the operations described herein. Instructions 266 may also be referred to as code. The terms “instructions” and “code” should be interpreted broadly to include any type of computer-readable statement(s). For example, the terms “instructions” and “code” may refer to one or more programs, routines, subroutines, functions, procedures, etc. “Instructions” and “code” may include a single computer-readable statement or many computer-readable statements.

[0100] The communication module 268 can include any electronic circuitry and / or logic circuitry to facilitate direct or indirect communication of data between the processor circuit 250, and other processors or devices. In that regard, the communication module 268 can be an input / output (I / O) device. In some instances, the communication module 268 facilitates direct or indirect communication between various elements of the processor circuit 250 and / or the ultrasound imaging system 100. The communication module 268 may communicate within the processor circuit 250 through numerous methods or protocols. Serial communication protocols may include but are not limited to United States Serial Protocol Interface (US SPI), Inter-Integrated Circuit (I2C), Recommended Standard 232 (RS- 232), RS-485, Controller Area Network (CAN), Ethernet, Aeronautical Radio, Incorporated 429 (ARINC 429), MODBUS, Military Standard 1553 (MIL-STD-1553), or any other suitable method or protocol. Parallel protocols include but are not limited to Industry Standard Architecture (ISA), Advanced Technology Attachment (ATA), Small Computer System Interface (SCSI), Peripheral Component Interconnect (PCI), Institute of Electrical and Electronics Engineers 488 (IEEE-488), IEEE-1284, and other suitable protocols. Where appropriate, serial and parallel communications may be bridged by a Universal Asynchronous Receiver Transmitter (UART), Universal Synchronous Receiver Transmitter (USART), or other appropriate subsystem.

[0101] External communication (including but not limited to software updates, firmware updates, model sharing between the processor and central server, or readings from the ultrasound imaging system 100) may be accomplished using any suitable wireless or wired communication technology, such as a cable interface such as a universal serial bus (USB), micro USB, Lightning, or FireWire interface, Bluetooth, Wi-Fi, ZigBee, Li-Fi, or cellular data connections such as 2G / GSM (global system for mobiles) , 3G / UMTS (universal mobile telecommunications system), 4G, long term evolution (LTE), WiMax, or 5G. For example, a Bluetooth Low Energy (BLE) radio can be used to establish connectivity with a cloud service, for transmission of data, and for receipt of software patches. The controller may beconfigured to communicate with a remote server, or a local device such as a laptop, tablet, or handheld device, or may include a display capable of showing status variables and other information. Information may also be transferred on physical media such as a USB flash drive or memory stick.

[0102] Figure 3A is a schematic, diagrammatic, cross-sectional view of a fetus 200 surrounded by amniotic fluid 210 within a uterus 220 accessible through a cervix 230 of a patient 300, according to aspects of the present disclosure. The placenta 240 is in a posterior position, e.g., toward the back of the abdomen of the patient 300. Automated determination of the position of the placenta 240 is an object of the present disclosure.

[0103] Figure 3B is a schematic, diagrammatic, cross-sectional view of a fetus 200 surrounded by amniotic fluid 210 within a uterus 220 accessible through a cervix 230 of a patient, according to aspects of the present disclosure. The placenta 240 is in an anterior position, e.g., toward the front of abdomen of the patient. Automated determination of the position of the placenta 240 is an object of the present disclosure.

[0104] Figure 3C is a schematic, diagrammatic, cross-sectional view of a fetus 200 surrounded by amniotic fluid 210 within a uterus 220 accessible through a cervix 230 of a patient, according to aspects of the present disclosure. The placenta 240 is in low-lying position, e.g., close to the cervix 230. Clinically, a placenta located within 2 cm of the cervix may be considered low-lying. Low-lying placenta can be a pregnancy complication, and thus, automated detection of low-lying placenta is an objective of the present disclosure.

[0105] Figure 3D is a schematic, diagrammatic, cross-sectional view of a fetus 200 surrounded by amniotic fluid 210 within a uterus 220 accessible through a cervix 230 of a patient, according to aspects of the present disclosure. The placenta 240 is in placenta previa condition, e.g., low-lying and blocking the cervix 230. Placenta previa can be a pregnancy complication, and thus, automated detection of low-lying placenta (including placenta previa) is an object of the present disclosure. In some aspects, placenta previa may be identified explicitly if present.

[0106] Figure 4 is a schematic, diagrammatic representation of a patient 300, according to aspects of the present disclosure. Visible on the abdomen 310 of the patient 300 is a desired sweep pattern 320 intended to capture images of desired features of the patient’s anatomy. The sweep pattern 320 includes multiple vertical sweep lines 330 and multiple horizontal sweep lines 340. Each sweep line 330, 340 represents a desired path for one imaging sweep of the abdomen 310. In the example shown in Figure 4, the sweep pattern includes three vertical sweep lines 330 labeled L (patient’s left), M (patient’s middle), and R(patient’s right), all in an upward direction with respect to the patient, and three horizontal sweep lines 340 labeled Cl (bottom), C2 (middle), and C3 (top), all in a right-to-left direction with respect to the patient. However, it is understood that a sweep pattern 320 may include more or fewer sweep lines 330, including vertical sweep lines 330, horizontal sweep lines 330, or combinations thereof, in any combination of upward, downward, left, or right directions, based on the patient’s fundal height. For example, if the patient’s belly is bigger in size, more sweeps may be needed. Furthermore, a sweep pattern 320 may cover other portions of the patient’s body, including but not limited to the head, neck, spine, limbs, etc. Examples of blind sweep protocol include but are not limited to obstetric sweep imaging (OSI), volume sweep imaging (VSI), 6-Stage, Fetal Age Machine Learning Initiative (FAMLI), and Philips.

[0107] These sweep patterns represent desired probe motion information, including desired positions, a desired velocity or velocities, and / or a desired orientation of the ultrasound probe while the ultrasound probe is obtaining a plurality of ultrasound image frames during the sweep. It is noted that the desired sweep patterns or blind sweep protocols stored in a memory of the processor may include only vertical sweeps, only horizontal sweeps, may include a grid (e.g., 3x3, 5x5, etc.) of vertical and horizontal sweeps, and may also include associated parameters such as desired probe motion (e.g., positions, velocities, and / or orientations) stored in the memory (e.g., blind sweep protocol 440 in Fig. 5). Depending on the implementation, sweeps may include curved, diagonal, and other types of sweeps. The protocols and their associated parameters can for example be based on standards established by authorities in the field (physician organizations, sonographer organizations, etc.), published in scholarly joumals / textbooks, etc.

[0108] Figure 5 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound sweep anatomy detection system 400, according to aspects of the present disclosure. An ultrasound probe 110 operated by a novice user 410 performs a blind sweep protocol 440 on the body of a patient 300 and sends ultrasound imaging data to a host 130, such as a tablet, smartphone, ultrasound cart, etc. In step 415, the host 130 generates ultrasound images using the ultrasound image data obtained by the ultrasound probe 110. The host 130 can control the ultrasound probe 110 to obtain the ultrasound image data (e.g., the host 130 establishes communication with the ultrasound probe 110, the host 130 sends control signals to start and / or stop acquisition of ultrasound image data, the host 130 sends power signals to power the ultrasound probe 110, etc.).

[0109] In step 420, the host 130 uses the ultrasound images generated in step 415 and / or the ultrasound image data used to generate the ultrasound images to determine if the patient’s placenta is low-lying, as described in more detail below. In step 435, the host 130 uses the ultrasound images generated in step 415 and / or the ultrasound image data used to generate the ultrasound images to determine the location of patient’s placenta (e.g., anterior / posterior, left / right lateral, etc.), as described in more detail below. In step 430, the host generates a visual representation and outputs the visual representation on a display (e.g., display 132 of Fig. 1), showing the position of the placenta (as determined in step 435) and / or an indication of whether the placenta is low-lying (as determined in step 420), as described in more detail below.

[0110] In step 450, the host 130 determines whether the planned path of the blind sweep protocol 440 has been followed adequately. If no, execution proceeds to steps 430 and 460, wherein the host 130 provides feedback (e.g., audio feedback through a speaker and / or visual feedback from a display) to repeat one or multiple sweeps of the blind sweep protocol 440. In some aspects, the determination in step 450 can be based on the results of analysis in step 420 of whether the patient’s placenta is low-lying and / or in step 435 of the placenta’s location. For example, if there is insufficient ultrasound image data to perform the determinations in step 420 and / or step 435 or if the ultrasound image data that has been obtained is not suitable to perform the determinations in step 420 and / or step 435, then step 450 can determine that the one or multiple sweeps of the blind sweep protocol 440 has not been completed correctly. The one or multiple sweeps that need to be performed again can be communicated to the user through visual or audio feedback in steps 430 and 460.

[0111] In some aspects, the planned path determined in step 450 of the blind sweep protocol 440 can be used by the host 130 to determine whether the patient’s placenta is low lying in step 420 and / or the location of the placenta in step 435. For example, the planned path determined in step 450 can provide absolute or relative spatial information (e.g., vertical location, horizontal location) about the ultrasound image data obtained by the ultrasound probe and / or the ultrasound images generated by the host. For example, referring again to Fig. 4, the planned path determined in step 450 can indicate that the M sweep is horizontally located (left-right direction in Fig. 4) between the R sweep and the L sweep. The host 130 can use this spatial information from step 450 to process the ultrasound image data and / or ultrasound images to determine whether the patient’s placenta is low lying in step 420 and / or the location of the placenta in step 435.

[0112] Based on the visual representation 430, the novice user 410 makes a triage decision to either rule out or rule in low-lying placenta. In step 470, the novice user 410 rules out low-lying placenta, and in step 480, the novice user 410 performs or recommends normal periodic evaluations for the patient 300. In step 490, the novice user 410 rules in low-lying placenta, and in step 495, the novice user 410 refers the patient to an expert user such as an experienced sonographer, radiologist, obstetrician, or other physician for follow-up imaging and diagnosis.

[0113] Flow diagrams and block diagrams are provided herein for exemplary purposes; a person of ordinary skill in the art will recognize myriad variations that nonetheless fall within the scope of the present disclosure. For example, any of the steps described herein may optionally include an output to a user of information relevant to the step, and may thus represent an improvement in the user interface over existing art by providing information not otherwise available. Similarly, block diagrams may show a particular arrangement of components, modules, services, steps, processes, or layers, resulting in a particular data flow. It is understood that some aspects of the systems disclosed herein may include additional components, that some components shown may be absent from some aspects, and that the arrangement of components may be different than shown, resulting in different data flows while still performing the methods described herein. The logic of flow diagrams may be shown as sequential. However, similar logic could be parallel, massively parallel, object oriented, real-time, event-driven, cellular automaton, or otherwise, while accomplishing the same or similar functions. In order to perform the methods described herein, a processor may divide each of the steps described herein into a plurality of machine instructions, and may execute these instructions at the rate of several hundred, several thousand, several million, or several billion per second, in a single processor or across a plurality of processors. Such rapid execution may be necessary in order to execute the method in real time or near-real time as described herein. For example, to provide an assessment of the placenta location and / or an indication of whether the placenta is low-lying, the system may need to generate the visual representation 430 within one second of completing the blind sweep protocol.

[0114] Figure 6 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound sweep anatomy detection method 420 to triage and / or monitor a low-lying placenta, according to aspects of the present disclosure. Images from a sweep of the blind sweep protocol can be organized into an image sequence known as a cine scan or cineloop (e.g., multiple image frames making up a video segment).The host thus receives or assembles cine scans Cl, C2, C3, R, M, and L, corresponding to the blind abdominal sweeps Cl, C2, C3, R, M, and L shown in Figure 4.

[0115] In step 510, these cine scans are fed into an object detector (e.g., a trained neural network) to detect the presence of and, if present, the location of the anatomical region of interest (e.g., placenta, cervix, etc.) in each ultrasound image frame of the cine scan. Depending on the implementation, execution then proceeds to either or both of steps 520 or 540.

[0116] In step 520, the method includes determining (e.g., using heuristics and / or a trained classifier) the placenta location within the uterus, and generating an output visual representation 430 that includes a visual or text representation of the position 530 of the placenta. The positioned of the placenta can include only the depth position (anterior or posterior), only the lateral position (left lateral, right lateral), or a combination of the depth position and the lateral position (e.g., one of the quadrants: anterior left lateral, anterior right lateral, posterior left lateral, or posterior right lateral) within the mother’s abdomen.

[0117] In step 540, the method includes (e.g., using a heuristic anomaly detector and / or a trained classifier) assessing whether or not the placenta is low-lying, and generating an output visual representation 550 including the information that either the placenta is not low-lying and no placenta previa is suspected, OR that the placenta is suspected to be low-lying and / or in a placenta previa condition.

[0118] Figure 7A is a schematic, diagrammatic overview, in block diagram form, of a training mode 700 for an untrained neural network 710a, according to aspects of the present disclosure. In the example shown in Figure 7A, a set of training data 705a includes ultrasound cineloops of probe sweeps annotated with the corresponding anatomy (placenta, amniotic fluid, cervix, etc.). The training data 705a is fed into an untrained neural network 710a in an iterative training process that will be familiar to a person of ordinary skill in the art.

[0119] The parameters of a network model (e.g., the weights at each artificial neuron) are initialized with initial values A that may be random values or with results from training on prior datasets. In an iterative process, the network is used to make detection inferences on the training images, the results are compared with the ground truth annotations, and an optimizer is used to adjust the network parameters B until a metric of accuracy is maximized.

[0120] Thus, an output of this training process 700 is a trained neural network 710b, wherein the parameters B (e.g., weights) are optimized for generating accurate bounding boxes for the anatomy imaged in the training data 705a.

[0121] Figure 7B is a schematic, diagrammatic overview, in block diagram form, of an inference mode or clinical usage mode 704 for the trained neural network 710b, according to aspects of the present disclosure. In clinical usage, an ultrasound video, cineloop, or cine sweep 720 of the blind sweep is fed to the trained and validated neural network 710b for analysis. The trained and validated neural network 710b then produces, as an output, anatomy detection bounding boxes 740 for each image (or the entire sweep). In some aspects, a confidence value can be determined as a normalized value in the range [0-1], where 0 indicates lowest confidence, and 1 indicates highest confidence that the detection is correct.

[0122] Figure 8 is a schematic, diagrammatic illustration, in block diagram form, of the detection of anatomy (e.g., the placenta, amniotic fluid, cervix, etc.), according to aspects of the present disclosure. A cineloop 810 comprising multiple frames 820 is fed into a trained object detector 830.

[0123] The object detector 830 may implement or include any suitable type of learning network. For example, in some aspects, the object detector 830 could include a neural network, such as a convolutional neural network (CNN). In addition, the convolutional neural network may additionally or alternatively be an encoder-decoder type network, or may utilize a backbone architecture based on other types of neural networks, such as an object detection network, classification network, etc. One example backbone network is the Darknet YOLO backbone, (e.g., Yolov3) which can be used for object detection. The CNN may for example include a set of N convolutional layers, where N may be any positive integer. Fully connected layers can be omitted when the CNN is a backbone. The CNN may also include max pooling layers and / or activation layers. Each convolutional layer may include a set of filters configured to extract features from an input (e.g., from a frame of the ultrasound video). The value N and the size of the filters may vary depending on the aspects. In some instances, the convolutional layers may utilize any non-linear activation function, such as for example a leaky rectified non-linear (ReLU) activation function and / or batch normalization. The max pooling layers gradually shrink the high-dimensional output to a dimension of the desired result (e.g., bounding boxes of a detected feature). Outputs of detection network may include numerous bounding boxes, with most having very low confidence scores and thus being filtered out or ignored. Fully connected layers may be referred to as perception or perceptive layers. In some aspects, perception / perceptive and / or fully connected layers may be found in object detector 830 (e.g., a multi-layer perceptron).

[0124] These descriptions are included for exemplary purposes; a person of ordinary skill in the art will appreciate that other types of learning models, with features similar to ordissimilar to those described above, may be used instead or in addition, without departing from the spirit of the present disclosure.

[0125] Outputs of the object detector 830 may include an annotated cineloop 840 made up of a plurality of annotated image frames 842, possibly including per-frame metrics 845 such as the confidence level of the detections.

[0126] The systems and methods disclosed herein are broadly applicable to different types of features, and can for example draw boxes around the placenta, amniotic fluid, cervix, fetus, or other anatomical features depending on the implementation. The object detector can be one class or multi-class, depending how the model is built. If another detector is trained separately, then both models can be run separately (e.g., one model for each feature type). Otherwise, multiple feature classes can be identified, and enclosed in detection boxes, at the same time. The ML model for placenta detection can use exactly the same structure as a model for cervix detection. One can either train / run a single detector that detects multiple feature types (a "multi-class detector") and provides their locations as an output, along with the confidence score and feature type (class) of each detection. Alternatively, one could run several "single-class" detectors, each trained to detect a single feature type / class. These separate single-class detectors may have the same architecture (e.g., layers and connections), but would have been trained with different data (e.g., different images and / or annotations).

[0127] Figure 9 is an ultrasound image frame 900 that has been annotated with bounding boxes for detected anatomy, according to aspects of the present disclosure. Visible is a bounding box 910 showing the detected location of the placenta. The centroid 920 of the placenta detection box is marked. The centroid 920 can also be referred to as a center point or a midpoint. While the centroid 920 is given as an example of a location used to identify / locate the bounding box 910, it is understood that any other location (e.g., a comer, a location along an edge, a location within the bounding box, etc.) or combination of locations can be used in other instances. Also visible is a bounding box 930 showing the detected location of amniotic fluid. The amniotic fluid can also be referred to as an amniotic fluid pocket. A detection of amniotic fluid can be used to determine the midline separating anterior and posterior in an ultrasound image frame, as described below.

[0128] Figure 10 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example placenta location determination method 520, for an aspect using only placenta detection and not automated detection of low-lying placenta, according to aspects of the present disclosure. The patient’s gestational age 1010 (indicative of expected fetus size) is used to select a depth setting and sector width setting for theultrasound probe, which in turn determines the fan beam coordinates 1015 for the captured ultrasound images. Gestational age may for example be stored and / or determined as part of the patient case, which is automatically provided to select fan beam coordinates when the patient’s case data is loaded into the host to control the ultrasound image probe to start the blind sweep protocol. Alternatively, the gestational age may be provided by a user input or calculated based on a user input. Execution then proceeds, in parallel, to steps 1020 and 1025.

[0129] In step 1020, the method 520 includes determining the depth midline by dividing the fan beam into two regions (e.g., anterior and posterior) based on the depth setting, to define the two depth quadrants. Execution then proceeds to step 1040.

[0130] In step 1025, the method 520 includes determining the lateral midline by dividing the fan beam into two regions based on the sector width setting, to define the two lateral quadrants. Execution then proceeds to step 1040.

[0131] In step 1035, the method 520 includes determining a midpoint for the placenta bounding boxes 1030 in each image frame where a placenta was detected. Execution then proceeds to step 1040.

[0132] In step 1040, the method 520 includes calculating the density of the placenta bounding box midpoints in each of the four sectors (anterior left, anterior right, posterior left, and posterior right). Execution then proceeds, in parallel, to steps 1045 and 1050.

[0133] In step 1045, the method 520 includes determining the depth position of the placenta using the bounding box midpoint density in all scans / sweeps (e.g., Cl, C2, C3, L, M, and R). Execution then proceeds to step 1055.

[0134] In step 1050, the method 520 includes determining the lateral position of the placenta using the bounding box midpoint density in all vertical scans / sweeps (e.g., L, M, and R). Execution then proceeds to step 1055.

[0135] In step 1055, the method 520 includes generating an output (e.g., a text or graphical output on a display) indicative of the placenta location, including and based on the depth position and the lateral position. Depending on the implementation, the output may include the depth position (posterior / anterior), the lateral position (left lateral / right lateral), a combination of the depth position and lateral position (e.g., the quadrant in which the placenta is determined to be located), and / or detailed information about the size, shape, location, and orientation of the placenta.

[0136] Figure 11A is a schematic, diagrammatic representation of an ultrasound image 1100, along with a depth setting 1110 and reference point 1115, according to aspects of thepresent disclosure. The ultrasound image 1110 can have a fan-shaped beam illustrated in Figure 11 A. The shape of the ultrasound image 1110 and / or the fan beam shape corresponds to the field of the view of the ultrasound probe for a given ultrasound image. It is understood that different types of ultrasound image probes (curved array, linear array, planar array, etc.) can have different beam shapes, and thus different shapes for the field of view and the ultrasound image 1110. The depth setting 1110 can be the depth of the field of the view of the ultrasound probe. For example, the depth setting 1110 can be defined the distance between ultrasound in the upper edge of the ultrasound image (proximate to the ultrasound probe) and the opposite, lower edge of field of view, which is the farthest location in depth that is depicted in the ultrasound image. This distance extends in the up and down direction in the ultrasound image. The depth setting 1110 can be automatic / pre-determined setting for blind sweep protocol, which depends, for example, on the gestational age, expected size of the fetus, stage / trimester of pregnancy, etc. In some instances, the depth setting 1110 can be manually selected and / or changed by the user. The depth setting 1110 can be used to determine a horizontal midline or depth midline 1125 that divides the image 1100 into an anterior portion 1120 (e.g., nearer to the ultrasound probe and / or to the mother’s belly) and a posterior portion 1130 (e.g., farther from the probe and / or nearer to the mother’s spine). For example, the depth midline 1125 can extend horizontally / laterally at a location corresponding to one-half of the depth setting 1110.

[0137] Figure 11B is a schematic, diagrammatic representation of an ultrasound image 1140, along with a sector width setting 1150, according to aspects of the present disclosure. The sector width setting 1150 can be the width of the field of the view of the ultrasound probe. For example, the sector width setting 1150 can be defined the lateral distance between ultrasound in the left edge of the ultrasound image and the opposite, right edge of field of view. This distance extends in the left and right direction in the ultrasound image. The sector width setting 1150 can be automatic / pre-determined setting for blind sweep protocol, which depends, for example, on the gestational age, expected size of the fetus, stage / trimester of pregnancy, etc. In some instances, the sector width setting 1150 can be manually selected and / or changed by the user. The sector width setting 1150 can be used to determine a vertical midline or lateral midline 1165 that divides the image 1140 into a right lateral portion (e.g., toward the mother’s right) and a left lateral portion 1170 (e.g., toward the mother’s left).

[0138] Figure 11C is a schematic, diagrammatic representation of an ultrasound image 1180 that has been divided into four quadrants, according to aspects of the present disclosure. A depth midline 1182 and vertical or lateral midline 1184 define the four quadrants: ananterior, right lateral quadrant 1192, an anterior, left lateral quadrant 1194, a posterior, right lateral quadrant 1198, and a posterior, left lateral quadrant 1196. These four quadrants may be used to describe the position of the placenta and help the user (e.g., a midwife) assess whether there is a risk of low-lying placenta.

[0139] Figure 12 is a schematic, diagrammatic representation of two-dimensional (x, y) plot 1200 that has been divided into the four quadrants defined in Figure 11C, according to aspects of the present disclosure. The plot 1220 thus includes the anterior, right lateral quadrant 1192, an anterior, left lateral quadrant 1194, a posterior, right lateral quadrant 1198, and a posterior, left lateral quadrant 1196. The x-axis can be representative of a lateral position and comprises values in, e.g. pixels (alternatively, can be centimeters, etc.). The y- axis can be representative of a depth position and comprises values in, e.g., pixels (alternatively, can be centimeters, etc.). The x-axis can extend at least as much as the depth setting 1110 and can be longer. The y-axis can extend at least as much as the sector width setting (1150) can be longer. An outline of an ultrasound image frame 1205 (similar to the ultrasound images 1100, 1140, 1180 from Figs. 11 A-l 1C) is overlaid on the plot 1200 to illustrate the spatial relationship between the plot 1200 and the fan beam shape / coordinates.

[0140] Fig. 14A, Fig. 14B, Fig. 15A, Fig. 15B, Fig. 18, and Fig. 19 include graphical representations based on the plot 1200 and includes lateral positions along the x-axis (e.g., in pixels, centimeters, etc.) and depth positions along the y-axis (e.g., in pixels, centimeters,). The graphical representations in Fig. 14A, Fig. 14B, Fig. 15A, Fig. 15B, Fig. 18, and Fig. 19 can be the result of many ultrasound image frames overlaid on top of one another. In particular, the centroids of the anatomy bounding boxes are shown when many ultrasound image frames (which include the anatomy bounding boxes) are overlaid on top of one another. However, the graphical representations in in Fig. 14A, Fig. 14B, Fig. 15 A, Fig. 15B, Fig. 18, and Fig. 19 omit the outline of the fan beam shape of the ultrasound image frame, which is included in Fig. 12.

[0141] Figure 13A is an ultrasound image 1300 that includes a placenta detection bounding box 910 with a centroid 920, according to aspects of the present disclosure. Because the centroid 920 is in the upper half of the image (closer to the probe), the placenta is identified as being in an anterior position (closer to the patient’s belly and the probe, which is moved along the patient’s belly during the blind sweep protocol).

[0142] Figure 13B is an ultrasound image 1310 that includes a placenta detection bounding box 910 with a centroid 920, according to aspects of the present disclosure. Because the centroid 920 is in the lower half of the image, the placenta is identified as beingin a posterior position (closer to patient’s spine, farther from patient’s belly and the ultrasound probe). Because Figures 13 A and 13B each show a single frame from a cine loop, only the anterior / posterior position of the placenta is indicated. To determine the lateral position, the density of placenta centroid / midpoints for the vertical sweep cine loops (L, M, R) can be considered, as described herein.

[0143] Figure 14A is a graphical representation 1400 of the locations of the placenta bounding box centroids 1410 across all horizontal sweeps (Cl, C2, and C3), according to aspects of the present disclosure. As discussed above, a depth midline 1125 separates the graph into an anterior portion 1120 and a posterior portion 1130. As can be seen in the graph, all of the centroids 1410 are located above the depth midline 1125, thus clearly indicating that the placenta is in an anterior position.

[0144] Figure 14B is a graphical representation 1420 of the locations of the placenta bounding box centroids 1410 across all horizontal sweeps (Cl, C2, and C3), according to aspects of the present disclosure. As discussed above, a depth midline 1125 separates the graph into an anterior portion 1120 and a posterior portion 1130. As can be seen in the graph, all of the centroids 1410 are located below the depth midline 1125, thus clearly indicating that the placenta is in a posterior position.

[0145] Figure 15A is a set of four graphical representations of the locations of the placenta bounding box centroids 1410 across multiple vertical sweeps (R0, M, L0, LI), according to aspects of the present disclosure. As discussed above, a depth midline 1125 separates the graph into an anterior portion and a posterior portion, while a lateral midline 1165 separates the graph into a left portion and a right portion. As can be seen in the graphs, the vast majority of the centroids 1410 are located above the depth midline 1125, thus indicating that the placenta is in an anterior position. Similarly, the centroids 1410 are located to the right of the lateral midline 1165 in the R0 and M sweeps, and to the left of the lateral midline 1165 in the L0 and LI sweeps, indicating that the placenta is in a left lateral position. Thus, the placenta may be reported to be in the anterior, left lateral quadrant.

[0146] Figure 15B is a set of four graphical representations of the locations of the placenta bounding box centroids 1410 across multiple vertical sweeps (R0, M, L0, LI), according to aspects of the present disclosure. As discussed above, a depth midline 1125 separates the graph into an anterior portion and a posterior portion, while a lateral midline 1165 separates the graph into a left portion and a right portion. As can be seen in the graphs, the majority of the centroids 1410 are located below the depth midline 1125, thus indicating that the placenta is in a posterior position. Similarly, the centroids 1410 are located to theright of the lateral midline 1165 in the RO and M sweeps, on both sides of the lateral midline 1165 in the LO sweep, and to the left of the lateral midline 1165 in the LI sweep, indicating that the placenta is in a left lateral position. Thus, the placenta may be reported to be in the posterior, left lateral quadrant.

[0147] Figure 16 is an ultrasound image frame 900 that has been annotated with bounding boxes for detected anatomy, according to aspects of the present disclosure. Visible is a bounding box 910 showing the detected location of the placenta. The centroid 920 of the placenta detection box is marked. Also visible is a bounding box 930 showing the detected location of amniotic fluid, with a centroid 1640. The detection of the amniotic fluid can be used to adjust the midline separating anterior and posterior in an ultrasound image frame. Rather than using the depth midline 1125, which has been determined based on the depth setting of the ultrasound probe, a depth midline 1660 can instead be defined such that it runs horizontally through the centroid of the amniotic fluid bounding box. Depending on the implementation, the depth midline 1660 may be more accurate than the depth midline 1125 for determining whether the placenta is anterior or posterior.

[0148] Figure 17 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example placenta location determination method 520, for an aspect using placenta location detection based on detections of both the placenta and the amniotic fluid, according to aspects of the present disclosure.

[0149] In step 1720, the method 520 includes determining a midpoint for the amniotic fluid bounding boxes in each image frame where amniotic fluid was detected. Execution then proceeds to step 1730.

[0150] In step 1730, the method 520 includes determining the depth midline based on the midpoints of the amniotic fluid bounding boxes, to define the two depth quadrants.Execution then proceeds, in parallel, to steps 1760 and 1780.

[0151] In step 1740, the method 520 includes determining a midpoint for the placenta bounding boxes 1910 in each image frame where a placenta was detected. Execution then proceeds, in parallel, to steps 1760 and 1780.

[0152] In parallel, the patient’s gestational age 1010 (indicative of expected fetus size) is used to select a depth setting and sector width setting for the ultrasound probe, which in turn determines the fan beam coordinates 1015 for the captured ultrasound images. Execution then proceeds to step 1750.

[0153] In step 1750, the method 520 includes determining the lateral midline by dividing the fan beam into two regions based on the sector width setting, to define the two lateral quadrants. Execution then proceeds, in parallel, to steps 1760 and 1780.

[0154] In step 1760, the method 520 includes calculating the placenta midpoint density for the two depth quadrants (anterior and posterior). Execution then proceeds to step 1770.

[0155] In step 1770, the method 520 includes determining the depth position of the placenta using the placenta midpoint density, in all scans (e.g., Cl, C2, C3, L, M, and R). Execution then proceeds to step 1795.

[0156] In step 1780, the method 520 includes calculating the placenta midpoint density for the two lateral quadrants. Execution then proceeds to step 1790.

[0157] In step 1790, the method 520 incudes determining the lateral position of the placenta using the placenta midpoint density in all vertical scans (e.g., L, M, and R). Execution then proceeds to step 1795.

[0158] In step 1795, the method 520 includes generating an output (e.g., a text or graphical output on a display) indicative of the placenta location, including the depth position and the lateral position. Depending on the implementation, the output may include only the quadrant in which the placenta is determined to be located, or may include detailed information about the size, shape, location, and orientation of the placenta.

[0159] Figure 18 is a set of three graphical representations 1700 of the locations of the placenta bounding box centroids 1410 and the amniotic fluid bounding box centroids 1840 across horizontal sweeps Cl, C2, and C3, according to aspects of the present disclosure. The old depth midline 1650 is defined by the depth setting of the ultrasound probe, whereas the new depth midline 1660 is defined as the vertical average of all the amniotic fluid midpoints 1840. In all three graphs 1700, the placenta midpoints 1410 fall consistently above the old depth midline, such that if the amniotic fluid were not considered, the placenta would be considered to be in an anterior position. However, the position of the old depth midline 1650 may be in error, based on incorrect positioning of the ultrasound probe during the Cl, C2, and C3 sweeps. Using knowledge of the amniotic fluid’s position can provide a more accurate measurement or estimate of the placenta’s position. As can be seen, in all three graphs 1600, the placenta midpoints 1410 fall above the new depth midline 1660, thus indicating that the placenta is in fact in a posterior position.

[0160] Figure 19 is a set of three graphical representations 1600 of the locations of the placenta bounding box centroids 1410 and the amniotic fluid bounding box centroids 1840 across vertical sweeps R, M, and L, according to aspects of the present disclosure. As inFigure 18, the old depth midline 1650 is defined by the depth setting of the ultrasound probe, whereas the new depth midline 1660 is defined as the vertical average of all the amniotic fluid midpoints 1840. In all three graphs 1700, the placenta midpoints 1410 fall consistently above the old depth midline, such that if the amniotic fluid were not considered, the placenta would be considered to be in an anterior position. However, in all three graphs 1700, the placenta midpoints 1410 fall above the new depth midline 1660, thus indicating that the placenta is in fact in a posterior position.

[0161] It is noted that the sweep / scans / cineloops in Figures 18 and 19 come from the same patient, and indicate an overall placement of “posterior”.

[0162] Figure 20 is a schematic, diagrammatic view of a scanning process 2000, according to aspects of the present disclosure. The scanning process 2000 includes horizontal scans 2010 labeled Cl, C2, and C3. A suspected low-lying area 2015 covers generally the same area as the Cl sweep. If the placenta is found in the suspected low-lying area 2015, then the placenta may be low-lying. The scanning process 2000 also includes vertical sweeps 2020, labeled R, M, and L, with the suspected low-lying region 2015 occupying a lower portion of each vertical sweep. The horizontal sweeps 2010 and vertical sweeps 2020 occur within a sweep area 2030 on the mother’s abdomen 2040, bounded by the upper abdomen 2050 and the cervix 2060, thus forming a sweep grid 2070 with the suspected low-lying region 2015 in roughly the bottom third, generally coincident with the area of the Cl sweep.

[0163] Figure 21A is a graphical representation of a horizontal scan or horizontal sweep 2100, according to aspects of the present disclosure. The horizontal sweep includes multiple ultrasound images 2110, some of which are images 2120 marked with a first color or pattern indicating no detections, and some of which are images 2130 marked with a second color or pattern indicating that a placenta is detected. Depending on which horizontal scan it is (Cl, C2, C3, etc.) and the position of the scan on the abdomen, detection of the placenta may indicate a low-lying placenta.

[0164] Figure 21B is a graphical representation of a horizontal scan or horizontal sweep 2100, according to aspects of the present disclosure. The horizontal sweep includes multiple ultrasound images 2110, some of which are images 2120 marked with a first color or pattern indicating no detections, some of which are images 2130 marked with a second color or pattern indicating that the placenta is detected, some of which are images 2140 marked with a third color or pattern indicating that both the placenta and the cervix are detected, and some of which are images 2150 marked with a fourth color or pattern indicating that only thecervix is detected. Detection of the placenta and cervix in the same image may, in some cases, occur in conjunction with a low-lying placenta.

[0165] Figure 22A is a graphical representation of a vertical scan or vertical sweep 2200, according to aspects of the present disclosure. The vertical sweep includes multiple ultrasound images 2210, some of which are images 2220 marked with a first color or pattern indicating no detections, and some of which are images 2230 marked with a second color or pattern indicating that a placenta is detected. Depending on where the placenta is detected within the scan (e.g., top, middle, or bottom) and the position of the scan on the abdomen, detection of the placenta may indicate a low-lying placenta.

[0166] Figure 22B is a graphical representation of a vertical scan or vertical sweep 2200, according to aspects of the present disclosure. The vertical sweep includes multiple ultrasound images 2210, some of which are images 2220 marked with a first color or pattern indicating no detections, some of which are images 2230 marked with a second color or pattern indicating that the placenta is detected, some of which are images 2240 marked with a third color or pattern indicating that both the placenta and the cervix are detected, and some of which are images 2250 marked with a fourth color or pattern indicating that only the cervix is detected. Detection of the placenta and cervix in the same image may, in some cases, occur in conjunction with a low-lying placenta.

[0167] Figures 22A and 22B can be representative of different patients. In Figure 22A, the cervix is not seen, and so a placenta-only algorithm may be used (e.g., described in Fig. 27B). For the scenario in Fig. 22B where cervix, is seen, an algorithm using both the placenta and cervix may be used instead or in addition (e.g., described in Fig. 27A).

[0168] Figure 23A is a graphical representation 2300 of a set of horizontal scans Cl, C2, and C3, marked with the suspected low-lying region 2015, according to aspects of the present disclosure. Within the horizontal scans Cl, C2, and C3 are light-colored regions 2310 where no detection was made, and darker-colored regions 2320 where the placenta was detected. In the example shown in Figure 23 A, no placenta detections occurred in the Cl sweep (e.g., within the suspected low-lying region 2015).

[0169] Figure 23B is a graphical representation 2330 of a set of vertical scans R, M, and L, marked with the suspected low-lying region 2015, according to aspects of the present disclosure. Within the vertical scans R, M, and L are light-colored regions 2310 where no detection was made, and darker-colored regions 2320 where the placenta was detected. In the example shown in Figure 23B, some of the placenta detections occurred within the suspected low-lying region 2015 within the R and M sweeps.

[0170] Figure 23C is a graphical representation 2340 of a set of combined vertical and horizontal scans, marked with the suspected low-lying region 2015, according to aspects of the present disclosure. Visible are light-colored regions 2310 where no detections occurred, lighter-colored regions 2320 where a placenta detection occurred in a horizontal scan or a vertical scan, but not both, and darker regions 2350 where the placenta was detected in both a vertical and a horizontal scan. Thus, the lighter-colored regions 2320 may be referred to as low-confidence detection regions (relatively lower confidence than region with detection both a vertical scan or a horizontal scan), whereas the darker-colored regions 2350 may be referred to as high-confidence detection regions (relatively higher confidence than region with detection in only a vertical scan or a horizontal scan, but not both). In the example shown in Figure 23C, some of the low-confidence detection regions 2320 overlap with the suspected low-lying region 2015, but none of the high-confidence detection regions 2350 overlap with the suspected low-lying region 2015. Thus, a low-lying placenta may be considered possible but not probable. Because only the placenta is detected in Figures 23A-23C, a placenta-based algorithm (e.g., not considering the cervix) may be used to rule in or rule out a low-lying placenta, as shown below in Figure 27B.

[0171] Figure 24A is a graphical representation 2400 of a high confidence detection region 2350 rendered as a heat map or spatial likelihood map 2410, according to aspects of the present disclosure. In an example, the heat map 2410 is generated by finding the centroid 2420 of the high-confidence detection region 2350 and then selecting a color for each point within the high-confidence detection region 2350 based on how far it is from the centroid. This can provide a further indication of the probability of the placenta being located at any of the given points within the high-confidence detection region 2350. Also visible is the depth midline 2415 below which the placenta may be considered low-lying. In the example shown in Figure 24A, no portion of the heat map 2410 is located below the depth midline 2415.

[0172] Figure 24B is a graphical representation 2400 of a high confidence detection region 2350 rendered as a heat map or spatial likelihood map 2410, according to aspects of the present disclosure. Visible are the centroid 2420 and the depth midline 2415. In the example shown in Figure 24B, no portion of the heat map 2410 is located below the depth midline 2415.

[0173] Figure 25A is a graphical representation 2400 of a high confidence detection region 2350 rendered as a heat map or spatial likelihood map 2410, according to aspects of the present disclosure. Visible are the centroid 2420 and the depth midline 2415. In the example shown in Figure 24B, the centroid 2420 and a majority (e.g., greater than 50%) ofthe heat map 2410 is located below the depth midline 2415, indicating a high probability that the placenta is low-lying.

[0174] Figure 25B is a graphical representation 2400 of a high confidence detection region 2350 rendered as a heat map or spatial likelihood map 2410, according to aspects of the present disclosure. Visible are the centroid 2420 and the depth midline 2415. In the example shown in Figure 24B, the centroid 2420 and a majority (e.g., around 75%) of the heat map 2410 is located above the depth midline 2415, while a minority (e.g., around 25%) of the heat map is located below the depth midline 2415, indicating a moderate probability that the placenta is low-lying. Depending on the location of the placenta and the locations of the scans, any percentage of the heat map 2410 may be located below the depth midline 2415, including but not limited to 0%, 10%, 25%, 50%, 75%, 100%, etc. The location of the region 2350 may be compared against a threshold to determine whether the placenta is low-lying. For example, the threshold can be that the centroid 2420 of the region 2350 must be below the midline 2415 in order for it to be considered a low-lying placenta. In another example, the threshold can be that a percentage area of the detection region 2350 (e.g., 30% or greater, 40% or greater, 50% or greater, 60% or greater) must be below the midline2415 in order for it to be low-lying placenta. In still another example, the threshold criterion could be a combination of these two criteria (e.g., the centroid 2420 must be below the midline 2415 and a certain percentage area of the detection region must be below the midline2415). Still other criteria are possible, and fall within the scope of the present disclosure.

[0175] Figure 26A is a graphical representation 2600 of a spatial likelihood map of placenta location, according to aspects of the present disclosure. Visible are regions 2310 where no detection occurred, low-confidence placenta detection regions 2320, high- confidence placenta detection regions 2350, and a centroid 2420. Also visible are regions 2610 where the cervix was detected. In the example shown in Figure 26A, the depth midline 2615 is set at the location of the top of the cervix 2610. Furthermore, the centroid 2420 of the high-confidence detection region 2350 is located below the depth midline 2615, indicating a high probability that the placenta is low-lying, and possibly in a placenta previa condition.

[0176] Figure 26B is a graphical representation 2600 of a spatial likelihood map of placenta location, according to aspects of the present disclosure. Visible are regions 2310 where no detection occurred, low-confidence placenta detection regions 2320, high- confidence placenta detection regions 2350, and a centroid 2420. Also visible are regions 2610 where the cervix was detected. In the example shown in Figure 26B, the depth midline2615 is set at the location of the top of the cervix 2610. Furthermore, the centroid 2420 of the high-confidence detection region 2350 is located well above the depth midline 2615, and only a small percentage of the high-confidence detection region 2350 and low-confidence detection region 2320 are located below the depth midline 2615. This may indicate a high probability that the placenta is not low-lying.

[0177] Figures 26A and 26B are examples associated with the algorithm in Fig. 27A, below.

[0178] Figure 27A is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example low-lying placenta determination method 520 based on detections of both the placenta and the cervix, according to aspects of the present disclosure.

[0179] In step 2715, the method 520A includes, based on bounding boxes 2170 for cervix detection, determining whether the cervix was detected in the Cl scan. If yes, execution then proceeds to step 2735. If no, execution proceeds to step 2720. Step 2715 can be an example of path planning step 450 (Fig. 5) in that step 2715 determines whether the path of the blind sweep protocol 440 (Fig. 5) was properly followed. In that regard, the expectation from the blind sweep protocol 440 and / or the path planning 450 can be that the Cl scan is taken low enough on the patient’s abdomen to capture the cervix, which is in a relatively lower position in the patient’s abdomen. If the user takes the Cl scan higher on the patient’s abdomen such that the cervix is not captured, the determination in step 2715 is a determination that the expected path (450 in Fig. 5) was not followed.

[0180] In step 2720, the method 520 includes generating an output to the user providing guidance to repeat the blind sweep protocol. Step 2720 is an example of the visual guidance provided on a display in step 430 (Fig. 5) and / or the audio guidance 460 (Fig. 5). The visual guidance and / or audio guidance, for example, can be to take the Cl sweep lower on the patient’s abdomen, which tries to ensure that the cervix is captured in the ultrasound image data or ultrasound images.

[0181] In step 2735, the method 520 includes using the placenta bounding boxes 2725, the blind sweep trajectory 2730, and the cervix bounding boxes 2710 to construct a spatial likelihood map for the placenta and the cervix (as shown for example in Figures 26A and 26B). Execution then proceeds, in parallel, to steps 2740 and 2765.

[0182] In step 2740, then method 520 includes defining the suspected low-lying zone in the spatial likelihood map. This may for example be defined as 2 cm above the top of the cervix. Execution then proceeds to step 2750.

[0183] In step 2750, the method 520 includes determining whether the placenta and cervix detections overlap in the suspected low-lying zone. If yes, execution proceeds to step 2760. If no, execution proceeds to step 2770.

[0184] In step 2760, the method 520 includes ruling in a suspected low-lying placenta. This may involve referring the patient to a more experienced practitioner.

[0185] In step 2765, the method 520 includes calculating the minimum distance between the placenta and the cervix. Execution then proceeds to step 2770.

[0186] In step 2770, the method 520 includes determining whether the minimum distance between the placenta and cervix is less than a threshold distance. If yes, execution proceeds to step 2760. If no, execution proceeds to step 2775.

[0187] In step 2775, the method 520 includes ruling out a low-lying placenta.

[0188] Figure 27B is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example low-lying placenta determination method 521 based on detections of the placenta (e.g., without relying cervix detections), according to aspects of the present disclosure.

[0189] In step 2736, the method 521 includes using the placenta bounding boxes 2725 and the blind sweep trajectory 2730 to construct a spatial likelihood map for the placenta (as shown for example in Figures 24A-25B). Execution then proceeds to step 2740.

[0190] In step 2741, then method 521 includes defining the suspected low-lying zone in the spatial likelihood map. If no cervix detections are available, then the low-lying zone can defined with dimensions (e.g., height and width) that is the same or similar to the Cl sweep. In some instances, the dimensions of the suspected low-lying zone made larger (by a distance, such as 1 cm, 2 cm, or by a percentage such as 5%, 10%, and / or other values) than the dimensions of the Cl sweep. Execution then proceeds to step 2750.

[0191] In step 2751, the method 521 includes determining whether the placenta is located in the suspected low-lying zone. As described above, step 2750 can be comparing the placenta detection region to one or multiple threshold criteria (e.g., the centroid of the detection region must be below the depth midline and / or a certain percentage area of the detection region must be below the depth midline). If yes, execution proceeds to step 2760. If no, execution proceeds to step 2775.

[0192] In step 2760, the method 521 includes ruling in a suspected low-lying placenta. This may involve referring the patient to a more experienced practitioner.

[0193] In step 2775, the method 521 includes ruling out a low-lying placenta.

[0194] The host 130 (Figs. 1 and 5) can choose the algorithm of Figure 27 A or that of Figure 27B, depending on whether there are cervix detections in addition to placenta detections (Figure 27A) or only placenta detections (Figure 27B). In some instances, the objection detection Al (e.g., CNN or deep learning network) is not trained to detect the cervix (but is trained to detect the placenta), so there will be placenta detections but no cervix detections as a result (Fig. 27B). In some instances, the objection detection Al (e.g., CNN or deep learning network) is trained to detect the cervix and the placenta, and the image data and / or ultrasound images from the blind sweep protocol include both placenta detections and cervix detections (Fig. 27A). In some instances, the objection detection Al (e.g., CNN or deep learning network) is trained to detect the cervix and the placenta, and the ultrasound image data and / or ultrasound images from the blind sweep protocol includes only placenta detections because of the locations on the patient’s abdomen that the user conducted sweeps. For example, the Cl sweep may not have been low enough on the abdomen to capture the cervix. No cervix detections would be generated with such ultrasound image data and / or ultrasound images, and Fig. 27B can be used in some instances. In other instances (e.g., when the host 130 is implemented to prefer or force use of Fig. 27A), Fig. 27A includes steps 2715 and 2720 to ensure that ultrasound image data or ultrasound images are obtained with cervix detections (e.g., instruct the user to repeat the Cl scan lower on the patient’s abdomen), such that Fig. 27A can be executed with the new / repeated scan that will produce cervix detections by the object detection Al.

[0195] Figure 28 is a schematic, diagrammatic representation of an example output screen display 2800 or the ultrasound sweep anatomy detection system, according to aspects of the present disclosure. In the example shown in Figure 28, the screen display 2800 includes a graphical illustration of the placenta location, as shown for example in Figures 14A and 14B. The screen display 2800 also includes one or more image frames 2820 showing the placenta bounding box and midpoint or centroid, as shown for example in Figures 13A and 13B. The screen display 2800 also includes a graphical representation 2830 of the blind sweep trajectory, as shown for example in Figure 4. The screen display 2800 also includes a graphical representation 2840 of the relationship between the ultrasound beam fan and the placenta positions, as shown for example in Figure 12. The screen display 2800 also includes one or more distribution plots 2850 of placenta bounding box midpoints, as shown for example in Figures 14A and 14B. The screen display 2800 also includes text 2860 describing the placenta location in terms of depth position and lateral position, as described above. The screen display 2800 also includes text 2870 assessing whether or not the placentais low-lying. The screen display 2800 also includes a spatial likelihood map 2880, as shown for example in Figures 23A - 26B. It is understood that a screen display may include more or fewer components than shown in Figure 28, that additional components may be shown, and / or that the components may be shown in different sizes, shapes, or positions than shown in Figure 28.

[0196] As will be readily appreciated by those having ordinary skill in the art after becoming familiar with the teachings herein, the ultrasound sweep anatomy detection system advantageously permits untrained and minimally trained users to perform an ultrasound blind sweep protocol to gather anatomical images of high quality, including automated detection of potential health conditions such as a low-lying placenta. This may result in higher accuracy and higher clinician trust in the results, while potentially improving health outcomes and / or decreasing the total cost of care.

[0197] The systems, methods, and devices described herein may be applicable in point of care and handheld ultrasound use cases such as with the Philips Lumify system. The ultrasound sweep anatomy detection system can be used for any handheld imaging applications, including but not limited to obstetrics, lung imaging, and echocardiography. The ultrasound sweep anatomy detection system could be deployed on handheld mobile ultrasound devices, and on portable or cart-based ultrasound systems. The ultrasound sweep anatomy detection system can be used in a variety of settings including emergency departments, ambulances, accident sites, and homes. The applications could also be expanded to other settings.

[0198] The invention is detectable from its functionality and output such as reporting of detected health conditions such as low-lying placenta. This invention increases the value proposition of ultrasound applications in the obstetrics context, especially for use by minimally trained users.

[0199] A number of variations are possible on the examples and aspects described above. For example, the systems, methods, and devices described herein are not limited to obstetric ultrasound applications. Rather, the same technology can be applied to images of other organs or anatomical systems such as the lungs, heart, brain, digestive system, vascular system, etc. Furthermore, the technology disclosed herein is also applicable to other medical imaging modalities where 3D data are available, such as other ultrasound applications, camera-based videos, X-ray videos, and 3D volume images, such as computer aided tomography (CT) scans, magnetic resonance imaging (MRI) scans, optical coherencetomography (OCT) scans, or intravascular ultrasound (IVUS) pullback sequences. The technology described herein can be used in a variety of settings.

[0200] Accordingly, the logical operations making up the aspects of the technology described herein are referred to variously as operations, steps, objects, layers, elements, components, algorithms, or modules. Furthermore, it should be understood that these may occur or be performed or arranged in any order, unless explicitly claimed otherwise or a specific order is inherently necessitated by the claim language.

[0201] All directional references e.g., upper, lower, inner, outer, upward, downward, left, right, lateral, front, back, top, bottom, above, below, vertical, horizontal, clockwise, counterclockwise, proximal, and distal are only used for identification purposes to aid the reader’s understanding of the claimed subject matter, and do not create limitations, particularly as to the position, orientation, or use of the ultrasound sweep anatomy detection system. Connection references, e.g., attached, coupled, connected, joined, or “in communication with” are to be construed broadly and may include intermediate members between a collection of elements and relative movement between elements unless otherwise indicated. As such, connection references do not necessarily imply that two elements are directly connected and in fixed relation to each other. The term “or” shall be interpreted to mean “and / or” rather than “exclusive or.” The word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. Unless otherwise noted in the claims, stated values shall be interpreted as illustrative only and shall not be taken to be limiting.

[0202] The above specification, examples and data provide a complete description of the structure and use of exemplary aspects of the ultrasound sweep anatomy detection system as defined in the claims. Although various aspects of the claimed subject matter have been described above with a certain degree of particularity, or with reference to one or more individual aspects, those skilled in the art could make numerous alterations to the disclosed aspects without departing from the spirit or scope of the claimed subject matter.

[0203] Still other aspects are contemplated. It is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative only of particular aspects and not limiting. Changes in detail or structure may be made without departing from the basic elements of the subject matter as defined in the following claims.

Claims

CLAIMSWhat is claimed is:

1. A system, comprising: a processor configured for communication with an ultrasound probe, wherein the processor is configured to: control the ultrasound probe to obtain a plurality of ultrasound image frames during a blind sweep protocol on a pregnant patient; provide the plurality of ultrasound image frames as an input to a deep learning network trained to detect anatomy of the pregnant patient; generate, as a first output of the deep learning network, a detection of a placenta within the plurality of ultrasound image frames; determine, using the detection of the placenta within the plurality of ultrasound image frames, whether the placenta is low-lying; and output, to a display in communication with the processor, an output representative of the determination of whether the placenta is low-lying.

2. The system of claim 1, wherein the processor is configured to generate, as a second output of the deep learning network, a detection of a cervix within the plurality of ultrasound image frames, and wherein the processor is further configured to determine whether the placenta is low- lying based on the detection of the cervix within the plurality of ultrasound image frames.

3. The system of claim 2, wherein the processor is configured to generate a spatial-likelihood map for the placenta and the cervix, and wherein the processor is further configured to determine whether the placenta is low- lying based on the spatial-likelihood map.

4. The system of claim 3, wherein the output representative of the determination comprises the spatial-likelihood map.

5. The system of claim 3,wherein the processor is configured to define a low-lying placenta zone in the spatial- likelihood map, wherein the processor is further configured to determine whether the placenta is low- lying based on the low-lying placenta zone in the spatial-likelihood map.

6. The system of claim 5, wherein the processor is configured to determine if overlap between the placenta and cervix is present in the low-lying placenta zone in the spatial-likelihood map, and wherein, when the overlap is present, the processor is configured to determine that the placenta is low-lying.

7. The system of claim 6, wherein the processor is configured to calculate a distance between the placenta and the cervix based on the spatial likelihood map, wherein, when the overlap is not present, the processor is configured to determine if the distance is less than a threshold distance, wherein, when the distance is less than the threshold distance, the processor is configured to determine that the placenta is low-lying, and wherein, when the distance is greater than the threshold distance, the processor is configured determine that the placenta is not low-lying.

8. The system of claim 1, wherein the processor is configured to: determine, using the detection of the placenta within the plurality of ultrasound image frames, a location of the placenta within a uterus; and output, to the display in communication with the processor, a visual representation of the location of the placenta, wherein the determination of the location of the placenta is distinct from the determination of whether the placenta is low-lying.

9. The system of claim 8, wherein the location of the placenta within the uterus comprises at least one of anterior, posterior, left lateral, or right lateral.

10. The system of claim 1, further comprising the ultrasound probe.

11. A system, comprising: a processor configured for communication with an ultrasound probe, wherein the processor is configured to: control the ultrasound probe to obtain a plurality of ultrasound image frames during a blind sweep protocol on a pregnant patient; provide the plurality of ultrasound image frames as an input to a deep learning network trained to detect anatomy of the pregnant patient; generate, as a first output of the deep learning network, a detection of a placenta within the plurality of ultrasound image frames; determine, using the detection of the placenta within the plurality of ultrasound image frames, a location of the placenta within a uterus; and provide, to a display in communication with the processor, a visual representation of the location of the placenta.

12. The system of claim 11, wherein, to determine the location of the placenta within the uterus, the processor is configured to: divide a field of view of the ultrasound probe into a plurality of regions; and plot, on the plurality of regions, the detection of the placenta within the plurality of ultrasound image frames.

13. The system of claim 12, wherein the location of the placenta comprises: one of anterior depth position or posterior depth position; and one of left lateral position or right lateral position.

14. The system of claim 13, wherein the plurality of regions is four quadrants corresponding respectively to anterior, posterior, left lateral, and right lateral.

15. The system of claim 12, wherein the detection of the placenta comprises a bounding box, wherein, to plot the detection of the placenta on the plurality of regions, the processor is configured to plot a location of the bounding box for each ultrasound image frame in which there is the detection of the placenta,wherein the processor is configured to determine a density of the plotted locations of the bounding boxes on the plurality of regions, wherein the processor is further configured to determine a region of the plurality of regions as the location of the placenta based on the density of the plotted locations for the region being greater than the density of the plotted locations for the other regions of the plurality of regions.

16. The system of claim 12, wherein, to divide the field of view into the plurality of regions, the processor is configured to: determine a depth midline dividing the field of view into two depth regions; and determine a lateral midline dividing the field of view into two lateral regions.

17. The system of claim 16, wherein the depth midline is based on a depth setting of the ultrasound probe.

18. The system of claim 16, wherein the processor is configured to generate, as a second output of the deep learning network, a detection of amniotic fluid within the plurality of ultrasound image frames, and wherein the depth midline is based on the detection of the amniotic fluid.

19. The system of claim 11, wherein the processor is configured to: determine, using the detection of the placenta within the plurality of ultrasound image frames, whether the placenta is low-lying; and provide, to the display in communication with the processor, an output representative of the determination of whether the placenta is low-lying, wherein the determination of whether the placenta is low-lying is distinct from the determination of the location of the placenta.

20. The system of claim 11, further comprising the ultrasound probe.

Images