Ultrasound-based verification of uterine imaging extent using blind sweep protocol

Abstract

A system includes a processor that communicates with an ultrasound probe. The processor controls the ultrasound probe to obtain ultrasound image frames during a blind sweep protocol on a patient with a pregnancy and determines whether the ultrasound image frames cover an entire distance across a uterus of the patient. The processor also provides an output to a display that is representative of whether the ultrasound image frames cover the entire distance across the uterus. When the determination is that the ultrasound images frames cover the entire distance across the uterus, the output includes an indication that the ultrasound image frames are accepted. When the determination is that the ultrasound images frames do not cover the entire distance across the uterus, the output includes an instruction to repeat acquisition by the ultrasound probe to obtain further ultrasound image frames.

Description

ULTRASOUND-BASED VERIFICATION OF UTERINE IMAGING EXTENT USING BLIND SWEEP PROTOCOLFIELD

[0001] The subject matter described herein relates to devices, systems, and methods for using ultrasound data from a blind abdominal imaging sweep to verify that the full extent of a uterus has been imaged.BACKGROUND

[0002] Ultrasound imaging is often used for diagnostic purposes in an office or hospital setting, but may also be used in resource-constrained care settings (e.g., homes, accident sites, ambulances, mobile health facilities, etc.) by emergency personnel, home health nurses, midwives, etc., who may lack ultrasound expertise. To facilitate ultrasound image acquisition by untrained or minimally trained users, a “blind sweep” protocol is often employed, in which the user follows pre-determined probe paths (e.g., sweeping out a pattern on the patient’s abdomen) during imaging.

[0003] Ultrasound imaging is a vital component of high-quality obstetric care. For example, detection of high-risk pregnancies through ultrasound can allow for appropriate referral for delivery care in highly resourced centers with providers trained to handle any associated risks and complications. However, in rural and under-resourced communities, the scarcity of ultrasound imaging results in a considerable gap in the healthcare of pregnant mothers.

[0004] Antepartum complications - including preterm labor, preterm premature rupture of the membranes, intrauterine growth restriction (IUGR), intrauterine fetal demise, gestational diabetes, and preeclampsia - develop in over 25% of pregnancies. While ultrasound can be very useful in identifying complications, access to ultrasound may be out of reach in low-resource settings. Access and skill to perform the ultrasound in low-resource setting is therefore deficient in many parts of the world.

[0005] The information included in this Background section of the specification, including any references cited herein and any description or discussion thereof, is included for technical reference purposes only and is not to be regarded as subject matter by which the scope of the disclosure is to be bound.SUMMARY

[0006] Disclosed is an ultrasound blind sweep uterine imaging extent verification system that, during or following a blind sweep protocol (sweeps at particular locations along body without trying to find specific anatomy), detects anatomical features in the captured images and uses them to identify whether an expected extent of the uterus (in some cases, the entire uterus) has been imaged by the sweeps. A benefit of the ultrasound blind sweep anatomy detection system is to improve the quality of care by using information obtained during the blind sweep protocol to identify medical conditions (e.g., pregnancy complications, gestational age of the fetus, etc.).

[0007] A system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.

[0008] One general aspect includes a system with a processor configured for communication with an ultrasound probe, where the processor is configured to: control acquisition by the ultrasound probe to obtain a plurality of ultrasound image frames during a blind sweep protocol on a patient with a pregnancy; determine whether the plurality of ultrasound image frames covers an entire distance across a uterus of the patient; and provide, to a display in communication with the processor, an output representative of the determination of whether the plurality of ultrasound image frames covers the entire distance across the uterus, where, when the determination is that the plurality of ultrasound images frames covers the entire distance across the uterus, the output may include an indication that the plurality of ultrasound image frames are accepted, and where, when the determination is that the plurality of ultrasound images frames does not cover the entire distance across the uterus, the output may include an instruction to repeat acquisition by the ultrasound probe to obtain a further plurality of ultrasound image frames. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0009] Implementations may include one or more of the following features. In some aspects, to determine whether the plurality of ultrasound image frames covers the entire distance across the uterus, the processor is configured to provide the plurality of ultrasoundimage frames to as an input to a deep learning network trained to output one or more detections of at least one of maternal anatomy or fetal anatomy. In some aspects, operation of the deep learning network to output the one or more detections may include generating a plurality of feature maps for the plurality of ultrasound image frames, and, to determine whether the plurality of ultrasound image frames covers the entire distance across the uterus, the processor is configured to perform edge detection in the plurality of feature maps to identify a uterine wall. In some aspects, the processor is configured to: generate a contour of the uterine wall from the plurality of feature maps; and determine whether the plurality of ultrasound image frames covers the entire distance across the uterus based on an extent of the generated contour. In some aspects, the plurality of ultrasound images may include a sweep in the blind sweep protocol. In some aspects, to determine whether the plurality of ultrasound image frames cover the entire distance across the uterus, the processor is configured to: determine whether the one or more detections corresponds to an expected location within the sweep. In some aspects, the expected location may include at least one of: a detection of the maternal anatomy at a beginning or an ending of the sweep; or a detection of the fetal anatomy at a middle of the sweep. In some aspects, to determine whether the plurality of ultrasound image frames covers the entire distance across the uterus, the processor is generate to: generate a statistical description of the one or more detections; and determine whether the plurality of ultrasound image frames covers the entire distance across the uterus based on the statistical description. In some aspects, the processor is configured to output, to the display, a real time indication during the acquisition of whether an ultrasound image frame of the plurality of ultrasound image frames is representative of anatomy outside the uterus or anatomy inside the uterus. In some aspects, to determine whether the plurality of ultrasound image frames covers the entire distance across the uterus, the processor is configured to: provide the plurality of ultrasound image frames as an input a deep learning network trained to determine output a determination of whether a sweep in the blind sweep protocol is complete or incomplete, where the deep learning network is trained using a plurality of sweeps labeled complete or incomplete based on a difference between an estimated value of a clinical feature and an actual value of the clinical feature.

[0010] One general aspect includes a method with a processor configured for communication with an ultrasound probe: control acquisition by the ultrasound probe to obtain a plurality of ultrasound image frames during a blind sweep protocol on a patient with a pregnancy; determine whether the plurality of ultrasound image frames covers an entire distance across a uterus of the patient; and provide, to a display in communication with theprocessor, an output representative of the determination of whether the plurality of ultrasound image frames covers the entire distance across the uterus, where, when the determination is that the plurality of ultrasound images frames covers the entire distance across the uterus, the output may include an indication that the plurality of ultrasound image frames are accepted, and where, when the determination is that the plurality of ultrasound images frames does not cover the entire distance across the uterus, the output may include an instruction to repeat acquisition by the ultrasound probe to obtain a further plurality of ultrasound image frames. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

[0011] Implementations may include one or more of the following features. In some aspects, determining whether the plurality of ultrasound image frames covers the entire distance across the uterus may include providing the plurality of ultrasound image frames to as an input to a deep learning network trained to output one or more detections of at least one of maternal anatomy or fetal anatomy. In some aspects, operation of the deep learning network to output the one or more detections may include generating a plurality of feature maps for the plurality of ultrasound image frames, and determining whether the plurality of ultrasound image frames covers the entire distance across the uterus may include performing edge detection in the plurality of feature maps to identify a uterine wall. In some aspects, the method may include: generating a contour of the uterine wall from the plurality of feature maps; and determining whether the plurality of ultrasound image frames covers the entire distance across the uterus based on an extent of the generated contour. In some aspects, the plurality of ultrasound images may include a sweep in the blind sweep protocol. In some aspects, determining whether the plurality of ultrasound image frames covers the entire distance across the uterus may include: determining whether the one or more detections corresponds to an expected location within the sweep. In some aspects, the expected location may include at least one of: a detection of the maternal anatomy at a beginning or an ending of the sweep; or a detection of the fetal anatomy at a middle of the sweep. In some aspects, the method determining whether the plurality of ultrasound image frames covers the entire distance across the uterus may include: generating a statistical description of the one or more detections; and determining whether the plurality of ultrasound image frames covers the entire distance across the uterus based on the statistical description. In some aspects, the method may include outputting, to the display, a real time indication during the acquisition of whether an ultrasound image frame of the plurality of ultrasound image frames isrepresentative of anatomy outside the uterus or anatomy inside the uterus. In some aspects, determining whether the plurality of ultrasound image frames covers the entire distance across the uterus may include: providing the plurality of ultrasound image frames as an input a deep learning network trained to determine output a determination of whether a sweep in the blind sweep protocol is complete or incomplete, where the deep learning network is trained using a plurality of sweeps labeled complete or incomplete based on a difference between an estimated value of a clinical feature and an actual value of the clinical feature. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0012] One general aspect includes a system with a processor configured for communication with an ultrasound probe. The processor is configured to: control the ultrasound probe to obtain one or more ultrasound image frames during a blind sweep protocol on a patient with a pregnancy; provide the one or more ultrasound image frames as an input to a deep learning network trained to detect at least one of maternal anatomy or fetal anatomy; determine, using the detection of at least one of maternal anatomy or fetal anatomy, whether the one or more ultrasound image frames cover an expected portion of a uterus of the patient; and provide, to a display in communication with the processor, an output representative of the determination of whether the one or more ultrasound image frames cover an expected portion of the uterus of the patient.

[0013] In some aspects, the one or more ultrasound image frames may include a single ultrasound image frame, and where the output may include a first indication if the single image frame is outside the uterus and a second indication if the single image frame is inside the uterus. In some aspects, the one or more ultrasound image frames may include a blind sweep. In some aspects, the expected portion of the uterus may include less than the entire uterus. In some aspects, the one or more ultrasound image frames may include an examination may include a plurality of blind sweeps. In some aspects, the expected portion of the uterus may include the entire uterus. In some aspects, the at least one of maternal anatomy or fetal anatomy may include a uterine wall. In some aspects, determining whether the one or more ultrasound image frames cover an expected portion of the uterus may include post-processing to assemble a 3D model of the uterine wall. In some aspects, the at least one of maternal anatomy or fetal anatomy may include at least one of a fetal head, a fetal abdomen, a fetal spine, a maternal bladder, or a maternal cervix. In some aspects, determining whether the one or more ultrasound image frames cover an expected portion of the uterus may include reducing a dimensionality of the detection. In some aspects,determining whether the one or more ultrasound image frames cover an expected portion of the uterus may include: calculating a confidence value of a clinical feature estimated from the one or more ultrasound image frames; and determining whether the confidence value exceeds a threshold value. In some aspects, the output may include a percentage of examinations that cover an expected portion of a uterus of the patient. In some aspects, the percentage is for at least one of a single examination or a group of examinations occurring over a course of one day, one month, or a history of the system. In some aspects, the percentage is for at least two different users. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0014] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter. A more extensive presentation of features, details, utilities, and advantages of the ultrasound blind sweep uterine imaging extent verification system, as defined in the claims, is provided in the following written description of various aspects of the disclosure and illustrated in the accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Illustrative aspects of the present disclosure will be described with reference to the accompanying drawings, of which:

[0016] Figure 1 is a schematic, diagrammatic representation of an ultrasound imaging system, according to aspects of the present disclosure.

[0017] Figure 2 is a schematic diagram of a processor circuit, according to aspects of the present disclosure.

[0018] Figure 3 is a schematic, diagrammatic representation of a patient on whose abdomen the ultrasound blind sweep protocol will be followed, according to aspects of the present disclosure.

[0019] Figure 4 is a schematic, diagrammatic representation, in block diagram form, of an example ultrasound blind sweep uterine imaging extent verification process, according to aspects of the present disclosure.

[0020] Figure 5 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound blind sweep uterine imaging extent verification system, according to aspects of the present disclosure.

[0021] Figure 6 is a schematic, diagrammatic representation, in block diagram form, of an example ultrasound blind sweep uterine imaging extent verification process, according to aspects of the present disclosure.

[0022] Figure 7 is a schematic, diagrammatic representation, in block diagram form, of at least a portion of an example ultrasound blind sweep uterine imaging extent verification system, according to aspects of the present disclosure.

[0023] Figure 8A is a schematic, diagrammatic overview, in block diagram form, of a training mode for an untrained neural network, according to aspects of the present disclosure.

[0024] Figure 8B is a schematic, diagrammatic overview, in block diagram form, of an inference mode or clinical usage mode for the trained neural network, according to aspects of the present disclosure.

[0025] Figure 9 is a schematic, diagrammatic illustration, in block diagram form, of the detection of anatomy, according to aspects of the present disclosure.

[0026] Figure 10 is a graphical representation of anatomical detections in middle sweeps and edge sweeps, according to aspects of the present disclosure.

[0027] Figure 11 is an ultrasound image frame or DL feature map that includes a detected uterine wall, according to aspects of the present disclosure.

[0028] Figure 12 is a schematic, diagrammatic representation, in flow diagram form, of an example uterine imaging extent verification method, according to aspects of the present disclosure.

[0029] Figure 13 is a schematic, diagrammatic view of an ultrasound blind sweep uterine imaging extent verification method for multiple sweeps, according to aspects of the present disclosure.

[0030] Figure 14 is a schematic, diagrammatic representation, in flow diagram form, of an example uterine imaging extent verification method, according to aspects of the present disclosure.

[0031] Figure 15 is a schematic, diagrammatic representation, in flow diagram form, of an example uterine imaging extent verification method, according to aspects of the present disclosure.

[0032] Figure 16 is a schematic, diagrammatic representation, in flow diagram form, of an example uterine imaging extent verification method, according to aspects of the present disclosure.

[0033] Figure 17 is a graph of the variance or skewness of amniotic fluid detections, according to aspects of the present disclosure.

[0034] Figure 18 is a schematic, diagrammatic illustration of a live uterine coverage feedback process, according to aspects of the present disclosure.

[0035] Figure 19 is a schematic, diagrammatic representation, in flow diagram form, of an example automatic location feedback method, according to aspects of the present disclosure.

[0036] Figure 20A is a schematic, diagrammatic representation, in flow diagram form, of an example first deep learning model training step 2000, according to aspects of the present disclosure.

[0037] Figure 20B is a schematic, diagrammatic representation, in flow diagram form, of an example second deep learning model training step 2050, according to aspects of the present disclosure.

[0038] Figure 20C is a schematic, diagrammatic representation, in flow diagram form, of an example deep learning inference mode 2075, according to aspects of the present disclosure.

[0039] Figure 21 is a screen display of an example uterine imaging extent verification system, indicating an accepted ultrasound blind sweep examination, according to aspects of the present disclosure.

[0040] Figure 22 is a screen display of an example uterine imaging extent verification system, indicating rejected ultrasound blind sweep examination, according to aspects of the present disclosure.

[0041] Figure 23 is a screen display of an example ultrasound blind sweep uterine imaging extent verification system, quantifying the uterine imaging completion success for different users across a span of time, according to aspects of the present disclosure.

[0042] Figure 24 is a is a schematic diagram of a deep learning network configuration, according to aspects of the present disclosure.DETAILED DESCRIPTION

[0044] In post-processing of images acquired during a blind sweep protocol, artificial intelligence (Al) methods may be used to detect clinically relevant metrics from the image data. However, if the user improperly places and / or moves the probe during the blind sweeps, parts of the uterus may not be imaged, and therefore insufficient image data may be acquired for the Al models to provide accurate findings. Furthermore, this challenge may limit adoption of automated methods, as stakeholders (e.g., healthcare systems, sonographers) must have confidence in the data acquired.

[0045] The present disclosure relates to the detection of sufficient imaging coverage of the uterus during antenatal ultrasound screening via the blind sweep method. The goal of this disclosure is to maximize Al performance in measuring pregnancy parameters such as gestational age (GA), by ensuring that blind sweep acquisitions contain sufficient data for accurate assessment of fetal and maternal anatomies. In this disclosure, several Al and image processing-based methods are detailed for assessing whether an obstetric blind sweep acquisition has sufficiently covered all relevant anatomies. Additionally, the present disclosure includes ways to provide user feedback in the form of haptic, audio, or visual cues, or summary outputs of exam completeness in a user interface format.

[0046] In accordance with at least one aspect of the present disclosure, an ultrasound blind sweep uterine imaging extent verification system is provided which can identify anatomy of interest, based on a blind sweep protocol with an ultrasound imaging probe. The ultrasound blind sweep uterine imaging extent verification system presents a novel approach to imaging quality control by ensuring that features such as the uterine wall or fetal anatomies are detected, and using the detected locations of these features to, for example, determine whether the blind sweep protocol has imaged the full extent of the uterus.

[0047] Ultrasound imaging is a vital component of high-quality obstetric care. In rural and under-resourced communities, the scarcity of ultrasound imaging results in a considerable gap in healthcare for pregnant mothers. Increased detection of pregnancy complications through ultrasound can allow for appropriate referral for delivery care in more- resourced centers with more highly trained providers. The present disclosure seeks to overcome this barrier to ultrasound access in a locally sustainable and resource-conscious way, using standardized blind-sweep scanning protocols combined with artificial intelligence, obviating the need for an interpreting provider (e.g., a radiologist or obstetrician) and an experienced sonographer in the remote location.

[0048] The obstetric blind sweep protocol can be taught to health care workers in a very short period, enabling them to acquire high-quality ultrasound images of pregnant mothers. There have been encouraging results on determination of gestational age and fetal presentation from this type of blind sweep protocol. A need exists for verification that the blind sweep protocol has been performed correctly, such that the entire extent of the uterus has been imaged.

[0049] Currently, there is a heavy dependence on the user to adhere to the blind sweep protocol. Given that blind sweep workflows are intended for novice users, there is a nontrivial risk for non-compliance. Failure to obtain complete imaging coverage may result in key anatomies being missed, with potentially negative implications for the clinical guidance provided by the system. Some examples of potential performance losses due to improper uterine coverage include, but are not limited to: (1) Incomplete coverage of the fetal body resulting in incorrect predictions of the fetal position (cephalic / non-cephalic, or cephalic / breech / oblique / transverse); (2) Incomplete coverage of the amniotic fluid space, resulting in skewed measurements of deepest vertical pocket (DVP) or amniotic fluid index (AFI); (3) Incomplete coverage of the uterus, resulting in missed detection of twins or triplets; (4) Lack of head detections, leading to faulty estimations of gestational age; (5) Lack of head detections, leading to false alarms for high risk scenarios such as anencephaly.

[0050] Furthermore, errors introduced by this challenge may limit adoption of the system, as stakeholders (e.g., healthcare systems, sonographers) must have confidence in the data acquired. The ultrasound blind sweep uterine imaging extent verification system described herein addresses this problem by providing confirmation that sufficient anatomical coverage has been achieved for accurate clinical decision making.

[0051] The present disclosure describes a software system for determining whether an exam was conducted with complete or incomplete coverage of the uterine volume. The system consists of an ultrasound transducer (e.g., Philips Lumify) and a signal processing unit. Complete or incomplete coverage of the uterus by the blind sweep protocol can be determined by the signal processing unit through quantification of the detectable uterine signal, particularly uterine wall and / or amniotic fluid signals, within a given sweep (e.g., within a given series of image frames). The signal processing unit may determine uterine extent via AL or non-ALbased methods. Feedback may be provided to the user post-sweep, or in the form of real-time guidance.

[0052] The first part is a method that uses a combination of traditional image processing techniques and Al (machine learning (ML) / deep learning (DL)) approaches to assess uterinecoverage based on a combination of ultrasound images and uterine anatomy detections derived from processed ultrasound data. In one aspect of this method, combined inputs of ultrasound blind sweep acquisitions and You Only Look Once (YOLO) model-based anatomy detections are fed to the uterine extent determiner, and the output is a classification of uterine coverage (e.g., complete or incomplete). A list of different aspects of this method, described in the next section, is provided below.

[0053] Aspect 1 : Direct detection of uterine wall using post-processing techniques on images (e.g., YOLO feature maps).

[0054] Aspect 2: Anatomy detection-based analysis of the uterine interior.

[0055] Aspect 3 : Assessment of uterine coverage using dimensionality reduction of anatomy detections.

[0056] Aspect 4: Automatic feedback of probe location during sweep.

[0057] Aspect 5: Al performance-based assessment of uterine coverage.

[0058] The second part is a user interface which provides a summary of uterine coverage from the blind sweep exam. This user interface can also provide feedback about specific parts of the exam that are missing information, or guidance about how to achieve more complete coverage in a subsequent exam. An embodiment, described in the next section, is listed below:

[0059] Aspect 6: Summary output user interface.

[0060] Is aspect 1, during YOLO inference, each frame of a blind sweep scan is processed through a trained YOLO model, which creates feature maps for object detection. The model takes the ultrasound image as input, then has several layers of feature maps. These feature maps extract information that helps to identify the location of specific fetal or maternal anatomies. Post-processing techniques are then performed on the YOLO feature maps to detect the uterine wall in each image or sweep. Assessment of the completeness of the detected wall can then guide the prediction of whether the full exam had complete or incomplete coverage of the uterus. Post-processing techniques that can be used include, but are not limited to edge detection algorithms (Canny Edge Detector, Sobel Edge Detector, etc.) or contrast manipulation algorithms (histogram equalization, etc.). A combination of these techniques can be used to more accurately detect the edges of uterine wall. For example, if the uterine wall is detected in multiple images in edge sweeps, this can show that the sweep adequately covered the uterine edges in the lateral (L-R) and anterior-posterior (A- P) directions. Similarly, if the uterine edge starts to be detected in the nth frame from the start of the sweep and stops being detected in the mth frame from the end of the sweep, thenthat may confirm adequate sweep coverage in the cranial-caudal direction. With all 3 directional components being confirmed, this sweep can be deemed to be adequately obtained. Such information can then be consolidated across all sweeps. Specific criteria may be used for each sweep, e.g. the middle vertical sweep should not contain lateral uterine edges, otherwise the sweep was incorrectly obtained. The determination of whether all sweeps were correctly obtained can then be used for an exam-level completeness determination.

[0061] In the second aspect, an existing YOLO-based machine learning model is currently applied to blind sweep exams to detect key intrauterine anatomies, including the fetal head, heart, abdomen, spine, bladder, amniotic fluid, placenta, etc. Key maternal (extrauterine) anatomies are also detected, including the maternal bladder and cervix. These detections provide landmarks indicating the complete or incomplete coverage of the uterus during an exam. Therefore, even without detection of the uterine edge, assessment of anatomy detections can be used to determine exam completeness. A list of Al inputs that could be extracted from these anatomy detections includes, but is not limited to:

[0062] Direct quantification of critical anatomies (e.g. head, heart, placenta), with a threshold number of detections to determine completeness.

[0063] Multi-hot encoding of fetal and maternal anatomies for each frame, or groups of adjacent frames, for input into a ML model.

[0064] Assessment of the lateral extent of intrauterine anatomies in edge and middle sweeps.

[0065] The anatomies listed in this example are representative and not the sole anatomies that could be required by an anatomy check approach.

[0066] In the third aspect, in addition to utilizing direct YOLO outputs as the features for a coverage detector, several methods may be used for dimensionality reduction prior to being input to the coverage detector. For example, the centroids of fetal / matemal anatomies such as the head, urinary bladder, and placenta may be calculated from their respective YOLO bounding box locations. Diffuse features such as amniotic fluid may be represented by features such as the percentage of exam images with amniotic fluid detections, the variance / skewness of amniotic fluid detections, etc. More complex features may also be engineered via topological data analysis, topographical or clustering methods, etc. The potential features described in this aspect are intended to be representative and are not the sole features that may be engineered for the purpose of dimensionality reduction.

[0067] The fourth aspect describes a way to provide real-time feedback during each sweep in the blind sweep exam. The basic premise of this aspect rests on the idea that a blind sweep exam is complete if each sweep traverses the full extent of the uterus. In this aspect, real-time feedback would let the user know whether the ultrasound beam is currently located inside or outside of the uterus. Using any of the methods described above, the ultrasound system can assess whether the current frame in a sweep is in the uterine interior or exterior. The system would then provide continuous audio / visual / haptic feedback to let the user know when they are moving from outside to inside the uterus (e.g., at the beginning of the sweep), crossing the inside of the uterus (e.g., in the middle of the sweep), or moving from inside to outside the uterus (e.g., at the end of the sweep).

[0068] In the fifth aspect, another way to evaluate whether the data collected had sufficient coverage or not is to relate it to the accuracy of the eventual output(s) produced. For example, if the goal is to estimate clinical parameters such as the gestational age of the fetus or the fetal lie (or other such relevant OB parameters), in the suggested embodiment, an Al approach is taken to leverage this information. In the proposed approach, a DL model is trained with the acquired images, or some subset of the acquired images, as input. The model is trained to assess exam completeness based on the accuracy of the estimated clinical features (difference between estimated value and the ground truth value), i.e. smaller the error, higher the likelihood of the data being deemed “sufficient”, and vice versa. During inference, the above trained Al model would take the ultrasound images from the different sweeps as input and provide an estimate of sufficiency / insufficiency for the data. This approach links the acquisition to the end clinical feature, which is complementary to the other approaches described earlier (which look directly at the uterine signal and / or anatomy detections therein).

[0069] In the sixth aspect, after image acquisition and assessment of completeness using any of the previous aspects, the ultrasound blind sweep uterine imaging extent verification system provides a summary output indicating completeness. For user guidance and feedback, this output may include:

[0070] An overall classification of the exam as complete or incomplete;

[0071] A map of sweeps that were complete or incomplete; or

[0072] Guidance / examples of how to repeat a sweep or exam that was incomplete.

[0073] Alternatively, this summary output can provide feedback for stakeholders to provide information about the reliability of the acquired data. This output might include:

[0074] Quantification of all complete and incomplete exams collected in a single imaging session, in the calendar day, or in the calendar month.

[0075] Quantification of all complete and incomplete exams collected by each individual user.

[0076] Tracking of the percentage of complete and incomplete exams, collected by an individual user or group of users over time.

[0077] This summary feedback can help the stakeholders to assess their own confidence in the measurements being acquired from blind sweep exams.

[0078] The present disclosure aids substantially in the capture of high-quality ultrasoundbased diagnoses by minimally trained users, by automatically detecting whether coverage of the uterus is complete, thus facilitating the diagnosis, prevention, and / or treatment of pregnancy complications and related diseases. Implemented on a processor in communication with an ultrasound probe, the ultrasound blind sweep uterine imaging extent verification system disclosed herein provides practical improvements in the quality of care available to patients in underserved areas. This improved imaging methodology transforms a process that is heavily reliant on professional experience into one that is accurate and repeatable even for minimally trained personnel, without the normally routine need to train clinicians such as emergency department personnel to recognize whether a blind sweep protocol has been performed correctly. This unconventional approach improves the functioning of the ultrasound imaging system, by providing reliable, repeatable imaging and diagnosis in hospital, office, vehicle, field, and home settings, as well as referral recommendations for patients suspected to have a pregnancy complication.

[0079] The ultrasound blind sweep uterine imaging extent verification system may be implemented as a process at least partially viewable on a display, and operated by a control process executing on a processor that accepts user inputs from a keyboard, mouse, or touchscreen interface, and that is in communication with one or more sensor probes. In that regard, the control process performs certain specific operations in response to different inputs or selections made at different times. Certain structures, functions, and operations of the processor, display, sensors, and user input systems provide novel features or aspects of the present disclosure.

[0080] These descriptions are provided for exemplary purposes only, and should not be considered to limit the scope of the ultrasound blind sweep multiple pregnancy detection system. Certain features may be added, removed, or modified without departing from the spirit of the claimed subject matter.

[0081] For the purposes of promoting an understanding of the principles of the present disclosure, reference will now be made to the aspects illustrated in the drawings, and specific language will be used to describe the same. It is nevertheless understood that no limitation to the scope of the disclosure is intended. Any alterations and further modifications to the described devices, systems, and methods, and any further application of the principles of the present disclosure are fully contemplated and included within the present disclosure as would normally occur to one skilled in the art to which the disclosure relates. In particular, it is fully contemplated that the features, components, and / or steps described with respect to one aspect may be combined with the features, components, and / or steps described with respect to other aspects of the present disclosure. For the sake of brevity, however, the numerous iterations of these combinations will not be described separately.

[0082] Figure 1 is a schematic, diagrammatic representation of an ultrasound imaging system 100, according to aspects of the present disclosure. The ultrasound imaging system 100 may for example be used to acquire ultrasound video sweeps, which can then be analyzed by a human clinician or an artificial intelligence to diagnose medical conditions.

[0083] The ultrasound imaging system 100 is used for scanning an area or volume of a subject’s body. A subject may include a patient of an ultrasound imaging procedure, or any other person, or any suitable living or non-living organism or structure. The ultrasound imaging system 100 includes an ultrasound imaging probe 110 in communication with a host 130 over a communication interface or link 120. The probe 110 may include a transducer array 112, a beamformer 114, a processor circuit 116, and a communication interface 118. The host 130 may include a display 132, a processor circuit 134, a communication interface 136, and a memory 138 storing subject information.

[0084] In some aspects, the probe 110 is an external ultrasound imaging device including a housing 111 configured for handheld operation by a user. The transducer array 112 can be configured to obtain ultrasound data while the user grasps the housing 111 of the probe 110 such that the transducer array 112 is positioned adjacent to or in contact with a subject’s skin. The probe 110 is configured to obtain ultrasound data of anatomy within the subject’s body while the probe 110 is positioned outside of the subject’s body. In some aspects, the probe 110 can be an external ultrasound probe, a transthoracic probe, and / or a curved array probe.

[0085] In other aspects, the probe 110 can be an internal ultrasound imaging device and may comprise a housing 111 configured to be positioned within a subject’s body. In some aspects, the probe 110 may be a curved array probe. Probe 110 may be of any suitable formfor any suitable ultrasound imaging application including both external and internal ultrasound imaging.

[0086] For an ultrasound imaging device, the transducer array 112 emits ultrasound signals towards an anatomical object 105 of a subject and receives echo signals reflected from the object 105 back to the transducer array 112. The ultrasound transducer array 112 can include any suitable number of acoustic elements, including one or more acoustic elements and / or a plurality of acoustic elements. In some instances, the transducer array 112 includes a single acoustic element. In some instances, the transducer array 112 may include an array of acoustic elements with any number of acoustic elements in any suitable configuration. For example, the transducer array 112 can include between 1 acoustic element and 10000 acoustic elements, including values such as 2 acoustic elements, 4 acoustic elements, 36 acoustic elements, 64 acoustic elements, 128 acoustic elements, 500 acoustic elements, 812 acoustic elements, 1000 acoustic elements, 3000 acoustic elements, 8000 acoustic elements, and / or other values both larger and smaller. In some instances, the transducer array 112 may include an array of acoustic elements with any number of acoustic elements in any suitable configuration, such as a linear array, a planar array, a curved array, a curvilinear array, a circumferential array, an annular array, a phased array, a matrix array, a one-dimensional (ID) array, a 1.x dimensional array (e.g., a 1.5D array), or a two- dimensional (2D) array. The array of acoustic elements (e.g., one or more rows, one or more columns, and / or one or more orientations) can be uniformly or independently controlled and activated. The transducer array 112 can be configured to obtain one-dimensional, two- dimensional, and / or three-dimensional images of a subject’s anatomy. In some aspects, the transducer array 112 may include a piezoelectric micromachined ultrasound transducer (PMUT), capacitive micromachined ultrasonic transducer (CMUT), single crystal, lead zirconate titanate (PZT), PZT composite, other suitable transducer types, and / or combinations thereof.

[0087] The object 105 may include any anatomy or anatomical feature, such an abdomen of a pregnant patient, one or multiple fetuses inside the abdomen of the pregnant patient, etc.

[0088] The beamformer 114 is coupled to the transducer array 112. The beamformer 114 controls the transducer array 112, for example, for transmission of the ultrasound signals and reception of the ultrasound echo signals. In some aspects, the beamformer 114 may apply a time-delay to signals sent to individual acoustic transducers within an array in the transducer 112 such that an acoustic signal is steered in any suitable direction propagating away from the probe 110. The beamformer 114 may further provide image signals to the processorcircuit 116 based on the response of the received ultrasound echo signals. The beamformer 114 may include multiple stages of beamforming. The beamforming can reduce the number of signal lines for coupling to the processor circuit 116. In some aspects, the transducer array 112 in combination with the beamformer 114 may be referred to as an ultrasound imaging component.

[0089] The processor 116 is coupled to the beamformer 114. The processor 116 may also be described as a processor circuit, which can include other components in communication with the processor 116, such as a memory, beamformer 114, communication interface 118, and / or other suitable components. The processor 116 may include a central processing unit (CPU), a graphical processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a controller, a field programmable gate array (FPGA) device, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 116 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The processor 116 is configured to process the beamformed image signals. For example, the processor 116 may perform filtering and / or quadrature demodulation to condition the image signals. The processor 116 and / or 134 can be configured to control the array 112 to obtain ultrasound data associated with the object 105.

[0090] The communication interface 118 is coupled to the processor 116. The communication interface 118 may include one or more transmitters, one or more receivers, one or more transceivers, and / or circuitry for transmitting and / or receiving communication signals. The communication interface 118 can include hardware components and / or software components implementing a particular communication protocol suitable for transporting signals over the communication link 120 to the host 130. The communication interface 118 can be referred to as a communication device or a communication interface module.

[0091] The communication link 120 may be any suitable communication link. For example, the communication link 120 may be a wired link, such as a universal serial bus (USB) link or an Ethernet link. Alternatively, the communication link 120 may be a wireless link, such as an ultra- wideband (UWB) link, an Institute of Electrical and Electronics Engineers (IEEE) 802.11 WiFi link, or a Bluetooth link.

[0092] At the host 130, the communication interface 136 may receive the image signals. The communication interface 136 may be substantially similar to the communicationinterface 118. The host 130 may be any suitable computing and display device, such as a workstation, a personal computer (PC), a laptop, a tablet, or a mobile phone.

[0093] The processor 134 is coupled to the communication interface 136. The processor 134 may also be described as a processor circuit, which can include other components in communication with the processor 134, such as the memory 138, the communication interface 136, an optional speaker 139, haptic feedback device 135, and / or other suitable components. The processor 134 may be implemented as a combination of software components and hardware components. The processor 134 may include a central processing unit (CPU), a graphics processing unit (GPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a controller, an FPGA device, another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 134 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration. The processor 134 can be configured to generate image data from the image signals received from the probe 110. The processor 134 can apply advanced signal processing and / or image processing techniques to the image signals. In some aspects, the processor 134 can form a three-dimensional (3D) volume image from the image data. In some aspects, the processor 134 can perform real-time processing on the image data to provide a streaming video of ultrasound images of the object 105. In some aspects, the host 130 includes a beamformer. For example, the processor 134 can be part of and / or otherwise in communication with such a beamformer. The beamformer in the in the host 130 can be a system beamformer or a main beamformer (providing one or more subsequent stages of beamforming), while the beamformer 114 is a probe beamformer or micro-beamformer (providing one or more initial stages of beamforming).

[0094] The memory 138 is coupled to the processor 134. The memory 138 may be any suitable storage device, such as a cache memory (e.g., a cache memory of the processor 134), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory, solid state memory device, hard disk drives, solid state drives, other forms of volatile and non-volatile memory, or a combination of different types of memory.

[0095] The memory 138 can be configured to store subject information, measurements, data, or files relating to a subject’s medical history, history of procedures performed,anatomical or biological features, characteristics, or medical conditions associated with a subject, computer readable instructions, such as code, software, or other application, as well as any other suitable information or data. The memory 138 may be located within the host 130. Subject information may include measurements, data, files, other forms of medical history, such as but not limited to ultrasound images, ultrasound videos, and / or any imaging information relating to the subject’s anatomy. The subject information may include parameters related to an imaging procedure such as an anatomical scan window, a probe orientation, and / or the subject position during an imaging procedure. The memory 138 can also be configured to store information related to the training and implementation of machine learning algorithms (e.g., neural networks) and / or information related to implementing image recognition algorithms for detecting / segmenting anatomy, image quantification algorithms, and / or image acquisition guidance algorithms, including those described herein.

[0096] The display 132 is coupled to the processor circuit 134. The display 132 may be a monitor or any suitable display. The display 132 is configured to display the ultrasound images, image videos, and / or any imaging information of the object 105.

[0097] The ultrasound imaging system 100 may be used to assist a sonographer in performing an ultrasound scan. The scan may be performed in a point-of-care setting. In some instances, the host 130 is a console or movable cart. In some instances, the host 130 may be a mobile device, such as a tablet, a mobile phone, or portable computer. During an imaging procedure, the ultrasound system can acquire an ultrasound image of a particular region of interest within a subject’s anatomy. The ultrasound imaging system 100 may then analyze the ultrasound image to identify various parameters associated with the acquisition of the image such as the scan window, the probe orientation, the subject position, and / or other parameters. The ultrasound imaging system 100 may then store the image and these associated parameters in the memory 138. At a subsequent imaging procedure, the ultrasound imaging system 100 may retrieve the previously acquired ultrasound image and associated parameters for display to a user which may be used to guide the user of the ultrasound imaging system 100 to use the same or similar parameters in the subsequent imaging procedure, as will be described in more detail hereafter.

[0098] In some aspects, the processor 134 may utilize deep learning-based prediction networks to identify parameters of an ultrasound image, including an anatomical scan window, probe orientation, subject position, identify and location of anatomical features, and / or other parameters. In some aspects, the processor 134 may receive metrics or perform various calculations relating to the region of interest imaged or the subject’s physiologicalstate during an imaging procedure. These metrics and / or calculations may also be displayed to the sonographer or other user via the display 132.

[0099] In some aspects, the host 130 may also include a speaker 180. The speaker 180 may for example be used to provide advisory tones, beeps, or other auditory feedback to the user.

[0100] Before continuing, it should be noted that the examples described above are provided for purposes of illustration, and are not intended to be limiting. Other devices and / or device configurations may be utilized to carry out the operations described herein.

[0101] Figure 2 is a schematic diagram of a processor circuit 250, according to aspects of the present disclosure. The processor circuit 250 may be implemented in the ultrasound imaging system 100, or other devices or workstations (e.g., third-party workstations, network routers, etc.), or on a cloud processor or other remote processing unit, as necessary to implement the method. As shown, the processor circuit 250 may include a processor 260, a memory 264, and a communication module 268. These elements may be in direct or indirect communication with each other, for example via one or more buses.

[0102] The processor 260 may include a central processing unit (CPU), a digital signal processor (DSP), a controller, or any combination of general-purpose computing devices, reduced instruction set computing (RISC) devices, application-specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other related logic devices, including mechanical and quantum computers. The processor 260 may also comprise another hardware device, a firmware device, or any combination thereof configured to perform the operations described herein. The processor 260 may also be implemented as a combination of computing devices, e.g., a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0103] The memory 264 may include a cache memory (e.g., a cache memory of the processor 260), random access memory (RAM), magnetoresistive RAM (MRAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read only memory (EPROM), electrically erasable programmable read only memory (EEPROM), flash memory, solid state memory device, hard disk drives, other forms of volatile and nonvolatile memory, or a combination of different types of memory. In an aspect, the memory 264 includes a non-transitory computer-readable medium. The memory 264 may store instructions 266. The instructions 266 may include instructions that, when executed by the processor 260, cause the processor 260 to perform the operations described herein.Instructions 266 may also be referred to as code. The terms “instructions” and “code” should be interpreted broadly to include any type of computer-readable statement s). For example, the terms “instructions” and “code” may refer to one or more programs, routines, subroutines, functions, procedures, etc. “Instructions” and “code” may include a single computer-readable statement or many computer-readable statements.

[0104] The communication module 268 can include any electronic circuitry and / or logic circuitry to facilitate direct or indirect communication of data between the processor circuit 250, and other processors or devices. In that regard, the communication module 268 can be an input / output (I / O) device. In some instances, the communication module 268 facilitates direct or indirect communication between various elements of the processor circuit 250 and / or the ultrasound imaging system 100. The communication module 268 may communicate within the processor circuit 250 through numerous methods or protocols. Serial communication protocols may include but are not limited to United States Serial Protocol Interface (US SPI), Inter-Integrated Circuit (I2C), Recommended Standard 232 (RS- 232), RS-485, Controller Area Network (CAN), Ethernet, Aeronautical Radio, Incorporated 429 (ARINC 429), MODBUS, Military Standard 1553 (MIL-STD-1553), or any other suitable method or protocol. Parallel protocols include but are not limited to Industry Standard Architecture (ISA), Advanced Technology Attachment (ATA), Small Computer System Interface (SCSI), Peripheral Component Interconnect (PCI), Institute of Electrical and Electronics Engineers 488 (IEEE-488), IEEE-1284, and other suitable protocols. Where appropriate, serial and parallel communications may be bridged by a Universal Asynchronous Receiver Transmitter (UART), Universal Synchronous Receiver Transmitter (US ART), or other appropriate subsystem.

[0105] External communication (including but not limited to software updates, firmware updates, model sharing between the processor and central server, or readings from the ultrasound imaging system 100) may be accomplished using any suitable wireless or wired communication technology, such as a cable interface such as a universal serial bus (USB), micro USB, Lightning, or FireWire interface, Bluetooth, Wi-Fi, ZigBee, Li-Fi, or cellular data connections such as 2G / GSM (global system for mobiles) , 3G / UMTS (universal mobile telecommunications system), 4G, long term evolution (LTE), WiMax, or 5G. For example, a Bluetooth Low Energy (BLE) radio can be used to establish connectivity with a cloud service, for transmission of data, and for receipt of software patches. The controller may be configured to communicate with a remote server, or a local device such as a laptop, tablet, or handheld device, or may include a display capable of showing status variables and otherinformation. Information may also be transferred on physical media such as a USB flash drive or memory stick.

[0106] Figure 3 is a schematic, diagrammatic representation of a patient 300 on whose abdomen the ultrasound blind sweep protocol will be followed, according to aspects of the present disclosure. Visible on the abdomen 310 of the patient 300 is a desired sweep pattern 320 intended to capture images of desired features of the patient’s anatomy. The sweep pattern 320 includes multiple vertical sweep lines 330 and multiple horizontal sweep lines 340. Each sweep line 330, 340 represents a desired path for one imaging sweep of the abdomen 310. In the example shown in Figure 4, the sweep pattern includes three vertical sweep lines 330 labeled L (patient’s left), M (patient’s middle), and R (patient’s right), all in an upward direction with respect to the patient, and three horizontal sweep lines 340 labeled Cl (bottom), C2 (middle), and C3 (top), all in a right-to-left direction with respect to the patient. However, it is understood that a sweep pattern 320 may include more or fewer sweep lines 330, including vertical sweep lines 330, horizontal sweep lines 340, or combinations thereof, in any combination of upward, downward, left, or right directions based on the patient’s fundal height. For example, if the patient’s belly is bigger in size, more sweeps may be needed. Furthermore, a sweep pattern 320 may cover other portions of the patient’s body, including but not limited to the head, neck, spine, limbs, etc. Types of blind sweep protocol include but are not limited to obstetric sweep imaging (OSI), volume sweep imaging (VSI), 6-Stage, Fetal Age Machine Learning Initiative (FAMLI), and otherwise.

[0107] These sweep patterns represent desired probe motion information, including desired positions, a desired velocity or velocities, and / or a desired orientation of the ultrasound probe while the ultrasound probe is obtaining a plurality of ultrasound image frames during the sweep. It is noted that the desired sweep patterns or blind sweep protocols stored in a memory of the processor may include only vertical sweeps, only horizontal sweeps, may include a grid (e.g., 3x3, 5x5, etc.) of vertical and horizontal sweeps, and may also include associated parameters such as desired probe motion (e.g., positions, velocities, and / or orientations) stored in the memory(e.g., blind sweep protocol 440 in Fig. 5). Depending on the implementation, sweeps may include curved, diagonal, and other types of sweeps. The protocols and their associated parameters can for example be based on standards established by authorities in the field (physician organizations, sonographer organizations, etc.), published in scholarly journals / textbooks, etc.

[0108] Figure 4 is a schematic, diagrammatic representation, in block diagram form, of an example ultrasound blind sweep uterine imaging extent verification process 345,according to aspects of the present disclosure. In the example shown in Figure 4, an ultrasound probe 110 is used to perform a blind sweep protocol 320 to image the contents and surroundings of a uterus 350, resulting in per-frame YOLO anatomy detections 360, which can then be compiled into per-sweep anatomy detections 370, and per-exam (e.g., across multiple sweeps) anatomy detections 380, of anatomy such as the fetal head 362, fetal spine 364, and / or fetal abdomen 366. These anatomy detections can then be used to perform any of the ultrasound blind sweep uterine imaging extent verification methods described herein.

[0109] Figure 5 is a schematic, diagrammatic representation, in hybrid block diagram / flow diagram form, of an example ultrasound blind sweep uterine imaging extent verification system 400, according to aspects of the present disclosure. An ultrasound probe 110 operated by a novice user 410 performs a blind sweep protocol 320 on the body of a patient 300 and sends ultrasound imaging data to a host 130 such as a tablet, smartphone, ultrasound cart, etc. In step 415, the host 130 generates ultrasound images using the ultrasound image data obtained by the ultrasound probe 110. The host 130 can control the ultrasound probe 110 to obtain the ultrasound image data (e.g., the host 130 establishes communication with the ultrasound probe 110, the host 130 sends control signals to start and / or stop acquisition of ultrasound image data, the host 130 sends power signals to power the ultrasound probe 110, etc.).

[0110] In step 420, the host 130 uses the ultrasound images generated in step 415 and / or the ultrasound image data used to generate the ultrasound images to determine if the full extent of the uterus has been imaged by the blind sweep protocol, as described in more detail below. In step 430, the host generates a visual representation on a display, showing an indication of whether the uterine imagine is complete or incomplete and / or a graphic associated with the determination that the imaging is complete or incomplete.

[0111] In step 470, if the imaging is incomplete, the system instructs the user to repeat the incomplete sweeps (thereby resulting in further ultrasound image frames). In step 490, if the imaging is complete, then the blind sweep protocol is also complete.

[0112] Flow diagrams and block diagrams are provided herein for exemplary purposes; a person of ordinary skill in the art will recognize myriad variations that nonetheless fall within the scope of the present disclosure. For example, any of the steps described herein may optionally include an output to a user of information relevant to the step, and may thus represent an improvement in the user interface over existing art by providing information not otherwise available. Similarly, block diagrams may show a particular arrangement of components, modules, services, steps, processes, or layers, resulting in a particular data flow.It is understood that some embodiments of the systems disclosed herein may include additional components, that some components shown may be absent from some embodiments, and that the arrangement of components may be different than shown, resulting in different data flows while still performing the methods described herein. The logic of flow diagrams may be shown as sequential. However, similar logic could be parallel, massively parallel, object oriented, real-time, event-driven, cellular automaton, or otherwise, while accomplishing the same or similar functions. In order to perform the methods described herein, a processor may divide each of the steps described herein into a plurality of machine instructions, and may execute these instructions at the rate of several hundred, several thousand, several million, or several billion per second, in a single processor or across a plurality of processors. Such rapid execution may be necessary in order to execute the method in real time or near-real time as described herein. For example, to provide an assessment of fetal or maternal anatomy locations and / or an indication of whether the coverage of the uterus is complete, the system may need to perform object recognition in real time, as the ultrasound images are acquired, at the rate of 60 cycles per second, 120 cycles per second, or other rate typical for ultrasound image acquisition.

[0113] Figure 6 is a schematic, diagrammatic representation, in block diagram form, of an example ultrasound blind sweep uterine imaging extent verification process 500, according to aspects of the present disclosure. In the example shown in Figure 6, an ultrasound probe 110 is used to perform a blind sweep protocol 320 to image the contents and surroundings of a uterus 350, which yields ultrasound sweep data 510 that can be used by the YOLO model 520 to generate YOLO anatomy detections 380. Both the ultrasound data 510 and the anatomy detections 380 may serve as inputs to the uterine volume coverage determination unit 420, which outputs a coverage assessment 540 indicating whether uterine coverage is complete or incomplete.

[0114] In that regard, the blind sweep protocol can include that, during a given sweep, the ultrasound probe start at locations of the patient’s body that are outside and / or nonoverlapping with the uterus, then move to locations of the patient’s body that inside and / or overlapping with the uterus, and then end at locations that are outside and / or non-overlapping with the uterus (on an opposite side of the uterus that the ultrasound probe started on). In this manner, when a sweep is complete, the ultrasound image frames from the sweep extend across a complete distance from one side of the uterus to the opposite side of the uterus. For example, in the horizontal sweeps (blind sweep protocol 320 in Figure 6; Cl, C2, and C3 shown in Figure 3), the ultrasound probe starts outside of the uterus on the patient’s right,crosses horizontally over the uterus from the patient’s right towards the patient’s left, and then ends outside of the uterus on the patient’s left. For example, in the vertical sweeps (blind sweep protocol 320 in Figure 6; R, M, and L shown in Figure 3), the ultrasound probe starts outside of the uterus on the side of the patient’s feet (down), crosses over the uterus from side of the patient’s feet (down) towards the side with the patient’s head (up), and then ends outside of the uterus on the side with the patient’s head (up). This is shown by the starting and ending points of the arrow, relative to the circular / ellipsoidal shape representing the uterus 350, in, e.g., Figure 4 (sweep pattern 320), Figure 6 (blind sweep protocol 320), and Figure 18 (complete sweep 1820).

[0115] It is understood that a sweep can extend in any direction (e.g., diagonal, curved, etc.) and not just longitudinally in the left-right / right-left direction or up-down / down -up direction. For example, if the sweep direction is diagonal, the sweep that is complete can extend from outside of the uterus on the upper right of the uterus, across the uterus diagonally, to outside of the uterus on the lower left of the uterus.

[0116] When a sweep is incomplete, the ultrasound image frames from the sweep do not extend across a complete distance from one side of the uterus to the opposite side of the uterus. For example, sweep can begin inside the uterus (instead of outside) and / or can end inside the uterus (instead of outside). This is shown by the starting and ending points of the arrow, relative to the circular / ellipsoidal shape representing the uterus 350, in, e.g., Figure 18 (incomplete sweep 1830).

[0117] The coverage assessment 540 described herein determines whether the plurality of ultrasound images frames covers the entire distance across the uterus (e.g., from outside the uterus on one side, across the distance spanning the uterus, to outside the uterus on the opposite side of the uterus). The coverage assessment 540 (determine of sweep as complete or incomplete) can be made on a per sweep basis (e.g., a single sweep among the many sweeps of the blind sweep acquisition). When the coverage assessment 540 is made on a per sweep basis, the ultrasound image frames from the respective sweep that is complete should include the opposite portions of the perimeter / surf ace of the uterus corresponding to the location of the sweep. A determination that the sweep is complete (incomplete) can be an indication that the ultrasound image frames from the respective sweep do (do not) include the opposite portions of the perimeter / surf ace of the uterus corresponding to the location of the sweep. The coverage assessment 540 can be made on multiple sweep basis (two or more sweeps, a majority of sweeps, all of the sweeps, etc., of the many sweeps of the blind sweep acquisition). When the coverage assessment 540 is made on the basis of all of the sweeps,the ultrasound image frames from all of the sweeps in the blind sweep acquisition collectively should include the entire perimeter / surf ace of the uterus. A determination that the sweep is complete (incomplete) can be an indication that the ultrasound image frames from all of the sweeps in the blind sweep acquisition do (do not) include the entire perimeter / surface of the uterus.

[0118] Figure 7 is a schematic, diagrammatic representation, in block diagram form, of at least a portion of an example ultrasound blind sweep uterine imaging extent verification system 400, according to aspects of the present disclosure. A set of ultrasound image sequences 610, also known as cineloops or cine scans (e.g., one cine scan per sweep of the blind sweep protocol) are received (whether one at a time, simultaneously, or in groups) by an object detector 620 (which may for example be the YOLO model 520). Each cine scan can include a plurality of ultrasound image frames (e.g., 100-1000 ultrasound image frames, and / or other values both larger and smaller). The object detector 620 may for example be a machine learning (ML) neural network as described below, although other types of object detectors may be used instead or in addition, including classical image recognition algorithms. The object detector 620 can be a deep learning network (e.g., convolutional neural network or CNN) trained to detect maternal anatomy and / or fetal anatomy. An output of the object detector may for example include annotated versions of the cineloops 610 that include bounding boxes for each detected anatomy in each frame where anatomy was detected. The object detector 620 can generate, as its output, a detection of a uterine wall within ultrasound image frames of the cine scan(s) and / or one or multiple fetal anatomical parts within the ultrasound image frames of the cine scan(s). Detected anatomy may for example include the fetal head, abdomen, heart, pelvis, and / or placenta 240.

[0119] The outputs of the object detector 620 are then passed to a uterine volume coverage determination unit or module 420, which may for example be software, hardware, firmware, analog / digital logic, analog / digital circuitry, or combinations thereof. The uterine volume coverage determination unit 420 determines, using the anatomy detections, whether the ultrasound sweep coverage of the uterine volume is complete or incomplete. In the example shown in Figure 7, the uterine volume coverage determination unit 420 includes a direct detection 632 of the uterine wall, detections 634 of anatomy within the uterine interior and / or exterior, an anatomy detection dimensionality reduction module 636, automatic feedback 638 that occurs during a sweep, and an Al performance assessment module 640.

[0120] The processor (e.g., processor 134 of Fig. 1, processor 116 of Fig. 1, and / or other processors) provide, to a display (e.g., display 132 of Fig. 1 and / or other displays) incommunication therewith, an output 430 that is representative of the determination of whether or not the uterine coverage is complete. The visual representation 430 can be or include a binary indicator 650 of whether or the uterine coverage is complete. The screen display 430 may also include a graphical representation 660 associated with the determination of whether the uterine coverage is complete. The graphical representation 660 may for example include drawings, ultrasound image frames, cineloops, or generated graphics, either with or without text or symbols as annotations, including graphics, images, text, and / or other visual representations described herein and / or the outputs of the individual determinations 632, 634, 636, 638, or 640).

[0121] Figure 8A is a schematic, diagrammatic overview, in block diagram form, of a training mode 700 for an untrained neural network 710a, according to aspects of the present disclosure. In the example shown in Figure 7A, a set of training data 705a includes ultrasound cineloops of probe sweeps annotated with the corresponding anatomy (head, heart, abdomen, pelvis, membrane, placenta, etc.). The training data 705a is fed into an untrained neural network 710a in an iterative training process that will be familiar to a person of ordinary skill in the art.

[0122] The parameters of a network model (e.g., the weights at each artificial neuron) are initialized with initial values A that may be random values or with results from training on prior datasets. In an iterative process, the network is used to make detection inferences on the training images, the results are compared with the ground truth annotations, and an optimizer is used to adjust the network parameters B until a metric of accuracy is maximized.

[0123] Thus, an output of this training process 700 is a trained neural network 710b, wherein the parameters B (e.g., weights) are optimized for generating accurate bounding boxes for the anatomy imaged in the training data 705a.

[0124] Figure 8B is a schematic, diagrammatic overview, in block diagram form, of an inference mode or clinical usage mode 704 for the trained neural network 710b, according to aspects of the present disclosure. In clinical usage, an ultrasound video, cineloop, or cine sweep 720 of the blind sweep is fed to the trained and validated neural network 710b for analysis. The trained and validated neural network 710b then produces, as an output, anatomy detection bounding boxes 740 for each image (or the entire sweep). In some aspects, a confidence value can be determined as a normalized value in the range [0-1], where 0 indicates lowest confidence, and 1 indicates highest confidence that the detection is correct.

[0125] Figure 9 is a schematic, diagrammatic illustration, in block diagram form, of the detection of anatomy (e.g., the head, heart, abdomen, pelvis, placenta, membrane, uterinewall, etc.), according to aspects of the present disclosure. A cineloop 810 comprising multiple frames 820 is fed into a trained object detector 830.

[0126] The object detector 830 may implement or include any suitable type of learning network. For example, in some aspects, the object detector 830 could include a neural network, such as a convolutional neural network (CNN). In addition, the convolutional neural network may additionally or alternatively be an encoder-decoder type network, or may utilize a backbone architecture based on other types of neural networks, such as an object detection network, classification network, etc. One example backbone network is the Darknet YOLO backbone, (e.g., Yolov3) which can be used for object detection. The CNN may for example include a set of N convolutional layers, where N may be any positive integer. Fully connected layers can be omitted when the CNN is a backbone. The CNN may also include max pooling layers and / or activation layers. Each convolutional layer may include a set of filters configured to extract features from an input (e.g., from a frame of the ultrasound video). The value N and the size of the filters may vary depending on the aspects. In some instances, the convolutional layers may utilize any non-linear activation function, such as for example a leaky rectified non-linear (ReLU) activation function and / or batch normalization. The max pooling layers gradually shrink the high-dimensional output to a dimension of the desired result (e.g., bounding boxes of a detected feature). Outputs of detection network may include numerous bounding boxes, with most having very low confidence scores and thus being filtered out or ignored. Fully connected layers may be referred to as perception or perceptive layers. In some aspects, perception / perceptive and / or fully connected layers may be found in object detector 830 (e.g., a multi-layer perceptron).

[0127] These descriptions are included for exemplary purposes; a person of ordinary skill in the art will appreciate that other types of learning models, with features similar to or dissimilar to those described above, may be used instead or in addition, without departing from the spirit of the present disclosure.

[0128] Outputs of the object detector 830 may include an annotated cineloop 840 made up of a plurality of annotated image frames 842, possibly including per-frame metrics 845 such as the confidence level of the detections.

[0129] The systems and methods disclosed herein are broadly applicable to different types of features, and can for example draw boxes around the head, heart, placenta, or other anatomical features depending on the implementation. The object detector can be one class or multi-class, depending how the model is built. If another detector is trained separately, then both models can be run separately (e.g., one model for each feature type). Otherwise,multiple feature classes can be identified, and enclosed in detection boxes, at the same time. In an example, the ML model for placenta detection can use exactly the same structure as a model for heart detection. One can either train / run a single detector that detects multiple feature types (a multi-class detector) and provides their locations as an output, along with the confidence score and feature type (class) of each detection. Alternatively, one could run several single-class detectors, each trained to detect a single feature type / class. These separate single-class detectors may have the same architecture (e.g., layers and connections), but would have been trained with different data (e.g., different images and / or annotations) and thus have different weights.

[0130] Figure 10 is a graphical representation 1000 of anatomical detections in middle sweeps 1002 and edge sweeps 1004, according to aspects of the present disclosure. In the middle sweeps 1002, for a low gestational age, complete coverage of the uterus for that sweep is indicated by detecting amniotic fluid 1010 surrounded on either side by exterior tissue 1020 (including but not limited to the uterine wall itself). For a high gestational age middle sweep, complete coverage of the uterus for that sweep is indicated by detecting amniotic fluid 1010 surrounded on either side by exterior tissue 1020, which in turn is surrounded on the outsides by decoupling 1030. Incomplete coverage for a middle sweep at any gestational age may for example be indicated by detection of amniotic fluid 1010 only, or amniotic fluid 1010 that is not surrounded on both sides by exterior tissue 1020. In the edge sweeps 1004, for low gestational age, complete coverage of the uterus for that sweep is indicated by detection of the edge of the amniotic fluid 1010 (e.g., amniotic fluid that does not extend laterally across the entire width of the sweep), surrounded by exterior tissue 1020. For high gestational age edge sweeps, complete coverage of the uterus for that sweep is indicated by detection of the edge of the amniotic fluid 1010 surrounded by exterior tissue 1020 and decoupling 1030. At any gestational age, incomplete coverage for an edge sweep is indicated by detection of amniotic fluid 1010 (whether or not surrounded by exterior tissue 1020) that extends across the entire width of the sweep. The features and indications shown in Figure 10 are exemplary; other features or indications may be used instead or in addition to identify the edges of the uterus, and whether such edges are fully captured by one or more sweeps.

[0131] Figure 11 is an ultrasound image frame 1100 or DL feature map 1110 that includes a detected uterine wall 1120 (dotted line), according to aspects of the present disclosure. Detection of the uterine wall 1120 is important for some aspects of the uterine imaging extent verification system. Specifically, in an edge sweep, detections of the uterinewall are expected across the approximate center of the sweep, as shown here in Figure 11. The absence of such a detection may be indicative that the edge sweep’s coverage of the uterus is incomplete. In middle sweeps, the uterine wall may be expected to be detected twice: first near the beginning of the sweep and again near the end, with amniotic fluid, fetal anatomy, or other non-uterine-wall tissue in between. The absence of such a detection may be indicative that the middle sweep’s coverage of the uterus is incomplete.

[0132] Figure 12 is a schematic, diagrammatic representation, in flow diagram form, of an example uterine imaging extent verification method 632, according to aspects of the present disclosure. It is understood that the steps of method 632 may be performed in a different order than shown in Figure 12, additional steps can be provided before, during, and after the steps, and / or some of the steps described can be replaced or eliminated in other aspects. One or more of steps of the method 632 can be carried by one or more devices and / or systems described herein, such as components of the ultrasound imaging system 100, ultrasound blind sweep uterine imaging extent verification system 400, and / or processor circuit 250.

[0133] In step 1210, the method 632 includes controlling the ultrasound probe to acquire images during a single sweep. Execution then proceeds to step 1220.

[0134] In step 1220, the method 632 includes performing anatomy detections on the image frames using an object detector (e.g., a machine-learning-based object detector as described above). Execution then proceeds to step 1225.

[0135] In step 1225, the method 632 includes post-processing, as described below in Figure 13. Execution then proceeds to step 1230.

[0136] In step 1230, the method 632 includes determining whether the uterine wall has been detected twice in the case of a middle sweep, or once in the case of an edge sweep (as described above in Figures 10 and 11). If yes, execution proceeds to step 1240. If no, execution proceeds to step 1250.

[0137] In step 1240, the uterine volume coverage for the current sweep is complete (e.g., the expected portion of the uterus has been imaged), and the method 632 includes accepting the sweep (e.g., notifying the user that the sweep is accepted). The method 632 is now complete.

[0138] In step 1250, the uterine volume coverage for the current sweep is incomplete (e.g., the expected portion of the uterus has not been imaged), and the method 632 includes rejecting the sweep (e.g., notifying the user that the sweep is rejected and needs to be performed again). The method 632 is now complete.

[0139] Figure 13 is a schematic, diagrammatic view of an ultrasound blind sweep uterine imaging extent verification method 1300 for multiple sweeps, according to aspects of the present disclosure. In the example shown in Figure 13, an ultrasound image 1310 is fed to an object detector 520 (e.g., a trained YOLO model). As part of the operation of the object detector 520 (e.g., a CNN), a feature map 1320 is generated. The feature map 1320 is the result of applying a filter (e.g., part of CNN) to an input image. A feature map can be generated at each layer, which the feature map being the output of the respective layer. The feature map 1320 is a visual representation of the features that a CNN detects at the respective layer. For example, the feature map 1320 can provide a visual representation that the CNN has detected parts of the anatomy and not others. Certain features in the image are emphasized in the feature map 1320 (e.g., features that are detected at the respective layer, which can include portions of the anatomy, such as the uterine wall) and other features in the image (e.g., features that are not detected at the respective layer, which can include portions of the body other than the anatomy) are de-emphasized. The feature map 1320 is then fed to a post-processor 1225, which outputs a post-processed feature map 1330 that includes a highlighted representation 1335 of the uterine wall 1120 (see Figure 11).

[0140] The system acquires and processes a large number of frames 1330, across multiple sweeps (e.g., across all horizontal and all vertical sweeps), and then, in a compile step 1340, compiles all of the uterine contours at the locations where they were detected, to yield a three-dimensional (3D) model 1350 of the uterine wall 1120. The 3D model is then received by a coverage estimator 1360, which uses the geometry of the 3D model 1350 to determine whether the 3D model includes any missing pieces of the uterine wall 1120.

[0141] For example, the edges of the uterine wall can be detected in the feature map using imaging processing algorithms that are not Al-based. Image-processing and / or postprocessing techniques that can be used include, but are not limited to edge detection algorithms (Canny Edge Detector, Sobel Edge Detector, etc.) or contrast manipulation algorithms (histogram equalization, etc.).

[0142] It is noted that a combination of these techniques can be used to more accurately detect the edges of uterine wall. For example, in the images belonging to the rightmost vertical sweep (pictured here in Figure 13), the uterine wall was detected in many of the images. This may for example show that the sweep adequately covered the uterine edges in the lateral (L-R) and anterior-posterior (A-P) directions. Similarly, if the uterine edge starts to be detected in the nth frame from the start of the sweep and stops being detected in the mth frame from the end of the sweep, then that detection (uterine wall surrounded by non-uterinewall tissue) confirms adequate sweep coverage in the cranial -caudal direction. With all 3 directional components being confirmed, this sweep can be deemed to be adequately obtained. Such information can then be consolidated across all sweeps. Some particular expectations may be unique to each sweep, e.g. the middle vertical sweep should not contain lateral uterine edges, otherwise the sweep was incorrectly obtained. By combining all of the uterine contours thus obtained, for all of the sweeps, the system then has enough information to make an exam-level completeness determination. The extent of the uterine contours can indicate whether the sweep is complete or not. For example, if the uterine contour has continuous edges forming a circular or elliptical shape (in 2D) or spherical or ellipsoidal shape (in 3D), then the sweep is complete. If the uterine contour has discontinuities (e.g., missing edges), then the sweep is incomplete.

[0143] Figure 14 is a schematic, diagrammatic representation, in flow diagram form, of an example uterine imaging extent verification method 1400, according to aspects of the present disclosure. It is understood that the steps of method 1400 may be performed in a different order than shown in Figure 14, additional steps can be provided before, during, and after the steps, and / or some of the steps described can be replaced or eliminated in other aspects. One or more of steps of the method 1400 can be carried by one or more devices and / or systems described herein, such as components of the ultrasound imaging system 100, ultrasound blind sweep uterine imaging extent verification system 400, and / or processor circuit 250.

[0144] In step 1410, the method 1400 includes controlling the ultrasound probe to acquire images during multiple sweeps of an examination (e.g., all of the sweeps of a complete examination). Execution then proceeds to step 1420.

[0145] In step 1420, the method 1400 includes performing anatomy detections on the image frames using an object detector (e.g., a machine-learning-based object detector as described above). Execution then proceeds to step 1425.

[0146] In step 1425, the method 1400 includes post-processing, as described above in Figure 13. Execution then proceeds to step 1430.

[0147] In step 1430, the method 1400 includes determining whether a complete uterine wall contour has been detected (as described above in Figure 13). If yes, execution proceeds to step 1440. If no, execution proceeds to step 1450.

[0148] In step 1440, the uterine volume coverage for the current examination is complete, and the method 1400 includes accepting the examination (e.g., notifying the user that the examination is accepted). The method 1400 is now complete.

[0149] In step 1450, the uterine volume coverage for the current examination is incomplete, and the method 1400 includes rejecting the examination (e.g., notifying the user that the examination is rejected, and that at least some portions of it need to be performed again). The method 1400 is now complete.

[0150] Figure 15 is a schematic, diagrammatic representation, in flow diagram form, of an example uterine imaging extent verification method 634, according to aspects of the present disclosure. It is understood that the steps of method 634 may be performed in a different order than shown in Figure 15, additional steps can be provided before, during, and after the steps, and / or some of the steps described can be replaced or eliminated in other aspects. One or more of steps of the method 634 can be carried by one or more devices and / or systems described herein, such as components of the ultrasound imaging system 100, ultrasound blind sweep uterine imaging extent verification system 400, and / or processor circuit 250.

[0151] In step 1510, the method 634 includes controlling the ultrasound probe to acquire images during a particular sweep. Execution then proceeds to step 1520.

[0152] In step 1520, the method 634 includes performing anatomy detections on the image frames using an object detector (e.g., a machine-learning-based object detector as described above). Execution then proceeds to step 1530.

[0153] In step 1530, the method 634 includes determining whether expected intrauterine and / or extrauterine anatomy has been detected. The expected detections may for example include maternal anatomy such as the bladder, cervix, etc., and / or fetal anatomy such as the head, spine, abdomen, etc., the detection of which may indicate that a sweep has fully covered the expected portion of the uterus. Expected anatomy can be location specific within a sweep. For example, maternal anatomy detections may be expected at the beginning and / or ending of a sweep, whereas fetal anatomy is expected in the middle of the sweep (because the uterus should be centered within the sweep). Expected anatomy may be particular to a given sweep. For example, detection of the cervix may be expected in the M and Cl sweeps, but not in the R, L, C2, or C3 sweeps. If yes, the expected anatomy is detected, then execution proceeds to step 1540. If no, execution proceeds to step 1550.

[0154] In step 1540, the uterine volume coverage for the current sweep is complete, and the method 634 includes accepting the sweep (e.g., notifying the user that the sweep is accepted). The method 634 is now complete.

[0155] In step 1550, the uterine volume coverage for the current sweep is incomplete, and the method 634 includes rejecting the sweep (e.g., notifying the user that the sweep is rejected and needs to be performed again). The method 634 is now complete.

[0156] Figure 16 is a schematic, diagrammatic representation, in flow diagram form, of an example uterine imaging extent verification method 636, according to aspects of the present disclosure. It is understood that the steps of method 636 may be performed in a different order than shown in Figure 16, additional steps can be provided before, during, and after the steps, and / or some of the steps described can be replaced or eliminated in other aspects. One or more of steps of the method 636 can be carried by one or more devices and / or systems described herein, such as components of the ultrasound imaging system 100, ultrasound blind sweep uterine imaging extent verification system 400, and / or processor circuit 250.

[0157] In step 1610, the method 636 includes controlling the ultrasound probe to acquire images during a particular sweep. Execution then proceeds to step 1620.

[0158] In step 1620, the method 636 includes performing anatomy detections on the image frames using an object detector (e.g., a machine-learning-based object detector as described above). Execution then proceeds to step 1630.

[0159] In step 1630, the method 636 includes reducing the dimensionality of the detections. Reducing the dimensionality can include using a statistical quantity / measure / description (e.g., variance, skewness, kurtosis, etc.) of the ultrasound image frames and / or anatomical detections from the ultrasound image frames. The statistical quantity / measure / description can be used to determine sweep completeness.

[0160] This may be done for example by calculating the centroids of fetal / maternal anatomies such as the head, urinary bladder, and placenta from their respective YOLO bounding box locations. Diffuse features such as amniotic fluid may be represented by features such as the percentage of exam images with amniotic fluid detections, the variance / skewness of amniotic fluid detections, etc. More complex features may also be engineered via topological data analysis, topographical or clustering methods, etc. The benefits of dimensionality reduction include extraction of key inter-frame features in the sweep, which provide insights into the 3D shape of intrauterine objects. This information can help to improve the accuracy of uterine coverage estimation. Execution then proceeds to step 1640.

[0161] In step 1640, the method 636 includes determining whether the reduced dimensionality data indicates the expected coverage of the uterine volume for the currentsweep. This may be done for example by examining the skew or kurtosis of the heights of key anatomies, e.g. the fetal head or abdomen, across multiple frame dimensions. Asymmetry in the shape of head detections across the frame dimension might indicate that the fetal head was not fully covered by a sweep, and therefore the sweep was incomplete. If yes, execution proceeds to step 1650. If no, execution proceeds to step 1660.

[0162] In step 1650, the uterine volume coverage for the current sweep is complete, and the method 636 includes accepting the sweep (e.g., notifying the user that the sweep is accepted). The method 636 is now complete.

[0163] In step 1660, the uterine volume coverage for the current sweep is incomplete, and the method 636 includes rejecting the sweep (e.g., notifying the user that the sweep is rejected and needs to be performed again). The method 636 is now complete.

[0164] Figure 17 is a graph 1700 of the variance or skewness of amniotic fluid (AF) detections, according to aspects of the present disclosure. Amniotic fluid detections may for example occur in step 1630 of Figure 16. In most cases, there should only be one AF detection per frame, and where the AF detections start and end is a highly salient piece of information. The graph 1700 includes a scatterplot 1710 of amin norm for incomplete (false) uterine coverage (where amin_norm = first AF frame / total # frames), a scatterplot 1720 of amin norm for complete (true) uterine coverage, a scatterplot 1730 of amax norm for incomplete (false) uterine coverage (where amax norm = last AF frame / total # frames), and a scatterplot 1740 of amax norm for complete (true) uterine coverage. Also visible are the medians 1715, 1725, 1735, and 1745 of each scatterplot.

[0165] Generally speaking, a sweep with complete uterine coverage should have no amniotic fluid detections in the first and last n frames, indicating that the sweep started and ended in an extra-uterine position and thus traversed the full uterine extent. Sweeps with incomplete uterine extent tend to have amniotic fluid detections at or near the first and / or last frame of the sweep. Thus, a value of amin norm that is close to zero (e.g., less than 0.05) may indicate that the sweep is incomplete, whereas a value of amin norm that is far from zero (e.g., 0.05 or higher) may indicate that the sweep is complete. Similarly, a value of amax norm that is close to 1.0 (e.g., greater than 0.9) may indicate that the sweep is incomplete, whereas a value of amax norm that is far from 1.0 (e.g., 0.9 or less) may indicate that the sweep is complete. As can be seen in the graph 1700, for “false” sweeps that do not provide full uterine coverage, the median value 1715 of amin norm is less than 0.05, indicating that half of the data points have a value smaller than this, indicating poor or incomplete uterine coverage. Conversely, for “true” or complete sweeps, the median value ofamin norm is greater than 0.05, indicating that more than half the data points are above 0.05, thus indicating that the associated sweeps are complete. Similarly, for incomplete sweeps, the median value 1735 of amax norm is extremely close to 1.0, indicating that half of the associated sweeps have amniotic fluid detection in the final frame, whereas for complete sweeps, the median value 1745 is approximately 0.8, indicating that the vast majority of the associated sweeps do not have amniotic fluid detections in the final frame or in, for example, the final 5 frames. This analysis may for example take place in step 1640 of Figure 16.

[0166] Figure 18 is a schematic, diagrammatic illustration of a live uterine coverage feedback process 1800, according to aspects of the present disclosure. The process 1800 provides automatic feedback of the location of the ultrasound probe 110 during a sweep. A complete sweep 1820 and incomplete sweep 1830 are shown, with real-time feedback. The determination of whether the ultrasound probe is currently imaging the inside or the outside of the uterus 350 can be made by any of the methods described herein. During the sweep, an indicator 1810, such as a light, sound, or display could show whether the current ultrasound image is inside the uterus 350 or outside the uterus 350. In the example shown in Figure 18, probe positions outside the uterus are marked with a first color 1840 and the letter “O” for “Outside”, whereas probe positions inside the uterus 350 are marked with a second color 1850 and the letter “I” for “Inside”. In an example, the desired duration for a sweep is about 10 seconds, such that the sweep should be about 50% complete after 5 seconds and 100% complete after 10 seconds. Thus, in an example of a complete sweep, the probe is outside the uterus at time 0 seconds, inside the uterus at time 5 seconds, and outside the uterus again at time 10 seconds. In an example of an incomplete sweep, the probe is outside the uterus at time 0 seconds, inside the uterus at time 5 seconds, and still remains inside the uterus at time 10 seconds. It is understood that other types of indicators 1810, as would occur to a person of ordinary skill in the art, may be used instead or in addition. By providing the indications described above, the uterine imaging extent verification system allows the user to see, for example, the amount of distance across the patient’s abdomen that comprises the uterus and its contents, and therefore to determine whether the current sweep has imaged an expected extent of the uterus. For example, in edge sweeps, it may be expected that a majority of the sweep is outside the uterus, whereas in middle sweeps, it may be expected that a majority of the sweep is inside the uterus. If these expectations are not met by the indications, then the patient may know to repeat the sweep or to repeat the entire blind sweep protocol (both resulting in further ultrasound image frames).

[0167] Figure 19 is a schematic, diagrammatic representation, in flow diagram form, of an example automatic location feedback method 638, according to aspects of the present disclosure. It is understood that the steps of method 638 may be performed in a different order than shown in Figure 19, additional steps can be provided before, during, and after the steps, and / or some of the steps described can be replaced or eliminated in other aspects. One or more of steps of the method 638 can be carried by one or more devices and / or systems described herein, such as components of the ultrasound imaging system 100, ultrasound blind sweep uterine imaging extent verification system 400, and / or processor circuit 250.

[0168] In step 1910, the method 638 includes controlling the ultrasound probe to acquire images during a particular sweep. Execution then proceeds to step 1920.

[0169] In step 1920, the method 638 includes performing anatomy detections on the image frames using an object detector (e.g., a machine-learning-based object detector as described above). Execution then proceeds to step 1930.

[0170] In step 1930, the method 638 includes determining whether the detections are inside or outside the uterus, by any of the methods or criteria described herein. For example, if the object / anatomy detector 620 (Figure 7) is trained to detect both maternal anatomy and fetal anatomy, then fetal anatomy would be detected when the probe is positioned during the sweep to be imaging the inside of the uterus and maternal anatomy would be detected (and / or there would be an absence of fetal anatomy detections) when the probe is positioned during the sweep to be imaging outside of the uterus. For example, if the object / anatomy detector 620 (Figure 7) is trained to detect only fetal anatomy, then fetal anatomy would be detected when the probe is positioned during the sweep to be imaging the inside of the uterus and there would be an absence of fetal anatomy detections when the probe is positioned during the sweep to be imaging outside of the uterus. If yes, execution proceeds to step 1940. If no, execution proceeds to step 1950.

[0171] In step 1940, the method 638 includes outputting an indication that the current probe image is outside the uterus. Such an indication may be visual, auditory, haptic, or otherwise. The method 638 is now complete.

[0172] In step 1950, the method 638 includes outputting an indication that the current probe image is inside the uterus. Such an indication may be visual, auditory, haptic, or otherwise. The method 638 is now complete.

[0173] The method 638 may run continuously during a sweep, such that the “O” indicator of Figure 18 is shown when the probe is outside of the uterus (e.g., an example of the outputfrom step 1940), and the “I” indicator of Figure 18 is shown when the probe is inside of the uterus (e.g., an example of the output from step 1950).

[0174] Figure 20A is a schematic, diagrammatic representation, in flow diagram form, of an example first deep learning model training step 2000, according to aspects of the present disclosure. In a training process or training mode 2000, ultrasound images 2005 are fed to pre-trained deep learning model “A” 2010. Pre-trained can indicate that DL model A 2010 has been previously trained to estimate a clinical feature (e.g., gestational age, fetal presentation, deepest vertical pocket, placenta previa, etc.) from input ultrasound images.

[0175] Aspects related to a deep learning model trained to estimate clinical features are described in U.S. Provisional Application No. 63 / 611,810, filed December 19, 2023, titled “Low-Lying Placenta and / or Placenta Location in Ultrasound Imaging With Blind Sweep Protocol”, U.S. Provisional Application No. 63 / 540,755, filed September 27, 2023, titled “Ultrasound Imaging With Follow Up Sweep Guidance After Blind Sweep Protocol”, U.S. Provisional Application No. 63 / 655,754, filed June 4, 2024, , titled “Multiple Pregnancy / Gestation Detection Based On Graphing And Skeletonization Of Fetal Anatomy Detections From Ultrasound Imaging Blind Sweep Protocol”, U.S. Provisional Application No. 63 / 620,385, filed January 12, 2024, titled “Multiple Pregnancy / Gestation Detection Using Ultrasound Imaging With Blind Sweep Protocol”, and U.S. Provisional Application No. 63 / 540,740, filed September 27, 2023, titled “Ultrasound Imaging With Ultrasound Probe Guidance In Blind Sweep Protocol”, each of which is incorporated by reference herein.

[0176] DL model A 2010 outputs an estimated clinical feature 2015. Ground truth 2020 is the actual value of the clinical feature (e.g., gestational age, fetal presentation, deepest vertical pocket, placenta previa, etc.). The estimate 2015 is then compared with ground truth value of the clinical feature 2020 (e.g., manual annotations on the images, or other association in memory storing ultrasound images and user annotation), and a difference 2030 is calculated. Based on the difference, a “sweep complete” label 2035 or a “sweep incomplete” label 2040 is generated based on the difference 2030, such that relatively larger differences are marked with the “sweep incomplete” label 2040, whereas relatively small differences are marked with the “sweep complete” label 2035.

[0177] When the difference 2035 is relatively large (e.g., difference greater than 10%, 20%, 25%, 50%, etc.), the sweep 2005 is assigned the label of “complete” 2035. When the difference 2035 is relatively small (e.g., difference smaller than 10%, 20%, 25%, 50%), the sweep 2005 is assigned the label of “incomplete” 2040. Sweep complete is an indication that the uterine volume has been completely covered during the sweep. Sweep incomplete is an2024P00110WQ indication that the uterine volume has not been completely covered during the sweep. In this aspect, accuracy of the estimated clinical feature is being used as a proxy for sweep completeness. That is, a more accurate clinical feature estimation (smaller difference 2030) is likely to result from a sweep that is complete. A less accurate clinical feature estimation (e.g., a larger difference 2030) is likely to result from a sweep that is incomplete. The label complete 2035 or incomplete 2040 is associated with the sweep 2005. The steps of Figure 20A may then be repeated for many sweeps, with each sweep being associated with a label complete 2035 or incomplete 2040. This data (sweeps + associated labels) becomes the training data for step 2 discussed in Fig. 20B

[0178] Figure 20B is a schematic, diagrammatic representation, in flow diagram form, of an example second deep learning model training step 2050, according to aspects of the present disclosure. The second training step 2050 is actually training DL model B to estimate sweep completeness. Data from the first training step (e.g., Figure 20A) is present - ultrasound images from sweep 2005 and the label complete 2035 or incomplete 2040. The assigned label complete 2035 or incomplete 2040 are used as the ground truth for training DL model B. Ultrasound images from sweep 2005 are provided to DL model B 2055 as input. In the second training step 2050, the ultrasound images 2005 from the sweep are fed into DL model “B” 2055, which generates an estimation of sweep completeness 2060 (e.g., an estimation of whether the sweep is complete or incomplete). These estimations 2060 are then compared against the ground truth labels 2035 or 2040, which were assigned in the step 2000 (Fig. 20A)..

[0179] Using model objectives / functions 2065, the prediction / estimation 2060 (output of the model 2055) may be compared with the ground truth 2035 or 2040. In some instances, the prediction / estimation 2060 and / or the ground truth 2035 or 2040 can be discrete variables (e.g., categorial, with a finite number of categories, binary, etc.). For example, sweep complete or sweep incomplete can be binary outputs of the model 2055. In such cases, model objectives / functions 2065 may be functions of discrete / categorical / binary inputs and / or non- differentiable. Reinforcement learning and / or other derivative-free optimization techniques may be used to update parameters 2070 of the model 2055.

[0180] In some instances, the prediction / estimation and / or the ground truth can be continuous variables (e.g., continuous numerical values). Model objectives / functions may include objectives / functions which penalize to a greater or lesser extent prediction / estimations that are further or closer to the ground truth respectively. In some aspects, the model objectives / functions may be a mean squared error or mean absolute error.

[0181] The DL model B 2055 is then updated (e.g., the weights are adjusted) to minimize the difference 2065. This updating may be done repeatedly on different images, sweeps, cineloops, or complete exams, until the model converges (e.g., until changes between one iteration and the next are less than a threshold value). At this point, the DL model 2055 may be considered a trained DL model 2055.

[0182] Figure 20C is a schematic, diagrammatic representation, in flow diagram form, of an example deep learning inference mode 2075, according to aspects of the present disclosure. In the inference process or inference mode 2075, new ultrasound images 2080 (e.g., live clinical data from a blind sweep examination of the patient’s abdomen) are fed to the trained DL model B 2055 as input. DL model B 2055 has been trained to estimate sweep completeness (Figs. 20A and 20B). Trained DL model B 2055 provides an output of either sweep complete 2085 or sweep incomplete 2090. If the sweep is complete 2085, the sweep can be accepted 2087. The display can provide an output indicating acceptance 2087. If the sweep is incomplete complete 2090, the sweep can be rejected 2092. The display can provide an output indicating rejection 2092 and / or instructions to repeat sweep (thereby resulting in further ultrasound image frames).

[0183] Figure 21 is a screen display 2100 of an example uterine imaging extent verification system, indicating an accepted ultrasound blind sweep examination, according to aspects of the present disclosure. The screen display 2100 includes a visual representation 2110 of the exam, including all of the successful sweeps 2120, as well as a coverage assessment indicator, showing that the uterine coverage is complete for this examination. Such a screen display (e.g., shown on the display 132 of the host 130 of Figure 1) allows the user to see, at a glance, that the blind sweep examination has successfully covered the volume of the uterus, and thus that clinical conclusions drawn from the blind sweep examination are more likely to be accurate.

[0184] Figure 22 is a screen display 2200 of an example uterine imaging extent verification system, indicating rejected ultrasound blind sweep examination, according to aspects of the present disclosure. The screen display 2200 includes a visual representation 2110 of the exam, including all of the successful sweeps 2120, as well as visual representations 2210 of rejected sweeps, along with instructions 2220 to the user to, e.g., repeat the rejected sweeps (thereby resulting in further ultrasound image frames), repeat the entire examination (thereby resulting in further ultrasound image frames), etc. The screen display 2200 also includes a coverage assessment indicator, showing that the uterine coverage is incomplete for this examination. Such a screen display (e.g., shown on thedisplay 132 of the host 130 of Figure 1) allows the user to see, at a glance, that the blind sweep examination been unsuccessful in covering the volume of the uterus, and thus that clinical conclusions drawn from the blind sweep examination are less likely to be accurate or valid, and the examination (or portions thereof) should be repeated. Related aspects are described for example in U.S. Provisional Application No. 63 / 540,740, filed September 27, 2023, titled “Ultrasound Imaging with Ultrasound Probe Guidance in Blind Sweep Protocol”, which is incorporated by reference as though fully set forth herein.

[0185] Figure 23 is a screen display 2300 of an example ultrasound blind sweep uterine imaging extent verification system, quantifying the uterine imaging completion success for different users across a span of time, according to aspects of the present disclosure. The screen display 2300 includes a “% complete” rating 2310 for the current imaging session, as well as for the current day 2320, current month 2330, and overall 2340 (e.g., across the entire history of a given instance of the ultrasound blind sweep uterine imaging extent verification system, e.g., a single device on which the system is installed). Results are shown for two different users 2350 and 2360. Totals 2370 are also shown, to provide a level of confidence in the measurements being acquired from blind sweep examinations. The screen display 2300 allows users to see, at a glance, whether the percentage of successful blind sweep examinations is improving over time, and also which users may require additional training in how to perform blind sweep examinations accurately.

[0186] Figure 24 is a schematic diagram of a deep learning network configuration, according to aspects of the present disclosure. The configuration 3600 can be implemented by a deep learning network. The configuration 3600 includes a deep learning network 3610, which may include one or more CNNs 3612. The CNN 3612 is one example of a type of predictive model and / or machine learning model, which may be used in the anatomy detector 1530, machine learning model 3210, artificial intelligence model 3410, and otherwise. For simplicity of illustration and discussion, Fig. 24 illustrates one CNN 3612. However, any suitable number of CNNs 3612 (e.g., about 2, 3 or more) may be included. The configuration 3600 can be trained for identification of various anatomy (organs, tissue, bone) and / or other features (natural and / or man-made) within a patient anatomy, for detection of motion between image frames, for determining a desired direction of probe movement, and otherwise. The configuration 3600 can be further trained for segmenting human anatomy, diagnosis of medical conditions or any number of other diagnostic or medical tasks.

[0187] The CNN 3612 may include a set of N convolutional layers 3620 followed by a set of K fully connected layers 3630, where N and K may be any positive integers. Theconvolutional layers 3620 are shown as 3620(i) to 3620(N). The fully connected layers 3630 are shown as 3630(i) to 3630(K). Each convolutional layer 3620 may include a set of filters 3622 configured to extract features from an input 3602 (e.g., x-ray images or other data). The values N and K and the size of the filters 3622 may vary depending on the use of the CNN. In some instances, the convolutional layers 3620(i) to 3620(N) and the fully connected layers 3630(i) to 3630(K-I) may be interspersed with rectified non-linear (ReLU) or leaky ReLU or other activation functions and / or batch normalization layers. The fully connected layers 3630 may gradually shrink the high-dimensional output to a lower dimension or the dimension of the predicted result 3640 (e.g., location for an object detection landmark, or location and dimension for an object detection box, number of classes for a classification output, etc.). The fully connected layers 3630 may also be referred to as a classifier. In some aspects, the fully convolutional layers 3620 may additionally be referred to as representation or encodings or features.

[0188] When the prediction result 3640 takes the form of classification output, it may indicate a confidence score (e.g., a probability) for each class 3642 based on the input image 3602. The classes 3642 are shown as 3642a, 3642b, . . ., 3642c. For example, when the CNN 3612 is trained for classification of present anatomical features, the classes 3462 may indicate a first anatomical feature class 3642a, a second anatomical feature class 3642b, a third anatomical feature class 3642c, a fourth anatomical feature class 3642d, or any other suitable class. A class 3642 indicating a high confidence score indicates that the input image 3602 or a section or pixel of the image 3602 is likely to include an anatomical object / feature of the class 3642. Conversely, a class 3642 indicating a low confidence score indicates that the input image 3602 or a section or pixel of the image 3602 is unlikely to include an anatomical object / feature of the class 3642.

[0189] The CNN 3612 can also output a feature vector 3650 at the output of the last convolutional layer 3620(N), though any of the layers in the CNN are feature vectors. A feature vector 3650 or encodings or representations may encode some representation of objects detected from the input medical image 3602 or other data. These representations can be decoded using a reversed CNN where the fully connected layers expand the lowdimensional representation to a higher dimension and transposed convolutional layers can expand feature layers up to the size of the original input, where pixel-wise outputs (e.g., binary segmentation map, multi-class segmentation map) can be generated.

[0190] The deep learning network 3610 may implement or include any suitable type of learning network. For example, in some aspects, and as described in relation to Fig. 36, thedeep learning network 3610 could include a convolutional neural network 3612. In addition, the deep learning network 3610 may additionally or alternatively be or include a multi-class classification network, an encoder-decoder type network, a fully connected deep learning network, or any suitable network or means of identifying features within an image.

[0191] In some aspects, when the deep learning network 3610 includes a fully-connected neural network, the fully connected neural network may transform the data not related to a medical image or it may transform data derived from an image (e.g., detected objects) generated by another network, program, or human annotator. Data for detected objects may be concatenated into a layer (e.g., as an additional channel. For simple numeric features like age, weight, and other data - these can be concatenated into fully connected layers. For example, the fully connected neural network may transform information about a patient, such as age, weight, or other low-dimensional data.

[0192] In some aspects, when the deep learning network 3610 includes an encoderdecoder network, the network may include two components. One component may be a constricting component or encoder, in which a large image, such as the image 3602, may be convolved by several convolutional layers 3620 such that the size of the image 3602 changes in relation to the depth of the network layer. For instance, the CNN 3612 may be the encoder. The image 3602 may then be represented in a low dimensional space, or a flattened space. From this flattened space, an additional component or decoder may expand the flattened space to the original size of the image 3602. For instance, the reverse of the CNN 3612 may be the decoder. In some aspects, the encoder-decoder network may reconstruct the input image 3602. In some aspects, the encoder-decoder network may segment the image 3602 into patches. In some aspects of the present disclosure, the deep learning network 3610 may include a multi-class classification network. In that aspect, the multi-class classification network may include an encoder path. For example, the image 3602 may be a high dimensional image. The image 3602 may then be processed with the convolutional layers 3620 such that the size is reduced. The resulting low dimensional representation of the image 302 may be used to generate the feature vector 3650 shown in Fig. 36. The low dimensional representation of the image 3602 may additionally be used by the fully connected layers 3630 to regress and output one or more classes 3642. In some regards, the fully connected layers 3630 may process the output of the convolutional layers 3620. The fully connected layers 3630 may additionally be referred to as task layers or regression layers, among other terms.

[0193] Any suitable combination or variations of the deep learning network 3610 described is fully contemplated. For example, the deep learning network may include fullyconvolutional networks or layers or fully connected networks or layers or a combination of the two. In addition, the deep learning network may include a multi-class classification network, an encoder-decoder network, or any combination of networks. The process of training the deep learning network 3610 includes adjusting its parameters, or more particularly the weights and biases, which control the operation of activation functions in the neural network. In supervised learning, the training process automatically adjusts the weights and the biases, such that when presented with the input data, the neural network accurately provides the corresponding expected output data. In order to do this, the value of the loss functions, or errors, are computed based on a difference between predicted output data and the expected output data. The value of the loss function may be computed using functions such as the negative log-likelihood loss, the mean squared error, or the Huber loss, or the cross-entropy loss. During training, the value of the loss function is typically minimized. Various methods are known for solving the loss minimization problem such as gradient descent, Quasi-Newton methods, and so forth. Various algorithms have been developed to implement these methods and their variants including but not limited to Stochastic Gradient Descent “SGD”, batch gradient descent, mini -batch gradient descent, Gauss-Newton, Levenberg Marquardt, Momentum, Adam, Nadam, Adagrad, Adadelta, RMSProp, and Adamax “optimizers”. These algorithms compute the derivative of the loss function with respect to the model parameters using the chain rule. This process is called backpropagation since derivatives are computed starting at the last layer or output layer, moving toward the first layer or input layer. These derivatives inform the algorithm how the model parameters must be adjusted in order to minimize the error function. The training process is performed iteratively by making adjustments to the weights and biases in each iteration. Training is terminated when the error, or difference between the predicted output data and the expected output data, is within an acceptable range for the training data, or for some validation data. Subsequently the neural network may be deployed, and the trained neural network makes predictions on new input data using the trained values of its parameters. If the training process was successful, the trained neural network accurately predicts the expected output data from the new input data.

[0194] As will be readily appreciated by those having ordinary skill in the art after becoming familiar with the teachings herein, the ultrasound blind sweep uterine imaging extent verification system advantageously permits untrained and minimally trained users to perform an ultrasound blind sweep protocol to gather anatomical images of high quality, including automated detection of health parameters such as gestational age. This may resultin higher accuracy and higher clinician trust in the results, while potentially improving health outcomes and / or decreasing the total cost of care. Potential benefits include detection of pregnancy complications via blind sweeps performed by novice ultrasound users. The solution can be a quick initial check scan for a center with high volume ultrasound turnover to triage patients for a more detailed obstetric scan, and can provide or support referral of the subject diagnosed with complications for further diagnosis and management to a tertiary care center. Early detection of pregnancy complications may be extremely helpful for follow-up and monitoring of the pregnancy.

[0195] Use of the present disclosure can be detected if an acquired sweep is detected to have insufficient coverage or data, and a user interface (UI) shows visualizations to prompt the user to repeat the sweep (thereby resulting in further ultrasound image frames). Similarly, while a sweep is being acquired, visualizations on UI or audio, provided based on image coverage in real-time as guidance, are indicative of the ultrasound blind sweep uterine imaging extent verification system being used. Additionally, if the system disclosed herein is implemented in an ultrasound imaging system, sweeps that fail to cover the entirety of the uterus may result in “incomplete coverage” feedback through UI or audio either after or during the acquisition, as mentioned above.

[0196] The system disclosed herein can be associated with portable Al-assisted obstetric screening platform for novice ultrasound users that leverages, e.g., the Philips Lumify ultrasound probe. However, the system and methods described herein may be applied to any ultrasound probe and application in which a region of interest is being imaged and in which a determination of complete coverage should be made based on anatomy detections or similar input described in the above aspects.

[0197] The systems, methods, and devices described herein may be applicable in point of care and handheld ultrasound use cases. The disclosed technology can be used for any handheld imaging applications, including but not limited to obstetrics and echocardiography. The technology could be deployed on handheld mobile ultrasound devices, and on portable or cart-based ultrasound systems. The ultrasound blind sweep uterine imaging extent verification system can be used in a variety of settings including emergency departments, ambulances, accident sites, and homes. The applications could also be expanded to other settings. The disclosed technology increases the value proposition of ultrasound applications in the obstetrics context, especially for use by minimally trained users.

[0198] Accordingly, the logical operations making up the aspects of the technology described herein are referred to variously as operations, steps, objects, layers, elements,components, algorithms, or modules. Furthermore, it should be understood that these may occur or be performed or arranged in any order, unless explicitly claimed otherwise or a specific order is inherently necessitated by the claim language.

[0199] All directional references e.g., upper, lower, inner, outer, upward, downward, left, right, lateral, front, back, top, bottom, above, below, vertical, horizontal, clockwise, counterclockwise, proximal, and distal are only used for identification purposes to aid the reader’ s understanding of the claimed subject matter, and do not create limitations, particularly as to the position, orientation, or use of the ultrasound blind sweep multiple pregnancy detection system. Connection references, e.g., attached, coupled, connected, joined, or “in communication with” are to be construed broadly and may include intermediate members between a collection of elements and relative movement between elements unless otherwise indicated. As such, connection references do not necessarily imply that two elements are directly connected and in fixed relation to each other. The term “or” shall be interpreted to mean “and / or” rather than “exclusive or.” The word “comprising” does not exclude other elements or steps, and the indefinite article “a” or “an” does not exclude a plurality. Unless otherwise noted in the claims, stated values shall be interpreted as illustrative only and shall not be taken to be limiting.

[0200] The above specification, examples and data provide a complete description of the structure and use of exemplary aspects of the ultrasound blind sweep uterine imaging extent verification system as defined in the claims. Although various aspects of the claimed subject matter have been described above with a certain degree of particularity, or with reference to one or more individual aspects, those skilled in the art could make numerous alterations to the disclosed aspects without departing from the spirit or scope of the claimed subject matter.

[0201] Still other aspects are contemplated. It is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative only of particular aspects and not limiting. Changes in detail or structure may be made without departing from the basic elements of the subject matter as defined in the following claims.

Claims

CLAIMSWhat is claimed is:

1. A system, comprising: a processor configured for communication with an ultrasound probe, wherein the processor is configured to: control acquisition by the ultrasound probe to obtain a plurality of ultrasound image frames during a blind sweep protocol on a patient with a pregnancy; determine whether the plurality of ultrasound image frames cover an entire distance across a uterus of the patient; and provide, to a display in communication with the processor, an output representative of the determination of whether the plurality of ultrasound image frames covers the entire distance across the uterus, wherein, when the determination is that the plurality of ultrasound images frames covers the entire distance across the uterus, the output is representative of the plurality of ultrasound image frames being accepted, and wherein, when the determination is that the plurality of ultrasound images frames does not cover the entire distance across the uterus, the output comprises an instruction to repeat acquisition by the ultrasound probe to obtain a further plurality of ultrasound image frames.

2. The system of claim 1, wherein, to determine whether the plurality of ultrasound image frames covers the entire distance across the uterus, the processor is configured to provide the plurality of ultrasound image frames to as an input to a deep learning network trained to output one or more detections of at least one of maternal anatomy or fetal anatomy.

3. The system of claim 2, wherein operation of the deep learning network to output the one or more detections comprises generation of a plurality of feature maps for the plurality of ultrasound image frames, and wherein, to determine whether the plurality of ultrasound image frames covers the entire distance across the uterus, the processor is configured to perform edge detection in the plurality of feature maps to identify a wall of the uterus.

4. The system of claim 3, wherein the processor is configured to: generate a contour of the wall of the uterus from the plurality of feature maps; and determine whether the plurality of ultrasound image frames covers the entire distance across the uterus based on the generated contour.

5. The system of claim 2, wherein the plurality of ultrasound images comprises a sweep in the blind sweep protocol.

6. The system of claim 5, wherein, to determine whether the plurality of ultrasound image frames covers the entire distance across the uterus, the processor is configured to: determine whether the one or more detections occur to an expected location within the sweep.

7. The system of claim 6, wherein the expected location comprises at least one of: a detection of the maternal anatomy at a beginning portion or an ending portion of the sweep; or a detection of the fetal anatomy at a middle portion of the sweep.

8. The system of claim 2, wherein, to determine whether the plurality of ultrasound image frames covers the entire distance across the uterus, the processor is generate to: generate a statistical description of the one or more detections; and determine whether the plurality of ultrasound image frames covers the entire distance across the uterus based on the statistical description.

9. The system of claim 2, wherein the processor is configured to output, to the display, a real time indication during the acquisition of whether an ultrasound image frame of the plurality of ultrasound image frames is representative of anatomy outside the uterus or inside the uterus.

10. The system of claim 1, wherein, to determine whether the plurality of ultrasound image frames covers the entire distance across the uterus, the processor is configured to:provide the plurality of ultrasound image frames as an input a deep learning network trained to determine output a determination of whether a sweep in the blind sweep protocol is complete or incomplete, wherein the deep learning network is trained using a plurality of sweeps labeled complete or incomplete based on a difference between an estimated value of a clinical feature and an actual value of the clinical feature.

11. A method, comprising: controlling, with a processor in communication with an ultrasound probe, acquisition by the ultrasound probe to obtain a plurality of ultrasound image frames during a blind sweep protocol on a patient with a pregnancy; determining, with the processor, whether the plurality of ultrasound image frames cover an entire distance across a uterus of the patient; and providing, to a display in communication with the processor, an output representative of the determination of whether the plurality of ultrasound image frames covers the entire distance across the uterus, wherein, when the determination is that the plurality of ultrasound images frames covers the entire distance across the uterus, the output comprises an indication that the plurality of ultrasound image frames are accepted, and wherein, when the determination is that the plurality of ultrasound images frames does not cover the entire distance across the uterus, the output comprises an instruction to repeat acquisition by the ultrasound probe to obtain a further plurality of ultrasound image frames.

12. The method of claim 11, wherein determining whether the plurality of ultrasound image frames covers the entire distance across the uterus comprises: providing the plurality of ultrasound image frames to as an input to a deep learning network trained to output one or more detections of at least one of maternal anatomy or fetal anatomy.

13. The method of claim 12, wherein operation of the deep learning network to output the one or more detections comprises generating a plurality of feature maps for the plurality of ultrasound image frames, andwherein determining whether the plurality of ultrasound image frames covers the entire distance across the uterus comprises performing edge detection in the plurality of feature maps to identify a wall of the uterus.

14. The method of claim 13, further comprising: generating a contour of the wall of the uterus from the plurality of feature maps; and determining whether the plurality of ultrasound image frames covers the entire distance across the uterus based on an extent of the generated contour.

15. The method of claim 12, wherein the plurality of ultrasound images comprises a sweep in the blind sweep protocol.

16. The method of claim 15, wherein determining whether the plurality of ultrasound image frames covers the entire distance across the uterus comprises: determining whether the one or more detections corresponds to an expected location within the sweep.

17. The method of claim 16, wherein the expected location comprises at least one of: a detection of the maternal anatomy at a beginning portion or an ending portion of the sweep; or a detection of the fetal anatomy at a middle portion of the sweep.

18. The method of claim 12, determining whether the plurality of ultrasound image frames covers the entire distance across the uterus comprises: generating a statistical description of the one or more detections; and determining whether the plurality of ultrasound image frames covers the entire distance across the uterus based on the statistical description.

19. The method of claim 12, further comprising outputting, to the display, a real time indication during the acquisition of whether an ultrasound image frame of the plurality of ultrasound image frames is representative of anatomy outside the uterus or inside the uterus.

20. The method of claim 12, wherein determining whether the plurality of ultrasound image frames covers the entire distance across the uterus comprises:providing the plurality of ultrasound image frames as an input a deep learning network trained to determine output a determination of whether a sweep in the blind sweep protocol is complete or incomplete, wherein the deep learning network is trained using a plurality of sweeps labeled complete or incomplete based on a difference between an estimated value of a clinical feature and an actual value of the clinical feature.

Images