Systems and methods for improving detection of fetal congenital heart defects

Abstract

Systems and methods are provided for assisting in the detection and diagnosis of critical cardiac defects during fetal ultrasound examinations in which image data (e.g., motion video clips and / or image frames) are analyzed with a machine learning algorithm to identify and select image frames within the image data that correspond to a standard view recommended by a fetal ultrasound guide, and analyzing the selected image frames with a machine learning algorithm to detect and identify morphological anomalies indicative of critical CHD associated with the standard view. Results of the analysis are presented to a clinician for review along with an overlay of the selected image frame that identifies the anomaly with graphical or textual markings. The overlay may further be annotated by the clinician and stored to create a file record of the fetal ultrasound examination.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to U.S. patent application No. 18 / 406,446, filed on January 8, 2024, U.S. Provisional Application No. 63 / 584,117, filed on September 20, 2023, U.S. patent application No. 18 / 183,937, filed on March 14, 2023 (now U.S. Patent No. 11,869,188), and European Patent Application No. 23305235.6, filed on February 22, 2023, the entire contents of each of which are incorporated herein by reference. Technical Field

[0003] The present invention relates to systems and methods for improving the detection of fetal congenital heart defects during and after an ultrasound examination by using machine learning algorithms to ensure creation of complete data sets, perform preliminary review of the complete data sets, and determine which data sets are designated for expert review. Background Art

[0004] Congenital heart disease (CHD) is the most common birth defect, affecting approximately 0.8 to 1% of all newborns. As of 2014, CHD accounted for 4% of neonatal deaths in the United States and 30% to 50% of deaths related to congenital anomalies. A study by Nayak et al., titled "Evaluation of fetalechocardiography as a routine antenatal screening tool for detection of congenital heart disease," published in Cardiovasc. Diagn. Ther., Vol. 6, pp. 44-49 (2016), showed that 92% of CHD cases occurred during pregnancy and were defined as "low risk." In a study conducted by Stümpflen et al., entitled "Effect of detailed fetalechocardiography aspart of routine prenatal ultrasonographic screening on detection of congenital heart disease," Lancet, 348, 854-857 (1996), it was observed that most CHDs were identified during the second trimester screening examination, supporting the need for universal fetal heart screening examinations during the second trimester.

[0005] Congenital heart disease (CHD) is often asymptomatic in the fetal period, but causes severe morbidity and mortality after birth. In addition to adverse cardiac outcomes, CHD is also associated with an increased risk of adverse neurodevelopmental outcomes, and is associated with factors such as associated chromosomal abnormalities, syndromes, postnatal cardiac dysfunction, and intrauterine hemodynamic abnormalities. Critical CHD (see Table 1), defined as requiring surgery or catheter intervention in the early years of life, accounts for approximately 25% of all CHD. See "Temporal trends in survival among infants with critical congenital heart defects" by Matthew E. Oster, ME et al., Pediatrics, Vol. 131, pp. e1502-1508 (2013). In infants with critical cardiac lesions, the risk of morbidity and mortality increases when diagnosis is delayed and timely referral to a tertiary center with expertise in treating these patients is not possible. See Kuehl, KS, et al., “Failure to Diagnose Congenital Heart Disease in Infancy,” Pediatrics 103:743–7 (1999); Eckersley, L., et al., “Timing of diagnosis affects mortality in critical congenital heart disease,” Arch. Dis. Child. 101, 516–520 (2016).

[0006] Table 1

[0007]

[0008]

[0009] Compared with postnatal diagnosis, fetal diagnosis can significantly improve neonatal outcomes by allowing for earlier planning of delivery care, surgery, and / or early intervention, and in some cases, consideration of intrauterine treatment. Furthermore, accurate prenatal diagnosis allows parents to make informed decisions about continuing the pregnancy.

[0010] Distinguishing a normal fetal heart from those exhibiting complex forms of CHD typically involves an initial screening examination performed by physicians, nurse practitioners, physician assistants, ultrasound technicians, and other providers trained in diagnostic obstetric ultrasound. Licensed medical providers who meet the guidelines of their training are responsible for interpreting the ultrasound examination. If the ultrasound is abnormal, further testing via fetal echocardiography is necessary for confirmation and diagnostic refinement. Further testing may also be necessary in situations such as a family history of congenital heart defects, maternal diabetes, or the use of in vitro fertilization. Only highly trained and / or experienced pediatric cardiologists, maternal-fetal medicine specialists, obstetricians, or radiologists with the appropriate knowledge base and skills should oversee and perform these fetal echocardiograms. Low sensitivity in this task can limit palliative care options, worsen postnatal outcomes, and hinder in utero treatment research, while low specificity can lead to unnecessary additional testing and referrals.

[0011] The World Health Organization (WHO) recommends that all pregnant women undergo an ultrasound scan before 24 weeks of pregnancy to estimate gestational age (GA), assess the location of the placenta, determine single or multiple pregnancies, improve detection of fetal anomalies, and improve pregnancy outcomes. WHO Recommendations for Antenatal Care for a Positive Pregnancy Experience (World Health Organization, 2016).

[0012] In 2013 and 2018, the American Institute of Ultrasound in Medicine (AIUM) and the International Society of Ultrasound in Obstetrics and Gynecology (ISUOG) both changed their practice guidelines for second-trimester fetal cardiac screening. See Carvalho et al., “ISUOG Practice Guidelines (updated): sonographic screening examination of the fetal heart: ISUOG Guidelines,” Ultrasound Obstet. Gynecol., 41, 348–359 (2013); and “AIUM-ACR-ACOG-SMFM-SRUPractice Parameter for the Performance of Standard Diagnostic Obstetric Ultrasound Examinations,” J. Ultrasound Med., 37, E13–E24 (2018). These updated guidelines specify a minimum of three views: a four-chamber view (4C) and views of the left ventricular outflow tract (LVOT) and right ventricular outflow tract (RVOT) (2,3). Unfortunately, several cardiac anomalies are poorly detected prenatally using this approach. Although three-vessel (3V) and three-vessel and trachea (3VT) views are not mandatory in the AIUM and ISUOG practice guidelines, both guidelines indicate that these views are desirable and should be attempted as part of routine screening. See Table 2. Many groups have performed additional views during routine screening and reported higher fetal heart malformation detection rates of 62 to 87.5%, compared to 40 to 74% using the recommended three views, as described in "Committee on Practice Bulletins—Obstetrics and the American Institute of Ultrasound in Medicine, Practice Bulletin No. 175: Ultrasound in Pregnancy," Obstet. Gynecol., vol. 128, pp. e241-e256 (2016).

[0013] Table 2

[0014]

[0015] Some critical CHDs are easier to visualize with ultrasound screening during pregnancy than others. Pinto et al., "Barriers to prenatal detection of congenital heart disease: a population-based study," Ultrasound Obstet. Gynecol. Off. J. Int. Soc. Ultrasound Obstet. Gynecol., Vol. 40, pp. 418-425 (2012), using data from the Utah Birth Defects Network from 1997 to 2007, observed that the defects most likely to be detected prenatally included those with an abnormal four-chamber view, while defects demonstrating outflow tract abnormalities were much less likely to be detected prenatally. In a study conducted from 2005 to 2010 among members of a large health maintenance organization (HMO) in California, Levy et al., “Improved prenatal detection of congenital heart disease in an integrated health care system,” Pediatr. Cardiol., 34, 670-679 (2013), showed that prenatal diagnosis rates were significantly higher among women who received care from HMO clinics that had a policy requiring outflow tract examination during prenatal ultrasound (59%) compared with women who did not (28%).

[0016] In the current triage workflow, the patient typically presents to the first point of care (obstetrician-gynecologist, midwife, or radiologist) where the fetus is evaluated, for example, via fetal ultrasound screening performed by a healthcare professional or sonographer. The image data is interpreted by the frontline practitioner in real time during the ultrasound examination or offline after the examination has been performed. A report is generated by the frontline practitioner and may be pre-filled by the sonographer. If a congenital heart defect is suspected, the patient is referred to a specialist who will review the report and perform specific tests (echocardiogram, genetic testing) intended to confirm the presence or absence of the underlying congenital defect. Depending on the results of that further examination or test, a decision is made regarding the patient's treatment and / or transfer to a subsequent point of care.

[0017] Previously known CHD screening workflows have numerous shortcomings, often including inaccuracies and low specificity caused by poor examination technique, time pressure, maternal obesity, and simple misdiagnosis. Specifically, CHD detection during second-trimester ultrasound examinations is often as low as 30%. Specificity is also suboptimal, ranging from 40 to 50%, attributed to a lack of skill in adapting ultrasound images (i.e., the ultrasound operator lacks the skills to acquire data from which a correct diagnosis can be made, resulting in approximately 49% of misdiagnoses); a lack of experience in making an accurate diagnosis (i.e., the image acquired is clear enough and prenatal pathology is visible, but the operator disagrees, resulting in approximately 31% of misdiagnoses); and pathologies that go undetected because they are not visible on the ultrasound image, accounting for approximately 20% of missed diagnoses. Time pressures associated with achieving adequate patient throughput in clinical settings can exacerbate these issues, particularly when patient transfer to a specialist is necessary.

[0018] While some efforts have been made to improve CHD detection during routine prenatal ultrasound examinations, much work remains to be done. For example, ultrasound technicians have access to considerable guidance describing how to acquire complete, high-diagnostic-quality image datasets during an examination and how to confirm the presence of cardiac structures in real time during the examination. For example, U.S. Patent No. 7,672,491 to Krishnan et al. describes a system for evaluating the diagnostic quality of images obtained during an ultrasound examination that uses machine learning to compare the obtained images with the expected images.

[0019] As discussed above, the ISUOG Practice Guidelines published in Ultrasound Obstet. Gynecol. 2013; 41: 348–359 recommend five axial positions that should be imaged during routine fetal cardiac ultrasound examinations, as well as the major organs and blood vessels and the orientation of each that should be confirmed at each position. European Patent Application Publication EP 3964136 by Voznyuk et al. describes a machine learning system that analyzes ultrasound images generated during an examination, uses a first convolutional neural network (CNN) to compare the acquired images with the views required by those guidelines, and uses a second CNN to analyze the images to identify potential abnormalities.

[0020] U.S. patent application publication No. US2021 / 0345987 to Ciofolo-Veit et al. describes an ultrasound imaging system that uses a machine learning algorithm to analyze acquired images to detect abnormal features, and if abnormal features are detected, uses the machine learning algorithm to determine and display other previously acquired ultrasound images that provide complementary views of the potential abnormal features to permit an improved diagnosis.

[0021] Furthermore, fetal ultrasound screening exams typically generate thousands of image frames across multiple structures in each single video "scan," so frames of diagnostic interest for CHD may be few and easily missed. Furthermore, the prevalence of CHD in the population (approximately 0.8 to 1%) is low enough that abnormal images are rarely seen by non-experts and may be overlooked or ignored. These factors combine to make CHD detection one of the most difficult diagnostic challenges in ultrasound, with a significant impact on postpartum outcomes and quality of life.

[0022] In view of the foregoing, it would be desirable to provide methods and apparatus for triaging prenatal ultrasound scans to improve the accuracy of congenital defect detection and subsequent management.

[0023] It would further be desirable to provide a machine learning system for prenatal fetal ultrasound that is configured to review recorded ultrasound videos and identify images from the videos that correspond to views recommended by guidelines.

[0024] It would be further desirable to provide methods and systems for performing prenatal ultrasound examinations that assist the sonographer in collecting high-quality data sets in accordance with applicable guidelines, assist the interpreting physician and / or technician in identifying potential abnormalities in the obtained data, and, further, guide the sonographer in real time in obtaining additional views to augment the image data set, for example, to facilitate specialist review.

[0025] It would still further be desirable to provide methods and systems for objectively evaluating a sonographer's performance across multiple examinations. Summary of the Invention

[0026] The present invention relates to systems and methods for performing fetal ultrasound examinations that aid in the detection of critical heart defects during second-trimester ultrasound examinations. The systems and methods of the present invention assist trained and qualified physicians in interpreting ultrasound recordings of motion video clips by identifying standard views that appear within the motion video clips. Furthermore, the systems and methods of the present invention assist in the detection and identification of morphological abnormalities that may indicate critical CHD.

[0027] Provided herein is a system for use with an ultrasound system for assisting a clinician in detecting and diagnosing heart defects during a fetal ultrasound examination, the system comprising: one or more computers configured to store non-transitory programming instructions, the one or more computers being programmed to: store view templates corresponding to standard guideline views, including at least view templates for 4C, LVOT, RVOT, 3V, and 3VT views; store data indicating one or more potential abnormalities associated with each of the view templates; receive multiple sets of image data generated by the ultrasound system during a fetal ultrasound examination, each set of image data comprising multiple frames; compare each of the multiple frames with the view templates to identify and select a corresponding image frame for each view template, if present; analyze each corresponding image frame to detect the presence of the one or more potential abnormalities; and present the corresponding image frame on a display screen for each standard view in response to a request by the clinician, including an overlay indicating the presence of the one or more potential abnormalities.

[0028] The one or more computers may include a display computer and a server computer. The non-transitory programming instructions may include a user interface component and an interpretation component. The user interface component may be configured to receive, store, and display the multiple sets of image data generated by the ultrasound system in real time. The user interface component may be configured to store and display analysis results returned by the interpretation component. The interpretation component includes a machine learning algorithm for identifying and selecting each corresponding image frame. The interpretation component may further include a machine learning algorithm for detecting the presence of the one or more potential anomalies in each corresponding image frame. The overlay may indicate the presence of the one or more potential anomalies, including one or more of a bounding box surrounding the potential anomaly, a graphical indicator, or a textual indicator.

[0029] The system may further include non-transitory programming instructions for enabling the clinician to annotate the overlay. The system may further include non-transitory programming instructions for generating a report documenting the clinician's observations during the fetal ultrasound examination. The report may include the clinician's observations during the fetal ultrasound examination. 4C represents four chambers, LVOT represents left ventricular outflow tract, RVOT represents right ventricular outflow tract, 3V represents three vessels, and 3VT represents three vessels and trachea.

[0030] The one or more computers may be further programmed to determine data indicating one or more potential abnormalities for each template of the view. The one or more computers may be further programmed to associate each of the data indicating one or more potential abnormalities with each corresponding view template. Each of the plurality of frames may be compared to the view template using a neural network. The one or more computers may be further programmed to: determine the data indicating one or more potential abnormalities for each view template; associate each of the data indicating one or more potential abnormalities with each corresponding view template; compare each of the one or more corresponding image frames to the data indicating the one or more potential abnormalities for the view template corresponding to each of the one or more corresponding image frames using a neural network to detect the presence of the one or more potential abnormalities; generate a report indicating the presence of one or more potential abnormalities for each of the one or more corresponding image frames; and cause the display screen to present the report, including an overlay indicating the presence of the one or more potential abnormalities, in response to a request by the clinician.

[0031] Provided herein is a method for use with an ultrasound system for assisting a clinician in detecting and diagnosing cardiac defects during a fetal ultrasound examination, the method comprising: providing a computer configured to store and execute non-transitory programming instructions; storing on the computer view templates corresponding to standard guideline views, including at least view templates for 4C, LVOT, RVOT, 3V, and 3VT views; storing on the computer data indicating one or more potential abnormalities associated with each of the view templates; generating a motion video clip with an ultrasound system during a fetal ultrasound examination and storing the motion video clip; receiving the motion video clip by the computer; comparing, with the computer, each frame of the motion video clip with the view template to identify and select a corresponding image frame for each view template, if present; analyzing, with the computer, each corresponding image frame to detect the presence of the one or more potential abnormalities; and presenting, for each standard view on a display screen associated with the computer, in response to a request by the clinician, the corresponding image frame including an overlay indicating the presence of the one or more potential abnormalities.

[0032] Providing a computer may include providing one or more computers including a display computer and a server computer. Providing the one or more computers may include providing non-transitory programming instructions for a user interface component to the display computer and providing non-transitory instructions for an interpretation component to the server computer. The method may further include displaying the multiple sets of image data generated by the ultrasound system to the clinician in real time using the display computer. The method may further include transmitting analysis results generated by the interpretation component from the server computer to the display computer. Comparing each of the multiple frames with the view template using the computer may include analyzing the multiple frames using a machine learning algorithm to identify and select each corresponding image frame. Analyzing each corresponding image frame using the computer to detect the presence of the one or more potential abnormalities may include analyzing each corresponding image frame using a machine learning algorithm to detect the presence of the one or more potential abnormalities.

[0033] Presenting the overlay on the display screen may include presenting a graphic or textual indicia indicating the presence of the one or more potential abnormalities. The method may further include creating an annotated overlay corresponding to the overlay, including additional graphic or textual information input by the clinician, and storing the annotated overlay. The method may further include generating, by a computer, a report for the fetal ultrasound examination documenting the annotated overlay. The method may further include generating, by a computer, a report having an entry for each standard view and the overlay indicating the presence of the one or more potential abnormalities. The method may further include determining, by the computer, the presence of one or more potential abnormalities and sending, to a specialist, at least a portion of the multiple sets of image data or one or more of the report. The method may further include determining, by the computer, a quality value for the fetal ultrasound examination based on the multiple sets of image data.

[0034] In one embodiment, the systems and methods are embodied in a computer-aided diagnostic aid for performing, for example, a two-dimensional prenatal fetal ultrasound examination, typically during the second trimester. Machine learning algorithms are employed to assist the user in identifying and interpreting standard views in fetal cardiac ultrasound image data (e.g., a motion video clip). Specifically, the present systems and methods are embodied in software that can be executed to support the identification of critical CHD. Additionally, information generated during the machine learning-enhanced analysis can be stored for later referral to a specialist (e.g., a specialist) to assist with further diagnosis and treatment planning.

[0035] In a preferred embodiment, the system of the present invention employs two components: a user interface component that provides clinicians with tools to analyze and review fetal ultrasound images and ultrasound image data (e.g., motion video clips); and a machine learning interpretation component that receives ultrasound image data (e.g., motion video clips and images) from a conventional fetal ultrasound screening system and identifies images within the image data (e.g., motion video clips) that correspond to fetal ultrasound screening guidelines. The interpretation component also analyzes the identified images to detect and identify the presence of morphological abnormalities and provides that information to the user interface component to highlight such abnormalities for clinician review. The interpretation component may be executed partially or entirely in real time on a local computer workstation. Alternatively, the interpretation component may reside on a cloud-based server and interact with the user interface component via a secure connection over a local or wide area network (e.g., the Internet).

[0036] According to another aspect of the present invention, the method and system provide a consistent process to ensure that all views recommended by fetal examination practice guidelines are obtained. Specifically, if the machine learning-based review of image data (e.g., a motion video clip) from a fetal ultrasound scan does not identify an image frame determined to be suitable for review, the system marks this view as unavailable or of insufficient quality to permit abnormality detection analysis, and the user interface instructs the clinician to re-perform the ultrasound scan to obtain the missing data. The new image data (e.g., a new motion video clip) is then transmitted to the interpretation component for analysis and the supplemental analysis is returned to the user interface for presentation to the clinician.

[0037] According to another aspect of the present invention, the analysis results returned to the user interface component can be displayed and further annotated by the clinician to include additional graphical markers or textual notes. The obtained analysis results and annotations can be stored for later referral to a specialist for further diagnosis or treatment planning.

[0038] According to another aspect of the present invention, the analysis and / or results (including detected morphological anomalies) can be used to generate a report. The report can automatically populate an entry for each standard view using the video clip frame, and the report can include a bounding box overlay. Information about the view can be included in the report to add context to the image.

[0039] According to another aspect of the invention, the system can recommend a referral to a clinician and / or specialist. According to another aspect of the invention, the system can perform a visual assessment of the technician performing the imaging (e.g., sonographer). According to another aspect of the invention, the system can automatically organize the results so that the most relevant information appears first or otherwise appears most prominently. Additionally or alternatively, the results can be organized by patient severity.

[0040] In another embodiment, a system and computer-implemented method for analyzing fetal ultrasound images are provided. The systems and methods may include: receiving multiple sets of image data generated by an ultrasound system during a fetal ultrasound examination, each set of image data in the multiple sets of image data including multiple frames; analyzing a set of image data in the multiple sets of image data to automatically determine that one or more frames of the set of image data correspond to a standard view from a plurality of standard views; analyzing the set of image data to automatically determine that the one or more frames indicate a first morphological anomaly from a plurality of morphological anomalies; and generating a user interface for display, wherein the user interface includes: an image data viewer adapted to visually present the set of image data; a standard view indicator corresponding to the set of image data presented on the image data viewer and visually indicating whether each standard view from the plurality of standard views is present in the set of image data; and a morphological anomaly indicator corresponding to the set of image data presented on the image data viewer and visually indicating whether each morphological anomaly from the plurality of morphological anomalies is present in the set of image data, wherein when the image data viewer visually presents the set of image data, the standard view indicator indicates the presence of the first standard view in the set of image data and the morphological anomaly view indicator indicates the presence of the first morphological anomaly.

[0041] The user interface may be generated on a display of the ultrasound system and / or on a display of a healthcare provider device. The standard view indicator may include a plurality of color indicators, each corresponding to one of the plurality of standard views, each color indicator being adapted to present a first color when a corresponding standard view of the plurality of standard views is present in the set of image data and a second color when the corresponding standard view of the plurality of standard views is not present in the set of image data. The morphological abnormality indicator may include a plurality of color indicators, each corresponding to one of the plurality of morphological abnormalities, each color indicator being adapted to present a first color when a corresponding morphological abnormality of the plurality of morphological abnormalities is present in the set of image data and a second color when the corresponding morphological abnormality of the plurality of morphological abnormalities is not present in the set of image data. Each color indicator of the plurality of color indicators may further be adapted to present a third color indicating that the presence of the corresponding morphological abnormality of the plurality of morphological abnormalities in the set of image data is uncertain.

[0042] The image data viewer may include a first time bar and a cursor on the time bar, wherein the cursor is adapted to move so that the image data viewer visually presents a plurality of image frames corresponding to a plurality of time points along the time bar. The standard view indicator may include a plurality of second time bars, each corresponding to the first time bar and each having a first visual indicator corresponding to the cursor and adapted to move synchronously with the cursor. Each of the plurality of second time bars may be adapted to visually indicate one or more time points on the second time bar, each corresponding to the presence of a corresponding standard view among the plurality of standard views. The morphological anomaly indicator may include a plurality of second time bars, each corresponding to the first time bar and each having a first visual indicator corresponding to the cursor and adapted to move synchronously with the cursor. Each of the plurality of second time bars may be adapted to visually indicate one or more time points on the second time bar, each corresponding to the presence of a corresponding morphological anomaly among the plurality of morphological anomalies.

[0043] The plurality of sets of image data may be generated by the ultrasound system and may include a plurality of image data (e.g., motion video clips) generated by the ultrasound system. The plurality of standard views may include four-chamber (4C), left ventricular outflow tract (LVOT), right ventricular outflow tract (RVOT), three-vessel (3V), and / or three-vessel and trachea (3VT) views. The plurality of morphological abnormalities may include an increased cardiothoracic ratio, a size discrepancy between the right and left ventricles, a size discrepancy between the tricuspid and mitral annuli, cardiac axis deviation, a septal defect in a critical cardiac location, a size discrepancy between the pulmonary and aortic annuli, arterial overriding, and / or an outflow tract relationship abnormality.

[0044] The user interface may include an exam summary adapted to present a list of standard views of the plurality of standard views determined to be present in the plurality of sets of image data and a list of morphological anomalies of the plurality of morphological anomalies determined to be present in the plurality of sets of image data. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figures 1A to 1C is a schematic diagram of an exemplary server-based model and / or local model for implementing the methods and systems of the present invention.

[0046] Figure 2 is an exemplary screen display presented to the clinician showing the results returned by the interpretation component to the user interface.

[0047] 3A-3B are exemplary flowcharts and data flows showing the analysis process performed by an interpretation component to analyze an image data set (eg, a motion video clip) generated during a fetal ultrasound examination.

[0048] Figure 4A and4B is an exemplary image presented to the clinician via the user interface module after the analysis results are returned from the interpretation module, wherein Figure 4A Displays the image selected to correspond to the four-chamber view and Figure 4B A similar view from a different fetus includes a bounding box identifying the large atrioventricular defect.

[0049] Figure 5 is an exemplary graphical user interface that includes a representation of a patient image as well as a list of standard views and anomalies.

[0050] Figure 6 An exemplary data flow for acquiring a medical image, identifying red flags, annotating the image, and providing the flagged and annotated image in a graphical representation is provided.

[0051] Figure 7 is an exemplary graphical user interface that includes a representation of a patient image, patient information, and an abnormality list.

[0052] Figures 8A to 8C is an exemplary user interface for displaying image data generated by an ultrasound system, along with standard views and abnormality indicators.

[0053] Figure 9 is an exemplary flow chart for dynamically requesting additional image views and recording the presence and absence of standard views and morphological anomalies. DETAILED DESCRIPTION

[0054] Disclosed are systems and methods for performing fetal ultrasound examinations that aid in the detection of critical heart defects during fetal ultrasound examinations, typically performed during the second trimester. Specifically, the present systems and methods assist a trained and qualified physician in interpreting ultrasound recording image data sets (e.g., motion video clips, images, etc.) by identifying and selecting image frames corresponding to standard guideline views for presentation to the physician. More specifically, the present systems and methods assist in detecting and identifying morphological abnormalities that may be indicative of critical CHD. Table 3 provides exemplary correspondences between representative CHDs, the views in which those CHDs typically appear, and the morphological abnormalities that are typically identifiable in those views.

[0055] exist Figure 1AIn the exemplary system depicted in FIG, the system may include a conventional ultrasound system 10, a display computer 20, and a server computer 30 that communicate with each other via a wide area network 40 (illustratively, the Internet or any other suitable network (e.g., a local area network)). In a preferred embodiment, the system and method are embodied in a computer-aided diagnostic aid for performing two-dimensional fetal ultrasound examinations, such as those typically performed during the second trimester. Machine learning algorithms are employed to assist the user in identifying and interpreting standard views in a fetal cardiac ultrasound image dataset (e.g., a motion video clip). Although an ultrasound system is described throughout, it should be understood that the same or similar methods may be used with any other suitable medical imaging system (e.g., CT (computed tomography) scans, magnetic resonance imaging (MRI), etc.).

[0056] In a preferred embodiment, the method and system of the present invention employ two software components: a user interface component and an interpretation component. The user interface computer preferably resides on the display computer 20 and provides the clinician with tools to analyze and review fetal ultrasound images and ultrasound motion video clips. The interpretation component preferably resides on the server computer 30, receives ultrasound motion video clips and image groups from the ultrasound system 10 or the display computer 20, and uses a machine learning algorithm to identify images (e.g., image frames) within the motion video clips and / or image data groups that correspond to fetal ultrasound screening guidelines. The interpretation component also analyzes the identified images and any unidentified images (e.g., corresponding to non-standard or non-recommended views) to detect and identify the presence of morphological abnormalities, and provides that information to the user interface component to highlight such abnormalities for clinician review. In an alternative embodiment, the interpretation component may be executed partially or entirely in real time on a local computer workstation.

[0057] Typically, the ultrasound system 10 includes a handheld probe that a clinician moves over a patient's abdomen during a prenatal fetal examination to generate image data sets (e.g., motion video clips) of the fetus that can be transferred to a display computer 20 during the scanning process for storage and display on a display screen associated with the display computer 20. The image data sets (e.g., motion video clips) generated during the examination can be uploaded directly from the ultrasound system 10 to the server system 30 via the wide area network 40 or can be transferred by a user interface module executed on the display computer 20.

[0058] The display computer 20 is preferably configured to display real-time video generated by the ultrasound system 10 and, in addition, to display analysis results generated by the interpretation component executing on the server system 30 to the clinician. The display computer may include a display screen, storage devices, a CPU, input devices (e.g., a keyboard, a mouse), and network interface circuitry for bidirectional communication with the server system 30 via the wide area network 40. In a preferred embodiment, the display computer 20 executes the user interface component of the present invention, which receives and stores physiological information about the patient. The display computer 20 also receives and stores real-time ultrasound video from the ultrasound system 10 and relays that image data, along with the patient's physiological information, to the interpretation component executing on the server system 30.

[0059] The server system 30 includes an interpretation component of the system of the present invention, including a machine learning algorithm for analyzing an image data set (e.g., a motion video clip) received from the display computer 20 to compare the ultrasound video clip with a set of preferred image templates corresponding to fetal ultrasound examination guidelines. In a preferred embodiment, the interpretation component includes an image template corresponding to each of the views recommended in the fetal thermal ultrasound screening guidelines set forth in Table 2, the views including: (1) transverse abdominal view; (2) four-chamber view (4C); (3) left ventricular outflow tract view (LVOT); (4) right ventricular outflow tract view (RVOT); (5) three-vessel view (3V); and (6) three-vessel and tracheal view (3VT). As described in further detail below, the interpretation component preferably employs machine learning to compare each frame of the input image data (e.g., a motion video clip) with the six aforementioned view templates and selects one or more high-quality image frames that correspond to the selected templates. If an abnormality is detected, an image frame displaying the abnormality may be selected. The interpretation component employs a machine learning model to analyze each of the image frames selected as representative of the guideline view, and optionally other unselected image frames, for the presence of anomalies known to exist in image templates such as those set forth in Table 3.

[0060] For example, once the interpretation component has identified and selected an image frame from an uploaded image dataset (e.g., a motion video clip) that is representative of a 3VT view, the machine learning feature will analyze the selected image frame for features identified in Table 3 as being visible in a 3VT view: an aorta larger than the pulmonary artery, which is associated with aortic coarctation and conotruncal lesions; a right aortic arch, which is associated with conotruncal lesions; abnormal vascular arrangement, which is associated with transposition of the great arteries; and additional visible vessels, which are associated with anomalous pulmonary venous connections.

[0061] If the interpretation component of the system identifies that one or more of the features described in Table 3 are present in the selected image frame, the system can further create an overlay on the selected image, the overlay comprising a bounding box surrounding the detected abnormality and, optionally, a text label associated with the suspected defect. The selected image frame and the analysis results are then transmitted back to the display computer 20 for presentation to the clinician and consideration. Because clinicians often have multiple patients, a clinician may be sent or otherwise assigned responsibility for reviewing the results for several patients. To facilitate efficient review by clinicians and / or experts, the system can automatically organize the results with the most relevant information (e.g., detected morphological abnormalities) appearing first or otherwise appearing most prominently. Additionally or alternatively, the results can be organized by patient severity.

[0062] The display computer 20 may provide the ability to annotate selected image frames with additional graphical or textual annotations, which are then saved with the results for later recall during preparation of a documented report regarding the fetal ultrasound examination.

[0063] If, during analysis by the interpretation component, no image frames corresponding to the image data (e.g., a motion video clip) are identified as corresponding to a standard view template, or the identified image frames are judged to be of too poor quality to warrant potential defect analysis, then that image template is identified as missing when the analysis results are transmitted back to the display computer 20. In this case, the clinician can be prompted by the display computer 20 to rescan the fetus to obtain the missing view, and that image data (e.g., a motion video clip) can be resubmitted to the interpretation component for additional analysis. The results of the additional analysis can then be sent back to the display computer 20 for presentation to the clinician and consideration.

[0064] Now refer to Figure 1B ,illustrate Figure 1A The system 15 can be used with Figure 1A The system described in the same manner as in the preceding claims may include an ultrasound system 10, a display computer 20, and a server 30. Figure 1B, the ultrasound scanning system 10 can be any suitable ultrasound scanning system for performing fetal anatomy ultrasound examinations (e.g., a second trimester fetal anatomy ultrasound examination between 18 and 24 weeks of gestation, a first trimester examination, a third trimester fetal examination, etc.), but the present invention software / programming described herein is stored and executed on a controller of the ultrasound scanning system 10. In one example, the ultrasound scanning system 10 can be a Samsung WS80A ultrasound system or any other suitable ultrasound scanning system. The ultrasound scanning system 10 can include an ultrasound probe (e.g., probe 35), a display, one or more computers, inputs (e.g., a keyboard, mouse, knobs, dials, switches, toggles, and the like), speakers, microphones, and the like. The system 15 can optionally also include a healthcare provider device 25, which can include a display 22.

[0065] The healthcare provider device 25 may be a standalone computer device that can display analysis results generated by the interpretation component executed on the server system 30 to a healthcare provider (e.g., a doctor, technician, specialist, etc.). The display computer may include a display screen, a storage device, a CPU, an input device (e.g., a keyboard, a mouse), and network interface circuitry for bidirectional communication with the server system 30 and / or the computer device 20 via any suitable wired or wireless connection. The display computer 20 and optionally the healthcare provider device 25 may execute the user interface components of the present invention. For example, the display computer 20 and / or the healthcare provider device 25 may display a graphical user interface 17, which may be any graphical user interface described herein (e.g., Figures 8A to 8C The healthcare provider device 25 may communicate with the server 30 and / or the computer device 20 to provide input, comment, edit, and otherwise adjust or modify the graphical user interface.

[0066] Now refer to Figure 1C ,illustrate Figures 1A to 1B The clinical workflow of the system is described in Figure 1C As shown in FIG, the clinical center 12 can communicate with a back end 14, which can be on a server (e.g., Figure 1A The clinical center 12 may include an ultrasound module 16 that can be used on an ultrasound scanning device (e.g., Figure 1A a picture archiving and communications (PACS) system 18, which may be running on the ultrasound scanning device or a device located in the same building or campus as the ultrasound scanning device and / or on a remote server; a digital imaging and communications in medicine (DICOM) viewer 22; and a DICOM router 24.

[0067] The ultrasound module 16 may generate, receive, acquire, and / or store ultrasound images (e.g., image data, such as motion video clips and image frames). The image data may be communicated from the ultrasound module 16 to the PACS system 18. The PACS system 18 may securely store the image data received from the ultrasound module 16. The image data stored in the PACS system 18 may be electronically annotated with the record based on user selection input. Once the image data is stored and / or annotated in the PACS system 18, the DICOM router 24 may connect to the PACS system 18 to retrieve the image data and may also connect to the back end 14, which may store the image data on a server (e.g., Figure 1A 30). For example, the DICOM router 24 can be connected to the implementation module 26 and can send the image data to the implementation module 26. In one example, the DICOM router 24 can pseudonymize the files so that only the pseudonymized files are sent to the back end 14. For example, all patient information except certain necessary variables (e.g., gestational age) can be removed, and a pseudonym identifier can be added to the file for each examination and / or for each record. Once the DICOM router 24 receives the output from the back end 14, it can then perform re-identification by replacing the pseudonym identifier with the patient information. The implementation module 26 can upload the image data to the storage device 28. For example, the storage device 28 can store encrypted and otherwise protected image data.

[0068] Implementation module 26 may retrieve certain image data from storage device 28 and may communicate such image data to analysis module 29. Analysis module 29 may process the image data using a machine learning algorithm to identify the presence of morphological abnormalities in the image data, as described in greater detail herein with respect to FIGURES 3A and 3B. In one example, the results and / or output of analysis module 29 may be stored in storage device 28 as an annotated DICOM file. The annotated DICOM file (e.g., indicating morphological abnormalities) may be communicated back to DICOM router 24 and stored in PACS 18. Once stored in PACS 18, a healthcare provider (e.g., a physician) uses DICOM viewer 22 to access the annotated DICOM file from PACS 18 and view the annotated DICOM file (e.g., using a healthcare provider device).

[0069] Now refer to Figure 2, depicts an exemplary display suitable for summarizing the results returned to the display computer 20 by the interpretation component residing on the server system 30. The display 50 includes three columns 51 through 53, which may contain links that may be activated using an input device (e.g., a mouse) associated with the display computer 20. Column 51 labeled "View" describes a standard guideline view (e.g., 4C, LVOT, etc.). Column 52 labeled "Image" contains a checkbox indicating whether the interpretation component of the system has identified and selected an image frame, such as one corresponding to a standard guideline view. Column 53 labeled "Observation" indicates whether any observations (e.g., potential abnormalities) have been detected in the selected image frame.

[0070] Activating a link in the View column (column 51), for example by clicking with a mouse on the view title, will display an idealized, generic image of a standard guideline view (e.g., those shown in Table 2). In column 52, the presence of a check box indicates that the image frame was selected by the interpretation component on the server computer 30. Clicking that check box will cause the display computer to display the original image selected by the interpretation component. The absence of a check box in column 52 indicates that the interpretation component was unable to locate an image in the image data (e.g., a motion video clip) suitable for analysis by machine learning features. For example, for Figure 2 Clicking on an empty check box in RVOT can be configured to display a prompt to the clinician to rescan a portion of the patient's abdomen to obtain new image data (e.g., a motion video clip) containing the desired view, which can then be sent to the server computer 30 for additional analysis.

[0071] Column 53 may contain a textual description of any observations annotated by the interpretation component in the selected image frame. Figure 2 , column 53 presents the label "None" for all views except the RVOT view. Because display 50 is visible to the patient, column 53 preferably uses a neutral label "Review" to indicate to the clinician that a potential abnormality was detected in that view, rather than a more descriptive label that could cause undue concern in the patient. In one embodiment, the text indicator for each standard view can be a clickable link. For example, clicking on the Figure 2 The 4C, LVOT, 3V, and 3VT views in the display indicate that the label "None" may display the image frame selected by the interpretation component along with labels indicating where the interpretation component determines the anatomical landmarks are located. Figure 2 The tab "Review" of the RVOT view in

[15] displays the selected image frame annotated with anatomical landmarks and a bounding box surrounding the suspected abnormality along with a text description of the potential defect.

[0072] In a fetal ultrasound examination performed in accordance with the principles of the present invention, after reviewing the real-time ultrasound image data (e.g., a motion video clip) generated by the ultrasound scanner 10, as displayed on the display computer 20, the clinician can then review the analysis results generated and returned by the interpretation component resident on the server computer 30. In this manner, the clinician can review Figure 2 The display computer 20 may then review the contents of the display 50 of the standard guideline view, review the selected raw image data corresponding to each standard guideline view (by clicking the checkbox in column 52), and review the detailed machine learning analysis of that selected image frame by clicking the label in column 53. The clinician may thus be able to confirm his or her own observations during the review of the real-time video clip or adjust his or her findings based on the machine learning analysis results. As described above, the display computer 20 may include the ability to open additional boxes associated with any of the image frames presented in column 53 to record and store additional findings for later recall in preparing a written or electronic report documenting the fetal ultrasound examination.

[0073] Turning now to FIG3A , an exemplary flowchart 60 illustrating the interpretation component of the analysis software is described. It should be understood that the tasks and / or operations performed in flowchart 60 can be performed on a server (e.g., server 30 of FIG1 ), an ultrasound scanning system (e.g., ultrasound scanning system 10 of FIG1 ), and / or a display (e.g., display 20 of FIG1 ). At step 61 , image data (e.g., motion video clips, image frames, and the like) is received from and / or determined by the ultrasound system 10 or display computer 20. At step 62 , a template corresponding to a standard guideline view, such as 4C, LVOT, RVOT, 3V, or 3VT, is selected. The template may consist of an idealized version of the images shown in Table 2. At step 63, the received image data (which may be a video image clip) is analyzed using a machine learning algorithm (e.g., a convolutional neural network, a deep neural network, and / or one or more suitable neural networks) to compare each frame (e.g., image frame) of the received image data (e.g., a motion data clip) to a standard template to determine which frame or frames best match the standard view guide template. It should be understood that at step 63, a video clip formed by a group of image frames or individual image frames can be analyzed. The interpretation component can also analyze one or more selected image frames to confirm that the image meets the specified quality requirements for clarity. If no frames in the image data are determined to correspond to a standard view, then at decision block 64, the process moves to step 65 (where a flag indicating that no suitable image frames are available is set), and the process continues with selecting the next standard view template at step 62.

[0074] If the interpretation component determines that a corresponding frame is available in the received image data, the process moves to step 66, where the selected image frame and optionally unselected image frames are analyzed by another machine learning algorithm (e.g., one or more suitable neural networks) to detect the presence or absence of anomalies associated with that standard view. For example, if the selected image frame corresponds to a 4C standard view template, the algorithm will analyze the selected frame for any of the defects and / or anomalies listed in Table 3 for that standard view. If a defect is detected in the selected image frame, the algorithm may check adjacent frames of the video clip to confirm the presence of the same defect.

[0075] In one example, morphological abnormalities may include: arterial overhang (e.g., an artery outflowing from the left ventricle is positioned over a ventricular septal defect), a septal defect in a critical cardiac location (e.g., a septal defect located in a critical cardiac location, either a primary atrial septum or an inflow ventricular septum), parallel great arteries, an increased cardiothoracic ratio (e.g., a ratio of the area of ​​the heart to the thoracic cavity measured at the end of diastole greater than 0.33), a discrepancy between the right and left ventricle sizes (e.g., a ratio of the area of ​​the left and right ventricles greater than 1.4 or less than 0. 5), tricuspid and mitral annular size discrepancy (e.g., a ratio between the tricuspid and mitral annuli at the end of diastole greater than 1.5 or less than 0.65), pulmonary and aortic annular size discrepancy (e.g., a ratio between the pulmonary and aortic annuli at the end of systole greater than 1.6 or less than 0.85), outflow tract relationship abnormality (e.g., the aorta and pulmonary arteries do not have a typical anterior-posterior crossing pattern), and cardiac axis deviation (e.g., the angle between the cardiac axis (the line bisecting the thoracic cage and the interventricular septum) is less than 25° or greater than 65°). Alternatively or additionally, at step 66, any other morphological abnormalities may be detected.

[0076] At optional step 67, an overlay may be created for the selected image frame, including a graphical pointer to the detected anatomical landmark and a bounding box surrounding the abnormality detected in the image frame. The overlay may also additionally or alternatively include textual information describing the specific abnormality and / or associated CHD class, as set forth in Table 3. At step 68, the information generated by the interpretation component (i.e., the overlay and graphical / descriptive information) is associated with the selected image frame and stored in the server computer 30 for later transmission to the display computer 20. At optional decision block 69, a determination is made as to whether all image data received at step 61 has been analyzed and / or whether all standard views have been determined to be present. If not all standard views have been determined to be present and / or all image data received at step 61 has not been analyzed, the process may return to step 62, where the next standard view template is selected for analysis. Alternatively, if it is determined at decision block 69 that all standard views have been determined to be present and / or all image data has been analyzed, the process may move to step 71, where the results are returned to the display computer 20 for presentation and review by the clinician. Alternatively, decision 69 may be optional and may be bypassed to initiate blocks 71 and / or 72. For example, a user may determine that analysis results are to be returned to the user interface for display and / or report generation even if all standard views have not been determined to be present and / or all image data has not been analyzed.

[0077] At optional step 72, the analysis and / or results may be used to generate a report. For example, the report may identify detected morphological abnormalities corresponding to certain image frames and / or may include an entry for each standard view. Alternatively, only entries for standard views determined to be present may be included in the report. For example, detected abnormalities may include one or more of abnormal ventricular asymmetry, coarctation of the aorta, pulmonary or aortic valve stenosis, ventricular hypoplasia, or a univentricular heart, and / or any other cardiovascular abnormality. The report may be pre-populated so that for each standard view entry, a representative image can be selected. If a morphological abnormality is detected, an image representing the morphological abnormality for a given standard view may be included in the report in the corresponding view entry. If a bounding box is generated for a given frame, this image with the bounding box overlaid may be included in the report. Information regarding the view, anatomical structure, any textual description of the detected morphological defect and / or abnormality, and / or any other relevant information may also be included in the report to add context to the image and otherwise generate a more informative report. The resulting analysis, results, annotations, and / or report may be stored for later reference. The system may cause a display screen to present the report. For example, in response to a clinician's request, a report may be caused to be displayed or otherwise presented.The report may include an overlay indicating the presence of one or more potential abnormalities.

[0078] The images, image frames, video clips, analyses, results, annotations, and / or reports may be shared with or otherwise made available to a specialist or clinician (e.g., after referral to a specialist or clinician). Each type of morphological abnormality may be associated with a specialist or clinician and their contact information. If a morphological abnormality is detected at step 66, a specialist or clinician corresponding to the morphological abnormality may optionally be recommended.

[0079] In addition to performing steps 61-72 illustrated in FIG3A , the system may perform an objective assessment of the examination, the images generated, and / or the technician performing the imaging (e.g., a sonographer). The system may consider data such as the average duration of the examination and / or image collection, the quality of the images obtained, the percentage of standard views for which suitable images were obtained, and / or any other information indicative of the quality of the examination, images, and / or sonographer. For example, this data may be processed using a model trained to generate one or more quality values ​​indicative of the quality of the examination and / or images. The data may be determined over the course of several examinations. For example, the percentage of standard views for which suitable images were obtained for multiple examinations performed by the same technician may be used to determine a quality value indicative of the rate at which the technician performs a comprehensive examination.

[0080] Now go to Figure 3B , describes the neural network output and post-processing steps of the analysis software. Figure 3B As shown in FIG, input 32 can be input into model 33, which can be one or more neural networks. Input 32 can be image data, such as a motion video clip, an image frame, and the like. In one example, input 32 can be raw digital data representing or forming the image data. Model 33 can include a convolutional base 34, a classification head 35, a segmentation head 36, and a keypoint detection head 37. Convolutional base 34 can be a convolutional neural network (CNN) that can process input 32. Convolutional base 34 can include a classification head 35, a segmentation head 36, and a keypoint detection head 37.

[0081] The classification head 35 may be a classification neural network that can be trained to process the input 32 to determine the probability of the presence or absence of one or more morphological anomalies and / or the likelihood that one or more of the morphological anomalies is uncertain. The segmentation head 36 may be a segmentation neural network that can be trained to determine counters, peripheries, and / or regions that may explain or otherwise correspond to an anatomical structure in the image data represented by the input 32. The keypoint detection head 37 may be a neural network that can be trained to determine the location of an anatomical structure and / or point in the image data represented by the input 32.

[0082] like Figure 3BAs shown in FIG, model 33 may output neural network outputs 39, which may include outputs 38, 40, and 41. Output 38 may be output from classification head 35 and may be a probability of the presence or absence of a morphological abnormality (e.g., an overriding artery, a septal defect in a critical cardiac location, parallel great arteries, etc.). Output 40 may be output from segmentation head 36 and may identify data points in the image data that form the outline of an anatomical structure shown in the image data (e.g., the outline of the left ventricle, right ventricle, heart, chest cavity, etc.). Output 41 may be output from keypoint detection head 37 and may identify data points in the image data that correspond to the location of features of an anatomical structure represented in the image data (e.g., the distal end of the tricuspid valve, the distal end of the mitral valve, the distal end of the pulmonary valve, the distal end of the aortic valve, the long axis of the heart, and / or the anterior-posterior axis of the chest cavity).

[0083] The neural network output 39 can then be processed by the post-processing module 42. For example, the output 38 can be processed by the module 43 to determine whether the morphological abnormality is present or absent, or whether the absence or presence is uncertain. For example, the output 38 can be one or more vectors and can include values ​​indicating the presence, probability of absence, and / or uncertainty of each morphological abnormality. The module 43 can process the vectors by comparing each vector to certain thresholds to determine whether each morphological abnormality is absent, present, or uncertain. For example, for the morphological abnormality "overriding artery," a vector can be output with a value of 0.95 representing presence, 0.1 representing absence, and 0.1 representing uncertainty. For each of presence, absence, and uncertainty, a threshold value can be set to 0.9, and a value of 0.95 representing presence can then satisfy the threshold value. Therefore, the module 43 can determine that the abnormality "overriding artery" is present. It should be understood that other thresholds and / or limits can be used to determine the presence, absence, and / or uncertainty of the morphological abnormality.

[0084] Output 40 may be processed by module 44, which may determine measurements (e.g., area, length, diameter, circumference, and the like) of the outlines of anatomical structures shown in the image data, such as, for example, left ventricular area, right ventricular area, cardiac circumference, and / or thoracic circumference. These measurements may then be provided to and processed by module 47, which may determine ratios and / or comparisons of the measurements (e.g., right ventricular area divided by left ventricular area, cardiac circumference divided by thoracic circumference, etc.). These ratios and / or comparisons may then be provided to and processed by module 50, which may determine the absence, presence, or uncertainty of certain abnormalities based on the ratios and / or comparisons determined at module 47 by comparing such ratios and the comparisons to thresholds and / or limits. In one example, module 50 may determine the presence, absence, or uncertainty of a right ventricular / left ventricular size discrepancy or the presence, absence, or uncertainty of an increased cardiothoracic ratio. For example, the values ​​determined by module 46 may be compared to thresholds to determine whether such values ​​exceed the thresholds. For example, the values ​​determined by module 47 may be compared to a threshold value to determine whether such values ​​exceed the threshold value.

[0085] Output 41 may be processed by module 45, which may determine certain measurements (e.g., lengths, angles, areas, etc.) based on features of the anatomical structures represented in the image data. For example, module 45 may determine values ​​such as tricuspid valve size (e.g., length, width, area), mitral valve size, pulmonary valve size, aortic valve size, and / or cardiac axis angle (e.g., the angle between the long axis of the heart and the anterior-posterior axis of the chest). The values ​​determined by module 45 may be provided to module 46, which may clean up ratios and / or comparisons based on the values. For example, a ratio such as tricuspid valve size divided by mitral valve size and / or pulmonary valve size divided by aortic valve size may be determined. The ratios and / or comparisons may then be provided to and processed by module 48, which may determine the absence, presence, or uncertainty of certain abnormalities based on the ratios and / or comparisons determined at module 46 by comparing such ratios and the comparisons to thresholds and / or limits. In one example, module 48 may determine the presence, absence, or uncertainty of tricuspid and mitral valve size discrepancies, pulmonary / aortic valve size discrepancies, or heart axis deviation. For example, the values ​​determined by module 46 may be compared to thresholds to determine whether such values ​​exceed the thresholds.

[0086] Now refer to Figure 4A and 4B , the description can correspond to click Figure 2 As described above, if a standard view is identified by the interpretation component as being present in the image data (e.g., a motion video clip) transmitted to the server computer 30, then Figure 2A check mark will appear in column 52. As previously described, clicking on the check box will cause the original image selected by the interpretation component to be displayed on the display computer 20. Clicking on the "None" label in column 53 will cause the selected image frame annotated with the overlay generated by the interpretation component to be displayed on the display computer 20. Figure 4A is an example of an image frame corresponding to a 4C standard view with no abnormality and the foramen ovale identified. Figure 4B is an example of a selected image frame corresponding to a standard 4C view, in which a large atrioventricular defect is detected. Figure 4B In the figure, the defect is surrounded by a white bounding box; other cardiac structures are identified by text and graphic labels.

[0087] Now refer to Figure 5 , depicts an exemplary graphical user interface including at least a portion of an image and / or video captured by an imaging system (e.g., an ultrasound imaging system), a list of standard views, and a list of abnormalities. The graphical user interface 80 may be a digital display and may be presented on a healthcare provider's device (e.g., a computing device, laptop, desktop, tablet, smartphone, monitor, etc.). The graphical user interface 80 may present information regarding whether an image corresponds to a standard view type and / or whether a certain abnormality or condition is present in the image from the imaging system.

[0088] like Figure 5 , image 81 may be an image frame from a video clip generated by an imaging system (e.g., ultrasound video frames and / or clips). A time bar 87 may include a cursor 90 that can be moved along the time bar to indicate a time point along the video clip corresponding to the image presented in image 81. The time bar 87 may further include an indicator 91 that can visually indicate (e.g., via a color or marker) the presence and location of an abnormality in the video clip. The cursor 90 can be moved along the time bar 87 to an image frame or other time point in the time bar corresponding to an abnormality and / or a standard view. Moving the cursor 90 can then cause the image 81 to change to an image frame at the corresponding time point. In one example, the image 81 can be annotated with a color, bounding box, text, or other visual indicator to identify the location of the abnormality in the image 81.

[0089] The standard view list 82 may include a list of standard imaging views (eg, 4C, LVOT, RVOT, 3V, 3VT, etc.). Figure 581. For each standard view, image 81 includes a record and frame indicator 83, which identifies whether a record exists for each view and whether a representative frame for each view is identified. Each view also includes a time bar 84, and the length of time bar 84 is comparable to the time length of a given video clip. For each time bar, a visual indicator (e.g., visual indicator 85) is included to show the position at which a given standard view appears in the video clip. If no visual indicator is provided for a given time bar, then the corresponding standard view does not appear in the video clip. A cursor bar 86 is also included on the time bar to indicate the position on the time bar 84 corresponding to the image frame presented at image 81.

[0090] Abnormality list 89 may include a list of abnormalities and / or conditions corresponding to image 81. For example, abnormality list 89 may include increased CTR, cardiac axis deviation, RV / LV size discrepancy, TM / MV size discrepancy, septal defect at a critical cardiac location, overriding arteries, parallel great arteries, PV / AV size discrepancy, outflow tract relationship abnormalities, and / or any other abnormalities and / or conditions. For each abnormality and / or condition, graphical user interface 80 includes a record and frame indicator 83 that identifies whether a record exists for each view and whether a representative frame for each view exists.

[0091] Each view also includes a time bar 95, with the length of time bar 95 corresponding to the duration of the given video clip. For each time bar of time bar 95, a visual indicator 96 is included to show the position of the given view in the video clip. If no visual indicator is provided for a given time bar, then the given anomaly or condition corresponding to that time bar does not occur in the corresponding video clip. A cursor bar 86 may also be included on the time bar to indicate the position on time bar 95 corresponding to the image frame presented at image 80.

[0092] Time bar 84 and time bar 95 below can also include time bar 94, and time bar 94 can indicate the position of cursor bar 86 along the length of corresponding video clip via cursor 93. Moving cursor 90, cursor bar 86 and / or cursor 93 can make other cursors and / or cursor bars move accordingly. Time bar 94 can include play and / or pause button. When the play button is pressed, video clip can be played, thereby showing the various image frames of video clip in image 81. As video clip advances in image 81, cursor bar 86 and cursors 90 and 93 can advance along their corresponding time bars. When the pause button is pressed, video clip can pause. Graphical user interface 80 can optionally include button 98 to move to the next or previous video clip.

[0093] Now refer to Figure 6, depicts an exemplary process flow for acquiring medical images, identifying red flags, annotating the images, and providing the flagged and annotated images in a graphical representation. Figure 6 As shown in FIG, fetal images (e.g., 2T fetal ultrasound images, videos, and / or recordings) are generated and transmitted to a platform, which may be a cloud-based platform on a server (which may be a remote server or a local server). The platform may process the images using the techniques described herein to identify red flags (e.g., the presence of anomalies / abnormalities and / or conditions in the image). The images may be annotated to indicate the presence of anomalies and / or conditions. The annotated images may be presented to a healthcare provider (e.g., a clinician) on a healthcare provider device for immediate or near-immediate (e.g., 1 minute, 2 minutes, 3 minutes, 4 minutes, 5 minutes, 10 minutes, etc.) review by the healthcare provider. The review by the healthcare provider may be performed in the presence of the patient. The device generating the images and / or the healthcare provider device may communicate wirelessly with the platform (e.g., via the Internet). The healthcare provider device and the imaging device may be the same or different devices.

[0094] Now refer to Figure 7 , depicts an exemplary graphical user interface including a representation of a patient image, patient information, and a list of abnormalities. The graphical user interface 100 may include patient information 101, including patient ID, healthcare provider center, date, time, and any other relevant information. The graphical user interface 100 may further include an image 102, which may include a play and / or pause button for playing an image video (e.g., an ultrasound video). The user can scroll to the next video or image by moving the graphical user interface up and down.

[0095] The graphical user interface 100 may further include an anomaly analysis 103 for each image 102. The anomaly analysis 103 may include a list of anomalies and may provide a time bar for each anomaly. A visual indicator on the time bar may indicate whether the corresponding anomaly is present in the video. The visual indicator may be a color bar that extends the portion of the time bar where the anomaly is present. The time bar may include a cursor that indicates the position along the time bar corresponding to the image frame presented on the image 102. The anomaly analysis may further include a time bar with a cursor and pause and play buttons. Moving the cursor and / or pressing pause or play may cause the image 102 to be moved to a specific point in time for pausing or playing.

[0096] Now refer to Figures 8A to 8C , illustrating an exemplary user interface for displaying image data generated by an ultrasound system along with standard views and abnormality indicators. For example, Figure 8A The user interface 200 may be provided by an ultrasound system (eg, Figure 1BThe user interface 200 may be presented on a display computer 20 of the ultrasound system 10 and / or a display 22 of a healthcare provider device 25.

[0097] The user interface 200 may include a user section 202 that may include an exam title, which may be an identifier of the ultrasound exam, and user information 206 that may include a user identifier (ID), date, center data, gestational age, mother's age, status (e.g., processed), and the like. The user section 202 may include a comments section 208 for a technician or other healthcare provider to annotate the exam and / or a set of image data (e.g., a video clip).

[0098] The user interface 200 may further include a thumbnail viewer 210, a detailed viewer 230, and an exam summary 214. The thumbnail viewer 210 may be a collection of thumbnails, each thumbnail corresponding to a video clip and / or image frame generated by the ultrasound device 10. For example, during an ultrasound exam, image data, such as image clips and / or image frames, may be generated. In one example, a thumbnail for each video clip generated for a given exam may be included in the thumbnail viewer 210. The thumbnail viewer 210 may further include a video indicator 222 that visually indicates whether the image data includes a video clip and / or an indicator 224 that visually indicates whether a fetal heart is interpretable in at least one frame of a corresponding set of image data. For example, if the set of image data is corrupted or if a fetal heart is not present in the set of image data, the indicator 224 may not be included in the thumbnail viewer 210, or the indicator 224 may visually indicate that the fetal heart is uninterpretable.

[0099] Each set of image data generated during an examination (e.g., a video clip and / or one or more image frames) can be viewed in detail viewer 230 (e.g., by clicking a thumbnail (e.g., thumbnail 220) in thumbnail viewer 210). For example, detail viewer 230 may correspond to thumbnail 220. A user can click a different thumbnail in thumbnail viewer 210 to update detail viewer 230 to present the set of image data corresponding to thumbnail 220.

[0100] Detailed viewer 230 may include image data viewer 232, standard view indicator 234, and morphological anomaly indicator 236. Image data viewer 232 may present a video clip and / or still frames of image data (e.g., the set of image data corresponding to thumbnail 220). Standard view indicator 238 may include standard view list 242 and color indicator 238 that indicates whether each standard view in standard view list 242 is present in the image data or whether the presence of such a standard view is uncertain. For example, a color indicator may indicate whether a certain standard view is present in the image data (e.g., using different colors to represent presence and absence).

[0101] The morphological anomaly indicator 236 may include a morphological anomaly list 244 and a color indicator 240 that indicates whether each morphological anomaly in the morphological anomaly list is present in the image data. For example, the color indicator 240 may indicate whether a particular morphological anomaly is present in the image data or, alternatively, whether the presence of the morphological anomaly is uncertain. The color indicator for the standard view indicator 234 may differ from the color indicator for the morphological anomaly indicator 236 (e.g., each using a different and unique color to represent presence and absence). Furthermore, a different and unique color may be used to represent uncertainty.

[0102] Now refer to Figure 8B , the user interface 200 may be adjusted by the user to present a detailed viewer 250, a thumbnail viewer 210, and an inspection summary 270. The user may select a thumbnail 240 of the thumbnail viewer 210 and / or otherwise navigate the user interface 200 to view the detailed viewer 250, which may correspond to the thumbnail 240. The detailed viewer 250 may be associated with Figure 8A Detailed viewer 250 may be the same as or similar to detailed viewer 230 of Detailed Viewer 250. Detailed viewer 250 may include an image data viewer 251, which may include still frames or video clips of image data (e.g., the set of image data corresponding to thumbnails 240). A time bar 252 may be included in detailed viewer 250, and a cursor 254 may indicate a point in time along the time bar corresponding to an image frame presented in image data viewer 251.

[0103] The detailed viewer 250 may further include a standard view indicator 253 and a morphological abnormality indicator 255, which may be displayed with the standard view indicator 253 and the morphological abnormality indicator 255. Figure 8A The standard view indicator 234 and the abnormal morphology indicator 236 are the same or similar. Figure 8BAs shown in FIG, standard view "4C" may be present in standard view indicator 253, with color indicator 261 indicating "yes" for presence, which is presented in blue. Several morphological abnormalities are indicated as not present in morphological abnormality viewer 255, such as increased CTR, RV / LV size discrepancy, TV / MV size discrepancy, and critical cardiac septal defect. Other abnormalities in morphological abnormality viewer 255 are indicated as indeterminate. Color indicator 263 may display green to indicate the presence of each of these abnormalities in the image data presented in image data viewer 251 and display white to indicate indeterminate.

[0104] The standard view indicator may also include a time bar for each standard view in the standard view list. For example, time bar 256 may correspond to standard view "4C." Similarly, morphology anomaly indicator 255 may include a time bar for each anomaly in morphology anomaly viewer 255. For example, time bar 258 may correspond to a CTR increase. Each time bar may display a color along some or all of the time bar when the corresponding standard view or anomaly is determined to be present or absent in the image data. For example, time bar 256 may be blue to indicate the presence of standard view "4C," and time bar 258 may be green to indicate the absence of a CTR increase.

[0105] Each time bar of the standard view indicator 253 and the morphological anomaly viewer 255 may include a visual indicator that moves along with the cursor 254. For example, the time bar 256 may include the visual indicator 262, and the time bar 258 may include the visual indicator 260. Additionally, below each time bar of the standard view indicator 253 and the morphological anomaly viewer 255 may include a time bar 264 that may be aligned with each time bar (e.g., the time bar 256 and the time bar 258) and may include a cursor that is aligned with the visual indicators 262 and 260, which the user may use to move the cursor 254 to different points in time along the time bar 252.

[0106] The user interface 200 may further include an exam summary 270, which may include a standard view summary 272 and an anomaly summary 276. The anomaly summary 276 may summarize the standard views and morphological abnormalities determined to be present, absent, or indeterminate in the image data set uploaded from the ultrasound system. The standard view summary 272 may include a list of standard views and a color indicator to indicate whether each standard view is present, absent, or indeterminate. The standard view summary 272 may include a forward button 274 for each standard view, which the user can use to advance the user interface 200 to a detailed viewer containing image data for which a standard view exists. The forward button 274 may automatically adjust the image data viewer so that the image frames for which a standard view exists are visible. For example, each time the forward button 274 is pressed, the next image frame determined to correspond to the corresponding standard view is displayed in the detailed viewer, allowing the user to efficiently view image frames and / or image data sets corresponding to the standard view. The exam summary 270 allows the user to efficiently determine whether a view and / or abnormality is absent, present, or indeterminate (e.g., whether an abnormality is indeterminate).

[0107] The inspection summary 276 may further include an exception summary 272. The exception summary 276 may include a list of exceptions and a color indicator to indicate whether each exception exists. The exception summary 276 may include a forward button 278 for each exception, which the user can use to advance the user interface 200 to a detailed viewer containing image data for which the exception exists, and the image data viewer may be automatically adjusted so that the image frame for which the exception exists is visible. For example, each time the forward button 278 is pressed, the next image frame determined to correspond to the corresponding exception will be displayed in the detailed viewer, thereby allowing the user to efficiently view the image frames and / or image data sets corresponding to the exception. The user interface 200 may further include a save button 279 to save any image data, images, determinations and / or data, settings, annotations, comments, or the like from the user interface 200.

[0108] Now refer to Figure 8C , the detail viewer 250 of the user interface 200 may be updated to indicate that an anomaly has been determined to be present in the image data. For example, the morphological anomaly indicator 255 may be updated to change the color indicator 284 corresponding to the heart axis offset to red to indicate that the image data presented in the image data viewer 251 is "heart axis offset." The time bar 282 may be changed to the same color as the color indicator 255 (e.g., red) to indicate the presence of a morphological anomaly. The time bar 282 may alternatively be changed to the same color as the "absent" indicator (e.g., green) to indicate one or more frames along the time bar corresponding to the absence of the anomaly. Alternatively, only the portion of the time bar 282 corresponding to the time point in the image data associated with the image frame showing the anomaly will be colored.

[0109] Now refer to Figure 9 , illustrates a process flow 300 for dynamically requesting additional image data based on the presence or absence of standard views and morphological abnormalities in the image data. Some or all blocks of the process flow in the present disclosure can be performed in a distributed manner across any number of devices. Some or all operations of the process flow may be optional and may be performed in a different order.

[0110] At optional block 302, a process stored on a device (e.g., a server and / or a computer (e.g., Figure 1B A request is made for image data (e.g., image frames, video clips, etc.) having a certain standard view (e.g., 4C) from a set of predetermined standard views. For example, a list of standard views that should be collected during an ultrasound examination may be predetermined. At block 304, the computer-executable instructions stored in the memory of a device (e.g., a server and / or computer of the ultrasound system) may be executed to determine the image data (e.g., image frames, video clips, etc.). For example, the ultrasound system may present a prompt for certain image data, and the user may generate the image data using the ultrasound system, which may include an ultrasound probe.

[0111] At block 306, computer-executable instructions stored in a memory of a device (e.g., a server and / or computer of the ultrasound system) may be executed to analyze the image data to determine the presence or absence of a standard view from the set of standard views. For example, the image data may be processed using the method described above with respect to FIG. 3A. At decision 307, computer-executable instructions stored in a memory of a device (e.g., a server and / or computer of the ultrasound system) may be executed to determine whether the image data satisfies a standard view (e.g., whether a standard view is present in the image data).

[0112] At block 310, if a standard view is determined to be absent or if it is uncertain whether a standard view exists in the image data, computer-executable instructions stored in a memory of a device (e.g., a server and / or computer of the ultrasound system) may be executed to record or otherwise annotate the absence of a standard view in the image data or, if it is unclear whether a standard view exists or does not exist, to record or otherwise annotate the uncertainty of the presence of a standard view. Furthermore, one or more time points (e.g., timestamps) in the image data may be associated with the absence or uncertainty of a standard view in the image data. At block 311, computer-executable instructions stored in a memory of a device (e.g., a server and / or computer of the ultrasound system) may be executed to present a request for additional image data corresponding to a standard view.

[0113] Alternatively, if a standard view is determined to be present in the image data, then at block 308, computer-executable instructions stored in a memory of a device (e.g., a server and / or computer of the ultrasound system) may be executed to record or otherwise annotate the presence of a standard view in the image data, and one or more time points (e.g., timestamps) in the image data may be associated with the standard view. At block 312, computer-executable instructions stored in a memory of a device (e.g., a server and / or computer of the ultrasound system) may be executed to analyze the image data to determine the presence or absence of certain morphological anomalies from the set of morphological anomalies. For example, the image data may be processed using the method described above with respect to FIG. 3A . Data indicating one or more potential anomalies may be determined for each view template. Optionally, block 312 may be initiated after block 310 and after block 308.

[0114] At decision 316, computer-executable instructions stored in a memory of a device (e.g., a server and / or computer of the ultrasound system) may be executed to determine whether a morphological abnormality is present or absent. If it is unclear whether a morphological abnormality is present or absent, then at block 314, computer-executable instructions stored in a memory of the device (e.g., a server and / or computer of the ultrasound system) may be executed to record or otherwise mark the morphological abnormality as uncertain because it is unclear whether the abnormality is present or absent at one or more time points (e.g., timestamps) associated with the image data. Alternatively, if it is determined that a morphological abnormality is present or absent, then at block 318, computer-executable instructions stored in a memory of the device (e.g., a server and / or computer of the ultrasound system) may be executed to record or otherwise mark the morphological abnormality as present or absent, as appropriate, and / or associate the presence or absence of the abnormality with one or more time points in the image data. For example, data indicating one or more morphological abnormalities may be associated with a view template corresponding to the image data.

[0115] At decision 320, computer-executable instructions stored on a memory of a device (e.g., a server and / or computer of an ultrasound system) may be executed to determine whether additional views of the anomaly are required. For example, when an anomaly is determined to be present, it may be desirable to generate additional imaging to further analyze the anomaly. Conversely, when it is determined that an anomaly is not present, it may be desirable to generate additional images to further confirm that the anomaly is not present. Whether additional views are required may depend on the type of anomaly detected and may be predetermined (e.g., if a certain anomaly is detected, the system may automatically request certain additional views). If additional views are not required at decision 320, blocks 302 and / or 304 may be restarted. Alternatively, if additional views of the anomaly are required, at block 322, computer-executable instructions stored on a memory of a device (e.g., a server and / or computer of an ultrasound system) may be executed to request additional views of the anomaly and block 304 may be restarted.

[0116] While various illustrative embodiments of the present invention have been described above, it will be apparent to those skilled in the art that various changes and modifications may be made thereto without departing from the present invention. The appended claims are intended to cover all such changes and modifications as fall within the true scope of the invention.

Claims

1. A system for use with an ultrasound system for assisting a clinician in detecting and diagnosing heart defects during a fetal ultrasound examination, the system comprising: One or more computers configured to store non-transitory programming instructions, the one or more computers being programmed to: Storing view templates corresponding to standard guideline views, including at least view templates for 4C, LVOT, RVOT, 3V, and 3VT views; storing data indicative of one or more potential exceptions associated with each of the view templates; receiving a plurality of sets of image data generated by the ultrasound system during a fetal ultrasound examination, each set of image data in the plurality of sets of image data comprising a plurality of frames; comparing each frame of the plurality of frames to the view template to identify and select a corresponding frame for each view template, if one exists; analyzing each corresponding frame to detect the presence of the one or more potential anomalies; and The corresponding frame is presented on a display screen for each standard view in response to the clinician's request, including an overlay indicating the presence of the one or more potential abnormalities.

2. The system of claim 1, wherein the one or more computers include a display computer and a server computer.

3. The system of claim 2, wherein the non-transitory programming instructions include a user interface component and an interpretation component. 4 . The system of claim 3 , wherein the user interface component is configured to receive, store, and display the multiple sets of image data generated by the ultrasound system in real time.

5. The system of claim 4, wherein the user interface component is configured to store and display analysis results returned by the interpretation component.

6. The system of claim 3, wherein the interpretation component comprises a machine learning algorithm for identifying and selecting each corresponding image frame.

7. The system of claim 3, wherein the interpretation component further comprises a machine learning algorithm for detecting the presence of the one or more potential anomalies in each corresponding image frame. 8 . The system of claim 1 , wherein the overlay indicating the presence of the one or more potential anomalies comprises one or more of a bounding box surrounding potential anomalies, a graphical indicator, or a textual indicator.

9. The system of claim 1, further comprising non-transitory programmed instructions that enable the clinician to annotate the overlay.

10. The system of claim 1, further comprising non-transitory programmed instructions for generating a report documenting the clinician's observations during the fetal ultrasound examination.

11. The system of claim 1, wherein the report comprises the clinician's observations during the fetal ultrasound examination.

12. The system of claim 1, wherein said 4C represents four chambers, LVOT represents left ventricular outflow tract, RVOT denotes right ventricular outflow tract, 3V triple vessel, and 3VT triple vessel and trachea.

13. The system of claim 1, wherein the one or more computers are further programmed to determine, for each template of the view, data indicative of one or more potential anomalies.

14. The system of claim 1, wherein the one or more computers are further programmed to associate each of the data indicative of one or more potential anomalies with each respective view template.

15. The system of claim 1, wherein each frame of the plurality of frames is compared to the view template using a neural network.

16. The system of claim 1, wherein the one or more computers are further programmed to: determining, for each of the view templates, the data indicative of one or more potential anomalies; associating each of the data indicative of one or more potential anomalies with each respective view template; using a neural network, comparing each corresponding image frame with the data indicative of the one or more potential anomalies for the view template corresponding to each corresponding image frame to detect the presence of the one or more potential anomalies; generating, for each corresponding image frame, a report indicating the presence of one or more potential anomalies; and The display screen is caused to present the report, including an overlay indicating the presence of the one or more potential abnormalities, in response to a request by the clinician.

17. A method for use with an ultrasound system for assisting a clinician in detecting and diagnosing heart defects during a fetal ultrasound examination, the method comprising: Providing a computer configured to store and execute non-transitory programming instructions; storing view templates corresponding to standard guideline views on the computer, including at least view templates for 4C, LVOT, RVOT, 3V, and 3VT views; storing on the computer data indicative of one or more potential anomalies associated with each of the view templates; generating, with an ultrasound system, a plurality of sets of image data during a fetal ultrasound examination and storing the plurality of sets of image data, each set of image data comprising a plurality of frames; receiving the plurality of sets of image data by the computer; comparing, with the computer, each frame of the plurality of frames to the view template to identify and select, for each view template, a corresponding image frame if one exists; analyzing, with the computer, each corresponding image frame to detect the presence of the one or more potential abnormalities; and The corresponding image frame is presented for each standard view on a display screen associated with the computer in response to a request by the clinician, including an overlay indicating the presence of the one or more potential abnormalities.

18. The method of claim 17, wherein providing a computer comprises providing one or more computers including a display computer and a server computer.

19. The method of claim 18, wherein providing the one or more computers comprises providing non-transitory programming instructions for user interface components to the display computer and providing non-transitory instructions for interpreting components to a server computer.

20. The method of claim 19, further comprising displaying the multiple sets of image data generated by the ultrasound system to the clinician in real time using the display computer.

21. The method of claim 19, further comprising transmitting analysis results generated by the interpretation component from the server computer to the display computer.

22. The method of claim 17, wherein comparing, with the computer, each of the plurality of frames to the view template comprises analyzing, with a machine learning algorithm, the plurality of frames to identify and select each corresponding image frame.

23. The method of claim 17, wherein analyzing each corresponding image frame with the computer to detect the presence of the one or more potential anomalies comprises analyzing each corresponding image frame with a machine learning algorithm to detect the presence of the one or more potential anomalies.

24. The method of claim 17, wherein presenting the overlay on the display screen comprises presenting a graphic or textual indicia indicating the presence of the one or more potential anomalies.

25. The method of claim 17, further comprising creating, with the computer, an annotated overlay corresponding to the overlay including additional graphical or textual information entered by the clinician and storing the annotated overlay.

26. The method of claim 17, further comprising generating, with the computer, a report for the fetal ultrasound examination documenting the annotated overlay.

27. The method of claim 17, further comprising generating, with the computer, a report having an entry for each standard view and the overlay indicating the presence of the one or more potential anomalies.

28. The method of claim 27, further comprising determining, with the computer, the presence of one or more potential abnormalities and sending one or more of at least a portion of the multiple sets of image data or the report to a specialist.

29. The method of claim 17, further comprising determining, with the computer, a quality value for the fetal ultrasound examination based on the multiple sets of image data.

30. A computer-implemented method for analyzing a fetal ultrasound image, the method comprising: receiving a plurality of sets of image data generated by an ultrasound system during a fetal ultrasound examination, each set of image data in the plurality of sets of image data comprising a plurality of frames; analyzing a set of image data from the plurality of sets of image data to automatically determine that one or more frames of the set of image data corresponds to a standard view from a plurality of standard views; analyzing the set of image data to automatically determine that the one or more frames are indicative of a first morphological abnormality of a plurality of morphological abnormalities; Generating a user interface for display, wherein the user interface comprises: (a) an image data viewer adapted to visually present the set of image data; (b) a standard view indicator corresponding to the set of image data presented on the image data viewer and visually indicating whether each standard view of the plurality of standard views is present in the set of image data; and (c) a morphological anomaly indicator corresponding to the set of image data presented on the image data viewer and visually indicating whether each morphological anomaly of the plurality of morphological anomalies is present in the set of image data, Wherein, when the image data viewer visually presents the set of image data, the standard view indicator indicates that a first standard view exists in the set of image data and the morphological abnormality view indicator indicates that a first morphological abnormality exists.

31. The computer-implemented method of claim 30, wherein the user interface is generated on a display of the ultrasound system.

32. The computer-implemented method of claim 30, wherein the user interface is generated on a display of a healthcare provider device.

33. The computer-implemented method of claim 30, wherein the standard view indicator comprises a plurality of color indicators each corresponding to one of the plurality of standard views, each color indicator of the plurality of color indicators being adapted to present a first color when a corresponding standard view of the plurality of standard views is present in the set of image data and to present a second color when the corresponding standard view of the plurality of standard views is not present in the set of image data.

34. The computer-implemented method of claim 30, wherein the morphological anomaly indicator comprises a plurality of color indicators each corresponding to one of the plurality of morphological anomalies, each color indicator of the plurality of color indicators being adapted to present a first color when a corresponding morphological anomaly of the plurality of morphological anomalies is present in the set of image data and to present a second color when the corresponding morphological anomaly of the plurality of morphological anomalies is not present in the set of image data.

35. The computer-implemented method of claim 34, wherein each color indicator of the plurality of color indicators is further adapted to present a third color indicating that presence of the respective one of the plurality of morphological anomalies in the set of image data is uncertain.

36. The computer-implemented method of claim 30, wherein the image data viewer comprises a first time bar and a cursor on the time bar, and wherein the cursor is adapted to move to cause the image data viewer to visually present a plurality of image frames corresponding to a plurality of time points along the time bar.

37. The computer-implemented method of claim 36, wherein the standard view indicator comprises a plurality of second time bars each corresponding to the first time bar and each having a first visual indicator corresponding to the cursor and adapted to move synchronously with the cursor.

38. The computer-implemented method of claim 37, wherein each of the plurality of second time bars is adapted to visually indicate one or more time points on the second time bar that each correspond to the presence of a respective standard view of the plurality of standard views.

39. The computer-implemented method of claim 36, wherein the morphological anomaly indicator comprises a plurality of second time bars each corresponding to the first time bar and each having a first visual indicator corresponding to the cursor and adapted to move synchronously with the cursor.

40. The computer-implemented method of claim 39, wherein each of the plurality of second time bars is adapted to visually indicate one or more time points on the second time bar corresponding to the presence of a respective morphological anomaly of the plurality of morphological anomalies.

41. The computer-implemented method of claim 30, wherein the plurality of sets of image data generated by the ultrasound system comprises a plurality of motion video clips generated by the ultrasound system.

42. The computer-implemented method of claim 30, wherein the plurality of standard views includes four-chamber (4C), left ventricular outflow tract (LVOT), right ventricular outflow tract (RVOT), three-vessel (3V) and / or three-vessel and trachea (3VT) views.

43. The computer-implemented method of claim 30, wherein the plurality of morphological abnormalities comprises an increased cardiothoracic ratio, a size discrepancy between the right and left ventricles, a size discrepancy between the tricuspid and mitral annuli, cardiac axis deviation, a septal defect at a critical cardiac location, a size discrepancy between the pulmonary and aortic annuli, arterial overriding, and / or outflow tract relationship abnormalities.

44. The computer-implemented method of claim 30, wherein the user interface further comprises an examination summary adapted to present a list of standard views of the plurality of standard views determined to be present in the plurality of sets of image data and a list of morphological anomalies of the plurality of morphological anomalies determined to be present in the plurality of sets of image data.

45. A system for analyzing fetal ultrasound images, the system comprising: a memory configured to store computer-executable instructions; and at least one computer processor configured to access the memory and execute the computer-executable instructions to: receiving a plurality of sets of image data generated by an ultrasound system during a fetal ultrasound examination, each set of image data in the plurality of sets of image data comprising a plurality of frames; analyzing a set of image data from the plurality of sets of image data to automatically determine that one or more frames of the set of image data corresponds to a standard view from a plurality of standard views; analyzing the set of image data to automatically determine that the one or more frames are indicative of a first morphological abnormality of a plurality of morphological abnormalities; Generating a user interface for display, wherein the user interface comprises: (a) an image data viewer adapted to visually present the set of image data; (b) a standard view indicator corresponding to the set of image data presented on the image data viewer and visually indicating whether each standard view of the plurality of standard views is present in the set of image data; and (c) a morphological anomaly indicator corresponding to the set of image data presented on the image data viewer and visually indicating whether each morphological anomaly of the plurality of morphological anomalies is present in the set of image data, Wherein, when the image data viewer visually presents the set of image data, the standard view indicator indicates that a first standard view exists in the set of image data and the morphological abnormality view indicator indicates that a first morphological abnormality exists.

46. ​​The system of claim 45, wherein the user interface is generated on a display of the ultrasound system.

47. The system of claim 45, wherein the user interface is generated on a display of a healthcare provider device.

48. The system of claim 45, wherein the standard view indicator comprises a plurality of color indicators each corresponding to one of the plurality of standard views, each color indicator of the plurality of color indicators being adapted to present a first color when a corresponding standard view of the plurality of standard views is present in the set of image data and to present a second color when the corresponding standard view of the plurality of standard views is not present in the set of image data.

49. The system of claim 45, wherein the morphological anomaly indicator comprises a plurality of color indicators each corresponding to one of the plurality of morphological anomalies, each color indicator of the plurality of color indicators being adapted to present a first color when a corresponding morphological anomaly of the plurality of morphological anomalies is present in the set of image data and to present a second color when the corresponding morphological anomaly of the plurality of morphological anomalies is not present in the set of image data.

50. The system of claim 49, wherein each color indicator of the plurality of color indicators is further adapted to present a third color indicating that presence of the respective one of the plurality of morphological anomalies in the set of image data is uncertain.

51. The system of claim 45, wherein the image data viewer comprises a first time bar and a cursor on the time bar, and wherein the cursor is adapted to move to cause the image data viewer to visually present a plurality of image frames corresponding to a plurality of time points along the time bar.

52. The system of claim 51, wherein the standard view indicator comprises a plurality of second time bars each corresponding to the first time bar and each having a first visual indicator corresponding to the cursor and adapted to move synchronously with the cursor.

53. The system of claim 52, wherein each of the plurality of second time bars is adapted to visually indicate one or more time points on the second time bar that each correspond to the presence of a respective standard view of the plurality of standard views.

54. The system of claim 51, wherein the morphology anomaly indicator comprises a plurality of second time bars each corresponding to the first time bar and each having a first visual indicator corresponding to the cursor and adapted to move synchronously with the cursor.

55. The system of claim 54, wherein each of the plurality of second time bars is adapted to visually indicate one or more time points on the second time bar corresponding to the presence of a respective morphological anomaly of the plurality of morphological anomalies.

56. The system of claim 45, wherein the plurality of sets of image data generated by the ultrasound system comprises a plurality of motion video clips generated by the ultrasound system.

57. The system of claim 45, wherein the plurality of standard views comprises four-chamber (4C), left ventricular outflow tract (LVOT), right ventricular outflow tract (RVOT), three-vessel (3V) and / or three-vessel and trachea (3VT) views.

58. The system of claim 45, wherein the plurality of morphological abnormalities comprises an increased cardiothoracic ratio, a size discrepancy between the right and left ventricles, a size discrepancy between the tricuspid and mitral annuli, cardiac axis deviation, septal defects at critical cardiac locations, a size discrepancy between the pulmonary and aortic annuli, arterial overriding, and / or outflow tract relationship abnormalities.

59. The system of claim 45, wherein the user interface further comprises an examination summary adapted to present a list of standard views of the plurality of standard views determined to be present in the plurality of sets of image data and a list of morphological abnormalities of the plurality of morphological abnormalities determined to be present in the plurality of sets of image data.

Images