Estimation of cardiac parameters using neural networks

Abstract

One embodiment of the present invention provides a system for automatically deriving parameters of a human heart from ultrasound results. A first neural network is configured to receive multiple echocardiographic images and classify the images into at least one of two-chamber and four-chamber views. A second neural network is configured to receive the echocardiographic images, including two-chamber or four-chamber views, and identify the endocardial border of the left ventricle (LV) for each view. End-systolic and end-diastolic images are then identified, and parameters such as LV volume, ejection fraction, global longitudinal strain, and regional longitudinal strain are calculated.

Description

[Technical Field]

[0001] The present invention relates to a neural network for determining parameters of the human heart from multiple echocardiographic images. [Background technology]

[0002] Echocardiography is a powerful tool for investigating the condition of the heart, but traditionally has relied heavily on expert radiologists to interpret the resulting images, which introduces delays and expense.

[0003] In an echocardiogram, ultrasound data of a particular patient's heart is typically collected for several different "views," that is, data collected by the probe from different points around the patient's body, with each view representing a different plane through the patient's body. These views may include the apical four-chamber view, the apical three-chamber view, the apical two-chamber view, the parasternal long-axis view, and the short-axis view. All views may be present in the patient data, or only a subset may be available. Also, there may be multiple ultrasound videos for a single view for a particular patient. Ultrasound results are often communicated via DICOM (Digital Images and Communication in Medicine) files, although those skilled in the art will recognize that other formats are available.

[0004] Some research has already been done on the use of trained neural networks to interpret individual ultrasound views, for example, to automatically delineate the contours of the heart walls. However, such neural networks only operate on specific views (e.g., apical four-chamber views), which means that any particular network can only accept ultrasound data associated with one type of input data (i.e., one "view").

[0005] Unfortunately, ultrasound files such as DICOM files typically do not contain any information about which views are present in any particular data set. It can take a human operator 20 minutes or more to sort through all the ultrasound data from a particular patient to ensure that data for the correct views is provided to the correct neural network to derive clinically applicable cardiac measurements. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] WO2017 / 216545 [Non-patent literature]

[0007] [Non-Patent Document 1] Keras(Chollet, Francois et. Al. 2015, https: / / keras.io) [Non-patent document 2] Google Tensorflow: A system for large-scale machine learning (Abadi, Martin et al., 2016 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), page 265-283, https: / / www.usenix.org / system / files / conference / osdi16 / osdi16-abadi.pdf) Summary of the Invention [Problem to be solved by the invention]

[0008] The object of the present invention is to remedy this drawback. [Means for solving the problem]

[0009] According to a first aspect of the present invention there is provided a system for providing parameters of a human heart, the system comprising: a first trained neural network having an input and an output, the first trained neural network configured to receive a plurality of echocardiographic images and classify the echocardiographic images into one of at least two different views, including at least a two-chamber view and a four-chamber view; a second trained neural network having an input and an output, the second trained neural network configured to receive images from at least one of a two-chamber or a four-chamber view and identify an endocardial border of the left ventricle for each view; a first responder responsive to the output of the second neural network to identify end-systolic and end-diastolic images; and a second responder responding to end-systolic and end-diastolic images to derive cardiac parameters. Includes:

[0010] By automatically detecting relevant views in the source data and segmenting the left ventricular endocardial border, a more accurate and efficient system is provided. By automatically providing parameters, a physician or subsequent automated system can more easily and efficiently diagnose the patient.

[0011] Most parameters require delineating (segmenting) the endocardial border of the left ventricle and combining at least two- and four-chamber views to form a volume, such as Simpson's biplane method to determine ventricular volume, which is further required to calculate parameters such as ejection fraction.

[0012] From the left ventricle segmentation, appropriate parameters are derived, including ejection fraction (EF) and global longitudinal strain (GLS). Additionally or alternatively, local (regional) strain values ​​may be derived. The view detection network may provide a confidence measure along with the decision regarding which views are present. This measure may be used to eliminate any redundant views (i.e., three-chamber apical and SAX views) and / or to select one set of views from multiple sets of views of the same type.

[0013] The first responder can be a means responsive to the output of the second neural network to identify end-systolic and end-diastolic images. The second responder can be a means responsive to the output of the second neural network to identify end-systolic and end-diastolic images.

[0014] According to a second aspect of the present invention there is provided a method of diagnosing a cardiac condition, the method comprising: receiving a plurality of echocardiographic images from a subject; using the system of the first aspect to analyze a plurality of echocardiographic images to derive a cardiac parameter; comparing the cardiac parameter to at least one predetermined threshold parameter; detecting a difference between the cardiac parameter and at least one threshold parameter, the difference being indicative of a cardiac pathology; Includes:

[0015] According to a third aspect of the present invention there is provided a method of treating a cardiac condition, the method comprising: receiving a plurality of echocardiographic images from a subject using the system of the first aspect to analyze the plurality of echocardiographic images to derive a cardiac parameter; comparing the cardiac parameter to at least one predetermined threshold parameter; detecting a difference between the cardiac parameter and at least one threshold parameter, said difference being indicative of a cardiac pathology; administering to the subject a therapeutically effective amount of a drug, wherein the drug alleviates one or more symptoms of the cardiac condition; Includes:

[0016] The methods and systems of the present invention may be implemented in conventional echocardiography devices or as stand-alone systems that receive echocardiography data.

[0017] The invention will now be described, by way of example, with reference to the following drawings: [Brief explanation of the drawings]

[0018] [Figure 1] FIG. 2 is a block schematic diagram illustrating a first configuration of a neural network, according to one embodiment of the present invention. [Figure 2] FIG. 2 is a block schematic diagram of a second embodiment of the present invention comprising six processing stages connected in series. [Figure 3] FIG. 10 illustrates the contouring of the left ventricle using an embodiment of the present invention. [Figure 4] FIG. 10 illustrates the selection process for end-diastolic and end-systolic frames identified by an embodiment of the present invention. [Figure 5] FIG. 10 illustrates end-diastolic and end-systolic frames identified by an embodiment of the present invention. [Figure 6] FIG. 1 illustrates a comparison between an embodiment of the present invention and conventional human processing of cardiac image data. DETAILED DESCRIPTION OF THE INVENTION

[0019] 1 illustrates a system 100 in which an input ultrasound image 102 is provided to a first trained neural network 104. Apparatus for acquiring ultrasound images is shown, for example, in WO 2017 / 216545, and well-known commercially available systems are available from companies including Philips, Siemens, and General Electric. In this example, the input to the embodiment includes multiple non-contrast echocardiographic video clips consisting of multiple instances of views acquired from a DICOM imaging study of a given subject. In an alternative example, the input to the embodiment includes multiple contrast echocardiographic video clips consisting of multiple instances of views acquired from a DICOM imaging study of a given subject. Of course, other imaging formats may be used as desired.

[0020] Two broad categories of imaging modalities are known to those skilled in the art of echocardiography. These are known as contrast and non-contrast imaging. Contrast echocardiography represents a variety of imaging techniques, all of which rely on the introduction of a contrast agent that is acoustically active and remains within the vascular space for a period of time. In some embodiments of the present invention, contrast echocardiographic images are echocardiographic images acquired during or after the introduction of one or more acoustically active contrast agents. Contrast images may be acquired by introducing a contrast agent, such as microbubbles, into the patient. Contrast agents generally improve the detection of ventricular boundaries in echocardiographic images compared to non-contrast echocardiographic images, which are echocardiographic images captured from subjects who have not received a contrast agent. Non-contrast imaging involves the acquisition of echocardiographic images in the absence of any contrast agent.

[0021] In some embodiments, any aspect of the invention utilizes non-contrast echocardiographic images. In some other embodiments, any aspect of the invention utilizes contrast echocardiographic images. In some embodiments, any aspect of the invention utilizes both contrast and non-contrast echocardiographic images. The systems and methods described herein, and embodiments thereof, are readily adapted to both imaging modalities.

[0022] The first neural network 104 is configured to determine to which view an input video clip is associated so that it can be provided to and correctly processed by subsequent stages of the system.

[0023] Preferably, the first neural network also provides a confidence measure for the associated view determination, i.e., an indication of the likelihood that the view is in fact of the determined type.

[0024] In this example, only the two-chamber and four-chamber apical views are needed. Therefore, any views related to other types of views, such as the three-chamber apical or SAX views, are ignored. This can be done in two different ways.

[0025] The first neural network can be trained to identify any number of possible views, e.g., two-chamber, three-chamber, four-chamber, and parasternal short axis (SAX) views, in which case it is straightforward to pass only the two-chamber and four-chamber apical views to subsequent processing stages.

[0026] Alternatively, the first neural network may be trained to identify only two-chamber and four-chamber apical views, along with their respective confidence measures. This means, for example, that three-chamber and SAX views may be (mistakenly) classified by the neural network as two-chamber or four-chamber apical views. However, the confidence measures associated with these incorrect classifications should be low. Data with low confidence measures are then ignored, meaning that only two-chamber and four-chamber apical data are passed on to subsequent processing stages.

[0027] Thus, the two-chamber or four-chamber apical data 106 is provided to a second convolutional neural network 112 that is trained to outline the left ventricle for each input view. Other methods for providing data to the second neural network will be apparent to those skilled in the art. For example, all of the image data may be provided to the second neural network, but the neural network is only activated to process the correct image in response to the output from the view detection neural network 104. The first neural network 104 also provides an indication 108 that it has detected a two-chamber view and an indication 110 that it has detected a four-chamber view.

[0028] The second network 112 determines the contour of the endocardial border of the left ventricle in each image. Figure 3 shows an example of such a contour being traced around the endocardial border.

[0029] Once the endocardial borders have been delineated for each of the input images, the two-chamber view 114 and the four-chamber view 116 are provided to first responders 118 and 120, which can perform respective cycle and frame selection operations. In other words, the two-chamber view 114 and the four-chamber view 116 are provided to the respective cycle and frame selection means 118 and 120. The first responder is configured to determine end-systolic and end-diastolic frames within the image data. In one example, the image data includes non-contrast image data. In an alternative example, the image data includes contrast-enhanced image data. Knowing the frames associated with a fully contracted or fully dilated left ventricle can then be used to determine important parameters such as the ejection fraction (EF) and global longitudinal strain (GLS) of the subject's heart. Other parameters, such as any one or more of volume, principal strain, shear strain, regional strain, left ventricular mass, left ventricular area, left ventricular length, rectangularity, and solidity, can also or alternatively be determined. The heart may be bovine, equine, porcine, murine, feline, canine, or primate. Those skilled in the art will understand that the heart may include, but is not limited to, a heart in a livestock animal. Those skilled in the art will further understand that a primate heart may include, but is not limited to, a human heart.

[0030] A first output 122 from the first responder for two-chamber cycle and frame selection (i.e., two-chamber cycle and frame selection means) 118 and a second output 124 from the first responder for four-chamber cycle and frame selection (i.e., four-chamber cycle and frame selection means) 120 are provided to a second responder 126 capable of performing parameter calculation. The second responder 126 calculates one or more parameters from any of volume, ejection fraction, global longitudinal strain, principal strain, shear strain, regional strain, left ventricular mass, left ventricular area, left ventricular length, rectangularity, and solidity. In other words, the output 122 from the two-chamber cycle and frame selection means 118 and the output 124 from the four-chamber cycle and frame selection means 120 are provided to a parameter calculation means 126 that calculates parameters. For example, the ejection fraction is calculated using a volume estimate such as Simpson's biplane method. Other methods of volume calculation may also be applied.

[0031] GLS uses the end-diastolic (baseline length) and end-systolic (shortening length) to calculate global longitudinal strain as follows: Global longitudinal strain (%) = ((shortened length - baseline length) / (baseline length)) × 100

[0032] In addition to GLS, regional strain parameters may be derived. Regional strain is calculated by dividing the left and right sides (anterior and posterior walls) of the 2 / 4-chamber view contour centered at the apex into three segments, creating a total of six contour segments. Regional strain at each time point for each segment is calculated by taking the length of each contour segment, subtracting the length of that contour segment at end-diastole, taking the ratio between this difference and the end-diastolic length, and multiplying by 100 to generate the percentage strain. This is formally expressed as follows:

[0033]

number

[0034] where t is the time index relative to the end-diastolic time frame at t0, and l i is the length of the i-th contour segment.

[0035] Regional strain over time generally produces curves that begin and end at approximately zero values ​​for the end-diastolic time point. These are then time-smoothed to account for error. The output parameters are then the strain curves for each segment, the peak strain for each segment (i.e., the most negative strain value in each curve), and the time between the beginning end-diastolic time point and the peak strain, referred to as the "time to peak." End-systolic strain values ​​can also be calculated from the identified end-systolic and end-diastolic frames.

[0036] The first responder 118, 120 may be a cycle and frame selector, which may also utilize other data, such as an ECG tracing, to identify end-systolic and end-diastolic frames within the contoured image data.

[0037] The two neural networks shown in Figure 1 are trained separately as follows: The first (view detection) neural network is provided with training ultrasound data whose relevant views have been identified by a human operator. In one example, the training ultrasound data includes non-contrast ultrasound data. In an alternative example, the training ultrasound data includes contrast ultrasound data.

[0038] From a sufficiently large multi-subject DICOM dataset, several apical-four-chamber (A4C), apical-three-chamber (A3C), apical-two-chamber (A2C), and parasternal short-axis mitral (PSAX-MV or SAX) views were present in each subject's DICOM study. The first neural network includes a multi-class convolutional neural network (CNN) configured to receive multiple echocardiographic video clips as input. In one example, the echocardiographic video clips include non-contrast echocardiographic video clips. In an alternative example, the echocardiographic video clips include contrast echocardiographic video clips. The first neural network is further configured to determine whether the image was an apical-four-chamber (A4C), apical-three-chamber (A3C), apical-two-chamber (A2C), or parasternal short-axis mitral (PSAX-MV) view. In one example, this included using multiple 2D convolutional layers using grayscale image input. Cross-entropy was used as the loss function. Class weighting factors were applied to the data to match the uneven distribution of training data per represented view. Inference using the classifier model was performed on all frames within the imaging study to assess which view the study was, and the sum of the classifier outputs was used as the class voting strategy. For preliminary validation in one example where the echocardiographic video clip consisted of non-contrast echocardiographic video clips, an unseen "holdout" data subset formed from 10% of the original data was applied to the view classifier to generate the view identification accuracy shown in Table 1 below. The true labels are placed on the vertical axis, and the predicted labels (by the neural network) are placed on the horizontal axis. There was only minor misclassification between 2C and 3C images.

[0039] [Table 1]

[0040] For preliminary validation in one example where the echocardiographic video clips consisted of contrast-enhanced echocardiographic video clips, an unseen "holdout" data subset formed from 10% of the original data was applied to the view classifier to produce the view identification accuracies shown in Table 2 below. As with Table 1, the true labels are placed on the vertical axis and the predicted labels (from the neural network) are placed on the horizontal axis. There was only minor misclassification between 2C and 3C views.

[0041] [Table 2]

[0042] In one example where the echocardiographic video clips were composed of non-contrast echocardiographic video clips, the view classifier was also tested using an independent test dataset, with the results shown in Table 3.

[0043] [Table 3]

[0044] In one example where the echocardiographic video clips consisted of contrast-enhanced echocardiographic video clips, the view classifier was also tested using an independent test dataset, with results as shown in Table 4.

[0045] [Table 4]

[0046] The left ventricle (LV) segmentation algorithm consisted of a U-net convolutional neural network (CNN) automated LV segmentation framework developed using Python 3.5 with Keras (Chollet, Francois et al. 2015, https: / / keras.io) and Google Tensorflow: A system for large-scale machine learning (Abadi, Martin et al., 2016 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pages 265–283, https: / / www.usenix.org / system / files / conference / osdi16 / osdi16-abadi.pdf) backends to segment A2C and A4C images. In one example, non-contrast-enhanced A2C and A4C images were segmented. In another example, contrast-enhanced A2C and A4C images were segmented. The images were used to train a U-net CNN, which was first manually contoured (i.e., LV endocardial border traced) by a British Society of Echocardiography (BSE)-certified echocardiographer. In one example, non-contrast images were used to train the U-net CNN. In an alternative example, contrast-enhanced images were used to train the U-net CNN. The images included frames for A2C and A4C views, respectively. To train the CNN, both datasets were split into an 80% training set and a 20% validation set. The raw images were filtered, normalized, and applied to the modified U-net CNN framework. The CNN generated contours that were able to smoothly track the endocardial wall through time (Figure 3). The effectiveness of the network's segmentation performance was evaluated using the Sorensen-Dice coefficient (DC), calculated from the ratio of intersections between the output contour (Y) and a known ground truth contour (X), as shown below. DC=2|X∩Y| / (|X|+|Y|)

[0047] The LV contour (segmentation) represents the output of this algorithm and the input of the cycle and frame selection stages.

[0048] Multiple LV contours, derived from the preceding LV segmentation algorithm, are received by the cycle and frame selection algorithm. To automatically calculate parameters such as EF and GLS, it is necessary to automatically identify cardiac cycles and end-diastolic and end-systolic frames before calculating physiological measurements. An automated method was established from an image clip, including the heart rate (commonly available in DICOM files as a DICOM tag), the number of frames (commonly available in DICOM files as a DICOM tag), and the evaluation of a series of contours, one per frame, for the view under consideration. This method, summarized in Figure 4, consists of cycle separation, cycle filtering and extraction of end-diastolic (ED) and end-systolic (ES) frames, and systolic cycle filtering. The resulting series of contours may be from a single cardiac cycle or multiple cardiac cycles. The first step is to separate the series of contours into multiple cardiac cycles, if possible. It is possible to calculate the number of complete cardiac cycles using a simple formula. numcycles=(timebetweenframes)×(heartrate)×(numframes)

[0049] where "timebetweenframes" is the time between the individual frames. Once the ES and ED frames are identified, image features are extracted from the two contours generated per view as output of this algorithm for input into the anatomical quantification algorithm.

[0050] In one example, all of the training data was based on non-contrast echocardiograms. In an alternative example, all of the training data was based on contrast echocardiograms.

[0051] FIG. 2 illustrates a second embodiment of the present invention, which differs from the first embodiment primarily by the inclusion of a clip selection algorithm in each path. FIG. 2 illustrates an embodiment 200 in which six processing stages are applied to input views. A plurality of image files 202 are provided to a first neural network 204, which has been trained to perform view detection as described above with reference to FIG. 1. In one example, the image files consist of non-contrast images. In a second example, the image files consist of contrast-enhanced images. The first neural network 204 may also provide a confidence measure indicating the confidence that the category into which the video clip has been placed is correct. In this embodiment, the first neural network is trained using two-chamber apical, three-chamber apical, four-chamber apical, and SAX views.

[0052] The outputs of the first network include an identified two-chamber view 206 and an identified four-chamber view 208, which are provided to respective video clip selection algorithms 210, 212. In one example, the video clip selection algorithm is configured to select the best (possible) ultrasound videos per view to provide to subsequent processing stages. This may be done based on a reliability measure. In an alternative example, the video clip selection algorithm is configured to select an appropriate video from the plurality of ultrasound videos per view to provide to subsequent processing steps. This may be done based on at least one characteristic of the echocardiographic image. The at least one characteristic includes at least one of a reliability measure, gain, granularity, resolution, level of motion in the image, field of view, size of the field of view, size of the image, or number of cavities in a given view. The outputs 214, 216 of the video clip selection algorithm 210 are connected to inputs to a frame selection stage 230 and a left ventricle (LV) segmentation neural network 224, respectively. The outputs 218, 220 of the video clip selection algorithm 212 are connected to inputs to a left ventricle (LV) segmentation neural network 224 and a frame selection stage 232, respectively. The outputs 226, 228 of this neural network are provided as inputs to respective cycle and frame selection algorithms 230, 232. These algorithms provide their outputs 234, 236 as inputs to respective anatomical quantification algorithms 238, 240, which provide outputs 242, 244 to a combination algorithm 246. The combination algorithm provides output parameters 248. The output parameters may be one or more of ventricular volume, ejection fraction, global longitudinal strain, principal strain, shear strain, regional (local) strain, left ventricular mass, left ventricular area, left ventricular length, rectangularity, and solidity.

[0053] Decoupling of the neural network means that the network designer can be confident that subsequent model training and application will take into account variables associated with, in this example, different echocardiographic views, or any other prior confounders that were addressed in previous steps.

[0054] Figure 4 shows the process for identifying ED and ES frames as follows: End-diastolic and end-systolic contours are extracted from the set of contours by considering many possible end-diastolic and end-systolic candidates (top diagram). After this filtering, the candidate with the largest area difference is selected (bottom diagram). See also Figure 5.

[0055] This process is repeated for both the two-chamber and four-chamber apical views.

[0056] Taking the ED and ES frames selected from the cycle and frame selection algorithm as input, physiological parameters such as volume, ejection fraction, or global longitudinal strain may be calculated as described. Additionally or alternatively, any one or more of principal strain, shear strain, regional strain, left ventricular mass, left ventricular area, left ventricular length, rectangularity, and solidity may be calculated.

[0057] Ejection fraction and global longitudinal strain are well-known parameters in echocardiography that are useful for detecting cardiac abnormalities in a subject, such as heart failure, coronary artery disease, amyloidosis, and hypertrophic cardiomyopathy. The above-described system can be used in a method for determining a prognosis indication for a subject with a cardiac abnormality. Additionally or alternatively, the above-described system can be used in a method for diagnosing a cardiac condition. Additionally, the above-described system can be used in a method for treating a cardiac condition. A series of echocardiographic images from a subject is analyzed by the above-described system to derive parameters. In one example, the series of echocardiographic images includes a series of non-contrast-enhanced images. In an alternative example, the series of echocardiographic images includes a series of contrast-enhanced images. The derived cardiac parameters can be one or more of volume, ejection fraction, global longitudinal strain, principal strain, shear strain, regional strain, left ventricular mass, left ventricular area, left ventricular length, rectangularity, and solidity, as known in the art. In one example, once one or more parameters are derived by the system, they are compared to a reference data set. The reference data set may include multiple reference contrast-enhanced echocardiographic images previously analyzed by the system to generate the threshold parameters. In an alternative example, once the one or more parameters are derived by the system, they are compared to at least one predetermined threshold parameter. Optionally, the predetermined threshold parameter is derived from multiple reference non-contrast-enhanced echocardiographic images previously analyzed by the system to generate the threshold parameters. Optionally, in an alternative example, the predetermined threshold parameter is derived from multiple reference contrast-enhanced echocardiographic images previously analyzed by the system to generate the threshold parameters. Additionally or alternatively, in one example, the at least one predetermined threshold parameter is derived from a known, medically defined parameter. In any of the previously described examples, the predetermined threshold parameter may be one or more of volume, ejection fraction, global longitudinal strain, principal strain, shear strain, regional strain, left ventricular mass, left ventricular area, left ventricular length, rectangularity, and solidity.

[0058] The multiple reference echocardiographic images may be from subjects without a cardiac pathology, in this example, the threshold parameter represents a normal value, and deviation from this value indicates a cardiac pathology.

[0059] The multiple reference echocardiograms may be from subjects with known cardiac pathologies, in this example, a threshold parameter represents an outlier, and if the parameter of the subject in whom the system is being used is within a set range of the threshold, this indicates a known cardiac pathology.

[0060] The multiple reference echocardiographic images may be from subjects with known conditions and subjects without known conditions, and since the predetermined threshold parameter may be derived from the multiple reference echocardiographic images, the predetermined threshold parameter may be updated over time to reflect newly diagnosed conditions in the subject.

[0061] The comparison between the parameters derived for the subject and the thresholds derived from the reference data set or from the predetermined threshold parameters can be performed by a physician, who then determines whether a cardiac condition is present in the subject and / or the subject's prognosis based on the comparison. Alternatively, the comparison can be performed by a system suitable for that purpose itself. Such a system is disclosed in WO2017 / 216545 A1. Diagnosis of a cardiac condition, such as heart failure, can be performed by a physician, by a system, or by the combined operation of the two.

[0062] Once a diagnosis of a cardiac condition (eg, heart failure) is obtained, a prescription for improvement, such as a prescription for medication and / or lifestyle changes, may be derived.

[0063] The present invention also includes a method for treating a cardiac condition in a subject, the method comprising obtaining a diagnosis of a cardiac condition in the subject as described above, and then administering to the subject a therapeutically effective amount of a drug to alleviate the symptoms of the cardiac condition. For example, after obtaining a diagnosis of heart failure as described above, the doctor then administers an appropriate treatment for heart failure. This treatment may involve the administration of one or more of angiotensin-converting enzyme (ACE) inhibitors, angiotensin receptor blockers, beta-blockers, mineralocorticoid receptor antagonists, diuretics, and other drugs commonly used to treat heart failure. The exact selection and dosage of the treatment will depend on the subject's clinical history, but will be readily apparent to those skilled in the art.

[0064] As an example, using the system described above, a physician derives the ejection fraction (EF) of the left ventricle of a patient's heart and compares it with a reference data set derived from multiple reference echocardiograms of subjects without cardiac pathology. The ejection fraction is significantly reduced compared to the reference data set, indicating heart failure. Therefore, the physician administers a course of ramipril (an ACE inhibitor) at an initial dose of 1.25 mg / day, which is gradually increased to 10 mg / day under supervision.

[0065] Alternatively, using the system described above, a physician derives the ejection fraction (EF) of the left ventricle of the patient's heart and compares it to a predetermined threshold parameter. Optionally, the predetermined threshold parameter is derived from multiple reference non-contrast echocardiograms of a subject without a cardiac pathology. Optionally, in an alternative example, the predetermined threshold parameter is derived from multiple reference contrast echocardiograms of a subject without a cardiac pathology. Additionally or alternatively, in one example, at least one predetermined threshold parameter is derived from a known, medically defined parameter. The ejection fraction is significantly reduced compared to the corresponding threshold parameter, indicating heart failure. Therefore, the physician administers a course of ramipril (an ACE inhibitor) at an initial dosage of 1.25 mg / day, which is gradually increased to 10 mg / day under monitoring.

[0066] Applying a continuous series of AI algorithms to image processing problems overcomes the problem of poor reproducibility in image quantification that exists in traditional echocardiographic quantification applications, resulting in increased efficiency (fewer processing failures) and improved outcomes (in this case, patient benefits).

[0067] For this illustration, the accuracy and precision of the described embodiment were evaluated against a conventional human operator performing routine image processing methods (i.e., manual view, cycle, and frame selection with manual LV segmentation and physiological parameter calculation). Data were collected consisting of a series of Digital Imaging and Communications in Medicine (DICOM) format videos visualizing the heart from various angles (views). From the collated dataset, the techniques described above were applied to quantify exemplary physiological measures (ejection fraction (EF) and global longitudinal strain (GLS)) using apical-four-chamber (A4C) and apical-two-chamber (A2C) videos. In addition, the same data was processed for left ventricular (LV) quantification by a qualified echocardiographer (operator). Data processed multiple times by several different operators was used to compare repeatability (defined as the difference in measurements provided by operators processing the same imaging study across repetitions). This study demonstrates that autonomous sequential AI imaging dramatically reduces the variability of EF and GLS, two important assessors of LV function (Figure 6). This is achieved by relying on full automation when contouring the endocardial border.

[0068] The term "neural network" is used herein to denote a network that is trained on a set of data. If a portion of a particular network is trained independently of the rest of the network, that portion of the network is considered a "neural network," and the overall particular network will include two (or more) "neural networks." [Explanation of symbols]

[0069] 100 systems 102 Input ultrasound images 104 First Trained Neural Network 106 Apical Data 108 Instructions 110 Instructions 112 Second Convolutional Neural Network 114 Two-chamber view 116 Four-chamber view 118 First Responder 118 Frame Selection Method 120 First Responder 120 Frame Selection Method 122 First Output 124 Second Output 126 Second Responder 126 Parameter Calculation Method 200 Embodiments 202 image files 204 The First Neural Network 206 Two-chamber view 208 Four-chamber view 210 Video Clip Selection Algorithm 212 Video Clip Selection Algorithm 214 Output 216 Output 218 Output 220 output 224 Left Ventricle (LV) Segmentation Neural Network 226 Output 228 output 230 Frame Selection Stage, Frame Selection Algorithm 232 Frame Selection Stage, Frame Selection Algorithm 234 output 236 Output 238 Anatomical Quantification Algorithm 240 Anatomical Quantification Algorithm 242 output 244 output 246 Combinatorial Algorithms 248 Output Parameters

Claims

1. A system (100; 200) for providing parameters of the human heart, comprising: receiving a plurality of echocardiographic images (102; 202); classifying said echocardiographic images (102; 202) into a plurality of different types of views; identifying at least one of a two-chamber view or a four-chamber view (106; 206, 208) in said plurality of echocardiographic images (102; 202); determining a confidence measure for a given view, said confidence measure indicating a confidence that the identified view is correct; a first trained neural network (104; 204) configured to: receiving an image from at least one of the two-chamber view or the four-chamber view; identifying an endocardial border of the left ventricle for the given view; a second trained neural network (112;224) configured to: identifying end-systolic and end-diastolic images based on the identified endocardial border of the left ventricle for the given view; a first responder (118, 120; 230, 232) configured to: deriving at least one first parameter (248) of the heart based on the end-systolic and end-diastolic images, wherein the at least one first parameter (248) is a global longitudinal strain, a principal strain, a shear strain, or a regional strain; and a second responder (126, 238; 240, 246) configured to Systems including (100;200).

2. The first responder: identifying one or more cardiac cycles, wherein the derivation of the at least one first parameter is further based on the identified one or more cardiac cycles. The system of claim 1 , further configured to:

3. 3. The system of claim 1 or claim 2, wherein the second responder (126; 246) is configured to combine the two-chamber and four-chamber endocardial boundaries of the left ventricle to derive volume-based parameters.

4. The second responder: deriving one first parameter of the heart based on the end-systolic and end-diastolic images, wherein the one first parameter is global longitudinal strain, principal strain, shear strain, or regional strain; deriving a second cardiac parameter based on the end-systolic and end-diastolic images, the second cardiac parameter being an ejection fraction; and 4. The system of claim 1, further configured to:

5. The system of claim 1 , wherein the regional strain parameters include six regions.

6. The system of claim 1 , wherein the plurality of echocardiographic images comprises a plurality of non-contrast images.

7. The system of claim 1 , wherein the plurality of echocardiographic images comprises a plurality of contrast-enhanced echocardiographic images.

8. 7. The method of training a system according to claim 1, wherein the neural network is trained with non-contrast data.

9. 8. The method of training a system according to claim 1, wherein the neural network is trained with imaging data.

Images