Evaluating cardiac parameters using neural networks

Abstract

Embodiments of the present invention provide a system for automatically deriving parameters of a human heart from ultrasound results. The first neural network is arranged to receive a plurality of echocardiography images and to classify the images as one of at least a two-lumen tangent plane and a four-lumen tangent plane. The second neural network is arranged to receive echocardiography images comprising two or four-lumen tangent planes and to identify endocardial boundaries of a left ventricle (LV) for each tangent plane. End systolic and end diastolic images are then identified and parameters such as LV volume, ejection fraction, overall longitudinal strain, and regional longitudinal strain are calculated.

Description

[0001] This application is a divisional application filed on March 5, 2021, with application number 202180019306.X, entitled "Evaluation of cardiac parameters using neural networks". Technical Field

[0002] This invention relates to a neural network for determining parameters of the human heart from multiple echocardiographic images. Background Technology

[0003] Echocardiography is a powerful tool for studying cardiac conditions, but traditionally it has relied heavily on the interpretation of the resulting images by a professional radiologist. This leads to delays and costs.

[0004] In echocardiography, ultrasound data of a specific patient's heart is typically collected across multiple distinct "sections"—data gathered from different points around the patient's body via a probe, with each section representing a different plane traversing the patient's body. These sections may include the apical four-chamber view, the apical three-chamber view, the apical two-chamber view, and the parasternal long-axis and short-axis views. Sometimes all sections will appear in the patient's data, and sometimes only a subset is available. Multiple ultrasound videos may also be available for a single section of a particular patient. Although skilled interpreters will recognize that other formats are available, ultrasound results are typically communicated via DICOM (Digital Imaging and Communications in Medicine) files.

[0005] Some work has been done on interpreting individual ultrasound sections using trained neural networks, such as automatically outlining the contours of the heart wall. However, such neural networks only operate on specific sections (e.g., the apical four-chamber view), meaning that any particular network can only accept ultrasound data related to one type of input data (i.e., one “section”).

[0006] Unfortunately, ultrasound files (such as DICOM files) typically do not include any information about which section is present in any particular dataset. Operators may need up to 20 minutes or more to sort through all ultrasound data from a particular patient to ensure that data associated with the correct section is fed into the correct neural network and derives clinically applicable cardiac measurements.

[0007] The purpose of this invention is to improve upon this shortcoming. Summary of the Invention

[0008] According to a first aspect of the present invention, a system for providing parameters of a human heart is provided, the system comprising: A first trained neural network having inputs and outputs, the first neural network being arranged to receive multiple echocardiographic images and classify the echocardiographic images into one of at least two different sections including at least a two-chamber section and a four-chamber section, and A second trained neural network having inputs and outputs, the second neural network being arranged to receive images from at least one of the two-chamber or four-chamber sections and identify the endocardial boundary of the left ventricle in each section. The first responder responds to the output of the second neural network to identify end-systolic and end-diastolic images, and The second responder responds to the end-systolic and end-diastolic images to derive parameters of the heart.

[0009] By automatically detecting relevant sections in the source data and segmenting the left ventricular endocardial boundary, a more accurate and effective system is provided. With automatically provided parameters, physicians or subsequent automated systems can diagnose patients more easily and effectively.

[0010] Most parameters require delineating the endocardial boundary (segmentation) of the left ventricle, combining at least two-chamber and four-chamber sections to form the volume, such as Simpson's two-plane method to determine ventricular volume, and also require calculating parameters (such as ejection fraction).

[0011] From the left ventricle segmentation, appropriate parameters are derived, including ejection fraction (EF) and global longitudinal strain (GLS). Additionally or optionally, local (regional) strain values ​​can be derived. A section detection network can provide a confidence metric and a determination of which section is present. This metric can be used to eliminate any redundant sections (i.e., the 3-chamber apical and SAX sections) and / or to select a set of sections from multiple sets of the same type.

[0012] The first responder can be a device that responds to the output of the second neural network to identify end-systolic and end-diastolic images. The second responder can be a device that responds to the output of the second neural network to identify end-systolic and end-diastolic images.

[0013] According to a second aspect of the present invention, a method for diagnosing cardiac pathology is provided, the method comprising: Receive multiple echocardiographic images from the subject. The first aspect of the system was used to analyze the multiple echocardiographic images to derive parameters of the heart. The parameters of the heart are compared with at least one predetermined threshold parameter, and The difference between the parameters of the heart and the at least one threshold parameter is detected, and the difference indicates the pathology of the heart.

[0014] According to a third aspect of the present invention, a method for treating cardiac pathology is provided, the method comprising: Multiple echocardiographic images are received from the subject, and the system of the first aspect is used to analyze the multiple echocardiographic images to derive parameters of the heart. The parameters of the heart are compared with at least one predetermined threshold parameter. The difference between the parameters of the heart and at least one threshold parameter is detected, the difference indicating cardiac pathology, and The subject is given a therapeutically effective dose of a drug that relieves one or more symptoms of the cardiac pathology.

[0015] The method and system of the present invention can be implemented on a conventional echocardiography device or as a stand-alone system for receiving echocardiographic data. Attached Figure Description

[0016] The invention will now be described by way of example with reference to the following figures: Figure 1 This is a schematic block diagram showing a first arrangement of a neural network according to an embodiment of the present invention. Figure 2 This is a schematic block diagram of a second embodiment of the present invention, which includes six sequentially connected processing stages. Figure 3 The left ventricular profile using an embodiment of the present invention is shown. Figure 4 The process for selecting end-diastolic and end-systolic frames, as identified by an embodiment of the present invention, is illustrated. Figure 5 The image shows end-diastolic and end-systolic frames identified by embodiments of the present invention, and... Figure 6 The embodiments of the present invention are shown in comparison with conventional manual processing of cardiac image data. Detailed Implementation

[0017] Figure 1 System 100 is shown, in which an input ultrasound image 102 is provided to a first trained neural network 104. Devices for acquiring ultrasound images, such as those shown in WO2017 / 216545 and well-known commercial systems, are available from companies including Philips, Siemens, and General Electric. In this example, the input to the implementation comprises multiple non-contrast echocardiographic video clips, consisting of multiple instances of acquisition sections from a DICOM imaging study of a given subject. In an alternative example, the input to the implementation comprises multiple contrast echocardiographic video clips, consisting of multiple instances of acquisition sections from a DICOM imaging study of a given subject. Of course, other imaging formats can be used as needed.

[0018] Those skilled in the art of echocardiography are aware of two main categories of imaging modalities. These are referred to as contrast-enhanced and non-contrast-enhanced imaging. Contrast-enhanced echocardiography describes a range of imaging methods, all of which rely on the introduction of an acoustically active contrast agent that persists in the vascular space for a period of time. In some embodiments of the present invention, contrast-enhanced echocardiographic images are echocardiographic images obtained during or after the introduction of one or more acoustically active contrast agents. Contrast-enhanced images can be obtained by introducing a contrast agent (e.g., microbubbles) into the patient. Contrast agents generally improve the detection of the central ventricular boundary in echocardiographic images compared to non-contrast-enhanced echocardiographic images (which are echocardiographic images captured in subjects who have never used contrast agents). Non-contrast-enhanced imaging involves collecting echocardiographic images in the absence of any contrast agent.

[0019] In some embodiments, any aspect of the invention utilizes non-contrast echocardiographic images. In some other embodiments, any aspect of the invention utilizes contrast echocardiographic images. In some embodiments, any aspect of the invention uses both contrast and non-contrast echocardiographic images. The systems and methods described herein, and their embodiments, are readily applicable to both imaging modalities.

[0020] The first neural network 104 is arranged to determine which aspect of the input video clip is associated with, so that it can be provided to the subsequent stages of the system for correct processing.

[0021] Preferably, the first neural network also provides a confidence metric for determining the relevant cross section, i.e., an indication of the probability that the cross section is actually of the determined type.

[0022] In this example, only 2-chamber and 4-chamber apical sections are required. Therefore, any sections related to other types of sections (such as 3-chamber apical sections or SAX sections) will be ignored. This can be done in two different ways.

[0023] The first neural network can be trained to recognize any number of possible sections, such as 2-chamber apical sections, 3-chamber apical sections, 4-chamber apical sections, and parasternal short axis (SAX) sections. The 2-chamber apical sections and 4-chamber apical sections are then directly passed to subsequent processing stages.

[0024] Optionally, the first neural network can be trained to recognize only 2-chamber and 4-chamber apical sections and the confidence measure for each section. For example, this means that 3-chamber and SAX sections might (misclassify) as 2-chamber or 4-chamber apical sections by the neural network. However, the confidence measures associated with these misclassifications should be low. Data with low confidence measures are then ignored, meaning that only 2-chamber and 4-chamber apical data are passed to subsequent processing stages.

[0025] Therefore, two-chamber or four-chamber apical data 106 is provided to a second convolutional neural network 112, which is trained to delineate the contour of the left ventricle for each input section. Other methods for providing data to the second neural network will be apparent to a skilled image reader. For example, all image data could be provided to the second neural network, but this network would only be activated in response to the output from the section detection neural network 104 to process the correct image. The first neural network 104 also provides an indication 108 that it has detected a two-chamber section and an indication 110 that it has detected a four-chamber section.

[0026] The second network 112 determines the contour of the endocardial boundary of the left ventricle in each image. Figure 3 An example of such a contour traced around the endocardial boundary is shown.

[0027] Once the endocardial boundary contours of each input image have been delineated, the two-chamber section 114 and the four-chamber section 116 are provided to first responders 118 and 120, which can perform corresponding period and frame selection operations. In other words, the two-chamber section 114 and the four-chamber section 116 are provided to their respective period and frame selection devices 118 and 120. The first responders are arranged to determine end-systolic and end-diastolic frames in the image data. In one instance, the image data includes non-contrast image data. In an alternative instance, the image data includes contrast image data. Understanding which frames are associated with a fully contracted or fully dilated left ventricle can then be used to determine key parameters such as the ejection fraction (EF) and global longitudinal strain (GLS) of the subject's heart. Other parameters can also be determined, or optionally, such as volume, principal strain, shear strain, regional strain, left ventricular mass, left ventricular area, left ventricular length, rectangularity, and solidity, among others. The heart can be the heart of a cow, horse, pig, rat, cat, dog, or primate. Those skilled in the art understand that the heart can include, but is not limited to, the heart of a domestic animal. Those skilled in the art further understand that the heart of a primate can include, but is not limited to, the human heart.

[0028] A first output 122 for a first responder used for 2-chamber cycle and frame selection (i.e., 2-chamber cycle and frame selection device) 118 and a second output 124 for a first responder used for 4-chamber cycle and frame selection (i.e., 4-chamber cycle and frame selection device) 120 are provided to a second responder 126, which can perform parameter calculations. The second responder 126 calculates one or more parameters based on any one of volume, ejection fraction, overall longitudinal strain, principal strain, shear strain, regional strain, left ventricular mass, left ventricular area, left ventricular length, rectangularity, and solidity. In other words, the output 122 from the 2-chamber cycle and frame selection device 118 and the output 124 from the 4-chamber cycle and frame selection device 120 are provided to the parameter calculation device 126 that calculates the parameters. For example, volume estimation (e.g., the Simpson two-plane method) is used to calculate the ejection fraction. Other volume calculation methods can also be applied.

[0029] For GLS, the end of diastole (base length) and the end of contraction (shortening length) are used to calculate the overall longitudinal strain, as shown below: Overall longitudinal strain (%) = ((shortening length - bottom length) / (bottom length)) × 100 In addition to GLS, regional strain parameters can also be derived. Regional strain is calculated by dividing the left and right sides (anterior and posterior walls) of the 2 / 4 chamber profile about the apex into three parts—a total of six segments are created for the profile. The regional strain of each segment at each time point is calculated as follows: take the length of each profile segment, subtract the length of the end-diastolic profile segment, take the ratio of this difference to the end-diastolic length, and then multiply by 100 to produce the strain percentage. This is formally expressed as:

[0030] in Is The time index relative to the end-diastolic time frame, and yes The length of the outline segment.

[0031] Regional strain over time typically produces curves that begin and end near the zero value at the end-diastolic time point. These are then time-smoothed to account for errors. The output parameters are then the strain curve for each segment, the peak strain for each segment (i.e., the maximum negative strain value in each curve), and the time between the onset of the end-diastolic time point and the peak strain—called the "time to peak." End-systolic strain values ​​can also be calculated from the identified end-systolic and end-diastolic frames.

[0032] First responders 118 and 120 can be period and frame selection devices. They can also utilize other data (such as ECG traces) to identify end-systolic and end-diastolic frames in the contour image data.

[0033] Figure 1 The two neural networks shown are trained as follows. The first (section detection) neural network is provided with training ultrasound data, where the relevant sections have been identified by the operator. In this example, the training ultrasound data includes non-contrast ultrasound data. In an optional example, the training ultrasound data includes contrast ultrasound data.

[0034] From a sufficiently large DICOM dataset of multiple subjects, each subject's DICOM study contained several apical four-chamber (A4C) sections, apical three-chamber (A3C) sections, apical two-chamber (A2C) sections, and parasternal short-axis mitral valve (PSAX-MV or SAX) sections. The first neural network comprised a multi-class convolutional neural network (CNN) arranged to receive multiple echocardiographic video clips as input. In this instance, the echocardiographic video clips included non-angiography echocardiographic video clips. In an optional instance, the echocardiographic video clips included angiography echocardiographic video clips. The first neural network was further arranged to determine whether an image was an apical four-chamber (A4C) section, an apical three-chamber (A3C) section, an apical two-chamber (A2C) section, or a parasternal short-axis mitral valve (PSAX-MV) section. In this instance, this involved multiple 2D convolutional layers using grayscale image input. Cross-entropy was used as the loss function. A class weighting factor was applied to the data to match the non-uniform distribution of the training data for each represented cross-section. A classifier model was used to infer which cross-section the study belonged to, and the sum of the classifier outputs was used as a classification voting strategy. For initial validation in an instance of echocardiographic video clips composed of non-contrast echocardiographic video clips, an invisible "reserved" subset of data, consisting of 10% of the original data, was applied to the cross-section classifier to produce cross-section recognition accuracy, as shown in Table 1 below. The true labels are located on the vertical axis, and the predicted labels (via the neural network) are located on the horizontal axis. There were only some minor misclassifications between the 2C and 3C cross-sections.

[0035]

[0036] Table 1 To conduct initial validation in an instance of echocardiographic video clips composed of contrast echocardiographic video clips, an invisible "reserved" subset of data, consisting of 10% of the original data, was applied to the section classifier to produce section recognition accuracy, as shown in Table 2 below. As shown in Table 1, the true labels are located on the vertical axis, and the predicted labels (via the neural network) are located on the horizontal axis. There are only some minor misclassifications between the 2C and 3C sections.

[0037]

[0038] Table 2 In the instance where the echocardiogram video clips consisted of non-contrast echocardiogram video clips, the section classifier was also tested using a separate test dataset, and the results are shown in Table 3.

[0039]

[0040] Table 3 In the example where the echocardiogram video clips consist of contrast echocardiogram video clips, the section classifier was also tested using a separate test dataset, and the results are shown in Table 4.

[0041]

[0042] Table 4 The left ventricle (LV) segmentation algorithm comprises an automatic LV segmentation framework using the U-net convolutional neural network (CNN) developed in Python 3.5, Keras (Chollet, Francois et. Al. 2015, https: / / keras.io), and a Google Tensorflow: Systems for Large-Scale Machine Learning backend (Abadi, Martin et al., 2016 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 16), pages 265-283, https: / / www.usenix.org / system / files / conference / osdi16 / osdi16-abadi.pdf) to segment A2C and A4C sections. In the example, non-angiography A2C and A4C sections are segmented. In an optional example, angiography A2C and A4C sections are segmented. Images were used to train the U-net CNN contours (i.e., tracking the endocardial boundary of the left ventricle) originally drawn manually by echocardiologists accredited by the British Society of Echocardiography (BSE). In the examples, U-netCNN was trained using non-contrast images. In optional examples, U-net CNN was trained using contrast images. These images included frames from A2C and A4C sections, respectively. Both datasets were split into 80% training sets and 20% validation sets for training the CNN. The original images were filtered, normalized, and applied to the modified U-net CNN framework. The contours generated by the CNN were able to smoothly track the endocardial wall over time. Figure 3 The effectiveness of the network segmentation performance was evaluated using the Sørensen-Dice coefficient (DC), which is calculated based on the ratio of the intersection points of the output contour (Y) with the known ground truth contour (X), as shown below: DC = 2|X∩Y| / (|X|+|Y|) LV contour (segmentation) represents the output of the algorithm as well as the inputs during the period and frame selection stages.

[0043] Multiple LV contours are received into a cycle and frame selection algorithm derived from the preceding LV segmentation algorithm. To automatically calculate parameters (e.g., EF and GLS), it is necessary to automatically identify the cardiac cycle and end-diastolic and end-systolic frames before calculating physiological measurements. An automated method is established from image clippings, including assessing heart rate (typically provided as a DICOM tag in the DICOM file), the number of frames (also typically provided as a DICOM tag in the DICOM file), and a series of contours, one per frame, for the section under consideration. This method... Figure 4 The process is summarized here, consisting of cycle separation, cycle filtering, extraction of end-diastolic (ED) and end-systolic (ES) frames, and filtering of systolic cycles. The resulting series of contours can come from a single cardiac cycle or from multiple cardiac cycles. The first step is to divide the series of contours into multiple cardiac cycles, where possible. The total number of cardiac cycles can be calculated using a simple formula: Number of cycles = (Time between frames) × (Heart rate) × (Number of frames) Here, "time between frames" refers to the time between each frame. Once the ES and ED frames are identified, image features are extracted from the two contours generated for each section and used as the output of the algorithm, which is then fed into the anatomical quantification algorithm.

[0044] In the example, all training data are based on non-contrast echocardiography. In the optional example, all training data are based on contrast echocardiography.

[0045] Figure 2 The second embodiment of the present invention is shown, the main difference from the first embodiment being that a clip selection algorithm is included in each path. Figure 2 Implementation scheme 200 is shown, in which six processing stages are applied to the input cross-section. Multiple image files 202 are fed to a first neural network 204, which is trained to perform cross-section detection, as previously referenced. Figure 1 The discussion continues. In one instance, the image file consists of non-contrast images. In a second instance, the image file consists of contrast images. The first neural network 204 can also provide a confidence metric indicating the confidence that the video clip has been correctly placed into the category. In this embodiment, the first neural network has been trained using apical 2-chamber, apical 3-chamber, apical 4-chamber, and SAX sections.

[0046] The output of the first network includes identified two-chamber sections 206 and identified four-chamber sections 208, which are provided to corresponding video clip selection algorithms 210 and 212. In one example, the video clip selection algorithm is configured to select the best of (possibly) multiple ultrasound videos for each section to provide to a subsequent processing stage. This can be based on a confidence metric measurement. In an alternative example, the video clip selection algorithm is configured to select the appropriate video of multiple ultrasound videos for each section to provide to a subsequent processing step. This can be done based on at least one feature of the echocardiographic image. The at least one feature includes at least one of confidence metric, gain, granularity, resolution, level of motion in the image, field of view, field of view size, image size, or number of chambers in a given section. The outputs 214 and 216 of the video clip selection algorithm 210 are respectively connected to the frame selection stage 230 and the input of the left ventricular (LV) segmentation neural network 224. The outputs 218 and 220 of the video clip selection algorithm 212 are respectively connected to the inputs of the left ventricular (LV) segmentation neural network 224 and the frame selection stage 232. The outputs 226 and 228 of this neural network are provided as inputs to the corresponding periodic and frame selection algorithms 230 and 232. These algorithms provide their outputs 234 and 236 as inputs to the corresponding anatomical quantification algorithms 238 and 240, which in turn provide their outputs 242 and 244 to the combination algorithm 246. The combination algorithm provides output parameter 248. This output parameter can be one or more of the following: ventricular volume, ejection fraction, global longitudinal strain, principal strain, shear strain, regional (local) strain, left ventricular mass, left ventricular area, left ventricular length, rectangularity, and solidity.

[0047] The separation of neural networks means that the network designer can be confident that subsequent model training and application have taken into account the variables associated with different echocardiographic sections, in this case, or any other of the aforementioned confounding factors that have been addressed in the preceding steps.

[0048] Figure 4 The process for identifying ED and ES frames is shown below. End-diastolic and end-systolic contours are extracted from a set of contours by considering many possible end-diastolic and end-systolic candidates (top row). After this filtering, the candidate with the largest area difference is selected (bottom row of the graph). See also Figure 5 .

[0049] This process is repeated for both the apical 2-chamber and apical 4-chamber views.

[0050] Using the ED and ES frames selected from the period and frame selection algorithm as input, physiological parameters such as volume, ejection fraction, or overall longitudinal strain can be calculated as described. Additionally or optionally, any one or more of the principal strain, shear strain, regional strain, left ventricular mass, left ventricular area, left ventricular length, rectangularity, and solidity can be calculated.

[0051] Ejection fraction and global longitudinal strain are well-known parameters in echocardiography and can be used to detect abnormalities in the heart of a subject, such as heart failure, coronary artery disease, amyloidosis, and hypertrophic cardiomyopathy. The system described above can be used as a method to determine prognostic indicators for subjects with cardiac abnormalities. Alternatively or alternatively, the system described above can be used as a method to diagnose cardiac pathology. Additionally, the system described above can be used as a method to treat cardiac pathology. A series of echocardiographic images from a subject will be analyzed by the system described above to derive parameters. In an example, the series of echocardiographic images includes a series of non-contrast echocardiographic images. In an alternative example, the series of echocardiographic images includes a series of contrast images. The derived cardiac parameters can be one or more of the following: volume, ejection fraction, global longitudinal strain, principal strain, shear strain, regional strain, left ventricular mass, left ventricular area, left ventricular length, rectangularity, solidity, etc., which are known in the art. In an example, once one or more parameters are derived by the system, they are compared with a reference dataset. This reference dataset may include multiple reference contrast echocardiographic images that have previously been analyzed by the system to generate threshold parameters. In optional instances, once the system derives one or more parameters, they are compared to at least one predetermined threshold parameter. Optionally, the predetermined threshold parameter is derived from multiple reference non-contrast echocardiographic images previously analyzed by the system to generate the threshold parameter. Optionally, in optional instances, the predetermined threshold parameter is derived from multiple reference contrast echocardiographic images previously analyzed by the system to generate the threshold parameter. Additionally or optionally, in instances, at least one predetermined threshold parameter is derived from a known medically defined parameter. In any of the foregoing instances, the predetermined threshold parameter may be one or more of volume, ejection fraction, global longitudinal strain, principal strain, shear strain, regional strain, left ventricular mass, left ventricular area, left ventricular length, rectangularity, and solidity.

[0052] Multiple reference echocardiographic images can be obtained from subjects without cardiac pathology. In this case, the threshold parameter will represent the normal value, and deviations from this value will indicate cardiac pathology.

[0053] Multiple reference echocardiographic images can be obtained from subjects with known cardiac pathology. In this case, the threshold parameter will represent outliers, and if the parameters of the subject using the system are within the set range of the threshold, this will indicate known cardiac pathology.

[0054] Multiple reference echocardiographic images can be obtained from subjects with known pathologies and subjects without known pathologies. Since predetermined threshold parameters can be derived from multiple reference echocardiographic images, these parameters can be updated over time to reflect newly diagnosed pathologies in subjects.

[0055] The comparison between parameters derived for a subject and thresholds derived from a reference dataset or from predetermined threshold parameters can be performed by a physician, who will then determine the presence of cardiac pathology in the subject and / or the subject's prognosis based on the comparison. Optionally, the comparison itself can be performed by a system suitable for this purpose. Such a system is disclosed in WO 2017 / 216545 A1. The diagnosis of cardiac pathology (e.g., heart failure) can be performed by a physician, a system, or a combination of both.

[0056] Once a diagnosis of cardiac pathology (such as heart failure) has been made, a prescription for remedies can be made, such as a prescription for medications and / or lifestyle changes.

[0057] The present invention also includes a method for treating cardiac pathology in a subject, the method comprising obtaining a diagnosis of cardiac pathology in the subject as described above, and then administering a therapeutically effective amount of a drug to the subject to alleviate the symptoms of cardiac pathology. For example, after obtaining a diagnosis of heart failure as described above, a physician will subsequently administer appropriate treatment for the heart failure. Such treatment may involve administering one or more of angiotensin-converting enzyme (ACE) inhibitors, angiotensin receptor blockers, beta-blockers, mineralocorticoid receptor antagonists, diuretics, and other commonly used drugs for treating heart failure. The exact choice and dosage of treatment depend on the subject's clinical history, but will be apparent to those skilled in the art.

[0058] For example, using the system described above, the physician determined the ejection fraction (EF) of the patient's left ventricle and compared it to a reference dataset derived from multiple reference echocardiographic images of subjects without cardiac pathology. The significantly reduced ejection fraction compared to the reference dataset indicated heart failure. Therefore, the physician prescribed a course of ramipril (an ACE inhibitor), starting at 1.25 mg daily and gradually increasing to 10 mg daily under supervision.

[0059] As an optional example, using the system described above, the physician derives the ejection fraction (EF) of the patient's left ventricle and compares it to a predetermined threshold parameter. Optionally, the predetermined threshold parameter is derived from multiple reference non-contrast echocardiographic images of a subject without cardiac pathology. Optionally, in an optional example, the predetermined threshold parameter is derived from multiple reference contrast echocardiographic images of a subject without cardiac pathology. Additionally or optionally, in the example, at least one predetermined threshold parameter is derived from a known medically defined parameter. A significantly reduced ejection fraction compared to the corresponding threshold parameter indicates heart failure. Therefore, the physician administers a course of ramipril (an ACE inhibitor) at an initial dose of 1.25 mg daily, gradually increased under supervision to 10 mg daily.

[0060] By applying a series of sequential AI algorithms to image processing problems, the poor reproducibility of images in traditional quantitative echocardiography applications is overcome. This leads to improved efficiency (reduced processing failures) and better outcomes (in this case, patient benefit).

[0061] To illustrate this, the accuracy and precision of the proposed implementation were evaluated against conventional operators performing routine image processing methods, i.e., manual sectionalization, period and frame selection using manual LV segmentation, and physiological parameter calculation. The collected data comprised a series of videos in Digital Imaging and Medical Communications (DICOM) format, visualizing the heart from different angles (sectionalizations). From the consolidated dataset, the aforementioned methods were applied to quantify example physiological measurements (ejection fraction (EF) and global longitudinal strain (GLS)) using apical four-chamber (A4C) and apical two-chamber (A2C) videos. Furthermore, the same data was processed by a qualified echocardiologist (operator) for left ventricular (LV) quantification. Using data processed multiple times by several different operators, we compared reproducibility (defined as the difference in measurements provided by operators through repeated processing of the same images). From this study, we demonstrate that autonomous sequential AI image processing significantly reduces the variability of two key evaluators of LV function: EF and GLS. Figure 6 This is achieved through full automation in delineating the endocardial boundary.

[0062] The term "neural network" is used in this article to refer to a network trained on a set of data. If a part of a particular network is trained independently of the rest of the network, that part of the network is considered a "neural network," and the entire particular network will consist of two (or more) "neural networks."

[0063] This application also relates to implementation schemes for the following items.

[0064] 1. A system (100; 200) for providing parameters of a human heart, said system comprising: A first trained neural network (104; 204) having inputs and outputs, the first neural network being arranged to receive multiple echocardiographic images and classify the echocardiographic images into one of at least two different sections including at least a two-chamber section and a four-chamber section; and A second trained neural network (112; 224) having inputs and outputs, the second neural network being arranged to receive images from at least one of the two-chamber section or the four-chamber section and identify the endocardial boundary of the left ventricle in each section; First responders (118, 120; 230, 232) respond to the output of the second neural network to identify end-systolic and end-diastolic images; and The second responder (126; 238, 240, 246) responds to the end-systolic and end-diastolic images to derive parameters of the heart.

[0065] 2. The system of claim 1, further comprising a third responder (210, 212) for identifying suitable echocardiographic images based on at least one feature of the echocardiographic images to be provided to a second neural network.

[0066] 3. The system as described in item 1 or item 2, wherein the second responder (126; 246) is arranged to combine the two-chamber and four-chamber endocardial boundaries of the left ventricle to derive volume-based parameters.

[0067] 4. The system of any one of items 1 to 3, wherein the parameter is ejection fraction.

[0068] 5. The system of any one of items 1 to 3, wherein the parameter is the overall longitudinal strain.

[0069] 6. The system as described in item 1 or item 2, wherein the parameter is regional strain.

[0070] 7. The system as described in item 6, wherein the regional strain parameter comprises six regions.

[0071] 8. The system of any one of claims 1 to 7, wherein the plurality of echocardiographic images comprises a plurality of non-contrast images.

[0072] 9. The system of any one of the preceding claims, wherein the plurality of echocardiographic images comprises a plurality of contrast echocardiographic images.

[0073] 10. A method for training a system as described in any one of items 1 to 8, wherein the neural network is trained based on non-contrast data.

[0074] 11. A method for training a system as described in any one of items 1 to 7 or 9, wherein the neural network is trained based on imaging data.

[0075] 12. A method for diagnosing cardiac pathology, the method comprising: Receive multiple echocardiographic images from the subject; The system according to any one of items 1 to 9 is used to analyze the plurality of echocardiographic images to obtain parameters of the heart; The parameters of the heart are compared with at least one predetermined threshold parameter; and The difference between the parameters of the heart and the at least one predetermined threshold parameter is detected, and the difference indicates the cardiac pathology.

[0076] 13. A method for treating cardiac pathology, the method comprising: Receive multiple echocardiographic images from a subject, and analyze the multiple echocardiographic images using the system according to any one of items 1 to 9 to derive parameters of the heart; The parameter is compared with at least one threshold parameter; The difference between parameters of the heart and at least one predetermined threshold parameter is detected, the difference indicating cardiac pathology. The subject is given a therapeutically effective amount of the drug, which alleviates one or more symptoms of the cardiac pathology.

Claims

1. A system for providing parameters of a human heart, the system comprising: a first trained neural network arranged to: receive a plurality of echocardiogram images; and identify at least one two-chamber or four-chamber cut-plane in the plurality of echocardiogram images; a third responder arranged to: select, based on at least one feature of the echocardiogram images, a subset of images for a given cut-plane, wherein the at least one feature is at least one of gain, granularity, resolution, level of motion in the images, field of view, field of view size, image size, or number of chambers in the given cut-plane; generate a first output based on the subset of images; a second trained neural network arranged to: receive the first output; outline the first output by identifying endocardial boundaries of a left ventricle of the given cut-plane; and generate a second output based on the outlined first output; a first responder arranged to: receive the second output; identify, based on the second output, end-systole and end-diastole images; and generate a third output based on the end-systole and end-diastole images; a second responder (126; 238, 240, 246) arranged to: receive the third output; and derive at least one first parameter of the heart based on the third output, wherein the at least one first parameter is global longitudinal strain, principal strain, shear strain, or regional strain.

2. The system of claim 1, wherein the second responder is arranged to combine two-chamber and four-chamber endocardial boundaries of the left ventricle to derive a volume-based parameter.

3. The system of claim 1, wherein the regional strain parameter includes six regions.

4. The system of claim 1, wherein the plurality of echocardiogram images includes a plurality of non-contrast images.

5. The system of claim 1, wherein the plurality of echocardiogram images includes a plurality of contrast echocardiogram images.

6. A method of training the system of claim 1, wherein the neural networks are trained based on non-contrast data.

7. A method of training the system of claim 1, wherein the neural networks are trained based on contrast data.

8. A method of diagnosing a cardiac pathology, the method comprising: receiving a plurality of echocardiogram images from a subject; analyzing the plurality of echocardiogram images using the system of claim 1 to derive parameters of the heart; comparing the parameters of the heart to at least one predetermined threshold parameter; and detecting a difference between the parameters of the heart and the at least one predetermined threshold parameter, the difference indicating the cardiac pathology.

9. A method of treating a cardiac pathology, the method comprising: receiving a plurality of echocardiogram images from a subject, analyzing the plurality of echocardiogram images using the system of claim 1 to derive parameters of the heart; comparing the parameters to at least one threshold parameter; detecting a difference between the parameters of the heart and the at least one predetermined threshold parameter, the difference indicating the cardiac pathology, and administering to the subject a therapeutically effective amount of a drug that reduces one or more symptoms of the cardiac pathology.

10. The system of claim 1, wherein the second responder is further arranged to: derive a first parameter of the heart based on end-systole and end-diastole images, wherein the first parameter is global longitudinal strain, principal strain, shear strain, or regional strain; and derive a second parameter of the heart based on end-systole and end-diastole images, wherein the second parameter of the heart is ejection fraction.

11. The system of claim 1, wherein the first responder is further arranged to: identify one or more cardiac cycles, wherein the third output is further based on the identified one or more cardiac cycles.

Images