Machine learning echocardiogram classification

Abstract

A method for analyzing a structure within a patient's heart includes receiving a patient study including multiple transthoracic and / or transesophageal echocardiographic images of the patient's heart obtained by an ultrasound device. The method includes extracting sets of pixel data from the images and analyzing the sets of pixel data using multiple trained machine learning algorithms to: a) identify images captured using a color Doppler modality and assign one or more of the images to a structure; b) identify images captured using a spectral Doppler modality and assign one or more of the images to a structure; c) determine a radius R of an inflow convergence zone from analysis of the color Doppler modality images assigned to the structure; and d) determine a maximum gradient PVreg of the structure from analysis of the spectral Doppler modality images assigned to the structure. The radius R of the inflow convergence zone, the maximum gradient PVreg of the structure, and the Nyquist value Va determined for the color Doppler modality are combined to determine an effective regurgitant valve area EROA for the structure.

Description

[Technical Field]

[0001] The present invention relates to echocardiogram classification using machine learning. [Background technology]

[0002] Valvular heart disease is a type of cardiovascular disease and includes any cardiovascular disease process involving one or more of the four valves of the heart (aortic, mitral, tricuspid, and pulmonary valves). Valvular heart disease is often associated with aging, but can also be the result of congenital abnormalities or certain diseases or physiological processes, including rheumatic heart disease and pregnancy.

[0003] Echocardiography is an important diagnostic aid in the morphological and functional evaluation of valvular heart disease. Typically, during an echocardiogram (echocardiogram) of a patient, a clinician or sonographer places an ultrasound scanning device in the patient's chest area to capture multiple 2D images of the patient's heart. The device beams sound waves into the chest so that reflected sound waves reveal details of the heart's internal structure and the velocity of blood flow within it. The position of the ultrasound probe on the chest and its angle relative to the chest are varied during the echocardiogram to capture different anatomical cross-sections as 2D slices of the heart from different perspectives.

[0004] Clinicians or technicians typically have the option of supplementing 2D images (hereafter referred to as brightness-mode or "B-mode" images) with data captured from one or more other modalities. These modalities include continuous-wave Doppler, pulsed-wave Doppler, and M-mode. Continuous-wave Doppler and pulsed-wave Doppler (hereafter referred to as "spectral Doppler") visualize blood flow velocity over a time interval and perform similar functions in the evaluation of valvular heart disease. Information obtained from B-mode images can be further enhanced by visualizing blood flow direction using Doppler imaging (hereafter referred to as "color Doppler"). A specific setting of pulsed-wave Doppler, called "tissue wave echocardiography," can capture myocardial (heart muscle) velocities.

[0005] Typically, captured B-mode images and other modality data are stored in DICOM TM The image is exported in a file format. [DICOM (Digital Imaging and Communications in Medicine) is an international standard for transmitting, storing, retrieving, printing, processing, and displaying medical imaging information.]

[0006] Although the type of modality (used to capture the B-mode image) is partially indicated in the metadata associated with the DICOM file, the position of the ultrasound machine defining the imaged cardiac structures remains undetermined within the DICOM file immediately after image capture.

[0007] After the patient examination, the clinician / technologist displays and reviews the images generated from the DICOM file, for example, by displaying 2D images side-by-side with color Doppler images, manually labeling and annotating cardiac structural or functional artifacts, such as the left ventricular endocardial border (LV) or the S-peak in tissue Doppler images, and taking measurements. This process relies on the operator's training and experience to recognize the cardiac view within each image and their ability to identify landmark features and take appropriate measurements. During subsequent review of the images, the cardiologist views and views the now labeled and annotated DICOM file and previously recorded measurements, compares them with guideline values, and makes a diagnosis based on their own interpretation. Of particular interest are metrics used in the quantitative analysis of valve dysfunction: the so-called effective regurgitant orifice area (EROA) and regurgitant volume.

[0008] This essentially manual workflow process for analyzing DICOM files requires a high degree of expertise, typically requires input from highly experienced cardiologists, is time-consuming, error-prone, and inevitably results in large variations in interpretation between different operators. While partial automation may help improve the process to some extent, the most efficient solution would be a fully automated one.

[0009] Proposed solutions for automated cardiac image interpretation to enable low-cost assessment of cardiac function by non-experts are discussed, for example, in Z. Akkus et al., "A Survey of Deep-Learning Applications in Ultrasound: Artificial Intelligence-Powered Ultrasound for Improving Clinical Workflow," Journal of the American College of Radiology 16(9):1318-1328, September 2019, DOI:10.1016 / j.jacr.2019.06.004. Akkus discusses recent technological advances in echocardiography, including color Doppler, convolutional neural networks (CNNs), and deep learning (DL). Akkus states, "Current deep learning models for ultrasound diagnosis only use two-dimensional cross-sectional images to make predictions. However, two-dimensional cross-sectional information is limited and does not fully represent the lesion. DL models trained on three-dimensional ultrasound data, ultrasound cine clips containing multiple views of the lesion, or spatiotemporal data may improve the diagnostic accuracy of the model and take the complete lesion into account. Furthermore, developing DL models trained on multimodal (B-mode, Doppler, contrast-enhanced ultrasound, SWE) images that provide mutually complementary information may also improve the diagnostic accuracy of DL models."

[0010] RJG van Sloun et al., "Deep learning in Ultrasound Imaging," 2020, Computer Science, Engineering Proceedings of the IEEE, describe a color Doppler IR neural network. Figure 4 (a) shows tissue Doppler processing using a deep encoder-decoder network for an exemplary cardiac ultrasound application displaying the wall between the right atrium and the aorta. Figure 4(b) shows a deep network architecture designed to encode input IQ data into a compressed latent space via a series of convolutional layers and spatial (max) pooling operations, while maintaining the functionality and performance of a typical Doppler processor (Kasai autocorrelator) using fully uncompressed IQ data. Figure 4(c) shows the convergence of network parameters during training, plotting the relative root mean square error (RMSE) on a test dataset for four data compression ratios.

[0011] J. Wang et al., "Auto-weighting for Breast Cancer Classification in Multimodal Ultrasound," published August 8, 2020, describes color Doppler as an ultrasound imaging modality and a neural network used to automate weighting between several different ultrasound modalities (B-mode, Doppler, SE, SWE). However, this method does not address classification of different cardiac views or object segmentation within classified frames, nor does it address segmentation of Doppler modality images.

[0012] WO 2020121014 A describes an automated workflow performed by software running on at least one processor, which includes receiving multiple echocardiographic images taken by an ultrasound device. A filter separates the multiple echocardiographic images into 2D images and Doppler modality images based on analysis of image metadata. The 2D images and Doppler modality images are classified by view type. Heart chambers are segmented in the 2D images, and the Doppler modality images are segmented to generate waveform traces, generating segmented 2D images and segmented Doppler modality images. Both sets of images are used to obtain measurements of cardiac features on both the left and right sides of the heart. The measurements are compared to international cardiac guidelines to generate a conclusion, and a report is output indicating calculated measurements that fall within or outside the guidelines.

[0013] The following publications provide further useful background information: https: / / atm.amegroups.com / article / view / 85237 / html F. Yang et al., "Self-supervised learning assisted diagnosis for mitral regurgitation severity classification based on color Doppler echology," January 2022, describes a diagnostic system that assists physicians in assessing the severity of MR based on color video Doppler echocardiography via a self-supervised learning algorithm. The proposed method segments the mitral regurgitation jet.

[0014] JW. Son et al., "Automated Quantification of Mitral Regurgitation by Three-Dimensional Real-Time Full-Volume Color Doppler Transthoracic Echocardiography: A Validation with Cardiac Magnetic Resonance Imaging and Comparison with Two-Dimensional Quantitative Methods," June 2013, describes a method for automatically assessing the severity of mitral regurgitation. However, this method can only be performed with three-dimensional full-volume color Doppler echocardiography, not with routine two-dimensional echocardiography.

[0015] https: / / www.siemens-healthineers.com / ultrasound / news-and-innovations / advancing-3d-tee Siemens ACUSON SC2000 TM Ultrasound system PRIME TM The platform features automated analysis of mitral regurgitation using three-dimensional volumetric color Doppler echocardiography.

[0016] The current American Society of Echocardiography (ASE) guideline, "Recommendations for Noninvasive Evaluation of Native Valve Regurgitation: A Report from the American Society of Echocardiography Developed in Collaboration with the Society for Cardiovascular Magnetic Resonance," recommends using inflow convergence to measure EROA and regurgitant volume. Summary of the Invention

[0017] According to a first aspect of the present invention, there is provided a computer-implemented method for analyzing structures within a patient's heart, performed by software components executing on at least one processor, comprising receiving from a memory a patient study including a plurality of transthoracic and / or transesophageal echocardiographic images of the patient's heart acquired by an ultrasound device, and extracting a set of pixel data from each of the images.

[0018] One or more trained machine learning algorithms are used to analyze the set of pixel data, including the steps of: a) identifying images captured using a color Doppler modality and assigning one or more of these images to a structure; b) identifying images captured using a spectral Doppler modality and assigning one or more of these images to a structure; c) determining a radius R of an inflow convergence zone from analysis of the color Doppler modality images assigned to the structure; and d) determining a maximum gradient PVreg of the structure from analysis of the spectral Doppler modality images assigned to the structure.

[0019] The radius of the inflow convergence zone R, the maximum gradient PVreg of the structure, and the Nyquist value Va determined for the color Doppler modality are combined to determine the effective regurgitant orifice area EROA for the structure.

[0020] The EROA of a structure may be determined according to the following formula: (2*π*R**2 x Va) / PVreg

[0021] The method may include determining a velocity time integral of regurgitant flow, VTIreg, from analysis of the spectral Doppler modality images assigned to the structure using the one or more trained neural networks, and the method may further include combining the EROA and the VTIreg to determine a regurgitant flow volume for the structure.

[0022] The following formula may be used to determine the reflux volume: EROA*VTIreg

[0023] The method may include using one or more trained machine learning algorithms to locate the Nyquist value within a color Doppler modality image assigned to the structure, and applying optical character recognition to the location to determine the Nyquist value.

[0024] The images may be cines, each cine comprising multiple frames.

[0025] For one or both of steps a) and b), the assignment of the one or more images to the structure may include using a trained classification neural network to individually analyze multiple frames of the image for the presence of the structure, and combining the outputs of the analyzed frames to assign the image to a structure.

[0026] For one or both of steps c) and d), a segmentation and landmark detection neural network may be used to identify a single frame of the image that is associated with the structure and has the largest inflow convergence zone radius R and / or the largest gradient PV of the structure.

[0027] The image may be a still image.

[0028] Each image may be received from the memory as a DICOM format file.

[0029] The structure analyzed may be one of the aortic valve, the mitral valve, the pulmonary valve, the tricuspid valve, the ventricular septum, and the atrial septum.

[0030] According to a second aspect of the present invention there is provided a method of diagnosing cardiovascular disease comprising carrying out the method of the first aspect above to determine the effective regurgitant orifice area, EROA, of a cardiac structure. [Brief explanation of the drawings]

[0031] [Figure 1] 1 is a flow chart illustrating a computer-implemented method for analyzing intracardiac structures of a patient performed by software components executing on at least one processor.

[0032] [Figure 2] 2 is a flow chart illustrating the selection of color Doppler and spectral Doppler cine within the method of FIG. 1;

[0033] [Figure 3] 2 is a flow chart illustrating the processing of color Doppler cine and extraction of inflow convergence zone radius and Nyquist value within the method of FIG. 1;

[0034] [Figure 4] 2 is a flow chart illustrating processing of spectral Doppler cine and extraction of maximum gradient of structures within the method of FIG. 1; DETAILED DESCRIPTION OF THE INVENTION

[0035] During a transthoracic echocardiogram (TTE) examination, the patient is instructed to remove all clothing covering the upper body and lie flat on a bed. The sonographer or clinician performing the TTE applies lubricating gel to the ultrasound probe, which is attached to a nearby ultrasound machine or wirelessly connected to a handheld device (e.g., a cell phone or tablet). The ultrasound machine or handheld device displays and records the images generated. By moving the ultrasound probe over the patient's chest, different parts of the patient's heart can be observed and focused on. The imaging mode can be changed to visualize blood flow or different cardiac features. B-mode is typically used to quantify the size of heart chambers and other cardiac structures, while spectral Doppler and color Doppler imaging modes are used to visualize blood flow. The video feed on the ultrasound machine or handheld device can be paused to allow measurement of specific features. The clinician or sonographer follows a protocol and obtains a list of cardiac parameters. Finally, the resulting images are saved in DICOM format and can be exported, for example, to a hospital image archive and communication system.

[0036] A computer-implemented method for analyzing images obtained using echocardiography and the process for determining the effective regurgitant orifice area (EROA) and regurgitant volume of the patient from whom the images were obtained will now be described. These images may be still images or video clips, hereinafter referred to as "cine images" or simply "cines." Still or cine images may be referred to as "echocardiograms." While the images may be obtained using any suitable echocardiography system, exemplary systems include the GE Healthcare Vivid S70, Philips Medical Systems EPIQ CVx, and Philips Medical Systems Lumify TM Handheld ultrasound system, Siemens ACUSON TM Sequoia, Esaote MyLabEight TMFor purposes of the following exemplary discussion, it is assumed that the images are cines, each containing a set of consecutive, chronologically ordered frames. A typical cine set may contain 40-120 frames stored in DICOM format.

[0037] The method may be implemented on one computer or set of computers with associated processors, memories, display means, etc. Alternatively, the method may be implemented on a server, such as a cloud-based server.

[0038] A semi-automated or fully automated network-based procedure is described for computing measurements to characterize the type and severity of alterations in one or more structures of the human (or animal) heart, and taking as input multiple cine sets (separately) stored in DICOM format. The procedure uses several stages, as follows, and is further illustrated in the flow chart of Figure 1.

[0039] 1. Classification of Capture Modalities A plurality of echocardiogram cines in DICOM format are received as initial input, each cine containing a chronologically ordered sequence of frames. This first classification stage aims to determine, for each cine, the capture modality of the cine.

[0040] For each cine, the following steps are performed:

[0041] (1a) Image pixel data is extracted from the cine. Typically, the pixel data comprises multiple arrays of two or more dimensions containing individual pixel intensity values ​​ranging from 0 to 255. Importantly, metadata from the received DICOM files is not used in this classification stage, providing an improvement over known procedures since the use of such metadata could introduce errors due to incorrectly encoded or recorded metadata.

[0042] (1b) Pixel data for the first frame of the cine is selected.

[0043] (1c) The selected pixel data is provided to a first classification neural network. This is a MobileNet-based neural network trained using cross-entropy loss for multi-class classification. [MobileNet is a class of convolutional neural networks (CNNs) open-sourced by GOOGLE®.] The neural network takes the input pixel data, resizes it to the input resolution, converts it into a tensor, and passes it to the neural network layers. [Resizing refers to scaling an image from one resolution to another using methods such as nearest neighbor interpolation (e.g., converting an image with height and width of 700x500 pixels to 224x244 pixels).] The model's output is a tensor containing the probability of the evaluated frame for each of the following categories: B-mode, M-mode, color Doppler, spectral Doppler, and other.

[0044] Steps (1a) through (1c) are repeated for each frame of the cine being evaluated to generate an output tensor for each frame. Based on an analysis (e.g., majority vote) of the probability tensors for all frames of the evaluated cine, an overall classification is given to the cine: B-mode, M-mode, color Doppler, spectral Doppler, and "other."

[0045] By repeating this for each of the input cines, the capture modality is determined for each cine without having to parse the metadata in each DICOM format file. [Note: In the following steps, only color Doppler and spectral Doppler cines are further analyzed, but cines of other modalities may also be further analyzed.]

[0046] 2. Classification of structures within the colored dorsal vein The cines identified by the first classification stage as being cines of the color Doppler modality are selected. For each selected identified cines, pixel data for up to five evenly distributed frames are extracted and the following steps are performed. Of course, the number five is shown here for illustrative purposes only, and more or fewer frames may be used.

[0047] (2a) The extracted pixel data is passed frame by frame (for the five selected frames) to a color Doppler classification neural network. This network is a MobileNet-based neural network trained using cross-entropy loss for multi-class classification. The neural network resizes the input (frame) data to the input resolution, converts it into tensors, and passes the tensors to the neural network layers. The output of the network is a tensor with the probability of one of the following categories for each frame of the cine being evaluated: aortic valve, mitral valve, pulmonary valve, tricuspid valve, ventricular septum, atrial septum, and "other." In other words, the network provides the probability of the presence of each of these structures in a particular frame of the cine.

[0048] (2b) The cine to be evaluated is associated with a structure (e.g., majority vote) based on the most likely structure indicated by a set of tensors in the selected frames of the cine.

[0049] By repeating this procedure for each cine (previously identified as captured with the color Doppler modality), all cines are associated with a structure (aortic valve, mitral valve, etc.). The cines are then grouped according to the structure they are associated with. Each group can contain 0, 1, or more cines.

[0050] 3. Color Doppler Landmark Detection and Segmentation Next, each group of cines (excluding groups with 0 cines and groups associated with "Other") is analyzed separately. For each cine in the group being analyzed, the following steps are performed:

[0051] (3a) Pixel data is extracted from the cine and associated with each individual frame of the cine.

[0052] (3b) The pixel data associated with each frame of the cine is then passed to a neural network for color Doppler instance segmentation and landmark detection. This is a Transformer-based neural network trained using the following multiple objectives: a) Localization of individual jet streams using bounding boxes; b) detection of pixels belonging to the localized jet stream; c) Localization of individual inflow convergence zones using bounding boxes; d) detection of pixels belonging to the localized inflow convergence zone; e) Detection of the location of the contraction in the localized inflow convergence zone and the radius of the inflow convergence zone.

[0053] The neural network resizes the input data to the input resolution, converts it to a tensor, and passes the tensor to the neural network layer. The neural network then generates an output tensor for the frame under consideration according to the training objectives described above. This indicates, for example, the presence of zero or more jet streams, zero or more inflow convergence zones, the location of vena contracta (venous constrictions), and the inflow convergence radius within the frame. To implement the neural network, a maximum number of jet streams, inflow convergence zones, vena contracta, and inflow convergence radii are defined for each. In this example, each limit is set to 3. The output tensor is resized to the same resolution as the original frame.

[0054] The neural network can detect up to three jet streams. If no jet streams are detected, the values ​​for each of the three possible jet streams are set to 0. If one or more jet streams are detected, the values ​​of the output tensor are further processed for each detected jet stream. Each location in the output tensor corresponds to a pixel in the original frame and is assigned a value. If the location value is 0.5 or greater, the pixel is assigned to the boundary of the jet stream. The number of locations with a value of 0.5 or greater (possibly comprising an individual jet stream) is counted. The resulting count is then multiplied by the physical delta x and physical delta y values ​​(for the two frame directions) to obtain the total area of ​​each jet stream. So, for example, if delta x is 0.2 mm, delta y is 0.4, and the number of pixels is 50, the jet stream area corresponds to 50 * 0.2 * 0.4 = 4 mm^2.

[0055] Similarly, if no inflow convergence zones are detected, the area value of each convergence zone is set to 0. If one or more convergence zones are detected, for each detected convergence zone, the number of locations with a value of 0.5 or greater are summed and multiplied by the values ​​of physical delta x and physical delta y to obtain the physical dimension of the convergence zone.

[0056] The contraction length is extracted from the output tensor and converted to physical units.

[0057] Repeating processes (3a) and (3b) for each frame of the evaluated cine produces a tensor containing the described data for each cine. The corresponding data is extracted from the frame tensor and assembled into a list. For example, the list [0, 0, 45, 64, 22] identifies, for a particular cine, the size, in pixels, of the inflow convergence zones detected in five frames of the cine. This indicates that no convergence zones were detected in the first or second frames, but a convergence zone was detected starting with the third frame. In this example, the largest convergence is in frame number 4. Therefore, that frame is marked as a frame of interest.

[0058] If multiple cines are associated with a particular structure group, the process of extracting the inflow convergence zone area for each frame is repeated for each cine in the group, and the final frame of interest is selected as the one with the largest inflow convergence zone area across all lists. For example, if a given structure group contains two cines, the following lists for inflow convergence zones are generated: [0, 0, 45, 64, 22] and [0, 10, 11, 68, 44]. In this case, the final frame for further analysis would be the fourth frame of the second cine, with a corresponding region of 68.

[0059] NOTE: If the jet area and vena contracta size cannot be determined in the selected frame, i.e., if either area or size is determined to be 0, the frame is discarded and the frame of interest is selected as the frame with the next highest value of inflow convergence zone area (in the example above, this would be the fourth frame of the first cine). This is repeated as necessary until the optimal frame for further analysis is identified. The radius of the inflow convergence zone in the optimal frame is converted to a physical dimension.

[0060] 4. Determining the Nyquist Limit in the Optimal Frame An important parameter when analyzing color Doppler cine is the Nyquist limit, which defines the maximum flow rate visible in the cine. This is typically presented within the pixel data of the frame and is conventionally read visually by the operator. Here, we propose to implement a neural network-based procedure to determine the Nyquist value from the identified optimal frame.

[0061] To achieve this, the optimal frame is passed to a Nyquist-threshold object detection neural network, which detects the region of the frame containing the Nyquist threshold. This is a transformer-based neural network trained using a localization loss. The neural network resizes the frame to the input resolution, converts it to a tensor, and passes the tensor through the neural network layers. The neural network then detects the location of the Nyquist threshold within the frame and generates an output tensor containing that location. The output tensor is resized to the same resolution as the original frame. The Nyquist threshold is then extracted from its location within the original frame using optical character recognition.

[0062] Overall, the following values ​​are generated for each structure in the optimum frame, if possible: a) Nyquist limit, b) length of one or more vena contracta sections, c) radius of one or more inlet convergence zones, d) area of ​​one or more jets, and e) area of ​​one or more inlet convergence zones.

[0063] 5.Spectral Doppler Classification Stage The cines identified by the first classification stage as cines of the spectral Doppler modality are selected, and for each selected cines, the last frame is extracted.

[0064] (5a) The pixel data from the last frame is passed to a spectral Doppler classification neural network to determine the observed structures. This is a MobileNet-based neural network trained using cross-entropy loss for multi-class classification. The neural network resizes the pixel data to the input resolution, converts it to a tensor, and passes the tensor through the neural network layers. The model's output is a tensor with the probability of each of the following categories: aortic valve, mitral valve, pulmonary valve, tricuspid valve, ventricular septum and atrial septum, and "other."

[0065] (5b) The cine being evaluated is associated with a structure based on the most likely structure indicated by the generated tensor (for the final frame of the cine).

[0066] By repeating this procedure for each cine (previously identified as captured with the spectral Doppler modality), each cine is further associated with a structure (aortic valve, mitral valve, etc.). The cines are then grouped according to the structure with which they are associated. Each group may contain 0, 1, or more cines.

[0067] 5. Spectral Doppler Segmentation Stage Next, each group of cines (excluding groups with 0 cines and groups associated with "Other") is analyzed separately. For each cine in the group being analyzed, the following steps are performed:

[0068] (6a) Pixel data is extracted from the cine and associated with each individual frame of the cine. The data for the last frame of the cine is selected.

[0069] (6b) The pixel data of the selected last frame is passed to a spectral Doppler segmentation neural network. This is a transformer-based neural network trained using Dice loss for semantic segmentation. In computer vision, semantic segmentation involves assigning a specific category (jet stream, inflow convergence zone, background, etc.) to each pixel in the input image. Dice loss is calculated using the following formula:

number

[0070] (6c) The output tensor is resized to the same resolution as the original frame.

[0071] (6d) Individual blood velocity waveforms are extracted from the output. The peaks and areas of the blood velocity waveforms, corresponding to the peak velocity and velocity time integral, respectively, are extracted for further analysis.

[0072] (6e) The peak velocity (PVreg - maximum reflux gradient of the structure) and velocity time integral (VTReg - velocity time integral of reflux) are converted to physical units.

[0073] For a given cine group, the maximum values ​​of PVreg and VTIreg are selected and identified as the group values. Repeating this procedure for each cine group results in PVreg and VTIreg values ​​for each evaluated structure, i.e., aortic valve, mitral valve, etc.

[0074] 7. Integration Stage This provided data for each structure evaluated for both color and spectral Doppler modalities. a) Nyquist limit, b) one or more vena contracta lengths, c) one or more inflow convergence zone radii, d) one or more inflow convergence zone areas for color Doppler modalities, and PVreg and VTIreg for spectral Doppler modalities.The effective regurgitant orifice area (EROA) and regurgitant volume are then calculated for each structure, i.e., aortic valve, mitral valve, etc., as follows:

[0075] (7a) The effective regurgitant orifice area (EROA) is then calculated using the following formula: (2*π*R**2 x Va) / PVreg, where: i. R is the radius of the inflow convergence zone, ii. Va is the Nyquist limit, iii. PVreg is the maximum slope of the structure.

[0076] (7b) Regurgitant volume is calculated using the following formula: EROA*VTIreg, where: i. VTIreg is the velocity time integral of the regurgitant flow.

[0077] (7c) EROA and regurgitant volume are metrics used in the quantitative analysis of valve dysfunction. The following publications provide examples of using this data to provide diagnostic and treatment recommendations:

[0078] https: / / academic.oup.com / view-large / 41542917 Mitral regurgitation (MR): Mild MR is defined as EROA (mm2) < 20 and regurgitant volume (mL) < 30. Mild to moderate MR is defined as an EROA (mm2) of 20-29 and a regurgitant volume (mL) of 30-44. Moderate to severe MR is defined as an EROA (mm2) of 30-39 and a regurgitant volume (mL) of 45-59. ●Severe MR is defined as EROA (mm2) ≥ 40 and regurgitant volume (mL) ≥ 60. Treatment of severe mitral regurgitation requires surgical intervention.

[0079] https: / / academic.oup.com / view-large / 41542736 Aortic Regurgitation (AR): Mild AR is defined as EROA (mm2) < 10 and regurgitant volume (mL) < 30. Mild to moderate AR is characterized by an EROA (mm2) of 10-19 and a regurgitant volume (mL) of 30-44. Moderate to severe AR is defined as an EROA (mm2) of 20-29 and a regurgitant volume (mL) of 45-59. Severe AR is defined as EROA (mm2) ≥ 30 and regurgitant volume (mL) ≥ 60.

[0080] https: / / academic.oup.com / view-large / 41542967 Tricuspid regurgitation (TR): Severe TR is defined as EROA (mm2) ≥ 40 and regurgitant volume (mL) ≥ 45. ●Cutoff values ​​for mild and moderate severity have not been defined.

[0081] https: / / academic.oup.com / view-large / 41542819 The cutoff value for pulmonary regurgitation (PR) has not been defined. (7d) Flow vena contracta is a metric used for semiquantitative analysis of valve dysfunction.

[0082] https: / / academic.oup.com / view-large / 41542917 Mitral regurgitation (MR): ●Mild MR is when the contraction area (mm) is <3. ●Moderate MR is a contraction (mm) of 4 to 6. • Severe MR is defined as a contraction (mm) ≥ 7.

[0083] https: / / academic.oup.com / view-large / 41542736 Aortic Regurgitation (AR): ●Mild AR is when the contraction area (mm) is <3. ●Moderate AR is a contraction area (mm) of 4 to 5. ●Severe AR is defined as a vena contracta (mm) ≥ 6.

[0084] https: / / academic.oup.com / view-large / 41542967 Tricuspid regurgitation (TR): ●The cutoff value for mild TR has not been defined. ●Moderate TR is when the contraction area (mm) is <7. ●Severe TR is defined as a vena contracta (mm) ≥ 7.

[0085] https: / / academic.oup.com / view-large / 41542819 The cutoff value for pulmonary regurgitation (PR) has not been defined. The following table provides more information related to the various neural networks mentioned above. [Table 1]

Claims

1. 1. A computer-implemented method for analyzing intracardiac structures of a patient, the method being performed by software components executing on at least one processor, the method comprising: receiving from memory a patient study including a plurality of transthoracic and / or transesophageal echocardiographic images of the patient's heart obtained by an ultrasound device; extracting respective sets of pixel data from said images; analyzing the set of pixel data using one or more trained machine learning algorithms; a) identifying images captured using a color Doppler modality and assigning one or more of these images to said structure; b) identifying images captured using the spectral Doppler modality and assigning one or more of these images to said structure; c) determining a radius R of the inflow convergence zone from an analysis of the color Doppler modality image assigned to said structure; d) determining the maximum gradient PVreg of the structure from the analysis of the spectral Doppler modality image assigned to said structure; and determining an effective regurgitant orifice area, EROA, for the structure by combining the radius, R, of the inflow convergence zone, the maximum gradient, PVreg, of the structure, and the Nyquist value, Va, determined for the color Doppler modality; A method comprising:

2. 2. The method of claim 1, comprising determining EROA for the structure according to the formula (2*π*R**2 x Va) / PVreg.

3. determining a velocity time integral of regurgitant flow (VTIreg) from analysis of the spectral Doppler modality images assigned to the structure using the one or more trained neural networks; 3. The method of claim 1, further comprising combining the EROA and the VTIreg to determine a regurgitant volume for the structure.

4. 4. The method of claim 3, comprising determining the regurgitant volume using the formula EROA*VTIreg.

5. 5. The method of claim 1, further comprising using one or more trained machine learning algorithms to locate the Nyquist value in a color Doppler modality image assigned to the structure, and applying optical character recognition to the location to determine the Nyquist value.

6. The method of any one of claims 1 to 5, wherein the images are cines, each cine comprising multiple frames.

7. 7. The method of claim 6, wherein for one or both of steps a) and b), the assignment of the one or more images to the structure comprises using a trained classification neural network to individually analyze multiple frames of the image for the presence of the structure, and combining the outputs of the analyzed frames to assign the image to the structure.

8. 8. The method of claim 6 or 7, wherein for one or both of steps c) and d), a segmentation and landmark detection neural network is used to identify a single frame of the image that is associated with the structure and has the largest inflow convergence zone radius R and / or the largest slope PVreg of the structure.

9. The method of any one of claims 1 to 5, wherein the image is a still image.

10. The method of any one of claims 1 to 9, wherein each image is received from the memory as a DICOM format file.

11. The method of any one of claims 1 to 10, wherein the structure is one of the aortic valve, the mitral valve, the pulmonary valve, the tricuspid valve, the ventricular septum, and the atrial septum.

12. 12. A method for diagnosing cardiovascular disease, comprising the step of carrying out a method according to any one of claims 1 to 11 to determine the effective regurgitant orifice area, EROA, for a cardiac structure.

Images