Method and system for predicting coronary artery disease based on echocardiography
The CAD prediction system uses end-to-end machine learning on ultrasound images to accurately predict CAD risk and intervention success, addressing the limitations of invasive imaging methods and enhancing clinical workflow.
Patent Information
- Application Number
- CN202380034225.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2022-04-15
- Filing Date
- 2023-04-06
- Publication Date
- 2024-12-03
AI Technical Summary
Current methods for detecting coronary artery disease (CAD) rely heavily on invasive and costly procedures like X-ray and CT angiography, which expose patients to radiation and are user-dependent, while echocardiography is challenging for direct coronary artery assessment.
A CAD prediction system using end-to-end machine learning processes 2D and 3D ultrasound images with patient metadata to generate feature maps, predicting CAD risk, regional wall motion abnormalities, and intervention success through a trained AI algorithm.
Provides accurate, non-invasive, and cost-effective CAD prediction, reducing the need for risky imaging procedures and improving clinical workflow efficiency by providing consistent and repeatable assessments.
Smart Images

Figure CN119072758A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure generally relates to methods and systems for predicting or detecting coronary artery disease using echocardiography. Background Art
[0002] Coronary artery disease (CAD) is one of the leading causes of death and one of the main types of CAD. Blockages in the main coronary arteries often lead to regional wall motion abnormalities (RWMA) in certain parts of the heart.
[0003] Early detection of coronary artery disease is critical for both preventive and treatment measures. However, the gold standard for evaluating coronary artery disease is invasive X-ray angiography, which can expose patients to unnecessary risk, especially when disease is not actually present. Instead, CT angiography is increasingly used as the gold standard. However, the procedure also exposes the patient to X-ray radiation, machine access may be limited, and the procedure can be expensive.
[0004] In contrast, ultrasound offers a noninvasive, inexpensive, and easily accessible opportunity for CAD assessment. However, direct imaging of the coronary arteries using echocardiography can be challenging. Therefore, assessment of regional wall motion abnormalities (RWMA) is often used in echocardiographic analysis as a surrogate for direct coronary artery disease assessment, as this condition is associated with blockages in the main coronary arteries.
[0005] However, the evaluation of RWMA is challenging because it is usually performed using clinician assessment results, which can be very user-dependent, relying on the analyst's skill and experience level.
[0006] US2005 / 020903 A1 discloses a system and method for automatically evaluating regional myocardial function and automatically diagnosing heart diseases.
[0007] US2021 / 085397 A1 discloses methods and systems for non-invasive assessment and treatment planning of coronary artery disease. Summary of the invention
[0008] Therefore, there is a need for methods and systems for more accurately predicting and / or detecting coronary artery disease. Various embodiments and implementations herein are directed to coronary artery disease (CAD) prediction methods and systems, which are configured to provide a patient with the possibility of CAD. A CAD prediction system receives patient metadata about a patient and receives a time series of 2D and / or 3D ultrasound images obtained based on an ultrasound analysis of the patient's heart. The system selects a plurality of ultrasound images from the time series and uses an artificial intelligence (AI) method to process the images to generate a feature map of the selected plurality of ultrasound images. The system uses a trained AI algorithm to analyze the generated feature map and the received patient metadata to generate a CAD prediction output, which includes one or more of the following: (i) the risk or probability that the patient currently has CAD or will develop CAD in the future; (ii) the probability of a patient's regional wall motion abnormality (RWMA); (iii) the patient's predicted X-ray and / or CT angiography score; (iv) a prediction of the success of a post-procedural CAD intervention for the patient; and / or (v) a prediction of the patient's survival with and / or without intervention. The system then provides the generated CAD prediction output to the user using a user interface.
[0009] In general, in one aspect, a method for using a coronary artery disease (CAD) prediction system to provide an analysis of CAD for a patient is provided. The method includes: (1) receiving patient metadata about the patient; (2) receiving a time series of 2D and / or 3D ultrasound images obtained based on ultrasound analysis of the patient's heart; (3) selecting a plurality of ultrasound images from the time series by the CAD prediction system; (4) processing the selected plurality of ultrasound images using a trained algorithm of the CAD prediction system to generate a feature map of the selected plurality of ultrasound images; (5) analyzing the generated feature map and the received patient metadata using the trained algorithm of the CAD prediction system to generate a CAD prediction output, the CAD prediction output including one or more of the following: (i) the risk or probability of CAD for the patient; (ii) the probability of regional wall motion abnormality (RWMA) for the patient; (iii) a predicted X-ray and / or CT angiography score for the patient; (iv) a prediction of the success of a post-procedural CAD intervention for the patient; and / or (v) a prediction of the patient's survival with and / or without intervention; and (6) providing the generated CAD prediction output via a user interface of the CAD prediction system.
[0010] In other words, the method uses end-to-end machine learning techniques to generate CAD prediction outputs based on multiple ultrasound images. Using end-to-end machine learning allows for the extraction of features that a human might not consider extracting (e.g., features that are not obviously relevant to CAD prediction). This improves the performance of the generated CAD prediction output by achieving an increased accuracy of the generated CAD prediction output.
[0011] The input to the trained algorithm for the CAD prediction system is the selected plurality of ultrasound images. This allows information from different views of the heart to be considered when generating the CAD prediction output, thereby further improving the accuracy of the generated CAD prediction output.
[0012] In some examples, the generated CAD prediction output includes a predicted X-ray and / or CT angiography score. Such an output allows monitoring the progression of CAD in a clinically meaningful manner. The predicted X-ray and / or CT angiography score also enables comparison with existing X-ray and / or CT data.
[0013] According to an embodiment, the ultrasound analysis of the patient's heart is a transthoracic examination (TTE) or a transesophageal examination (TEE), and further wherein the ultrasound analysis of the patient's heart is a stress test, or a resting examination, ie a non-stress examination.
[0014] In some examples, the ultrasound analysis is a transesophageal examination (TEE). Although TEE is more invasive than a transthoracic examination (TEE), since the patient must be intubated with an imaging probe, TEE ultrasound images are collected from a position more suitable for evaluating valve function than other types of cardiac ultrasound imaging, so the use of TEE is beneficial for evaluating CAD. In particular, when using TTE, many anatomical features and motion features that were originally visible using TEE will not be visible or less visible. For example, compared with TTE, TEE provides better visualization of the left atrium, mitral valve, pulmonary veins, and atrial septum. In addition, compared with TTE images, TEE allows more precise viewing of various parts of the heart.
[0015] In some examples, the time series of 2D and / or 3D ultrasound images includes contrast-enhanced images.Contrast-enhanced ultrasound images improve visibility of endocardial structures, thereby enabling more accurate CAD prediction output generation.
[0016] According to an embodiment, the provided CAD prediction output also includes one or more ultrasound images of the plurality of ultrasound images.
[0017] According to an embodiment, one or more of the provided plurality of ultrasound images comprises a saliency map.
[0018] According to an embodiment, the provided CAD prediction output also includes a confidence score.
[0019] According to an embodiment, the trained algorithm of the CAD prediction system processes the selected multiple ultrasound images to generate feature maps of the selected multiple ultrasound images in a spatial direction.
[0020] According to an embodiment, the trained algorithm of the CAD prediction system processes the selected multiple ultrasound images to generate feature maps of the selected multiple ultrasound images in a time dimension.
[0021] For example, the trained algorithm may be a 4D convolutional neural network (CNN). By using a spatiotemporal network, temporal aspects such as heart movement can be taken into account when generating CAD prediction outputs, thereby improving the accuracy of the generated CAD prediction outputs. Heart movement is effectively used as a proxy for vessel properties (e.g., vessel radius). This recognizes that the characteristics of heart movement can be more easily determined from cardiac ultrasound images than requiring high spatial resolution to determine vessel properties.
[0022] According to an embodiment, the method further comprises the step of administering CAD treatment to the patient based on the provided CAD prediction output.
[0023] According to another aspect, a system for providing an analysis of coronary artery disease (CAD) in a patient is provided. The system includes: patient metadata about the patient; a time series of 2D and / or 3D ultrasound images obtained based on ultrasound analysis of the patient's heart; a trained algorithm configured to analyze multiple ultrasound images to generate feature maps of the multiple ultrasound images; a trained algorithm configured to generate a CAD prediction output; a processor configured to: (i) select multiple ultrasound images from the time series; (ii) use the trained algorithm of the CAD prediction system to process the selected multiple ultrasound images to generate feature maps of the selected multiple ultrasound images; (iii) use the trained algorithm of the CAD prediction system to analyze the generated feature map and the received patient metadata to generate a CAD prediction output, the CAD prediction output including one or more of the following: the risk or probability of CAD for the patient; the probability of regional wall motion abnormality (RWMA) for the patient; a predicted X-ray and / or CT angiography score for the patient; a prediction of success of post-procedural CAD intervention for the patient; and / or a prediction of patient survival with and / or without intervention; and a user interface configured to provide the generated CAD prediction output.
[0024] According to another aspect, a non-transitory computer-readable storage medium having computer-readable program code embedded therein is provided, the computer-readable program code being used to cause a coronary artery disease (CAD) prediction system to provide an analysis of coronary artery disease for a patient by: receiving patient metadata about the patient; receiving a time series of 2D and / or 3D ultrasound images obtained based on an ultrasound analysis of the patient's heart; selecting a plurality of ultrasound images from the time series; processing the selected plurality of ultrasound images using a trained AI algorithm of the CAD prediction system to generate a feature map of the selected plurality of ultrasound images; analyzing the generated feature map and the received patient metadata using the trained algorithm of the CAD prediction system to generate a CAD prediction output, the CAD prediction output comprising one or more of the following: (i) a risk or probability of CAD for the patient; (ii) a probability of regional wall motion abnormality (RWMA) for the patient; (iii) a predicted X-ray and / or CT angiography score for the patient; (iv) a prediction of success of a post-procedural CAD intervention for the patient; and / or (v) a prediction of survival of the patient with and / or without intervention; and providing the generated CAD prediction output via a user interface of the CAD prediction system.
[0025] It should be understood that all combinations of the above concepts and the additional concepts discussed in more detail below (as long as these concepts do not conflict with each other) are considered part of the inventive subject matter disclosed herein. In particular, all combinations of the claimed subject matter appearing at the end of this disclosure are considered part of the inventive subject matter disclosed herein. It should also be understood that the terms explicitly used herein that may also appear in any disclosure incorporated by reference should be given the meaning most consistent with the specific concepts disclosed herein.
[0026] These and other aspects of the various embodiments will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS
[0027] In the drawings, the same reference numerals generally refer to the same parts in different views. The drawings showing the features and modes of implementing various embodiments should not be interpreted as limiting other possible embodiments falling within the scope of the appended claims. Moreover, the drawings are not necessarily drawn to scale, but the emphasis is generally placed on illustrating the principles of the various embodiments.
[0028] Figure 1 is a flow chart of a method for providing a likelihood of CAD for a patient, according to an embodiment.
[0029] Figure 2is a schematic diagram of a CAD prediction system according to an embodiment.
[0030] Figure 3 is a flow chart of a method for providing a likelihood of CAD for a patient, according to an embodiment.
[0031] Figure 4 is a flow chart of a method for providing a likelihood of CAD for a patient, according to an embodiment.
[0032] Figure 5 is a schematic representation of the output of a CAD prediction system according to an embodiment.
[0033] Figure 6 is a flow chart of a method for training a machine learning algorithm for a CAD prediction system according to an embodiment. DETAILED DESCRIPTION
[0034] The present disclosure describes various embodiments of systems and methods configured to generate predictions of coronary artery disease. More generally, the applicant has recognized and appreciated that it would be beneficial to provide improved methods and systems for non-invasive analysis and prediction of coronary artery disease. Accordingly, a CAD prediction system receives patient metadata about a patient and receives a time series of 2D and / or 3D ultrasound images obtained from an ultrasound analysis of the patient's heart. The system selects a plurality of ultrasound images from the time series and processes the images using a trained algorithm to generate a feature map of the selected plurality of ultrasound images. The system uses a trained algorithm to analyze the generated feature map and the received patient metadata to generate a CAD prediction output, the CAD prediction output including one or more of the following: (i) the patient's risk or probability of CAD; (ii) the patient's probability of regional wall motion abnormality (RWMA); (iii) the patient's predicted X-ray and / or CT angiography score; (iv) the patient's prediction of post-procedural CAD intervention success; and / or (v) the patient's prediction of survival with and / or without intervention. The system then provides the generated CAD prediction output to a user using a user interface.
[0035] Thus, according to an embodiment, a novel end-to-end learning pipeline is provided to predict findings from X-ray / CT angiography based on 2D / 3D rest / stress transthoracic (TTE) or transesophageal (TEE) echocardiography images and patient metadata. The algorithm is able to process one or more sequences at a time. The algorithm is robust to different representations of input image quality, resolution, vendor characteristics, etc. Furthermore, the tool enables prediction of global or local RWMA with classification or regression of occlusion of the main coronary arteries.
[0036] According to an embodiment, the method and the system utilize an AI solution to detect CAD using echocardiography. The method is intended to detect the presence of RWMA in a 2D / 3D TTE / TEE echo loop. Next, the system and the method are intended to predict the presence of CAD based on CT angiography images. That is, the system and the method utilize AI (e.g., 3D and / or 4D convolutional neural networks (CNNs)) to predict CAD assessment based on CT angiography using ultrasound images and associated patient metadata (age, weight, ECG features, etc.) as input. The system is capable of using global average pooling within the channel dimension of the CNN to aggregate in the time dimension. Alternatively, the system is capable of using a CNN with a combination of recurrent units (e.g., RNNs) to aggregate in the time dimension. The algorithm for CAD / RWMA assessment can be applied on a cart or through a cloud service - potentially enabling more patients to access CAD assessment. The output is connected to one or more classification layers or prediction layers. The output of the algorithm is more consistent than the manual assessment of RWMA. Furthermore, ultrasound imaging is safer, cheaper, and more accessible than other imaging modalities for evaluating CAD. The output of the pipeline can include predictions for several observables and significance maps, indicating decisive regions in the input sequence with respect to individual observables. Finally, the algorithm can help determine the next step in the clinical pathway by predicting success / survival with / without intervention.
[0037] According to an embodiment, the method and the system can be used to support a cardiologist in selecting / filtering patients / studies or sequences within a study for further evaluation. The algorithm can also indicate regions of interest within the image that show possible signs of RWMA. In this way, the method and the system can significantly improve the productivity of the cardiology workflow.
[0038] According to an embodiment, the method and the system produce as output one or more of the following:
[0039] 1. Classification of global or local RWMA assessment, wherein the classification of RWMA cases can be defined as normal or abnormal. In addition, the abnormal category can be further divided into: hypokinetic type, hyperkinetic type, or dyskinesia type.
[0040] 2. Prediction of CT angiography report findings based on echocardiography images + patient metadata. Depending on the embodiment, the output can be a classification of predicted obstruction / no obstruction, and / or regression of the amount of obstruction in any coronary artery (eg, predicted 80% obstruction in LAD).
[0041] 3. Prediction of X-ray angiography report findings based on echocardiography images + patient metadata. Depending on the embodiment, the output can be a classification of predicted obstruction / no obstruction, and / or regression of the amount of obstruction in any coronary artery (eg, predicted 80% obstruction in LAD).
[0042] 4. Prediction of success of post-procedural CAD intervention, where markers include likelihood of success of atherectomy (shock wave, rotational atherectomy) or likelihood of success of stent intervention.
[0043] 5. Prediction of patient survival with or without intervention (Cox hazard prediction).
[0044] According to embodiments, some of the existing disadvantages overcome by the methods and systems disclosed herein or otherwise contemplated are:
[0045] 1. End-to-end learning, thus eliminating the need for feature extraction.
[0046] 2. Use 2D / 3D rest / stress TTE / TEE ultrasound cycles with associated patient metadata (age, weight, ECG features, etc.) as input and employ a spatiotemporal deep learning architecture.
[0047] 3. Cheaper and safer than X-ray / CT angiography because ultrasound itself is relatively cheap, more accessible, and does not expose the patient to radiation.
[0048] 4. Consistent and repeatable identification of RWMA, rather than qualitative clinician assessments, which may vary from user to user.
[0049] 5. Predicting X-ray / CT angiography results based on echo images, potentially reducing the number of normal patients who choose those more expensive, more invasive and risky procedures.
[0050] 5. The ability to track long-term patient progression, starting with early assessment of CAD (as opposed to CT or angiography).
[0051] 6. Make CAD risk assessments more accessible to more people using automated or cloud-based approaches.
[0052] 7. Improve specialist cardiac workflow with pre-evaluation of echo images.
[0053] 8. Predicting the success of post-procedural CAD intervention, where markers include the likelihood of successful atherectomy (shock wave, rotational atherectomy) or the likelihood of successful stent intervention.
[0054] 9. Prediction of patient survival with or without intervention (Cox hazard prediction).
[0055] According to embodiments, in some non-limiting embodiments, the systems and methods described herein or otherwise contemplated can be implemented as a component of a commercial product for ultrasound imaging or analysis, or as a commercial product for cardiovascular analysis (e.g., Elements of the IntelliSpace Cardiovascular (ISCV) (available from Royal Philips of the Netherlands), or implemented as elements of a commercial product for patient analysis or monitoring (e.g., Philips Patient Flow Capacity Suite (PFCS)), or any suitable system.
[0056] refer to Figure 1 In one embodiment, a flow chart of a method 100 for using a coronary artery disease (CAD) prediction system to provide a likelihood of CAD for a patient is provided. The methods described in conjunction with the figures are provided as examples only and should be understood not to limit the scope of the present disclosure. The CAD prediction system can be any system described herein or otherwise contemplated. The CAD prediction system can be a single system or a plurality of different systems.
[0057] At step 110 of the method, a CAD prediction system 200 is provided. Figure 2 In the depicted embodiment of the CAD prediction system 200, the system includes one or more of the following: a processor 220, a memory 230, a user interface 240, a communication interface 250, and a storage device 260, which are interconnected via one or more system buses 212. It should be understood that Figure 2 In some respects, an abstraction is constituted, and the actual organization of the components of the system 200 may be different and more complex than that illustrated. In addition, the CAD prediction system 200 can be any system described or otherwise contemplated herein. Other elements and components of the patient analysis system 200 are disclosed and / or contemplated elsewhere herein.
[0058] At step 120 of the method, the CAD prediction system receives patient metadata about the patient on whom the ultrasound examination will be performed. Patient metadata can be any information about the patient that the CAD prediction system can or can be used to analyze as described herein or otherwise contemplated. According to an embodiment, the patient metadata includes one or more of the following: demographic information about the patient, the inpatient or outpatient status of the patient, the patient's medical history, the patient's diagnosis, and the reason for performing the ultrasound examination. For example, the demographic information may include information about the patient, such as name, age, body mass index (BMI), and any other demographic information. The inpatient or outpatient status of the patient may include any information about whether the patient is inpatient or outpatient, or any other status of the patient. The patient's medical history may be any historical admission or discharge information, historical treatment information, historical diagnostic information, historical examination or imaging information, and / or any other information. The diagnosis of the patient can be any information about the patient's medical diagnosis (historical and / or current information). The reason for performing the ultrasound examination may be any purpose, reason, demand, or other examination motivation.
[0059] Patient metadata is received from one or more different sources. According to an embodiment, patient metadata is received, retrieved, or otherwise obtained from an electronic medical record (EMR) database or system 270. The EMR database or system can be local or remote. The EMR database or system can be a component of the CAD prediction system, or can communicate locally and / or remotely with the CAD prediction system. The received patient metadata can be used immediately, or can be stored in a local or remote storage device for use in further steps of the method.
[0060] At step 130 of the method, the CAD prediction system receives a time series of 2D and / or 3D ultrasound images obtained based on an ultrasound analysis of the patient's heart. The ultrasound analysis of the patient's heart can be any ultrasound examination, including but not limited to a transthoracic examination (TTE) or a transesophageal examination (TEE), and can be a stress test or a resting examination. Ultrasound image data is sent to the system, obtained by the system, or otherwise received by the system. The ultrasound image includes a time series of ultrasound image data, for example, a video including multiple frames (also referred to as a "loop"), and / or multiple videos (each video including multiple frames). Ultrasound image data can be obtained for a single region, or ultrasound image data can be obtained for multiple different zones or regions. The system can receive ultrasound image data in real time, or the ultrasound image data can be stored in a local and / or remote memory and the system receives the ultrasound image data in the future.
[0061] Any ultrasound device or system may be used to obtain ultrasound image data, which may be any device or system suitable for obtaining or otherwise receiving ultrasound image data of a patient. A healthcare professional may be able to set, adjust, preprogram or otherwise determine one or more parameters of the ultrasound device. The ultrasound device or system may be remote, local, or a component of the CAD prediction system.
[0062] At step 140 of the method, the CAD prediction system selects a plurality of ultrasound images from the time series. The plurality of images may be all of one or more time series of ultrasound examinations or segments of the time series (e.g., segments in which a specific region or segment of the heart, blood vessels, or other anatomical region is imaged during the examination). The selection of a plurality of ultrasound images from the time series can be automatic selection, selection from a user, or a combination of the two. For example, the CAD prediction system can be designed or programmed to select a specific number of images from the sequence (e.g., a minimum number of images and / or a specific number of images spanning a minimum amount of examination or imaging time). The plurality of ultrasound images selected from the time series can be used immediately by the CAD prediction system, or can be stored in local and / or remote memory and used by the system in the future.
[0063] refer to Figure 3 , provides an overview of embodiments of the methods and systems described herein or otherwise contemplated. Dashed lines indicate possible or optional embodiments. According to an embodiment, the system and the method include: (1) an input layer with preprocessing, (2) a spatiotemporal neural network part with temporal aggregation, (3) a neural network classification head, and (4) decision visualization with a saliency map. Accordingly, the system and the method can be divided into three different parts: (1) an input interface, where one or more 2D or 3D ultrasound sequences (e.g., AP4+AP3+AP2, etc.) can be selected; (2) the main image processing algorithm, which includes image preprocessing, neural network reasoning, and temporal aggregation, and finally multiple prediction heads; (3) various observables for assessing the risk of CAD. Finally, there can be a module (4) for visualizing features on the input image that are important to the neural network using a saliency map.
[0064] According to an embodiment, the method and system include an interface for input (which interface can be or include a Digital Imaging and Communications in Medicine (DICOM) interface, other image source interface, raw data interface, or other interface). For example, the interface can be a visual selection of DICOM US sequences in stand-alone software, or integrated into existing hardware or software. In an alternative embodiment, the method and system can be applied directly to one or more files in a directory. The interface can include an indication of whether the input image data / patient metadata is within the data domain used to train the algorithm (i.e., an out-of-distribution indication).
[0065] According to an embodiment, the selected plurality of images are subjected to preprocessing. For example, the method and the system can be designed and tuned to extract image information from the input data and perform image processing transformations on the image sequence. This includes spatial and / or temporal image resizing and resampling, intensity normalization, cropping, and other preprocessing options. The hyperparameters of these processing steps can be selected so that the neural network portion is insensitive to various inputs.
[0066] At step 150 of the method, the trained AI algorithm of the CAD prediction system processes the selected multiple ultrasound images to generate feature maps of the selected multiple ultrasound images. According to an embodiment, the trained AI algorithm of the CAD prediction system is a convolutional neural network, but it may also be many other types of AI algorithms. According to an embodiment, the trained algorithm of the CAD prediction system processes the selected multiple ultrasound images to generate feature maps of the selected multiple ultrasound images in the spatial direction, and / or processes the selected multiple ultrasound images by temporally aggregating the selected multiple ultrasound images in the time dimension to generate feature maps.
[0067] According to an embodiment, the CAD prediction system includes an end-to-end learning pipeline characterized by a trained convolutional neural network. These end-to-end learning pipelines can be 3D or 4D algorithms that learn feature maps of input ultrasound sequences. Time correlation can be accessed in two different embodiments (first, convolution on the time dimension; or second, aggregation with a recursive unit (e.g., RNN)). In the first method, convolution is performed on the spatial dimension and the time dimension (thus in 3D or 4D), depending on the input (2D / 3D+time). In this case, time aggregation is accomplished by the following operation: average pooling is performed on the feature map on the time dimension, leaving the feature map as input for the classification head. In the second method, convolution is performed only on the spatial dimension, thereby creating an embedding vector containing basic information of the input frame (for the neural network). Time aggregation is accomplished by feeding these embedding vectors frame by frame into the recursive neural network. Then, the output of the recursive neural network is used again as input for the classification head neural network.
[0068] According to an embodiment, the spatiotemporal network aims to combine spatial patterns within frames and the temporal dependencies between them when compared to other CNNs that only take images as input to the network. The basic idea is that motion, flow and dynamics carry information about the nature of the sequence at hand. Moreover, in addition to 2D information, the system and the method also utilize 3D echo loops and patient metadata (e.g., age, gender, etc.), which may contain more predictive information than 2D imaging data alone.
[0069] According to an embodiment, the output of step 160 of the method is a feature map that can be analyzed as described herein or otherwise contemplated. The generated feature map can be used immediately, or can be stored in local and / or remote memory and used by the system at a future point in time.
[0070] At step 160 of the method, a trained algorithm of the CAD prediction system analyzes the generated feature map and the received patient metadata to generate a CAD prediction output. According to an embodiment, the trained algorithm is a neural network head that utilizes the feature map created in step 150 of the method as input. According to an embodiment, these classification heads and CNN parts are trained simultaneously for optimal performance. For training, the system utilizes a patient database with 2D / 3D ultrasound image cycles, and the following corresponding indicators for each patient: coronary artery / CT angiography findings (acquired within + / - 6 months of echocardiography image acquisition); interventional procedure results (atherectomy, patient survival, etc.). This data set is used to train the network to infer various proposed endpoints based on the input sequence. Preferably, the data set includes contrast-enhanced ultrasound images and non-contrast ultrasound images, because an algorithm trained using only non-contrast images will provide less accurate results when used on contrast-enhanced images, and vice versa.
[0071] According to an embodiment, the trained algorithm of the CAD prediction system can analyze the generated feature map and the received patient metadata to generate a CAD prediction output, wherein the CAD prediction output can be one or more of the following:
[0072] 1. The patient's risk or probability of CAD;
[0073] 2. The patient's probability of regional wall motion abnormality (RWMA);
[0074] 3. The patient's predicted X-ray and / or CT angiography score;
[0075] 4. Prediction of the success of a patient's post-procedural CAD intervention; and / or
[0076] 5. Prediction of patient survival with and / or without intervention.
[0077] The predicted X-ray and / or CT angiography score may include a prediction of any type of X-ray and / or CT angiography score used to assess coronary artery disease (e.g., a whole heart Agaston score, an aortic calcium score, or any other suitable angiography score determined from X-ray and / or CT imaging data). X-ray and CT angiography scores are widely used by clinicians to monitor coronary artery disease, so the predicted X-ray and / or CT angiography score generated based on ultrasound imaging data provides a measure of coronary artery disease that is meaningful to clinicians, but without the risks associated with X-ray and CT imaging.
[0078] Preferably, a 4D CNN is used to generate a predicted X-ray and / or CT angiography score so that the generated feature map analyzed to generate the predicted score includes motion features and / or intensity features. These motion features and / or intensity features are automatically identified using the end-to-end machine learning method described herein.
[0079] refer to Figure 4 In one embodiment, a flowchart is provided that includes possible embodiments of a trained algorithm for a CAD prediction system at steps 150 and 160 of the method that are independently focused on specific goals. According to embodiments, the trained algorithm for a CAD prediction system need not simultaneously deliver all of the features described herein. Figure 4 As shown, it is also possible to independently develop specific versions that focus on a subset of the described features. Accordingly, the CAD prediction system can include a single trained algorithm that generates and / or processes the feature map created in step 150 of the method to generate one or more of the outputs described herein or otherwise contemplated, or the CAD prediction system can include multiple trained algorithms to generate one or more of the outputs described herein or otherwise contemplated, wherein each of the multiple trained algorithms is configured to generate a specific one of the outputs described herein or otherwise contemplated.
[0080] At step 170 of the method, the generated CAD prediction output of one or more trained algorithms is displayed to a medical professional or other user via a user interface of the CAD prediction system. The generated CAD prediction output can be provided to the user by any mechanism that displays, visualizes or otherwise provides information via a user interface. According to an embodiment, information can be transmitted to the user interface and / or another device via wired and / or wireless communication. For example, the system can transmit information to a mobile phone, a computer, a laptop, a wearable device, and / or any other device configured to allow display and / or otherwise transmit a report. The user interface can be any device or system that allows the transmission and / or reception of information, and can include a display, a mouse, and / or a keyboard for receiving user commands.
[0081] According to an embodiment, the display may also include one or more of the following: the name of the subject, one or more details about the ultrasound analysis and / or treatment recommendations, and many other types of information. According to an embodiment, the CAD prediction output provided also includes one or more of the multiple ultrasound images obtained during the examination and / or used during the analysis by the CAD prediction system.
[0082] According to an embodiment, one or more of the plurality of ultrasound images provided includes a saliency map. Accordingly, the system can generate a saliency output for each classification head, which is displayed over the input data region associated with the individual decision. In this section, the display can also include an indication of the confidence of the classification. Therefore, according to an embodiment, the CAD prediction output provided also includes a confidence score.
[0083] refer to Figure 5 In one embodiment, a possible display of the generated CAD prediction output provided via a user interface of the CAD prediction system is provided. Although not shown, a colored area can be superimposed on the original input sequence, which indicates the significance strength of the selected observable with respect to the model. Figure 5 The WMA assessment in the display includes anterior wall motion abnormality detection with a confidence score of (0.84) and lateral wall motion abnormality detection with a confidence score of (0.75). The display includes an overall CAD assessment generated by one or more trained algorithms of the CAD prediction system, which indicates that invasive angiography is likely to reveal a finding that CAD is present (with a likelihood or confidence score of 0.77).
[0084] According to an embodiment, the methods and systems described herein or otherwise contemplated focus on predicting patient CAD and RWMA. Since this is a cheaper, safer and more accessible alternative to X-ray / CT angiography, the system utilizes various outputs from the trained algorithm to perform non-invasive CAD risk stratification for patients. In addition, it is also possible to evaluate patients longitudinally to observe whether the patient is progressing / regressing in CAD. Therefore, the indicators generated by the methods and systems described herein or otherwise contemplated can be used for the following clinical activities: (i) evaluating treatment progress for CAD maintenance; (ii) screening patients who should undergo stress echo examination; (iii) screening patients who need CT angiography examination; (iv) screening patients who need X-ray angiography examination; (v) expansion to other cardiovascular diseases (e.g., peripheral arterial disease); (vi) improving cardiology workflow via faster assessment of RWMA; and / or (vii) reducing inter-observer / intra-observer variability in clinical RWMA assessment. The methods and systems described herein or otherwise contemplated can be used in a variety of systems, including premium systems, point-of-care systems, value-based systems, and multi-provider systems.
[0085] exist Figure 1 At optional step 180 of the depicted method 100, a healthcare professional utilizes the generated CAD prediction output provided via a user interface of the CAD prediction system to implement a healthcare treatment for the subject. For example, a clinician or other decision maker utilizes the displayed generated CAD prediction output (e.g., a probability or likelihood of CAD, a probability or likelihood of a patient's regional wall motion abnormality (RWMA), a patient's predicted X-ray and / or CT angiography score, a prediction of the success of a post-procedural CAD intervention for the patient, and / or a prediction of the patient's survival with and / or without intervention) to make a patient care decision. For example, a healthcare professional can utilize the probability or likelihood of the disease alone or in combination with the prediction or likelihood of the patient's CAD intervention success and / or the prediction of the patient's survival with and / or without intervention to initiate, continue, or stop a specific treatment configured to address the patient's CAD. Implementations can include prescriptions, orders, additional tests, and / or another implementation. Many other implementations are also possible.
[0086] refer to Figure 6In one embodiment, a flow chart of a method 600 for training one or more trained algorithms of a CAD prediction system is provided. At step 610 of the method, the system receives a training data set, the training data set comprising training data (e.g., historical patient data) about a plurality of patients. The training data can include inputs, for example, ultrasound data of ultrasound examinations for patients with CAD and patients without CAD, clinical diagnoses, and results of patient treatments (including the success of CAD interventions for patients and / or the survival of patients with and without interventions). Preferably, the training data set comprises zero-contrast ultrasound data and contrast-enhanced ultrasound data (i.e., ultrasound data collected from ultrasound examinations in which contrast agents are administered to patients) for both the set of patients with CAD and the set of patients without CAD. The training data may be stored in and / or received from one or more databases. The database may be a local and / or remote database. For example, the CAD prediction system may include a database of training data.
[0087] In some examples, the training data set can be configured to train the algorithm of the CAD prediction system to generate a specific CAD prediction output. For example, to train the algorithm to generate a predicted X-ray and / or CT angiography score, a training data set can be used, the training data set comprising: ultrasound data of ultrasound examinations for patients with CAD and patients without CAD, and the corresponding angiography score of the patient determined from the X-ray and / or CT data.
[0088] According to an embodiment, the CAD prediction system may include a data preprocessor or similar component or algorithm that is configured to process the received training data. For example, the data preprocessor analyzes the training data to remove noise, bias, errors, and other potential problems. The data preprocessor can also analyze the input data to remove low-quality data. Many other forms of data preprocessing or data point identification and / or extraction are also possible.
[0089] At step 620 of the method, the system trains a machine learning algorithm, which will be an algorithm for use in analyzing input information as described or otherwise contemplated. The machine learning algorithm is trained using a training data set according to known methods for training machine learning algorithms. According to an embodiment, the algorithm is trained using the processed training data set to utilize the feature map to generate one or more of the possible CAD outputs described herein or otherwise contemplated (including confidence, likelihood, or probability scores). According to an embodiment, the algorithm is also trained using the processed training data set to generate one or more intervention recommendations based on the determined one or more possibilities.
[0090] At step 630 of the method, the trained model of the CAD prediction system is stored for future use. Depending on the embodiment, the model may be stored in a local or remote storage device.
[0091] refer to Figure 2 , a schematic representation of a CAD prediction system 200 is provided. The system 200 can be any system described herein or otherwise contemplated, and can include any components described herein or otherwise contemplated. It should be understood that Figure 2 An abstraction is constituted in some respects, and the actual organization of components of system 200 may be different and more complex than that illustrated.
[0092] According to an embodiment, the system 200 includes a processor 220, which can run instructions stored in the memory 230 or the storage device 260 or process data in other ways, thereby, for example, performing one or more steps of the method. The processor 220 can be formed by one or more modules. The processor 220 can take any suitable form, including but not limited to a microprocessor, a microcontroller, multiple microcontrollers, a circuit, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a single processor, or multiple processors.
[0093] Memory 230 can take any suitable form, including non-volatile memory and / or RAM. Memory 230 may include various memories, for example, L1, L2 or L3 cache or system memory. As such, memory 230 may include static random access memory (SRAM), dynamic RAM (DRAM), flash memory, read-only memory (ROM) or other similar storage devices. Memory can store operating systems, etc. RAM is used by processors to temporarily store data. According to an embodiment, the operating system may include code that controls the operation of one or more components of system 200 when run by the processor. It will be apparent that in embodiments where the processor implements one or more of the functions described herein in hardware, software described in other embodiments as corresponding to such functions may be omitted.
[0094] The user interface 240 may include one or more devices for realizing communication with the user. The user interface can be any device or system that allows to transmit and / or receive information, and may include a display, a mouse and / or a keyboard for receiving user commands. In certain embodiments, the user interface 240 may include a command line interface or a graphical user interface that may be presented to a remote terminal via a communication interface 250. The user interface may be located at one place with one or more other components of the system, or may be away from the system and communicate via a wired and / or wireless communication network.
[0095] The communication interface 250 may include one or more devices for implementing communication with other hardware devices. For example, the communication interface 250 may include a network interface card (NIC) configured to communicate according to the Ethernet protocol. In addition, the communication interface 250 may implement a TCP / IP stack for communicating according to the TCP / IP protocol. Various alternative or additional hardware or configurations for the communication interface 250 will be apparent.
[0096] The storage device 260 may include one or more machine-readable storage media, such as a read-only memory (ROM), a random access memory (RAM), a magnetic disk storage medium, an optical storage medium, a flash memory device, or a similar storage medium. In various embodiments, the storage device 260 may store instructions for the processor 220 to execute or data on which the processor 220 may operate. For example, the storage device 260 may store an operating system 261 for controlling various operations of the system 200.
[0097] It will be apparent that various information described as being stored in storage device 260 may also be stored in memory 230 in addition or in lieu thereof. In this regard, memory 230 may also be considered to constitute a storage device, and storage device 260 may be considered to be a memory. Various other arrangements will be apparent. In addition, both memory 230 and storage device 260 may be considered to be non-transitory machine-readable media. The term "non-transitory" as used herein will be understood to exclude transient signals, but include all forms of storage devices (including both volatile and non-volatile memories).
[0098] Although system 200 is shown as including one component of each described component, various components may be replicated in various embodiments. For example, processor 220 may include multiple microprocessors that are configured to independently run the methods described herein, or are configured to perform the steps or subroutines of the methods described herein, so that the multiple processors operate in coordination to implement the functions described herein. In addition, in the case where one or more components of system 200 are implemented in a cloud computing system, various hardware components may belong to separate physical systems. For example, processor 220 may include a first processor in a first server and a second processor in a second server. Many other variations and configurations are also possible.
[0099] According to an embodiment, the electronic medical record system 270 is an electronic medical record database from which information about the patient (including clinical information and ultrasound data) can be obtained or received. The electronic medical record database can be a local or remote database and communicate directly and / or indirectly with the CAD prediction system 200. Therefore, according to an embodiment, the CAD prediction system includes the electronic medical record database or system 270.
[0100] According to an embodiment, the system includes one or more ultrasound devices 280 capable of acquiring the desired ultrasound images or analysis. According to another embodiment, the CAD prediction system 200 communicates wired and / or wirelessly with a local or remote ultrasound device 280 capable of acquiring the desired ultrasound images or analysis. According to another embodiment, the CAD prediction system 200 communicates wired and / or wirelessly with a local or remote database 280 storing ultrasound images or analysis. The CAD prediction system 200 can obtain the desired ultrasound images or analysis from one or more of these sources.
[0101] According to an embodiment, the storage device 260 of the system 200 may store one or more algorithms, modules and / or instructions to perform one or more functions or one or more steps of the methods described herein or otherwise contemplated. For example, the system may include a trained pre-processing algorithm 262, a trained CAD prediction algorithm 264, and / or reporting instructions 264, in addition to other instructions or data.
[0102] According to an embodiment, the trained preprocessing algorithm 262 includes one or more trained algorithms configured to receive the selected multiple ultrasound images as input and configured to generate the feature map of the selected multiple ultrasound images as output. The preprocessing algorithm 262 can be trained using any known method (including the method described herein or otherwise conceived). The AI algorithm 262 can be any type of trainable algorithm (including but not limited to neural networks and many other types of algorithms). According to an embodiment, the trained convolutional neural network of the CAD prediction system processes the selected ultrasound image to generate a feature map by convolving the selected multiple ultrasound images in the spatial direction. According to another embodiment, the trained convolutional neural network of the CAD prediction system processes the selected ultrasound image to generate a feature map by temporally aggregating the selected multiple ultrasound images in the time dimension. According to an embodiment, the output of the trained preprocessing algorithm 262 is a feature map that can be analyzed as described herein or otherwise conceived. The generated feature map can be used immediately, or can be stored in a local and / or remote memory and used by the system in the future.
[0103] According to an embodiment, the trained CAD prediction algorithm 264 includes one or more trained algorithms configured to receive the generated feature map as input and configured to generate a CAD prediction output as described herein or otherwise contemplated as output. The CAD prediction algorithm 264 can be trained using any known method (including the method described herein or otherwise contemplated). The CAD prediction algorithm 264 can be trained using a training data set as described herein or otherwise contemplated. According to an embodiment, the trained algorithm is a neural network head using the feature map as input. According to an embodiment, these classification heads and CNN parts are trained simultaneously for optimal performance. For training, the system utilizes a patient database with 2D / 3D ultrasound image cycles, and the following corresponding indicators for each patient: coronary artery / CT angiography findings (acquired within + / - 6 months of echocardiography image acquisition); interventional procedure results (atherectomy, patient survival, etc.). This data set is used to train the network to infer various proposed endpoints based on the input sequence. According to an embodiment, the trained CAD prediction algorithm 264 analyzes the generated feature maps and the received patient metadata to generate a CAD prediction output, wherein the CAD prediction output can be one or more of the following: 1. the patient's risk or probability of CAD; 2. the patient's probability of regional wall motion abnormality (RWMA); 3. the patient's predicted X-ray and / or CT angiography score; 4. the prediction of the success of the patient's post-procedural CAD intervention; and / or 5. the prediction of the patient's survival with and / or without intervention, as well as other possible CAD prediction outputs.
[0104] According to an embodiment, the report instruction 264 instructs the system to generate a CAD prediction output and provide information including the generated CAD prediction output to the user via a user interface. According to an embodiment, the display may also include one or more of the following: the name of the subject, one or more details about the ultrasound analysis and / or treatment recommendations, and many other types of information. According to an embodiment, the CAD prediction output provided also includes one or more ultrasound images of the multiple ultrasound images obtained during the examination and / or used during the CAD prediction system analysis. According to an embodiment, one or more of the multiple ultrasound images provided include a saliency map. Accordingly, the system can generate a saliency output for each classification head, which is displayed above the input data area related to the individual decision. In this section, the display can also include an indication of the classification confidence. Therefore, according to an embodiment, the CAD prediction output provided also includes a confidence score. The CAD prediction output provided can also include treatment recommendations and many other types of information.
[0105] The generated CAD prediction output can be provided to the user by any mechanism that displays, visualizes or otherwise provides information via a user interface. According to an embodiment, information can be transmitted to a user interface and / or another device by wired and / or wireless communication. For example, the system can transmit information to a mobile phone, a computer, a notebook computer, a wearable device and / or any other device that is configured to allow display and / or otherwise transmit reports. The user interface can be any device or system that allows transmission and / or reception of information, and can include a display, a mouse and / or a keyboard for receiving user commands.
[0106] Thus, within the context of the disclosure herein, aspects of the embodiments may take the form of a computer program product embedded in one or more non-transitory computer readable media having computer readable program code embodied thereon. Thus, according to an embodiment, a non-transitory computer-readable storage medium comprising computer program code instructions is provided, wherein the computer program code instructions, when executed by a processor, enable the processor to perform the following method, the method comprising: (i) receiving patient metadata about a patient; (ii) receiving a time series of 2D and / or 3D ultrasound images obtained based on an ultrasound analysis of the patient's heart; (iii) selecting a plurality of ultrasound images from the time series; (iv) processing the selected plurality of ultrasound images using a trained convolutional neural network of a CAD prediction system to generate a feature map of the selected plurality of ultrasound images; (v) analyzing the generated feature map and the received patient metadata using a trained algorithm of the CAD prediction system to generate a CAD prediction output, wherein the CAD prediction output comprises one or more of the following: (a) the risk or probability of CAD for the patient; (b) the probability of regional wall motion abnormality (RWMA) for the patient; (c) a predicted X-ray and / or CT angiography score for the patient; (d) a prediction of the success of a post-procedural CAD intervention for the patient; and / or (e) a prediction of the patient's survival with and / or without intervention; and (vi) providing the generated CAD prediction output via a user interface of the CAD prediction system. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
[0107] According to an embodiment, the CAD prediction system is configured to: process thousands or millions of data points in the input data for training the system, and process and analyze the received time series of 2D and / or 3D ultrasound images obtained according to the ultrasound analysis of the patient's heart. For example, to generate a functional and well-trained system, the system includes a trained convolutional neural network configured to generate a feature map and a trained algorithm configured to generate a CAD prediction output, it is necessary to process millions of data points from the input data to train the algorithm. This requires millions or billions of calculations to generate a novel trained system based on millions of data points and millions or billions of calculations. As a result, based on the input data and parameters of the machine learning algorithm, the trained system is novel and unique, thereby improving the functionality of the CAD prediction system. Therefore, generating a functional, well-trained system includes a process with a large number of calculations and analyses, which cannot be completed by the human brain in one or more lifetimes. By providing an improved non-invasive CAD prediction system, the CAD prediction system has a huge positive impact on patient analysis, patient diagnosis and prognosis, and patient care compared to prior art systems.
[0108] Example
[0109] The following is provided as a non-limiting example of one possible embodiment of the methods and systems described or otherwise contemplated herein.
[0110] According to an embodiment, a preliminary network was developed to demonstrate the performance of this method using approximately 5500 apical 2 chamber (AP2) images. These images belonged to the category "anterior RWMA present" (1322 images) or "anterior RWMA not present" (4139 images). For this example, only non-contrast images were used.
[0111] A spatiotemporal 3D CNN was trained on these data using 3-fold patient stratified cross validation. Each fold was trained for 15 epochs using the Adam optimizer. The model is a VGG-inspired architecture with batch normalization. After the initial convolutional layer, there are three processing stages, each with two convolutional layers followed by stride-based downsampling. This is followed by a global average pooling layer and a fully connected layer with two outputs for the two classes. The entire pipeline was built in Pytorch.
[0112] The present invention may be a system, method and / or computer program product. It should be understood that any embodiment described as a system, method or computer program in the present disclosure can also be embodied as any of the other two embodiments. The computer program product may include (one or more) non-transient computer-readable storage media having computer-readable program instructions for causing a system or processor to perform various aspects of the present invention. The computer-readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer-readable storage medium can be, for example, but not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any combination of the above devices, as well as other possibilities. The computer-readable program instructions described herein can be downloaded from the computer-readable storage medium to various computing / processing devices, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network and / or a wireless network). Various aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, devices, systems and computer program products according to embodiments of the present invention. It should be understood that each box of the flowchart illustration and / or block diagram and the combination of boxes in the flowchart illustration and / or block diagram can be implemented by computer-readable program instructions.
[0113] All definitions, as defined and used herein, should be understood to control over dictionary definitions, definitions in documents incorporated by reference, and / or ordinary meanings of the defined terms.
[0114] Unless expressly indicated to the contrary, the terms "a" and "an" as used herein in the specification and claims should be understood to mean "at least one."
[0115] As used herein in the specification and claims, the phrase "and / or" should be understood to mean "one or both" of the elements so connected, i.e., elements that are present in some cases connected and in other cases separated. Multiple elements listed with "and / or" should be interpreted in the same manner, i.e., "one or more" of the elements so connected. In addition to the elements specifically identified by the "and / or" clause, other elements may optionally be present, whether related or unrelated to those elements specifically identified.
[0116] As used herein in the specification and claims, "or" should be understood to have the same meaning as "and / or" defined above. For example, when separating the items in the list, "or" or "and / or" should be interpreted as inclusive, that is, including at least one, but also including more than one element or item in several elements or element lists and optional other unlisted items. Only the term (for example, "only one of ..." or "exactly one of ...", or "consisting of ..." used in the claims) that clearly indicates the opposite situation refers to including exactly one element in several elements or element lists. In general, when there is an exclusive term (for example, "any one of ...", "one of ...", "only one of ..." or "exactly one of ...") in front, the term "or" as used herein should only be interpreted as indicating exclusive alternatives (that is, "one or the other, but not both").
[0117] The phrase "at least one" as used herein in the specification and claims with respect to a list of one or more elements should be understood to mean at least one element selected from any one or more elements in the list of elements, but not necessarily including each and every element specifically listed in the list of elements and at least one of each element, and not excluding any combination of elements in the list of elements. This definition also allows for the optional presence of elements other than the elements specifically identified in the list of elements to which the phrase "at least one" refers, whether or not these elements are related or unrelated to those elements specifically identified.
[0118] It should also be understood that, unless explicitly indicated to the contrary, in any method claimed herein that includes more than one step or action, the order of the steps or actions of the method is not necessarily limited to the order in which the steps or actions of the method are recorded.
[0119] In the claims and the foregoing description, all transitional phrases (e.g., "includes," "comprising," "carrying," "having," "containing," "involving," "holding," "consisting of," etc.) should be understood as open-ended, i.e., meaning including but not limited to. Only the transitional phrases "consisting of" and "consisting essentially of" should be closed transitional phrases or semi-closed transitional phrases, respectively.
[0120] Although several inventive embodiments have been described and illustrated herein, a person of ordinary skill in the art will readily conceive of various other means and / or structures for performing the functions described herein and / or obtaining the results and / or one or more advantages described herein, and each such variation and / or modification is considered to be within the scope of the inventive embodiments described herein. More generally, it will be readily understood by those skilled in the art that all parameters, dimensions, materials, and configurations described herein are exemplary, and that actual parameters, dimensions, materials, and / or configurations will depend on one or more specific applications for which the teachings of the present invention are used. Those skilled in the art will recognize that or will be able to determine many equivalents to the specific inventive embodiments described herein using only routine experiments. Therefore, it should be understood that the above embodiments are presented only by way of example and that within the scope of the appended claims and their equivalents, inventive embodiments may be practiced in a manner different from that specifically described and claimed. The inventive embodiments of the present disclosure are directed to each individual feature, system, article, material, kit, and / or method described herein. In addition, if two or more such features, systems, articles, materials, kits, and / or methods are not mutually contradictory, any combination of such features, systems, articles, materials, kits, and / or methods is included within the scope of the invention of the present disclosure.
Claims
1. A method (100) for providing an analysis of CAD for a patient using a coronary artery disease (CAD) prediction system (200), comprising: receiving (120) patient metadata about the patient; receiving (130) a time series of 2D and / or 3D ultrasound images obtained based on ultrasound analysis of the patient's heart; selecting (140) a plurality of ultrasound images from the time series by the CAD prediction system; Processing (150) the selected plurality of ultrasound images using the trained AI algorithm of the CAD prediction system to generate feature maps of the selected plurality of ultrasound images; analyzing (160) the generated feature map and the received patient metadata using a trained algorithm of the CAD prediction system to generate a CAD prediction output, the CAD prediction output comprising one or more of: (i) a risk or probability of CAD for the patient; (ii) a probability of regional wall motion abnormality (RWMA) for the patient; (iii) a predicted X-ray and / or CT angiography score for the patient; (iv) a prediction of success of a post-procedural CAD intervention for the patient; and / or (v) a prediction of patient survival with and / or without intervention; The generated CAD prediction output is provided (170) via a user interface of the CAD prediction system.
2. The method according to claim 1, wherein: The ultrasound analysis of the patient's heart is a transesophageal examination (TEE), and further wherein the ultrasound analysis of the patient's heart is a stress test, or a resting examination, ie, a non-stress examination.
3. The method according to claim 1 or 2, wherein: The temporal sequence of 2D and / or 3D ultrasound images comprises contrast enhanced images.
4. The method according to any one of claims 1 to 3, wherein: The provided CAD prediction output also includes one or more ultrasound images of the plurality of ultrasound images.
5. The method according to any one of claims 1 to 4, wherein: One or more of the provided plurality of ultrasound images comprises a saliency map.
6. The method according to any one of claims 1 to 5, wherein: The CAD prediction output provided also includes a confidence score.
7. The method according to any one of claims 1 to 6, wherein: The trained AI algorithm of the CAD prediction system processes the selected multiple ultrasound images to generate feature maps of the selected multiple ultrasound images in a spatial direction.
8. The method according to any one of claims 1 to 7, wherein: The trained AI algorithm of the CAD prediction system processes the selected multiple ultrasound images by temporally aggregating the selected multiple ultrasound images in a time dimension to generate a feature map.
9. The method according to claim 8, wherein: The trained AI algorithm is a 4D convolutional neural network.
10. The method according to any one of claims 1 to 9, further comprising the steps of: CAD treatment is administered (180) to the patient based on the provided CAD prediction output.
11. A system (200) for providing an analysis of coronary artery disease (CAD) in a patient, comprising: patient metadata about the patient; a time series of 2D and / or 3D ultrasound images obtained from an ultrasound analysis of the patient's heart; A trained AI algorithm (262) configured to analyze a plurality of ultrasound images to generate a feature map of the plurality of ultrasound images; a trained algorithm (263) configured to generate a CAD prediction output; A processor (220) configured to: (i) select a plurality of ultrasound images from the time series; (ii) using the trained AI algorithm of the CAD prediction system to process the selected multiple ultrasound images to generate feature maps of the selected multiple ultrasound images; (iii) using the trained algorithm of the CAD prediction system to analyze the generated feature map and the received patient metadata to generate a CAD prediction output, the CAD prediction output comprising one or more of: the patient's risk or probability of CAD; the patient's probability of regional wall motion abnormality (RWMA); the patient's predicted X-ray and / or CT angiography score; the patient's prediction of the success of a post-procedural CAD intervention; and / or the patient's prediction of survival with and / or without intervention; as well as A user interface (240) is configured to provide output of the generated CAD prediction.
12. The system according to claim 11, wherein: The ultrasound analysis of the patient's heart is a transesophageal examination (TEE), and further wherein the ultrasound analysis of the patient's heart is a stress test, or a resting examination, ie, a non-stress examination.
13. The system according to claim 11 or 12, wherein: The temporal sequence of 2D and / or 3D ultrasound images comprises contrast enhanced images.
14. A system according to any one of claims 11 to 13, wherein: The trained AI algorithm of the CAD prediction system processes the selected multiple ultrasound images to generate feature maps of the selected multiple ultrasound images in the spatial direction, and / or processes the selected multiple ultrasound images by temporally aggregating the selected multiple ultrasound images in the time dimension to generate a feature map.
15. A non-transitory computer readable storage medium (260) having computer readable program code embedded therein, the computer readable program code for causing a coronary artery disease (CAD) prediction system (200) to provide an analysis of coronary artery disease for a patient by: receiving patient metadata about the patient; receiving a time series of 2D and / or 3D ultrasound images obtained from an ultrasound analysis of the patient's heart; selecting a plurality of ultrasound images from the time series; Processing the selected multiple ultrasound images using the trained AI algorithm of the CAD prediction system to generate feature maps of the selected multiple ultrasound images; analyzing the generated feature maps and the received patient metadata using a trained algorithm of the CAD prediction system to generate a CAD prediction output, the CAD prediction output comprising one or more of: (i) a risk or probability of CAD for the patient; (ii) a probability of regional wall motion abnormality (RWMA) for the patient; (iii) a predicted X-ray and / or CT angiography score for the patient; (iv) a prediction of success of a post-procedural CAD intervention for the patient; and / or (v) a prediction of patient survival with and / or without intervention; The generated CAD prediction output is provided via a user interface of the CAD prediction system.
