Machine-based risk prediction for perioperative myocardial infarction or complications based on medical data

Abstract

Machine-based risk prediction or assistance is provided for perioperative complications, such as perioperative myocardial infarction (PMI). Machine learning models are used to predict the risk of PMI and / or recommend a course of action to avoid PMI in PCI. In the prediction, various combinations of types or modes of information are used, such as both imaging and non-imaging data. Machine learning models can be used to make predictions before, during, and / or after PCI to reduce the chance of PMI more quickly. Workflows for pre-PCI, during PCI, and / or post-PCI incorporate risk predictions and / or risk-based recommendations to reduce PMI in patients.

Description

Background Art

[0001] This embodiment relates to machine-based assistance in percutaneous coronary intervention (PCI). A common complication of catheterization in PCI is peri-procedural myocardial infarction (PMI), which has a prevalence of 5% to 30% depending on the evaluation criteria. Patients with PMI have a higher risk of fatality than those without PMI.

[0002] PMI can be predicted based on clinical symptoms, ECG abnormalities, and blood biomarkers (e.g., CK-MB or troponin). The preferred biomarker for myocardial necrosis is cardiac troponin (I or T), which has high myocardial tissue specificity and high clinical sensitivity. The disadvantage of these methods is that PMI can only be diagnosed when it is clearly indicated in blood biomarkers or ECG. Existing methods may not perform as well as early risk assessment. Summary of the invention

[0003] Provided are systems, methods, and computer-readable media with instructions for machine-based risk prediction or assistance for perioperative myocardial infarction. Machine learning models are used to predict the risk of PMI and / or recommend a course of action to avoid PMI in PCI. In the prediction, various combinations of types or patterns of information are used, such as both imaging and non-imaging data. Machine learning models can be used before, during, and / or after PCI to make predictions to reduce the chance of PMI more quickly. Workflows for before, during, and / or after PCI incorporate risk predictions and / or risk-based recommendations to reduce PMI in patients.

[0004] In a first aspect, a method for machine-based risk prediction for perioperative myocardial infarction is provided. A medical diagnostic imaging system images a patient. The imaging provides first image data representing an internal heart region of the patient. The first image data is from a first mode of data acquisition. Second data of the patient is acquired. The second data is for a second mode of data acquisition, the second mode being different from the first mode. A processor determines a risk of perioperative myocardial infarction by inputting first data responsive to the first image data and the second data into a machine learning model. Information based on the risk is displayed.

[0005] In one embodiment, the imaging includes computed tomography imaging, magnetic resonance imaging, or intravascular imaging. The first data is anatomy, heart disease, and / or cardiac function measurement results, and is based at least in part on detection in the first image data. The second data can be blood test measurement results, ECG measurement results, genetic data, demographic data, clinical data, or percutaneous coronary intervention characteristics. For example, the second data is a blood test measurement result. Other modes of data acquisition can be used, such as acquiring imaging data, blood test data, and third data for a third mode. For example, the third data is one of ECG measurement results, genetic data, demographic data, clinical data, or percutaneous coronary intervention characteristics.

[0006] In another embodiment, risk is determined as a prediction for a future time, such as the risk of PMI 30 days after PCI. The risk can be determined as the likelihood of readmission, revascularization, length of hospital stay, death, and / or major adverse cardiac events (MACE).

[0007] In yet another embodiment, a recommendation for treatment of the patient is identified. The recommendation corresponds to a risk.

[0008] Various machine learning models can be used to determine risk. For example, a machine learning model is a neural network.

[0009] The determination and / or display is performed at any one or more different times relative to the PCI. For example, the determination and display is performed before percutaneous coronary intervention is performed on the patient. As another example, the determination of risk is performed multiple times during the percutaneous coronary intervention on the patient, such as when another image is acquired during the intervention. The display is performed in the event that the risk from one of the times is above a threshold level. In another example, the determination is performed multiple times after the percutaneous coronary intervention is performed on the patient, such as each time additional data becomes available. In the event that the risk is above the threshold level, a recommended course of action is displayed.

[0010] In a second aspect, a method is for machine-based assistance for perioperative myocardial infarction. First data of a patient is accessed for percutaneous coronary intervention. An action policy is indicated by an artificial intelligence in response to input of the first data to the artificial intelligence. The action policy limits the risk of perioperative myocardial infarction. The action policy is displayed.

[0011] In one embodiment, accessing the first data includes: accessing medical imaging of a cardiac region of a patient; calculating anatomical, cardiac disease and / or cardiac function measurements based on the imaging as a first part of the first data; and obtaining blood test measurements, ECG measurements, and demographic data as a second part of the first data.

[0012] In another embodiment, the artificial intelligence is a machine learning neural network.

[0013] Various indications can be provided at different times relative to PCI. For example, the type of stent, medication, and / or device used in the percutaneous coronary intervention is indicated before the percutaneous coronary intervention. As another example, a correction is indicated during the percutaneous coronary intervention. In another example, a medication change or surgical intervention is indicated after the percutaneous coronary intervention.

[0014] In a third aspect, a method for machine-based assistance for perioperative complications is provided. First data for a patient undergoing percutaneous coronary intervention is accessed. A machine learning model determines a risk of a perioperative complication in response to input of the first data. Medical personnel involved in the percutaneous coronary intervention are notified of the risk.

[0015] In one embodiment, where the risk is above a threshold, medical personnel are notified of the risk and a recommended course of action.

[0016] In another embodiment, the determination and notification occurs as part of a workflow before, during, or after a percutaneous coronary intervention.

[0017] Any one or more aspects or concepts described above may be used alone or in combination. Aspects or concepts described for one embodiment may be used in other embodiments or aspects. Aspects or concepts described for a method or system may be used in other systems, methods, or non-transitory computer-readable storage media.

[0018] These and other aspects, features and advantages will become apparent from the following detailed description of the preferred embodiments which will be read in conjunction with the accompanying drawings. The present invention is defined by the following claims, and nothing in this section should be construed as a limitation on those claims. Further aspects and advantages of the present invention are discussed below in conjunction with the preferred embodiments, and these aspects and advantages may be claimed later, independently or in combination. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The components and drawings are not necessarily drawn to scale, emphasis instead being placed upon illustrating the principles of the embodiments. Moreover, in the drawings, like reference numerals designate corresponding parts throughout the different views.

[0020] Figure 1 illustrates the use of machine learning models at different times relative to PCI to assist in reducing PMI risk;

[0021] Figure 2 is a flow chart of one embodiment of a method for machine-based risk prediction for perioperative complications;

[0022] Figure 3 shows an example machine learning architecture for risk prediction; and

[0023] Figure 4 is a block diagram of one embodiment of a system for PMI risk prediction or PMI avoidance assistance. DETAILED DESCRIPTION

[0024] Automated risk prediction is provided for other complications of PMI or PCI. Early risk assessment or prevention of PMI can reduce the incidence of major adverse cardiovascular events (MACE) after surgery, length of hospital stay, and / or associated costs. Data-driven methods automatically extract and quantify complication-related features from conventional medical examinations (e.g., coronary angiography examinations), and combine these features with other clinical biomarkers, thereby simplifying risk assessment and complication prevention. PMI will be used as an example of complication in this article.

[0025] PMI risk scoring uses an AI model that relies on multimodal features. AI-based companions indicate action guidelines for preventing or limiting PMI. Favorable workflows for pre-PCI, during PCI, and post-PCI integrate AI-based PMI assessments.

[0026] exist Figure 1In one embodiment shown in , a comprehensive system and method based on artificial intelligence (AI) is provided for continuous or periodic assessment of PMI. One or more (such as, multiple) modes of data acquisition and corresponding data 10 are collected for input into a machine learning model as part of PMI assessment. PMI assessment is performed at one or more of various clinical stages, which are: before PCI, PCI procedure, and after PCI. The assessment is performed continuously, periodically, or when data is acquired. The assessment outputs a risk level, such as outputting a diagnosis for a given patient as one of spontaneous MI, perioperative MI, or no MI. The distinction between spontaneous MI (acute CAD) and perioperative MI may be based on various considerations, such as any given CK-MB or troponin level. The absolute risk from spontaneous MI may be higher than the absolute risk from perioperative MI. The output may be a PMI risk score at any given time point (before PCI, after PCI, during PCI), which indicates the risk of the patient suffering from PMI. One or more indications (eg, preferred or suggested courses of action) for reducing the risk of PMI may be output at any given point in time relative to PCI.

[0027] In approximately 20% of PMI cases, a mechanical cause for PMI may not be found. Microemboli of thrombus or atherosclerotic material may be the cause, but may not be detected in coronary angiography. Given multiple modes of data acquisition, an AI-based system may be more likely to be able to detect at least a subset of the 20% of PMI cases in which a clear mechanical cause of PMI may not be identified. Measures for prevention may be provided in situations where such measures cannot be triggered otherwise. PMI assessment and prevention by AI-based systems can reduce the incidence of post-operative MACE, length of hospital stay, associated costs, and / or failure to identify PMI risk. A disadvantage of existing methods is that PMI can only be diagnosed when it is clearly indicated in blood biomarkers or ECG. AI-based systems can perform early risk assessments, or indicate measures for prevention in other cases.

[0028] To assist in the prevention of PMI in PCI patients, AI automatically detects imaging and / or non-imaging features and characteristics associated with PMI, provides a risk score for a fine-grained assessment of PMI risk, provides instructions during the PCI procedure (e.g., in cases of high risk of PMI), and / or self-improves in a timely manner as data accumulates. Improved CAD patient care and outcomes can be provided, such as during cathlab interventions. An elevated risk or high risk of PMI can be automatically detected. As an increased risk is detected, countermeasures can be proposed and PMI can be avoided.

[0029] For training, historical training data is used. The training data includes many samples (e.g., hundreds or thousands) of input data sets, such as images, information derived from images, other measurements (e.g., laboratory, blood and / or genetic measurements), patient information (e.g., demographic or clinical information), and / or other data from different acquisition modes. The acquisition mode represents different types of information and / or measurements, such as different types of imaging (angiography, computed tomography, magnetic resonance, ultrasound and / or optical modes). The training data also includes basic facts, such as diagnosis (e.g., spontaneous MI, PMI, no MI), PMI risk, and action guidelines established by experts for each data set. For multi-task machine learning models, basic facts may include basic fact segmentation or feature locations. Basic facts may be annotations from experts or data mined from patient records. In one embodiment, troponin levels are used as basic facts for the risk or possibility of PMI.

[0030] An architecture of a model to be trained is defined. In one embodiment, the architecture is defined as a multi-task architecture. The network architecture includes an output layer for each task, such as one output layer for segmentation or estimation of image features (e.g., lesions and / or other anatomical features associated with PMI), and another one or more output layers for risk, diagnosis, and / or recommended course of action. Figure 3 An example embodiment is shown having a network architecture that includes an encoder 32, a bottleneck 34, and a decoder 36 (e.g., an autoencoder that together forms a U-Net or image-to-image neural network) for training for segmentation or image feature estimation. For applications, the decoder 36 may not be used. For risk, diagnosis, and / or action policy prediction, the network architecture includes a convolutional neural network 38 that is connected to receive the output of the encoder 32 or the bottleneck 34 as input. The convolutional neural network 38 outputs a classification of risk. The encoder 32 receives input data 30, which the encoder 32 encodes. The resulting abstraction or feature output by the encoder 32 at the bottleneck 34 is a fingerprint that will be learned based on an optimization that uses the ground truth of the output of the decoder 36 as one task and uses one or more outputs of the result prediction network 38 for another task or tasks.

[0031] Other network architectures can be used. In other embodiments, a generative, image-to-image or generative adversarial network (GAN) network architecture is used, such as a U-net type arrangement with downsampling and upsampling blocks or layers with or without skip connections. For multiple tasks in GAN, GAN includes two or more discriminator networks or layer arrangements. A discriminator network is provided for each task. In other methods, a network for learning a strategy for action decision-making in reinforcement learning is used. Convolutional neural networks, recursive neural networks, or fully connected neural networks can be used. In the case where the machine learning model will be applied continuously or at different time points relative to the PCI, a recursive neural network using long short-term memory or other time arrangements can use information from different times for the current output.

[0032] In order to machine train the model, a machine learning architecture arrangement is defined. The definition is performed by programming or configuring the learning. The number of layers or units, the type of learning, the order of layers, connections, and other characteristics of the network are controlled by the programmer or user. In other embodiments, one or more aspects of the architecture (e.g., the number of nodes, the number of layers or units, or connections) are defined and selected by the machine during learning.

[0033] The machine uses the training data to implement training of the defined architecture. Input features can be manually defined. Some or all features can be learned from the raw data input, such as for deep learning. A machine (e.g., a processor, computer, workstation, or server) performs machine training on a defined network (e.g., a defined multi-task generator). The network is trained to generate outputs for one or more tasks (e.g., multiple tasks).

[0034] To train any of these networks, various optimizers such as Adadelta, SGD, RMSprop, or Adam can be used. The weights, connections, filter kernels, and / or other learnable variables of the network are randomly initialized, but another initialization can be used. End-to-end training is performed, but one or more features can be set. The network for one task can be initially trained separately and then used to further train the network for one task and another network for another task. Separate losses can be provided for each task. Joint training can be used. Any multi-task training can be performed. Batch normalization, dropout, and / or data augmentation are not used, but they can be used (e.g., using batch normalization and dropout). During optimization, different distinguishing features are learned. Features that provide an indication of the result and / or an indication of another task are learned.

[0035] The optimizer minimizes an error or loss, such as mean squared error (MSE), Huber loss, L1 loss, or L2 loss. The same or different losses can be used for each task. In one embodiment, machine training uses a combination of losses from different tasks.

[0036] Store trained models. Store network parameters, such as connections, convolution kernels, weights, or other learned values ​​for the network. The network is stored in memory for use in applications or testing. Once trained, the machine learning model can be applied to estimate results (e.g., diagnosis, PMI risk, and / or indication or recommendation of action to be taken). Use many samples in the training data to learn to output given unseen samples of input data for the future or a given patient.

[0037] Figure 1 An AI-based system is shown for continuous or updated assessment of PMI embedded in a continuous learning framework. Although initially based on an existing large database with patient-specific records, the system can be updated, such as to continuously learn or retrain from new data. The retrained machine learning model is redeployed for use in other patient cases. As further examples or samples for different patients are provided, such as due to application of the machine learning model and observation of complications from PCI, these further examples are used to update the training and corresponding machine learning model.

[0038] Figure 2 is a flow chart of one embodiment of a method for machine-based risk prediction for complications such as PMI. AI is used to provide assistance for PMI, such as providing diagnosis, risk and / or action guidelines. Multiple (such as, five or more) modes of data acquisition are provided to enable more informed, or more comprehensive AI assistance. The workflow for PCI is modified to include this assistance, especially when the AI ​​may operate quickly on a time frame that matches PCI, where it is impossible for a human to provide such rapid analysis of a large amount of information.

[0039] The method is performed in the order shown (e.g., from top to bottom or by number), but other orders may be used. For example, actions 21 and 22 are performed in any order. As another example, action 24 is performed before action 23.

[0040] Additional, different or fewer actions may be provided. For example, actions 21, 22, 23, 24, 25, 26 and / or 27 are not performed. One of actions 23 or 24 / 25 may be provided without the other. Actions 26 / 27 may not be provided, such as in the case where the estimated risk is low.

[0041] The method is performed by a medical diagnostic scanner, workstation, server or computer. Remote (such as cloud-based) or local applications of AI can be used. For example, a scanner and / or memory is used to obtain or access the patient's data. A processor (such as an image processor of a scanner or a separate computer) predicts risk, diagnosis and / or action plan. The image processor uses a display screen or printer for display or notification. The physician can use the output information to assist in making treatment decisions for the patient. This information can lead to better treatment and / or diagnosis for the patient.

[0042] In act 20, the patient's data is accessed. The processor accesses the memory. Using mining, searching or finding, data stored in a medical record database, a computerized patient or health record, a laboratory database and / or other memory is accessed. Access may be provided by receiving a transmission, such as in response to a query or a push. Access may be provided by manual entry of information. Access may be provided by scanning, sensing or measurement from the patient. Different data may be accessed from the same or different means, locations or devices, such as data from different acquisition modes.

[0043] The data is for PCI or associated complications. Access any data related to PCI, such as data collected before, during and / or after PCI. Access any data indicating risk of PMI.

[0044] The data may be imaging data of act 21 and / or non-image data of act 22. For act 21, one or more medical diagnostic imaging systems image a patient. The imaging provides image data representing an internal cardiac region of the patient, such as a point, line, area, or volume of the patient.

[0045] Computed tomography imaging, magnetic resonance imaging, x-ray angiography, fluoroscopy, optics, ultrasound, optical coherence tomography, and / or intravascular imaging and corresponding scanners may be used. Each type of scanning provides one or more modes of data acquisition. Any number of modes may be provided.

[0046] The image data itself can be accessed for input to a machine learning model. Data derived from image processing can be accessed (e.g., created or calculated) for input to a machine learning model. The image data is used to determine anatomy, heart disease, and / or heart function measurements. These measurements are detected, at least in part, from the image data. Any of these features can be automatically extracted (e.g., image processing is performed on available invasively or non-invasively acquired images).

[0047] In one embodiment, non-invasive imaging performed prior to PCI is used. A high coronary artery calcium score (CACS) measured using CCTA can be determined for input. The presence of a napkin-ring sign and / or lesion length can be detected based on CCTA. Plaque composition, such as lipid-rich lesions, as assessed using CCTA can be detected.

[0048] Other non-invasive imaging performed before or after PCI can be used. For example, PMI after PCI can be visualized and quantified using cardiac magnetic resonance imaging (MRI). Cardiac MRI is used to determine the size of the ventricular infarct. Contrast-enhanced (CE) MRI can be used to quantify infarcted myocardium <1 g.

[0049] Intravascular or non-invasive imaging (e.g., x-ray angiography) acquired during the PCI procedure can be used. For example, intravascular ultrasound (IVUS), or optical coherence tomography (OCT), and / or metrics extracted from intravascular images (e.g., lumen and / or plaque-related features) can be used.

[0050] Anatomical features assessed before PCI (from non-invasive images) and during the PCI procedure can include vessel tortuosity, number of lesions, lesions in specific vessels (LAD, LM), bifurcation lesions and / or Medina classification, bifurcation angles (e.g., wide bifurcation angles are associated with a higher risk of side branch occlusion and, therefore, a higher risk of PMI), main branch to side branch diameter ratios, long lesions, overall disease burden, and / or thrombus. Functional assessments (e.g., fractional flow reserve, higher pre-PCI microvascular resistance index (IMR), or other flow or pressure measurements) can include invasive measurements and / or measurements determined from non-invasive imaging.

[0051] In action 22, non-image data is accessed. Data from other acquisition modes of the patient is obtained. The data is automatically and / or manually extracted. The non-image data comes from sensors, computerized patient medical records, manual input, pathology databases, laboratory databases, and / or other sources. The non-image data represents one or more characteristics of the patient, such as family history, medications taken, temperature, body mass index, and / or other information. For example, the patient's genomic (genetic), ECG, blood test (e.g., troponin level), demographic, clinical, measurement, molecular, and / or family history data are obtained from memory, transformation, data mining, and / or manual input. In another example, the characteristics of the patient's PCI surgery are obtained, such as the PCI process including a sequence of therapy events, the equipment used, and / or the area of ​​application for each event. Data from one or more of these different acquisition modes is obtained, such as obtaining two, more, or all of the patient's blood test, ECG measurement results, genetic data, demographic data, clinical data, or PCI characteristics.

[0052] Exemplary PCI procedure-specific features include: total procedure time (which can also be predicted during the pre-PCI phase using the AI ​​model), number and type of stents, number of vessels treated, stent over-expansion, side branch occlusion, dissection, (distal) embolism, spasm, no or little reflux (as assessed by TIMI flow grading), and / or side branch flow interruption. Example blood test measurements include: CK-MB and (hS) troponin, glucose variability, high-density lipoprotein cholesterol (HDL-C) and / or HDL-C subfractions, such as HDL2 cholesterol (HDL2-C), or HDL3 cholesterol (HDL3-C). Since PMI changes the ECG waveform, ECG measurements such as for spontaneous MI (acute CAD) can be used. Genetic information is obtained by sequencing as a direct or indirect indicator. Demographic and clinical risk characteristics include: age, sex, hypertension, and / or renal dysfunction. Diagnosis of proteinuria (a pathological condition in which the protein albumin is abnormally present in the urine) can be used as non-image data. The type of catheter intervention can be accessed (e.g., CTO vs. non-CTO treatment). A generic cardiovascular disease-related risk score (e.g., Framingham) or a PMI-specific risk score (e.g., a risk score for side branch occlusion) can be calculated. Data can be extracted from radiology reports, such as by using natural language processing (NLP). Measurements from non-invasive stress tests can be used. Patient conditions, such as stable, ACS (STEMI / NSTEMI, UA), or CTO can be used. The patient's clinical and / or medication history can be used. Other comorbidities in addition to the disease being treated can also be used.

[0053] In act 23, the processor determines a risk and / or diagnosis of PMI. The accessed data of the patient (e.g., data from different acquisition modes, such as image and non-image data) is input to a machine learning model. In response to the input, the machine learning model outputs a result (e.g., risk and / or diagnosis of PMI). The prediction or estimation of the result uses AI. The machine learning model (such as a machine learning neural network) makes a prediction based on the input. Scan data and / or non-image data are input for prediction performed by AI.

[0054] Any machine learning model resulting from machine training can be used. For example, support vector machines, clustering, or other generators are provided for generating estimates of outcomes based on inputs. Machine learning classifiers can be used. In one embodiment, a neural network is trained as a generator, such as a trained convolutional neural network (CNN). A reinforcement model, a recurrent neural network model, a convolutional neural network, a fully connected neural network, U-Net, densenet, or other model can be used. Any forward model, such as Figure 3 For reinforcement learning based approaches, a machine learning or AI agent proposes an action at any given point in time. Example actions include: doing nothing or implanting an SES stent based on the risk level as incorporated into the learned strategy. For machine learning recurrent neural networks, data or features are acquired for different times (e.g., discrete or continuous). Recurrent neural networks make analyses and predictions based on the evolution in process time.

[0055] The output can be the patient's PMI risk, diagnosis (no PMI, PMI, or spontaneous MI), and / or PMI level. PMI can range from mild elevations in cardiac enzymes to large infarctions. Thus, the AI ​​can estimate the type of PMI that may occur and the likelihood (e.g., by assessing the size and / or volume of the myocardium that is at risk of occlusion and other input data).

[0056] Risk is the probability of a PMI occurring. Risk can be predicted for any time, such as the current time or a future time. For example, the risk output can refer to risk at one or more of various time points: 1 day, 3 days, 30 days, 6 months, 12 months, 3 years, 5 years, and / or 10 years.

[0057] Risk is the risk of PMI occurring. Other PMI risks may be the risk of readmission, revascularization, length of hospital stay, major adverse cardiac events (MACE), death, or other events due at least in part to PMI.

[0058] In action 24, the processor identifies a recommendation for treatment of the patient. The recommendation is generated as an output of the machine learning model. Alternatively, the diagnosis and / or risk output by the machine learning model is used to generate the recommendation with or without other information about the patient. A coded program is used to make the recommendation, such as using a guideline given a risk level of a PMI.

[0059] Various types of indications (e.g., recommendations) can be provided by an AI-based system. The type of stent to be implanted can be recommended (e.g., bioresorbable stent (BRS) versus first-generation sirolimus eluting stent (SES)). BRS have been associated with a higher incidence of PMI, primarily due to a higher abluminal strut surface area. If the risk of PMI is below a threshold, BRS is recommended to prevent acute recoil or occlusion, allowing sealing of postoperative dissection following acute barotrauma, and providing inhibition of in-segment restenosis through effective drug elution. In cases of high risk of PMI, BRS may represent a suboptimal solution, and therefore SES is recommended. In patients with stable angina, the use of PCB (paclitaxel-coated balloon) is associated with a significant reduction in the risk of PMI compared to the deployment of DES (drug eluting stent). PCB is recommended in cases where the risk of PMI is above a threshold. For higher risks of PMI, high-dose statin pretreatment may be recommended. Based on the level of PMI risk, antithrombotic and antiplatelet agents (such as clopidogrel, prasugrel) may be recommended. For higher PMI risk, devices to prevent distal embolism may be recommended.

[0060] In act 25, the processor indicates one or more action policies. The AI ​​outputs one or more action policies in response to the input of data. Alternatively, the risk and / or diagnosis of PMI is output and used to select one or more action policies, such as selecting from a lookup table. The action policy is selected to limit the patient's risk of PMI.

[0061] In act 25, a recommendation for a course of action is indicated, such as by display in act 26 or notification in act 27. Before PCI indicates the type of stent, medication, and / or device used in PCI. During PCI indicates correction (e.g., repositioning stent, replacing stent, ablation, removing occlusion, stripping, embolization, fixation of spasm, altering regurgitation, or altering cardiac flow) or medication. After PCI indicates a medication change or surgical intervention.

[0062] In act 26, a display screen or report (e.g., a printed page) displays risk-based information. The information is a diagnosis, risk, and / or course of action. The display is a visual output. An image processor generates an image. The image can be output to a display, to a patient medical record, and / or to a report.

[0063] The risk, diagnosis and / or action can be an annotation, alphanumeric text, graphic or chart. For example, the risk of PMI is displayed as a probability graph as a function of time. As another example, the stratification results are displayed. The output is projected to a display for the physician.

[0064] Input data may be output, such as the most important or risk-reflecting data as estimated by the machine learning model. For example, an image showing anatomical structures indicating higher risk is output along with annotations of PMI risk values. Other information may be displayed along with the predicted risk. More than one prediction may be output, such as predictions of outcomes for different therapies and / or times. The risks resulting from taking different courses of action may be output, such as in a table.

[0065] The diagnosis, risk and / or action can be presented with or on (e.g., overlaid or annotated) an image of the patient. For example, an image of a slice through a cardiac region is displayed. The results are provided on a display screen along with an image of an anatomical structure of interest, such as the heart, blood vessels, or valves. Where 3D scan data is provided, the 3D scan data is rendered to a 2D plane. Any rendering, such as a projected rendering, may be used. The display may include information from one or more similar patients.

[0066] In act 27, the processor notifies medical personnel involved in the PCI of the risk. The personnel are notified visually and / or audibly. For example, a warning of the PMI risk is displayed on a display screen as part of act 26, and / or displayed through an audio output of a speaker. In response to the input of the multimodal patient data, the determined PMI risk, diagnosis, and / or action policy from the machine learning model is output to a medical professional, such as a physician or nurse involved in the PCI or patient treatment.

[0067] In response to a risk being above a threshold level, information such as a recommended course of action with or without the risk is provided. Multiple thresholds may be provided, with each range of PMI risk mapped to a different recommended course of action. In the event that the risk can be mitigated, a threshold is provided to notify those responsible for PCI or patient care of the mitigation. Alternatively, a recommended course of action (which may be to make no change) is provided based on risk, even for situations with low PMI risk. This information is provided for any PMI risk level to assist in treatment. Still in other alternatives, the machine learning model outputs a recommended course of action without setting a threshold, which is output to a medical professional to notify them of the recommendation.

[0068] The determination of action 23 and the notification of action 27 occur as part of a PCI workflow before, during, and / or after PCI. For example, the application of the machine learning model and the display of the output results are performed as part of a pre-PCI assessment (such as an assessment for PMI). The workflow includes four steps. First, examine a patient with suspected coronary artery disease (CAD) (e.g., chest pain). Demographic, clinical, genetic, and / or other risk features are evaluated by a physician. Second, a CCTA imaging examination is performed and blood tests are run. Third, if the patient is diagnosed with CAD, the above-mentioned AI-based system is used to evaluate the risk of PMI. The AI-based system can be used once, or repeatedly as additional data is collected. Fourth, the patient is scheduled for invasive coronary angiography, and the interventional cardiology team is notified of the PMI risk, PMI diagnosis, and / or the recommended action plan before the procedure. In one embodiment, the risk notification is performed only when the PMI risk is enhanced (above a threshold, or enhanced to a confirmed diagnosis).

[0069] In another example, risk determination is performed multiple times during PCI, such as each time an image scan is performed. Whenever the risk is above a threshold or in response to each risk determination, the risk, diagnosis and / or recommendation is displayed to a medical professional. During PCI, a PMI assessment is performed. First, a patient with acute CAD is scheduled for an urgent invasive coronary angiography examination. Optionally, the patient's past medical images (e.g., past CCTA examinations) are input into an AI-based PMI assessment system. PMI risk, diagnosis and / or recommendation are provided at the beginning of PCI. Second, invasive coronary angiography is performed, and stents are implanted into the culprit lesions. During the intervention, PMI risk is automatically assessed continuously, periodically, and / or multiple times. PMI risk is assessed by an AI-based system in response to new data. Higher risks may reflect stent over-expansion, side branch occlusion, dissection, (distal) embolism, spasm, no or small reflux, or side branch flow interruption. If the risk of PMI exceeds a certain threshold (either a priori or dynamically defined), the interventional cardiology team is notified during the procedure. Due to the speed of AI-based risk determination, the cardiology team has the opportunity to intervene to reduce that risk. One or more possible causes of PMI risk can be mitigated.

[0070] As another example, risk determination is performed one or more times (e.g., multiple times) after PCI is performed for a patient. Whenever the risk is above a threshold or in response to each risk determination, the risk, diagnosis, and / or recommendation is displayed to a medical professional. For after PCI, a PMI assessment is performed. After an invasive coronary angiography procedure is performed for a patient with acute or stable CAD, the patient is continuously monitored after PCI for PMI risk using an AI-based assessment system. PMI risk is determined when new data becomes available or periodically, such as when new blood test results are available and / or in response to an image scan to determine PMI risk. In one embodiment, the patient wears a wearable sensor, such as a pressure, heart rate, or ECG sensor. PMI risk is regularly determined as new sensor data is acquired. After repeated blood tests, CK-MB and (hS) troponin may be found to be elevated. There may be other issues indicating PMI risk. The AI-based system determines that the PMI risk exceeds a threshold, so the proposed action plan is presented to the medical professional, such as modifying medications, getting the patient to the cathlab immediately, increasing monitoring, and / or other actions.

[0071] In other embodiments, the machine learning model is trained and used to assess other perioperative complications. Thrombosis, stroke, hemorrhage, or death may be assessed. A combination of multiple different complications may be assessed by different machine learning models, one for each complication, and / or by one machine learning model trained to output risk, diagnosis, and / or action guidelines for multiple complications.

[0072] The physician treats the patient based in part on the machine learning model output. The physician uses information about the patient and the output risk, diagnosis, and / or recommendation to select a therapy or not. The physician uses the output to assist in making treatment decisions.

[0073] Figure 4 A medical system for PMI or perioperative complication risk assessment is shown. The system generates a risk prediction, diagnosis and / or recommended course of action, such as before, during or after PCI.

[0074] The medical system includes a display 40, a memory 44, and an image processor 42. The display 40, the image processor 42, and the memory 44 may be part of a medical imager 48, a computer, a server, a workstation, or other system for image processing of medical images from patient scans. A workstation or computer without a medical imager 48 and / or a computerized medical record database 49 may be used as the medical system.

[0075] Additional, different, or fewer components may be provided. For example, a computer network is included for remote estimation based on locally captured scans and / or other data. As another example, a user input device (e.g., a keyboard, button, slider, dial, trackball, mouse, or other device) is provided for user input of data for estimating risk.

[0076] The medical imager 48 is a computed tomography, magnetic resonance, ultrasound, positron emission tomography, optical coherence tomography, intracardiac imaging, or single photon emission computed tomography scanner. For example, the medical imager 48 is a computed tomography system having an x-ray source and detector connected to a movable gantry on opposite sides of the patient bed.

[0077] The medical imager 48 is configured by the settings to scan the patient. The medical imager 48 is configured to perform a scan for a given clinical question, such as a cardiac scan. The scan produces scan or image data that can be processed to generate an image of the interior of the patient on the display 40. The scan or image data can represent a three-dimensional distribution of locations (e.g., voxels) in the patient's volume and can be used to detect anatomical structures and / or derive information indicating a risk of PMI.

[0078] The image processor 42 is a control processor, a general purpose processor, a digital signal processor, a three-dimensional data processor, a graphics processing unit, an application specific integrated circuit, a field programmable gate array, an artificial intelligence processor or accelerator, a digital circuit, an analog circuit, a combination thereof, or other devices now known or later developed for processing medical image data. The image processor 42 is a single device, multiple devices, or a network. For more than one device, parallel or sequential processing partitioning can be used. Different devices that make up the image processor 42 can perform different functions. In one embodiment, the image processor 42 is a control processor or other processor of a medical diagnostic imaging system (such as, a medical imager 48). The image processor 42 operates according to stored instructions, hardware, and / or firmware to perform various actions described herein.

[0079] In one embodiment, the image processor 42 is configured to train one or more machine learning networks. Based on the network architecture and training data provided by the user or other sources (e.g., data from the database 49 and / or the imager 48), the image processor 42 learns the characteristics of the encoder, decoder or other network parts to train the network.

[0080] Alternatively or additionally, the image processor 42 is configured to apply one or more machine learning models 45. For example, the image processor 42 applies scan data from the imager 48 and data from the patient-specific database 49 to the machine learning model 45. The network estimates PMI risk, diagnosis and / or treatment.

[0081] The image processor 42 estimates risk in response to input of data from different acquisition modes using a machine learning model 45. The input data is accessed from an imager 48, a database 49, and / or another source (eg, user input or a network interface).

[0082] The image processor 42 is configured to generate an image. An image is generated showing the risk, diagnosis and / or recommendation of PMI. The results may be displayed along with an image of the interior of the patient, such as a computed tomography image.

[0083] Display 40 is a CRT, LCD, projector, plasma, printer, tablet, smart phone, or other display device now known or later developed for displaying output, such as an image with risks and recommendations.

[0084] Training data, network definitions, features, machine learning models 45, image data, non-image data, detected anatomical structures, risks, diagnoses, recommended courses of action, and / or other information are stored in a non-transitory computer-readable memory such as memory 44. Memory 44 is an external storage device, RAM, ROM, database, and / or local memory (e.g., a solid-state drive or hard drive). The same or different non-transitory computer-readable media can be used for instructions and other data. Memory 44 can be implemented using a database management system (DBMS) and resides on a memory such as a hard disk, RAM, or removable media. Alternatively, memory 44 is internal to processor 42 (e.g., cache).

[0085] Instructions for implementing the training or application processes, methods, and / or techniques discussed herein by execution by processor 42 are provided on a non-transitory computer-readable storage medium or memory, such as a cache, buffer, RAM, removable media, hard drive, or other computer-readable storage medium (e.g., memory 44). Computer-readable storage media include various types of volatile and non-volatile storage media. In response to one or more instruction sets stored in or on a computer-readable storage medium, functions, actions, or tasks illustrated in the accompanying drawings or described herein are performed. These functions, actions, or tasks are independent of a particular type of instruction set, storage medium, processor, or processing strategy, and may be performed by software, hardware, integrated circuits, firmware, microcode, etc., operating alone or in combination.

[0086] In one embodiment, the instructions are stored on a removable media device for reading by a local or remote system. In other embodiments, the instructions are stored in a remote location for transmission over a computer network. Still in other embodiments, the instructions are stored in a given computer, CPU, GPU, or system. Because some of the constituent system components and method steps depicted in the accompanying drawings can be implemented in software, the actual connection between the system components (or process steps) may vary depending on the method used to program the present embodiment.

[0087] The various improvements described herein may be used together or separately. Although illustrative embodiments of the present invention have been described herein with reference to the accompanying drawings, it should be understood that the present invention is not limited to these precise embodiments, and that various other changes and modifications may be made therein by those skilled in the art without departing from the scope or spirit of the present invention.

Claims

1. A method for machine-based risk prediction for periprocedural myocardial infarction associated with percutaneous coronary intervention, the method comprising: imaging a patient by a medical diagnostic imaging system, the imaging providing first image data representative of an internal cardiac region of the patient, the first image data being from a first mode of data acquisition; acquiring second data of the patient, the second data being non-image data and being for a second mode of data acquisition, the second mode being different from the first mode; determining, by a processor, a risk of periprocedural myocardial infarction associated with the percutaneous coronary intervention by inputting first data responsive to the first image data and second data into a machine learning model; determining, by a processor, a predicted outcome based on the risk from at least the following: no myocardial infarction, spontaneous myocardial infarction, and periprocedural myocardial infarction associated with the percutaneous coronary intervention; and An indication of the predicted outcome is displayed.

2. The method of claim 1, wherein the imaging comprises: Computed tomography imaging, magnetic resonance imaging, x-ray angiography or intravascular imaging, and wherein the first data comprises anatomy, heart disease and / or heart function measurements, the first data being based at least in part on detections in the first image data.

3. The method according to claim 1, wherein obtaining the second data comprises: Obtain blood test measurements, ECG measurements, genetic data, demographic data, clinical data, or percutaneous coronary intervention characteristics.

4. The method of claim 3, wherein obtaining the second data comprises obtaining a blood test measurement result; The method further comprises: Third data of a third modality is acquired, the third data comprising one of ECG measurements, genetic data, demographic data, clinical data, or percutaneous coronary intervention characteristics.

5. The method of claim 1 , wherein determining the risk comprises: The risk is determined as a forecast for a future time.

6. The method of claim 1 , wherein determining the risk comprises: The likelihood of readmission, revascularization, length of stay, death, and / or major adverse cardiac events (MACE) was determined.

7. The method according to claim 1, further comprising: A recommendation for treatment of the patient is identified, the recommendation corresponding to the risk.

8. The method of claim 1, wherein the determining comprises: The determination is made using machine learning models including neural networks.

9. The method of claim 1, wherein the determining and displaying are performed before the percutaneous coronary intervention is performed on a patient.

10. The method of claim 1, wherein the determining is performed a plurality of times during the percutaneous coronary intervention performed on a patient, and wherein the method further comprises displaying information when the determined risk from one of the times is above a threshold level.

11. The method of claim 1, wherein the determining is performed multiple times after the percutaneous coronary intervention is performed on a patient, and wherein the method further comprises displaying information as a suggested course of action if the determined risk is above a threshold level.

12. An AI-based system configured to perform the method according to any one of claims 1 to 11.

13. A non-transitory computer-readable storage medium storing instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 11.

Images